Chaos – “behavior so unpredictable as to appear random, owing to great sensitivity to small changes in conditions.”
There are ample videos of Marines in combat — US Marines Intense Combat Footage In Fallujah | Operation Phantom Fury – YouTube.
There are ample videos of law enforcement resolving hostage situations – https://youtu.be/hNHNhqG3a28
There are ample videos of police dogs and their handlers — Witnesses Describe Chaotic Scene After K-9 Police Dog and Suspect Shot to Death – NBC Chicago
There is a common theme in most of these videos. There is a lack of understanding of what is truly happening in any point of time. There is confusion because of the lack of real time, accurate, active intelligence. And when there is confusion, mistakes are made.
This post and following posts will show how machine learning, edge computing, capture of stereo video streams, speech to text audio conversion, radar, point clouds, Kafka to minimize latency, Azure Media Server, IOT and the cloud in a secure environment can be applied to reduce the lack of understanding context. With more effective real time analysis, better understanding of context leads to more folks surviving and going home at the end of the day. All of this is possible with off the shelf hardware and software and at a relative cheap cost.
The key is to detect and classify to determine intent. Time is a constraint. Seconds determine success or failure. The sensors used to collect and process data are mobile and mountable on both humans and working dogs. The dogs become a platform providing the handler with streams of real time processed intelligence.
In this post, a building block view is illustrated. Blocks are described at a high level. In future posts, each block will be described in greater detail.
Edge Computing:
“Edge computing allows devices in remote locations to process data at the “edge” of the network, either by the device or a local server. And when data needs to be processed in the central datacenter, only the most important data is transmitted, thereby minimizing latency.”
Edge Computing devices are hardware devices with sufficient compute power to process real time streams of cameras, audio, and radar.
Raspberry Pi 4 8 Gig Edge Device:
The Raspberry Pi 4 was chosen based upon its outstanding price/performance.
This product’s key features include a high-performance 64-bit quad-core processor, dual-display output via two Micro HDMI ports, up to 4K resolution, hardware video decoding at up to 4Kp60, up to 4GB of RAM, dual-band 2.4/5.0 GHz wireless LAN, Bluetooth 5.0, Gigabit Ethernet, USB 3.0, and PoE capability.”
On the edge, a Raspberry Pi 4 B processes stereo video streams, audio streams, radar point clouds and determines intent.
The 4 B has Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.8GHz.
As price performant as the 4 B is, it lacks it lacks the processing power to process the required streams with frames being captured every 30 milliseconds.
To meet sub second response time USB sticks from Intel are attached to the 4 B. Additional information about the Intel Movidius compute stick can be found here. https://www.intel.com/content/www/us/en/support/articles/000033354/boards-and-kits/neural-compute-sticks.html
It is possible to get a fairly accurate classification of intent with just video, but being able to determine location of speakers and capture audio streams and fusing the video with audio, increases the probability of an accurate classification of intent.
Additional information about the ReSpeaker Audio Array Microphone can be found here.
There are courses and papers on leveraging radar to satisfy autonomous vehicle requirements. There is a great deal that can be learned from ThinkAutonomous. When I read the papers referenced on the site, the thought was always in the back of my head, the models, frameworks, and hardware were all equally applicable to law enforcement and the door kickers in both law and the military.
Then there are point clouds. As understanding and leveraging point clouds are important to autonomous vehicles, point clouds are of equal value when humans and working dogs are trying to understand an unknown context.
More can be learned about point clouds on ThinkAutonomous and uRad. On uRad look at doppler radar and point clouds under the automotive board — replace automotive with human beings and working dogs.
“The OAK-D Pro is an upgraded version of OAK-D, featuring an IR laser dot projector (active stereo) and IR illumination LED (for night-vision). OAK-D Pro is from the Series 2 of OAK cameras. It’s smaller and lighter than the OAK-D.”
The OAK-D Pro is a stereo camera, has an IMU, comes with a Y connector to support IR illumination, and its foot-print is ideal as a mobile camera — much less physical space required than the Microsoft or Intel 3D stereo cameras. Support provides hundreds of models that have been ported to run on the Intel Movidius processor which is internal to the camera.
Over a decade ago, IPhone, Windows, Nokia, Blackberry, and Pixel were the major cell phone manufacturers. Integrating and controlling so many versions of cell phones into the enterprise was a complex problem. A number of third party solution providers rose to the challenge providing mobile deployment applications.
About a decade ago, Azure, AWS, IBM, and Google started adding cloud-based services — pay for what you use.
Even before cell phones, the term Internet of Things (IOT) was coined.
“The Internet of things (IoT) describes physical objects (or groups of such objects) with sensors, processing ability, software and other technologies that connect and exchange data with other devices and systems over the Internet or other communications networks.[1][2][3][4][5] Internet of things has been considered a misnomer because devices do not need to be connected to the public internet, they only need to be connected to a network,[6] and be individually addressable.[7][8]“
Fast forward a decade, devices get smaller, cost of compute power becomes cheaper, and what are defined as IOT devices, by the billions, are made operational around the globe. There is opportunity to make money. These reasons, coupled with the fact that clouds are finite, edge devices, providing services for the edge devices, is a win for cloud service providers.
The devices, Lidar, Radar, stereo camera, microphone arrays, and Raspberry Pi with Movidius sticks, are all IOT devices and can be made more effective by leveraging cloud-based IOT services.
As time permits, more details about each device and integration with cloud IOT services to follow.
Discover more from Threat Detection
Subscribe to get the latest posts to your email.