BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Smarter Cities: Will Autonomous AI Surveillance And IoT Now Automate Law Enforcement?

Following
This article is more than 5 years old.

Picture a CCTV camera installed onto the side of a building or onto a telegraph pole in a busy urban space. The camera has no pre-set configuration or coded rules, it simply observes its environment, studying and classifying patterns of life, continually learning. The camera will detect anomalies in the behavior and movement of people and vehicles and objects, in environmental conditions, without ever having been instructed as to what such an anomaly might look like. Every object will be detected and classified. Metadata will be captured under strict privacy rules. No imagery will be stored or streamed unless it relates to an incident or an anomaly. Unless it’s marked.

Now, connect that camera to others, to local clusters of cameras on one level, to entire networks of cameras on another, and the depth of machine learning is staggering. Sensors share data, they compare results, they train one another. They work as a system, quietly and unobtrusively, to learn and protect their environments. Within the multi-trillion-dollar Smart City market, the ongoing fusion of cloud and edge is progressing us towards this level of distributed intelligence, towards autonomous surveillance, whether we're ready or not.

The Fusion Of Cloud And Edge

Just as the cloud completes its mopping up and analysis of all the data known to humankind, here comes the IoT. This race to the edge will network billions of intelligent devices and automate our world. The IoT "is growing at a breath-taking pace," says Intel. "From 2 billion objects in 2006 to a projected 200 billion by 2020." The IoT underpins the many varied applications of artificial intelligence that Accenture predicts "could double annual economic growth rates [by] 2035," leading to "an economic boost of $14 trillion in additional gross value added." IoT in all its guises will drive the annual growth in data transmission from 25% to 50%. It will also shift processing from the cloud to the edge. There is too much data, it’s too indiscriminate, too centralized, and it takes too long to access. According to Accenture’s 2018 Technology Vision Report, this ‘internet of thinking’ extends intelligence from cloud to edge. “To fully enable real-time intelligence, businesses must shift event-driven analysis and decision processing closer to points of interaction and data generation. Delivering intelligence in the physical world means moving closer to the edge of networks.”

Despite the Cloud Vs Edge debate that has emerged in some quarters, what we will see in practice is a fusion of Cloud and Edge, shaped by the imperatives of AI. Professor Stephen Hawking said of AI that “every aspect of our lives will be transformed. In short, success in creating AI could be the biggest event in the history of our civilization.” Real-time video is fundamental to much of this, whether steering vehicles, fighting battles or smartening cities. And intelligent video analytics means high resolution, which means serious bandwidth and latency. This has been a major driver of the fusion of cloud and edge. "Computing will become an increasingly movable feast," says the Economist. "Processing will occur wherever it is best placed for any given application."

Executed properly, autonomous surveillance necessitates the combination of cloud and edge computing, it requires an end-to-end AI chain that can apply levels of processing flexibly based on need and equipment, as well as an intelligently distributed architecture where captured data and reference datasets can be shared and synchronized in real time.

The Staged Autonomy Of AI

The progression of AI commonly follows a four-stage pattern:

  1. It begins with some combination of smarter search, machine learning and natural language processing to help humans modify a particular process or activity to make it easier and quicker and cheaper.
  2. Beyond the initial stage, continued learning enables AI to part-automate, to filter out lower-level elements, until the process or activity becomes even better, even cheaper, even faster; people act as the interface between AI engines, applying point actions and decisions based on filtered information and streamlined workflows.
  3. Then comes the ‘AI of AI’, bringing together multiple part-automated processes, where people are no longer required to interfere or interface; the process or activity is fully automated and self-improving; beyond setting an objective, direct human intervention is not required.
  4. All the AI fear-mongering centers on the fourth and final (and thus far theoretical) stage. Here the process or activity is replaced: AI deduces a different way of reaching the objective, then AI sets the objective.

Vehicles have followed this path: voice-driven sat-nav, AI-powered route management and logistics, driver assistance tools, safety aids, predictive maintenance and driver monitoring, towards integrating all these processes and many more into fully automated driving. In short: first, make the commute quicker; then make it safer and more fun; finally, make it chauffeured. And stage four? Ferrying a human from home to office is not the most efficient way of completing the human’s work product.

In the military, Battlefield 2.0 is emerging as a chess game between algorithms. Full automation will only be stopped by policy and fear, not by technological limitations. With objectives set, battles can be fought between autonomous machines. Certainly, there will be more thinking machines than thinking humans in any theater of war. Human leaders will set battle objectives. And human fighters will deploy with machines to direct their efforts. But, as I've written before, there's a fine line between pointing out the enemy and taking out the enemy. And stage four? Set against the context of the AI Cold War between the US and China, algorithms will calculate that first strikes equal first mover advantage, negating the need for battlefields. Cue policy and fear.

In surveillance, stage one was filtering data, metadata-tagging imagery, saving hours from mundane tasks such as manually sifting video footage, using offline algorithms to classify and match objects, to narrow the haystack. Now we have reached stage two. Algorithms are more advanced, matching faces, gaits, vehicles. Now we can automate lower-level decisions, drawing conclusions based on inferences and filtering intelligence for humans. We can detect threats by learning what such threats look like. We can mine data to inform and predict. We can manage scarce human resources based on AI-derived prioritization.

The progression to autonomous surveillance is stage three.

Smart(er) City Surveillance

Street crime and antisocial behavior: a painful inconvenience for citizens; a resourcing issue for the agencies protecting them. As a result, lower-level crime is filtered and we have predictive toolsets to focus policing. Now add new autonomous surveillance networks into the mix. Incidents will be caught and captured by cameras, by multiple AI algorithms applied in combination to infer complex anomalies. As long as one camera has captured and classified an anomaly, all cameras can look out for it.

But it doesn’t stop there. Human intervention, even involvement, can be pushed way down the process. The interesting thing about policing is the level to which it depends on repeatable processes. And repeatable processes are ideal candidates for automation. Cue AI.

Following an incident: facial recognition; gait recognition; logo recognition on items of clothing. All automated. All in real time. The system enrolls the suspect(s) on its watchlist, based on their actions and subject to strict policy and protocols. For example, the system recognizes a street robbery and immediately triggers an automated process that includes processing the suspect and summoning care for the victim. No law enforcement person is yet in the loop. The network is alerted. The suspect(s) can be tracked, identified, located; the evidence case managed. An appropriate response team can be sent to the right place at the right time. The AI’s intelligently distributed architecture has tiered and filtered the data. People still make the final decisions. But the deductions on which those decisions are based are immediate and systemized.

Store Less, See More

Networked surveillance video is expected to see a seven-fold increase over the next four years. And, in addition to bandwidth constraints, there is also a limit to cloud storage. The video data produced in a single year by the world’s currently deployed CCTV cameras, most of which are still SD, exceeds the planet’s current data center storage capacity. We couldn’t stream and store all that data even if we wanted to. Not only does analytics provide immediate intelligence, but it also filters the video that needs to be stored from the rest that can be discarded or not even captured in the first place. There is a school of thought in surveillance that if you capture too much data, you capture too much data. Simply put, if you hoover up everything then you won’t be able to find anything. If clever thinking is done upfront, and with AI not all that thinking has to be human, then the data that is captured and shared is likely to have significantly more value.

With advanced smart city surveillance, steady-state analysis can remain at the edge, but there is always the need to stream video for further analysis and higher-level decision making. In real-time response to events, in law enforcement, in public safety, people in control rooms and the cloud processing supporting them need to see critical visual data. And the focus is giving them exactly what they need, exactly when they need it. Just in time surveillance, rather than the petabytes of just in case video clogging data centers today. Digital buildings are the same. As enterprise security and safety systems morph to the cloud, there remains a need for tiered analysis of data. Access control is different to intrusion detection for known threats. Dedicated edge processing can be overly simplified or fixed. As a system learns and develops, its outlying nodes should do the same.

Hardware To The Rescue? Not Quite

One answer to live video analytics is to upscale edge hardware. Dell launched an IoT division last year, which included a $1 billion investment over three years. "In an age where every type of device, from phones to cars to oil rigs to robots to heart monitors are alive and intelligent," they said on launch. “These devices simply cannot wait for a response from centralized cloud infrastructure that may be seconds away.” According to chairman Michael Dell, his company would lead the way “with a new distributed computing architecture that brings IoT and artificial intelligence together in one, interdependent ecosystem from the edge to the core to the cloud.”

This rebalancing of Cloud and Edge has become the trend. Amazon’s AWS Greengrass, Microsoft’s Azure Data Box Edge and Google’s Cloud IoT Edge push the same theme: distribute AI across the network, share the processing and storage load, push data around intelligently.

The growth in the development of edge-based AI chips from Qualcomm, Horizon Robotics and others, along with ever more powerful GPUs, reflects this. As does open IoT edge platforms like SAST from Bosch. Analyze video at the edge; strip the metadata; don’t push raw data across networks in real time. But, although edge processing in silicon drives efficiency and reliability, it also prioritizes local processing over a distributed, intelligent architecture. It fixes the selection of an AI engine at the edge. It prevents open-access innovation without a hardware investment. Not easy to blend edge and center when the data is live video at a spiky bitrate inviting latency and packet loss. And so this shift to AI at the edge addresses latency and bandwidth, but it comes at a cost.

To deliver the full cloud-edge architecture for live video, there have to be limits on the amount of data that needs to move at zero latency in real time. Systems at the edge sit within a distributed, fully connected architecture that can operate in either an online or offline mode. A distributed architecture is not simply a structure, it needs to be real-world proofed and adaptable. Data needs to be focused and cut into manageable packets for live transmission and processing. Edge and central systems need to work in lock-step. Available bandwidth needs managing efficiently.

One example of just such a distributed architecture in action comes from the military’s increasing reliance on drones. Picture an urban battlefield, with a number of drones dispatched to locate hostile combatants known to be hiding out. Lightweight processing on the drones can detect where people are detected in inferred hiding places. These detections and possible locations can then be sent to the cloud for more rigorous analytics, including deeper processing and cross-referencing input from multiple sensors, all to distinguish between combatants and cowering locals: essentially, object detection and classification at the edge, with object matching on result sets at the center. As the military moves forwards with ever more focus on drones, including the management of swarms of flying battlefield objects, physical communication and coordination become a major challenge. The equation is the balance of weight, power and processing. It comes down to architecture.

The Intelligently Distributed Architecture

Seamless, intelligent connectivity is the real vision for Edge-AI. Not the rush to equip edge devices and sensors with ever-cheaper AI silicon and dedicated GPUs. Distributed processing, designed to balance a lightweight edge with a limitless center, built to deliver split-second decision making on real-time data, relies on that. Many IoT video devices will be mobile, with the additional stress on video codecs from the frame to frame scene change with a moving sensor. 5G is imminent but not a panacea for the sheer scale of the networking challenge. Quality of service and universality of any specific offering will be front of mind. Solutions need to override issues with congestion and coverage.

Networking between cloud and edge has to be designed into the architecture of solutions. And, in addition to efficiency, this brings further benefits:

  • Live, real-time systems. The distributed cloud-edge architecture, with AI running seamlessly end-to-end, provides for live control data, such as watchlists and datasets. Results from edge decisions or detections can also be shared across the network. Applications can run on this common data across multiple platforms in multiple locations to deliver a connected system of systems without bandwidth becoming an impediment. Such platforms can have varying levels of onboard processing and storage, essentially a trade-off between the price of an edge device and the cost of bandwidth. This also allows for metadata streaming, with raw data kept at the edge or pushed to the cloud, a key enabler of business analytics.
  • Secondary processing. An integrated system can have lightweight AI processing at the edge, with selected data passed to the cloud at zero latency for additional, deeper processing. This might be a facial recognition or object classification match that can be further processed on a more capable system. By only pushing matches, the connection is not overwhelmed.
  • Multiple AI engines. If data can be pushed to the cloud, then multiple AI engines can be made available. AI engines become a pay-per-use utility, with a distributed architecture interfacing between them. If an edge device starts the processing chain and is equipped with the capability to move lightweight data, then multiple AI engines can be used without any edge infrastructure replacement. Conversely, if an edge sensor is deployed with a single vendor’s AI embedded in silicon that is a long-term lock-in.
  • Clustering. If sensors can move lightweight, real-time data to the center they can do the same to one another locally. This delivers clustering. An architecture where the value of a local set of sensors becomes greater than the sum of its parts. An example would be multiple cameras sharing analytics. If several local cameras armed with facial recognition each register a 30% chance that they detected a specific individual, none would alert on their own. But the combination might. This approach borrows from integrated surveillance platforms used within the military, where rules run across multiple sensors.

Cloudy Vision To Cutting Edge

Intelligently distributed architectures are a prerequisite for live video analytics. The fusion of cloud and edge, with efficient use of bandwidth and processing, underpins the next stage of development. This means cloud and IoT and networks working in lock-step. And it means evolving our approach, such that surveillance AI operates across systems, rather than being locked on sensors.

What will be interesting is the impact these developments have on the broader AI landscape, beyond video and surveillance. If heavy data can be intelligently packaged and moved around a network instantly, at zero latency, there will be many benefits. The core underlying software that drives these high-end use cases will find its way onto open platforms and networks. Stage three of surveillance AI with its autonomous cameras will depend on it.

Whether or not we are ready for cameras in our streets, public spaces, homes and workplaces that think for themselves, as well as for each other, we will soon find out. Autonomous surveillance systems will raise fundamental questions about the balance between public safety and individual privacy, generating serious debate about where we should draw the line.

So, what about stage four of surveillance AI? What happens beyond autonomous cameras? Well, why should a smarter city empower humans to set and enforce laws when an AI system can do so much more efficiently and effectively without much human involvement at all?

Follow me on Twitter or LinkedIn