There are several options to consider when it comes to analytics for video surveillance systems: deploy analytics on the edge, or “in-camera”; use a dedicated server; or use a hybrid approach which leverages both edge and server implementations.

The Power at the Edge

As more powerful in-camera chipsets are developed, edge devices continue to evolve. They are capable of utilizing powerful analytics that can inform operators on a wide range of real-time video or audio events requiring attention. Some cameras include video analytics like loitering, line crossing, face detection, enter/exit and direction detection. Audio classification can warn of glass breaks, gunshots and screams. More sophisticated analytics like queue management and heat maps bring new opportunities for business and traffic intelligence.

The next step in the evolution will come from a form of AI (Artificial Intelligence) called Deep Learning. Deep learning takes machine learning to a new level based on neural network theory that mimics the complexity of the human brain. Previously only available as a server-side process which would require video to be decompressed and processed, algorithms and companion data sets created from deep learning can be embedded on the edge to perform real-time detection and alerts for the specific conditions they have been trained to recognize. While edge devices cannot continue to self-learn, they can provide a useful function for recognizing known objects or behaviors. Still in its infancy, this technology will take in-camera analytics to a new level for both security and retail use cases.

Servers Have an Important Role to Play

Even with the increased analytic power available on the edge, there are certain tasks that benefit from running on a powerful server. As an example, license plate recognition demands more than optical character recognition, it also requires an interface to a large database where plates can be stored, cross-referenced and flagged. Likewise, facial recognition requires sophisticated database analytics for the same reasons. Dedicated servers are more expensive than edge devices with in-built analytics, so it’s important to clearly understand the needs of a business – Do we need to detect an object or a specific person? For now, machine learning and deep learning algorithms that continuously learn require the computational horsepower and storage that only a server can provide.

Bandwidth Limits – Server vs. Edge

As powerful as servers are, they do have limits. If a server is going to analyze video, it must first decode it, which consumes a fixed amount of CPU/GPU resources. After the video is decoded, then it can be analyzed. As the video channel count grows, it’s possible to saturate even a powerful server very quickly. Edge devices are immune to this problem because analytics are typically run right after the image is captured and before it is encoded into an IP video stream. The results of the analytics are then passed as data to the VMS, NVR or messaging system as metadata, which is quite small compared to a video stream.

Enter the Hybrid

For the reasons above, you can imagine that for powerful analytics processes, such as number plate or facial recognition, an efficiency can be derived from combining edge analytics with server-side processing and database management.

This hybrid approach is already in use by some vendors as a way to deliver powerful analytics at a fraction of the previous cost. In order to enable this, some camera manufacturers have reserved space on their cameras to allow third-party plugin analytics to be installed which pass data directly to the server. The video doesn’t need to be decoded which saves precious CPU/GPU cycles. This represents an ideal scenario that uses the best of both worlds: Edge analytics harvesting data before video is encoded, and servers receiving lightweight metadata from the edge to perform analysis. Whereas a single server might be able to decode and analyze 20 channels of video, the same server processing metadata provided by the cameras could analyze over 250 channels. The cost savings of such an implementation is substantial.

Conclusion

The edge will continue to deliver compelling analytics capabilities that suit a majority of use cases. The evolution of AI and deep learning will accelerate both edge and server capabilities to new levels in our industry. For the most demanding analytics, database processes and AI implementations, a hybrid approach can yield the best business value.

Written by Paul Kong who is the technical director for Hanwha Techwin America