The Computers Have a Thousand Eyes: Towards a Practical and Ethical Video Analytics System for Person Tracking

Abstract

With the rise of computer vision, video analytics systems have become more prevalent in the real-world, which automatically process video camera footage to produce information for human users. However, the majority of the existing research has looked at small parts of the overall puzzle in isolation, often focusing on accuracy while ignoring other practical requirements such as computation time and protecting privacy. The research presented in this thesis focuses on the development of a person tracking video analytics system that moves towards what is needed for real-world implementation with embedded systems. This is contextualised in a motivating scenario based on commercial market research, a modern application of computer vision moving away from traditional state-controlled surveillance. The aim is to balance the trade-off between accuracy and speed, with a modular pipeline to allow for better control and transparency into the algorithms being used. The developed video analytics system consists of a novel super pixel-based background estimation algorithm, person detection using off-the-shelf methods, an unsupervised person re-identification approach for classifying identities across multiple camera views, a Kalman Filter-based spatio-temporal model for tracking people by their positions, and a model fusion module that combines the appearance-based re-identification and spatio-temporal models together to improve the classification accuracy. In addition to the algorithms, this thesis also investigates the privacy loss encountered by these types of video analytics systems, firstly through a survey into public perceptions of privacy around surveillance cameras, and secondly through the proposal of a system architecture that uses computer vision and embedded systems to help protect privacy by default. This is then implemented with the use of smart cameras, and the impacts on accuracy, speed, and networking constraints are discussed. Lastly, further techniques for accelerating computer vision tasks in embedded system contexts are presented, with a case study demonstrating the use of Hardware/Software Co-design. The combination of all of these different factors brings a holistic view to the development of practical and ethical video analytics systems for person tracking, making progress towards overcoming the challenges faced by system designers and developers in real-world implementation.