Objects Detection and Tracking Using mmWave Technologies

Abstract

Smart sensing systems have been a significant focus in the industrial community. With the integration of intelligent sensors, controlling buildings and factories autonomously has become both economical and straightforward. However, the central concern in this field remains to identify effective detecting and tracking mechanisms to make real-time decisions and overcome environmental challenges. Therefore, millimeter-wave radar technology (mmWave) has garnered enormous attention from researchers worldwide. The design of efficient object detection, tracking and classification models based on mmWave technology emerges as a possible solution to the challenges of smart sensing systems. This study presents several strategies for detecting, tracking, and classifying multi-objects. These methodologies have been developed to overcome some significant obstacles in smart sensing systems, including accuracy and computing constraints. In the first phase of this research, we developed a small and fast object detection and tracking system for indoor applications. The system operates on an embedded platform in real time with high accuracy. One of the most common challenges of sensing technology is ensuring portability and scalability due to computing constraints. To overcome this challenge, we propose a system with two efficient clustering algorithms for rapid processing and high accuracy. The Recursive Kalman Filter (RKF) tracking algorithm also helps reduce algorithmic complexity and computation time. We evaluated the proposed method through several real indoor scenarios based on a Raspberry Pi platform, demonstrating its effectiveness. The subsequent phase of this research proposes a multisensor-based fusion system for indoor object detection and tracking. The system employs two noise reduction stages to distinguish cluster groups and remove noise effectively. The proposed data fusion method effectively estimates the transformation of the data alignment, synchronizes the results, and visualizes object information acquired using one radar on another. The tracking algorithm with data association accurately tracks multiple objects simultaneously. We demonstrated the proposed system's effectiveness and advantages over commonly used methods in scenarios involving occlusions and weak data due to sensors returning weak signals or being lost from the sensor's view by low signal reflections. This fusion system consisting of dual sensors has the potential to improve tracking accuracy significantly. In the final phase of our research, we proposed an identification and classification system that utilizes mmWave radar and deep learning. The point cloud data undergoes processing using a refined density-based clustering algorithm to accurately extract ground truth in 3D space. We used a bi-directional long short-term memory (BiLSTM) network trained with center loss added to the softmax loss to recognize the identities of objects and address the scattered features issue. A Random Forest layer predicted a binary output identifying intruders or insiders. We evaluated our proposed system through a comparative experiment against existing architectures in the literature. We extended the sensing technology into a dynamic platform and examined the potential of mmWave radar sensors in robotic navigation for indoor object avoidance and navigation purposes due to the increasing demand for such services. The performance of the proposed strategies was evaluated and tested within practical operating limits throughout the thesis. Although the proposed object detection and tracking mechanisms offer a promising alternative to the currently established methods, there is still potential for improvement in future work concerning theory and application.