Egomotion Estimation for Dynamic 3D Roadside Reconstruction

Abstract

In recent years, stereo analysis and video understanding technologies have become model examples for applying mature vision technologies to various academic areas or industries to improve accuracy, productivity, and reliability. Computer vision technology has been recognised as a suitable and proper solution for many expensive tasks that require a great amount of effort on observations and precise calculations. The demand of such systems is only likely to increase in the near future. Vision-based egomotion estimation, also known as visual odometry, is considered one of the mentioned expensive tasks. Estimating egomotion is a necessary step to understand the actual scene from the input images, it tells the positional (where) and rotational (how) data of the camera(s) movements, in order to solve a wide range of major problems in computer vision. The focus of our study is stereo-vision based egomotion estimation. Our proposed visual odometry method follows the traditional workflow of visual odometry algorithms: it firstly establishes the correspondences between the keypoints of every two frames, then it uses the depth information from the stereo matching algorithms, and it finally computes the best description of the cameras’ motion. However, instead of simply using keypoints from consecutive frames, we propose a novel technique that uses a set of augmented and selected keypoints, which are carefully tracked by Kalman-filter fusion. We also study the possibilities of using the multi-sensory integration and multirun scenarios to effectively improve 3D shape reconstruction results towards 3D roadside reconstruction. We propose to use the GPS data for key frames in the input sequence, in order to reduce the positioning errors of the estimations, so that the drift errors can be corrected at each key frame. We aim to bound the overall growth of the build-up errors over the increasing travel distance. A least-squares process is used to minimise the reprojection error and to ensure a good pair of translation and rotation measures, frame by frame; see the fourth chapter for details. Keywords: Egomotion estimation, visual odometry, stereo vision, feature tracking, optical flow, Kalman filter, bundle adjustment, 3D reconstruction, multi-sensory integration, multi-run scenario.