Abstract:
This thesis describes an investigation into the utility of facial expression information for the task of automated deception detection, using methods from computer vision. We present a thorough review of the current understanding of deception in Psychology, and then document previous e orts within computer vision to address the problem. We develop a deceptive interview scenario based on the widely used mock crime paradigm, as a way to obtain data containing facial behaviour during deception. Subjects are recorded using the Microsoft Kinect and Fuji lm W3 cameras, with depth maps obtained from each. All image sequences and corresponding depth maps are registered with respect to face location. For this, we obtain an a ne transformation for each frame, derived from ducial face feature points tracked via template matching. From each registered frame, and optionally for each depth frame, a feature vector is constructed, containing a number of concatenated histograms of oriented spatio-temporal gradients. Each component histogram represents an isolated sub-region of the face that provides strong expression information. This formulation allows the feature vector to account for both structural and dynamic information in the image sequence. Following this, we develop a deception detection scheme based on dimensionality reduction through principal component analysis, followed by classi cation via the nearest-neighbour algorithm. The accuracy of the system at detecting deception is evaluated by leave-onesubject- out cross-validation. The results of the validation testing did not support the e cacy of our algorithm on the dataset, nor did they support the inclusion of the depth channel as a way to increase accuracy. Finally, we identify some shortcomings of the work, providing suggestions and directions for future research in the area.