Abstract:
Extracting human silhouettes from video sequences is often a first step for highlevel vision analysis tasks, such as video surveillance, identification, people tracking, or activity recognition. Although there is numerous research on how to detect human silhouettes in video sequences, there are still challenges in the field due to background local motion, moving shadows, or changes in lighting conditions. This thesis presents a new robust human silhouette extraction system. In our system we proposed a shadow detection method for silhouette extraction of a person in grey-level video sequences. We use a shadow evaluator to verify whether a candidate shadow pixel, detected by Gaussian distribution analysis, is actually confirmed to belong to a shadow region. The evaluator considers a candidate shadow pixel initially as being a false-positive, and marks it as a silhouette pixel if it is enclosed or semi-enclosed by moving occlusion boundaries of a person. Those boundaries were extracted by subtracting edges in the current frame from edges of the background. We also propose a silhouette-compensation technique to recover some missing (i.e. removed) silhouette pixels by using a similarity criterion between silhouette pixels and their neighbours. The thesis also proposes a silhouette tracking algorithm based on a silhouette feature vector. Experimental results show that the proposed shadow detection method detects a silhouette of a person more accurately compared to other methods. Methods suggested by other researchers in YUV or RGB colour space typically identify silhouette pixels incorrectly as being in a shadow if the colour of these pixels is similar to that of the surrounding background. The effectiveness of our proposed compensation algorithm and of our suggested silhouette tracking method are confirmed by experimental results. There can be various applications of the proposed techniques such as 3D human pose understanding for evaluating the performance of body movements (e.g. for disabled children, or for sportsmen), up to artistic video rendering (e.g. when replacing a moving silhouette by a “ghost effect”). The work on this thesis led to two publications at international conferences, see [1] and [2]. Keywords: silhouette extraction, silhouette compensation, silhouette tracking, shadow removal, shadow evaluation, occlusion boundary.