Practical Real-time Hyperspectral Video Stream Processing

Abstract

Hyperspectral imaging allows quantitative evaluation of material composition and spatial distribution, and finds numerous applications in areas such as remote sensing and military reconnaissance. Hyperspectral images contain quantitative spectral information for each pixel and a useful model is to assume all pixels are linear combinations of so-called “pure” endmembers. Traditionally, hyperspectral algorithms have considered only static images and existing “real-time algorithms” process single frames without regard for sequential similarities or correlations. Recent advances suggest that hyperspectral video streaming hardware will be technically feasible in the near future. This will stimulate applications in novel areas such as real-time crime scene analysis. The difficulty in capturing and processing hyperspectral video sequences in real-time can be traced directly to the high-dimensionality of the data. To realise these applications and explore potential benefits, further work on hyperspectral video algorithms is required. This thesis focuses on hyperspectral video processing using linear mixing and geometric endmember determination. This broadly involves four steps: dimension reduction, endmember determination, abundance map generation and visualisation. An overarching aim of the work has been to exploit inter-frame redundancies to make hyperspectral video a reality. Methods to reduce the computational complexity of dimension reduction and endmember determination are presented. These methods rely on information obtained from previously captured hyperspectral frames and exploit temporal correlations inherent in the data. The widely-used convex combination of endmembers is shown to be ineffective in the presence of non-uniform illumination. An improvement to Abundance Guided Endmember Selection (AGES) is proposed. This improvement, termed Shadow-corrected AGES (SAGES), uses a modified simplex that compensates for illumination variations and is shown to be more effective across a range of scenes. The problem of visualising changing abundance maps in the video stream is addressed and a novel method to display abundance information is presented. This method uses a reduced colour-space to visualise areas of interest, intelligently updates the display to reduce viewer fatigue, and caters for colour blind operators. Finally, a hardware implementation using CUDA-enabled devices is considered. AGES-like algorithms are shown as effective alternatives to existing real-time algorithms. Recommendations are made for further development of video streaming hyperspectral imaging systems.