Abstract:
This thesis presents and describes the steps to build a benchmark for evaluating the reliability and performance of a lane-border detection algorithm. Building a reliable benchmark is important as it should consist of three components: 1) a variety of data-sets, 2) a reliable tool to extract and generate ground-truth, and 3) a reliable quantitative measure to evaluate the performance of an algorithm on a data-set with available ground-truth. For these reasons, a ground-truth generation approach was developed by Borkar et al.; this approach was selected to be further developed and tested. When testing the original approach on different types of data-sets, a few problems were identified and the approach needed to be improved to enable it to generate ground-truth on any input data showing a lane scenario. A number of data-sets were selected from three different sources, each with different characteristics. These were based on: visual appearance, shown scenarios, and image quality. The improvements worked well on most of these sequences but still had some problems with really noisy images, as the original approach relies on the availability of clearly visible markers in the images. This thesis also presents a test protocol and two novel evaluation measures selected to compare the generated ground-truth with calculated lane-borders. Two graphical representations can be generated from the results of the evaluation. A graph of a lane-border performance on an entire sequence is presented, together with a graph comparing the performances of different algorithms on the same dataset. Both help identifying inconsistencies between generated ground-truth and calculated lane-borders and between the tested algorithms. The benchmark was tested by applying two lane-border detection algorithms, provided by their authors. The availability of ground-truth data and test protocols clearly helps understanding the performance of these algorithms. The experiments also highlight a few detected issues found for each of them on the selected data-sets. Keywords: Lane-border detection, Ground-truth, Performance evaluation, Measures, Edge operators, Cubic spline, Benchmarks. Co-authored publications: [2] Image and Vision Computing New Zealand (IVCNZ) 2008, Christchurch [3] International Conference on Computer Vision and Graphics (ICCVG) 2014, Warsaw, Poland