Mixed Reality Simulation for Mobile Robots

Abstract

In recent years robots have dramatically improved in functionality, but as a result their designs have also become more complicated. The increase in the complexity of the tasks and the design of the robot result in a challenging and time-consuming robot development process. Sophisticated simulation models, expensive hardware setup, and careful human supervisions are necessary to provide a safe and close-to-real world testbed that helps to draw insights to the actual operation. However, the considerable cost and time for creating well-designed tests and for meeting safety requirements present a barrier to rapid development of complex robot systems. This thesis analyses the robot development process and identifies areas for improvement. Based on the requirements analysis, it proposes that, by applying the concept of Mixed Reality (MR) to robot simulation, it is possible to create a realistic, synthetic environment that enables robot developers to experiment freely and safely. The new simulation approach, named MR simulation, offers the flexibility of creating simulation environments composed of real and virtual objects that seamlessly interact with one another. This enables robot developers to mix virtual objects into the real experimental setup for cost and safety reasons. A generic MR framework is proposed that provides the basis for creating MR simulations. The most novel element of the framework is a behaviour based interaction scheme that formalises interactions between real and virtual objects. To validate the framework, a general purpose MR robot simulator is implemented, and it is accompanied by MR interfaces designed to provide users intuitive views of MR simulations. Notably, markerless Augmented Reality (AR) techniques are integrated for real time visualisation. Through three case study evaluations, the use of MR simulations has been demonstrated to produce results that reliably represent the real world. The thesis provides an insight to how MR simulations can help to minimise resource requirements in an industrial application, and enable efficient testing of a prototype aerial robot system. Findings from an initial user study indicate positive acceptance to the technology, while an observational user study identifies a strong contribution of MR simulation to the later stages of robot development.