A Virtual Environment for Medical Teamwork Training with Support for Non-Verbal Communication using Consumer-Level Hardware and Software

Abstract

Simulation is an increasingly important component of the medical curriculum due to procedures becoming more complex and the use of real patients being constrained by ethical restrictions, costs, availability of patients, and logistics. A large number of simulators are available, mostly for training of technical skills, such as suturing or cutting, but increasingly also for non-technical skills, such as teamwork and decision making. The majority of those simulators are very expensive and unaffordable for smaller hospitals, healthcare providers, and education institutes, especially in third world countries. Furthermore the hardware and content are usually proprietary, making it difficult and expensive to add new functionalities. In this thesis, a virtual environment (VE) for medical teamwork training, based on consumer-level hardware and software, is presented and evaluated. Advanced graphics, physics and multi-user capabilities are provided by a game engine, and non-verbal communication, an important aspect of teamwork, is facilitated by using a standard webcam for tracking the movement of the user's head and mirroring that movement onto the avatar. Different game engines are evaluated with regard to their suitability for medical simulations and guidelines and ideal parameters for stable head tracking using a web-cam are presented. Based on these results, a simulation framework is developed and evaluated for its suitability for medical teamwork training. A user study is presented, demonstrating that participants perceive non-verbal communication cues well and can intuitively control the view perspective with head movement. A multi-user study with a medical simulation scenario shows that the inclusion of view and avatar control by head tracking improves the perceived realism and naturalness of the simulation and the communication. The results demonstrate that medical simulators can be constructed using cheap consumer level hardware and software. Game engines provide a suitable software foundation and allow users to design their own content. Peripheral devices, such as web-cams in combination with computer vision techniques, can be used to improve realism and immersion. Constraints centre predominantly on the simulation of complex technical skills and complex multi-user interactions.