Computer vision-based phantom deformation modelling and ablation probe navigation

Abstract

Augmented reality (AR) and image-guidance are being increasingly used in robotic surgery, minimally invasive surgery (MIS) and ablation procedures. The aim of the project is to address an important topic in this research area, i.e. to model the deformation of a soft organ, and to provide reliable predictions of the spatial location of its internal structures. To reach this goal, firstly a novel computational framework has been developed that couples Computer Vision (CV) method and linear elasticity Finite Element Method (FEM) to form an Augmented Finite Element Method (AFEM). The framework includes a 2D AFEM and 3D AFEM in the initial investigation. The 2D AFEM system has an underlying dynamic solver and is validated against a gelatin phantom, achieving an accuracy of 0:36mm. The 3D AFEM is implemented using the Graphics Process Unit (GPU) that solves an energy minimisation problem. In conclusion a hybrid CV and FEM framework is implemented. With further improvements it has the potential to be used in Augmented Reality based intra-operational navigations. Secondly, an ablation tracking system is developed. A rig for the ablation probe was prototyped and the applicator was visualised in two avenues; the first, it was shown in a 3D viewer that incorporated a virtual liver model plus its internal structures, and the second, it was visualised in an AR Head-Mounted Display (HMD) environment. Both systems were evaluated and provided a solid foundation for future developments.