Augmented reality with topological changes


The visualization of internal structures of organs in minimally invasive surgery is an important avenue for improving the perception of the surgeon, or for supporting planning and decision systems. However, current methods dealing with non-rigid augmented reality only provide augmentation when the topology of the organ is not modified. In the work we presented at MICCAI 2015, we solved this shortcoming by introducing a method for physics-based non-rigid augmented reality. Singularities caused by topological changes are detected and propagated to the pre-operative model. This significantly improves the coherence between the actual laparascopic view and the model, and provides added value in terms of navigation and decision making. Following this work, we plan to extend the method to three-dimensional, volumetric, partial cuts. Since we only rely on monocular images, additional information about the cutting process will be extracted by analyzing instrument motion and by using context awareness to identify the instrument characteristics.




Stéphane Cotin

Research Director