Kidney Augmented Reality Surgery

Project Info

Fundings IHU

Objective Marker-based Augmented Reality for Kidney Surgery

Date 01/02/2016 – 30/07/2016




People Involved

Project Description

Kidney Augmented Reality Surgery project focused on the capability of AR to achieve this fusion in real-time and to compute the tumor position with high accuracy (less than 2mm). To achieve this accuracy, three studies are necessary:

1. Identification of salient markers in both the pre-operative 3D model and intra-operative images
Motion estimation of the kidney surface using advanced computer vision algorithms. This component will particularly relies on the tracking of the previously cited markers, to ensure a long-term and robust detection and tracking.

2. Modeling and real-time simulation of kidney tissue, that account for the vascular network and tumors: Despite the fact that the kidney is considered as a quasi-rigid organ, the need of a volume representation to visualize the tumors involve the complete bio-mechanical modeling of kidneys in order to faithfully translate the kidney behavior.

3. In addition, since we only relies on a surface information from laparoscopic images, the bio-mechanical model will play an important role to propagate the kidney surface motion to internal structures such as tumors.


Robust Augmented Reality registration method for Localization of Solid Organs’ Tumors Using CT-derived Virtual Biomechanical Model and Fluorescent Fiducials
Seong-Ho Kong, Nazim Haouchine, Renato Soares, Andrey S Klymchenko, Bohdan Andreiuk, Bruno Marques, Galyna Shabat, Thierry Piéchaud, Michele Diana, Stéphane Cotin, Jacques Marescaux. Surgical Endoscopy, Springer Verlag (Germany), 2016, 〈10.1007/s00464-016-5297-8〉

Project Gallery