Publications of Nazim Haouchine

 

Publications HAL de haouchine de la structure shacra;mimesis

2017

Conference papers

Screen%20Shot%202017-06-19%20at%203.30.12%20PM.png
titre
DejaVu: Intra-operative Simulation for Surgical Gesture Rehearsal
auteur
Nazim Haouchine, Danail Stoyanov, Frederick Roy, Stéphane Cotin
article
Medical Image Computing and Computer Assisted Interventions Conference MICCAI 2017, Oct 2017, Quebec City, Canada
resume
Advances in surgical simulation and surgical augmented reality have changed the way surgeons prepare for practice and conduct medical procedures. Despite considerable interest from surgeons, the use of simulation is still predominantly confined to pre-operative training of surgical tasks and the lack of robustness of surgical augmented reality means that it is seldom used for surgical guidance. In this paper, we present DejaVu, a novel surgical simulation approach for intra-operative surgical gesture rehearsal. With DejaVu we aim at bridging the gap between pre-operative surgical simulation and crucial but not yet robust intra-operative surgical augmented reality. By exploiting intra-operative images we produce a simulation that faithfully matches the actual procedure without visual discrepancies and with an underlying physical modelling that performs real-time deformation of organs and surrounding tissues, surgeons can interact with the targeted organs through grasping, pulling or cutting to immediately rehearse their next gesture. We present results on different in vivo surgical procedures and demonstrate the feasibility of practical use of our system.
Accès au texte intégral et bibtex
https://hal.archives-ouvertes.fr/hal-01542395/file/dejavu-249.pdf BibTex
titre
Image-driven Stochastic Identification of Boundary Conditions for Predictive Simulation
auteur
Igor Peterlik, Nazim Haouchine, Lukáš Ručka, Stéphane Cotin
article
20th International Conference on Medical Image Computing and Computer Assisted Intervention, Sep 2017, Québec, Canada. 2017
resume
In computer-aided interventions, biomechanical models reconstructed from the pre-operative data are used via augmented reality to facilitate the intra-operative navigation. The predictive power of such models highly depends on the knowledge of boundary conditions. However , in the context of patient-specific modeling, neither the pre-operative nor the intra-operative modalities provide a reliable information about the location and mechanical properties of the organ attachments. We present a novel image-driven method for fast identification of boundary conditions which are modelled as stochastic parameters. The method employs the reduced-order unscented Kalman filter to transform in real-time the probability distributions of the parameters, given observations extracted from intra-operative images. The method is evaluated using synthetic, phantom and real data acquired in vivo on a porcine liver. A quantitative assessment is presented and it is shown that the method significantly increases the predictive power of the biomechanical model.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01570811/file/miccai2017BCDA.pdf BibTex
titre
Template-based Monocular 3D Recovery of Elastic Shapes using Lagrangian Multipliers
auteur
Nazim Haouchine, Stéphane Cotin
article
Computer Vision and Pattern Recognition (CVPR), Jul 2017, Honolulu, Hawai, United States
resume
We present in this paper an efficient template-based method for 3D recovery of elastic shapes from a fixed monocular camera. By exploiting the object’s elasticity, in contrast to isometric methods that use inextensibility constraints , a large range of deformations can be handled. Our method is expressed as a saddle point problem using La-grangian multipliers resulting in a linear system which unifies both mechanical and optical constraints and integrates Dirichlet boundary conditions, whether they are fixed or free. We experimentally show that no prior knowledge on material properties is needed, which exhibit the generic usability of our method with elastic and inelastic objects with different kinds of materials. Comparisons with existing techniques are conducted on synthetic and real elastic objects with strains ranging from 25% to 130% resulting to low errors.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01524609/file/CVPR%202017%20-%20Template-based%20Monocular%203D%20Recovery%20of%20Elastic%20Shapes%20using%20Lagrangian%20Multipliers.pdf BibTex

2016

Journal articles

im.png
titre
Robust Augmented Reality registration method for Localization of Solid Organs’ Tumors Using CT-derived Virtual Biomechanical Model and Fluorescent Fiducials
auteur
Seong-Ho Kong, Nazim Haouchine, Renato Soares, Andrey S Klymchenko, Bohdan Andreiuk, Bruno Marques, Galyna Shabat, Thierry Piéchaud, Michele Diana, Stéphane Cotin, Jacques Marescaux
article
Surgical Endoscopy, Springer Verlag (Germany), 2016, <10.1007/s00464-016-5297-8>
resume
Accurate localization of solid organs tumors is crucial to ensure both radicality and organ function preservation. Augmented Reality (AR) is the fusion of computer-generated and real-time images. AR can be used in surgery as a navigation tool, by creating a patient-specific virtual model through 3D software manipulation of DICOM imaging (e.g. CT-scan). The virtual model can be superimposed to the real-time images to obtain the enhanced real-time localization. However, the 3D virtual model is rigid, and does not take into account inner structures’ deformations. We present a concept of automated navigation system, enabling transparency visualization of internal anatomy and tumor’s margins, while the organs undergo deformation during breathing or surgical manipulation.
Accès au texte intégral et bibtex
https://hal.archives-ouvertes.fr/hal-01314963/file/surg-endosc.pdf BibTex
thumbnail.png
titre
Handling Topological Changes during Elastic Registration: Application to Augmented Reality in Laparoscopic Surgery
auteur
Christoph Joachim Paulus, Nazim Haouchine, Seong-Ho Kong, Renato Vianna Soares, David Cazier, Stéphane Cotin
article
International Journal of Computer Assisted Radiology and Surgery (IJCARS), Springer, 2016
resume
Purpose: Locating the internal structures of an organ is a critical aspect of many surgical procedures. Minimally invasive surgery, associated with augmented reality techniques, offers the potential to visualize inner structures, allowing for improved analysis, depth perception or for supporting planning and decision systems. Methods: Most of the current methods dealing with rigid or non-rigid augmented reality make the assumption that the topology of the organ is not modified. As surgery relies essentially on cutting and dissection of anatomical structures, such methods are limited to the early stages of the surgery. We solve this shortcoming with the introduction of a method for physics-based elastic registration using a single view from a monocular camera. Singularities caused by topological changes are detected and propagated to the pre-operative model. This significantly improves the coherence between the actual laparoscopic view and the model, and provides added value in terms of navigation and decision-making, e.g. by overlaying the internal structures of an organ on the laparoscopic view. Results: Our real time augmentation method is assessed on several scenarios, using synthetic objects and real organs. In all cases, the impact of our approach is demonstrated, both qualitatively and quantitatively. Conclusion: The presented approach tackles the challenge of localizing internal structures throughout a complete surgical procedure, even after surgical cuts. This information is crucial for surgeons to improve the outcome for their surgical procedure and avoid complications.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01397409/file/ijcars2016.pdf BibTex

Conference papers

im.jpg
titre
Using Contours as Boundary Conditions for Elastic Registration during Minimally Invasive Hepatic Surgery
auteur
Nazim Haouchine, Frederick Roy, Lionel Untereiner, Stéphane Cotin
article
International Conference on Intelligent Robots and Systems, Oct 2016, Daejeon, South Korea
resume
We address in this paper the ill-posed problem of initial alignment of pre-operative to intra-operative data for augmented reality during minimally invasive hepatic surgery. This problem consists of finding the rigid transformation that relates the scanning reference and the endoscopic camera pose, and the non-rigid transformation undergone by the liver w.r.t its scanned state. Most of the state-of-the-art methods assume a known initial registration. Here, we propose a method that permits to recover the deformation undergone by the liver while simultaneously finding the rotational and translational parts of the transformation. Our formulation considers the boundaries of the liver with its surrounding tissues as hard constraints directly encoded in an energy minimization process. We performed experiments on real in-vivo data of human hepatic surgery and synthetic data, and compared our method with related works.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01353185/file/haouchine_iros2016.pdf BibTex
im.jpg
titre
Simultaneous Pose Estimation and Augmentation of Elastic Surfaces from a Moving Monocular Camera
auteur
Nazim Haouchine, Marie-Odile Berger, Stephane Cotin
article
International Symposium on Mixed and Augmented Reality, Sep 2016, Merida, Mexico
resume
We present in this paper an original method to estimate the pose of a monocular camera while simultaneously modeling and capturing the elastic deformation of the object to be augmented. Our method tackles a challenging problem where ambiguities between rigid motion and non-rigid deformation are present. This issue represents a major lock for the establishment of an efficient surgical augmented reality where endoscopic camera moves and organs deform. Using an underlying physical model to estimate the low stressed regions our algorithm separates the rigid body motion from the elastic deformations using polar decomposition of the strain tensor. Following this decomposition, a constrained minimization, that encodes both the optical and the physical constraints, is resolved at each frame. Results on real and simulated data are exposed to show the effectiveness of our approach.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01353189/file/haouchine_ISMAR2016.pdf BibTex
seg_labels.png
titre
Segmentation and Labelling of Intra-operative Laparoscopic Images using Structure from Point Cloud
auteur
Nazim Haouchine, Stephane Cotin
article
International Symposium on Biomedical Imaging : “From Nano to Macro” (ISBI 2016), Apr 2016, Prague, Czech Republic
resume
We present in this paper an automatic method for segmenting and labelling of liver its surrounding tissues in intra-operative laparoscopic images. The goal is to be able to distinguished between the different structure that compose a common intra-operative hepatic surgery scene. This will permits to improve the registration between pre-operative data and intra-operative images for task such as Augmented Reality. Our segmentation method consider the scene as a 3D structured point cloud instead of a laparoscopic images in order to exploit powerful informations such as curvature and normals, in addition to visual cues that permits to efficiently classify the scene. Our approach works well on sparse and noisy point clouds, thanks to a surface approximation stage, and unlike existing approaches, is independent of organs texture in the image. Experiements performed on challenging human hepatic surgery confirm that accurate segmentation and labelling are possible using 3D structure information and appropriate visual cues.
Accès au texte intégral et bibtex
https://hal.archives-ouvertes.fr/hal-01314970/file/Template_ISBI2016.pdf BibTex

2015

Journal articles

3D_model_reconstruction%20%281%29.png
titre
Patient-specific Biomechanical Modeling for Guidance during Minimally-invasive Hepatic Surgery
auteur
Rosalie Plantefève, Igor Peterlik, Nazim Haouchine, Stéphane Cotin
article
Annals of Biomedical Engineering, Springer Verlag, 2015
resume
During the minimally-invasive liver surgery, only the partial surface view of the liver is usually provided to the surgeon via the laparoscopic camera. Therefore, it is necessary to estimate the actual position of the internal structures such as tumors and vessels from the pre-operative images. Nevertheless, such task can be highly challenging since during the intervention, the abdominal organs undergo important deformations due to the pneumoperitoneum, respiratory and cardiac motion and the interaction with the surgical tools. Therefore, a reliable automatic system for intra-operative guidance requires fast and reliable registration of the pre- and intra-operative data. In this paper we present a complete pipeline for the registration of pre-operative patient-specific image data to the sparse and incomplete intra-operative data. While the intra-operative data is represented by a point cloud extracted from the stereo-endoscopic images, the pre-operative data is used to reconstruct a biomechanical model which is necessary for accurate estimation of the position of the internal structures, considering the actual deformations. This model takes into account the patient-specific liver anatomy composed of parenchyma, vascularization and capsule, and is enriched with anatomical boundary conditions transferred from an atlas. The registration process employs the iterative closest point technique together with a penalty-based method. We perform a quantitative assessment based on the evaluation of the target registration error on synthetic data as well as a qualitative assessment on real patient data. We demonstrate that the proposed registration method provides good results in terms of both accuracy and robustness w. r. t. the quality of the intra-operative data.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01205194/file/ABME.pdf BibTex
Screen%20Shot%202015-08-23%20at%2011.56.21%20AM.png
titre
Monocular 3D Reconstruction and Augmentation of Elastic Surfaces with Self-occlusion Handling
auteur
Nazim Haouchine, Jeremie Dequidt, Marie-Odile Berger, Stephane Cotin
article
IEEE Transactions on Visualization and Computer Graphics, Institute of Electrical and Electronics Engineers, 2015, pp.14. <10.1109/TVCG.2015.2452905>
resume
This paper focuses on the 3D shape recovery and augmented reality on elastic objects with self-occlusions handling, using only single view images. Shape recovery from a monocular video sequence is an underconstrained problem and many approaches have been proposed to enforce constraints and resolve the ambiguities. State-of-the art solutions enforce smoothness or geometric constraints, consider specific deformation properties such as inextensibility or resort to shading constraints. However, few of them can handle properly large elastic deformations. We propose in this paper a real-time method that uses a mechanical model and able to handle highly elastic objects. The problem is formulated as an energy minimization problem accounting for a non-linear elastic model constrained by external image points acquired from a monocular camera. This method prevents us from formulating restrictive assumptions and specific constraint terms in the minimization. In addition, we propose to handle self-occluded regions thanks to the ability of mechanical models to provide appropriate predictions of the shape. Our method is compared to existing techniques with experiments conducted on computer-generated and real data that show the effectiveness of recovering and augmenting 3D elastic objects. Additionally, experiments in the context of minimally invasive liver surgery are also provided and results on deformations with the presence of self-occlusions are exposed.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01186011/file/haouchine_tvcg_2015.pdf BibTex
scanned-3.jpg
titre
Impact of Soft Tissue Heterogeneity on Augmented Reality for Liver Surgery
auteur
Nazim Haouchine, Stephane Cotin, Igor Peterlik, Jeremie Dequidt, Mario Sanz-Lopez, Erwan Kerrien, Marie-Odile Berger
article
IEEE Transactions on Visualization and Computer Graphics, Institute of Electrical and Electronics Engineers, 2015, 21 (5), pp.584 – 597. <10.1109/TVCG.2014.2377772>
resume
This paper presents a method for real-time augmented reality of internal liver structures during minimally invasive hepatic surgery. Vessels and tumors computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Compared to current methods, our method is able to locate the in-depth positions of the tumors based on partial three-dimensional liver tissue motion using a real-time biomechanical model. This model permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Experimentations conducted on phantom liver permits to measure the accuracy of the augmentation while real-time augmentation on in vivo human liver during real surgery shows the benefits of such an approach for minimally invasive surgery.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01136728/file/haouchineTVCG2014-low.pdf BibTex

Conference papers

titre
Framework for augmented reality in Minimally Invasive laparoscopic surgery
auteur
Bruno Marques, Rosalie Plantefeve, Frédérick Roy, Nazim Haouchine, Emmanuel Jeanvoine, Igor Peterlik, Stéphane Cotin
article
HealthCom 2015, Oct 2015, Boston, United States. 2015 17th International Conference on E-health Networking, Application & Services (HealthCom) 2015, <10.1109/HealthCom.2015.7454467>
resume
This article presents a framework for fusing pre-operative data and intra-operative data for surgery guidance. This framework is employed in the context of Minimally Invasive Surgery (MIS) of the liver. From stereoscopic images a three dimensional point cloud is reconstructed in real-time. This point cloud is then used to register a patient-specific biomechanical model derived from Computed Tomography images onto the laparoscopic view. In this way internal structures such as vessels and tumors can be visualized to help the surgeon during the procedure. This is particularly relevant since abdominal organs undergo large deformations in the course of the surgery, making it difficult for surgeons to correlate the laparoscopic view with the pre-operative images. Our method has the potential to reduce the duration of the operation as the biomechanical model makes it possible to estimate the in-depth position of tumors and vessels at any time of the surgery, which is essential to the surgical decision process. Results show that our method can be successfully applied during laparoscopic procedure without interfering with the surgical work flow.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01315574/file/article.pdf BibTex
thumbnail.jpg
titre
Surgical Augmented Reality with Topological Changes
auteur
Christoph Paulus, Nazim Haouchine, David Cazier, Stéphane Cotin
article
Medical Image Computing and Computer Assisted Interventions, Oct 2015, München, Germany. <http://miccai2015.org/>
resume
The visualization of internal structures of organs in minimally invasive surgery is an important avenue for improving the perception of the surgeon, or for supporting planning and decision systems. However, current methods dealing with non-rigid augmented reality only provide augmentation when the topology of the organ is not modified. In this paper we solve this shortcoming by introducing a method for physics-based non-rigid augmented reality. Singularities caused by topo-logical changes are detected and propagated to the pre-operative model. This significantly improves the coherence between the actual laparascopic view and the model, and provides added value in terms of navigation and decision making. Our real time augmentation algorithm is assessed on a video showing the cut of a porcine liver’s lobe in minimal invasive surgery.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01184498/file/2015MICCAI.pdf BibTex
thumbnail.jpg
titre
Augmented Reality during Cutting and Tearing of Deformable Objects
auteur
Christoph Paulus, Nazim Haouchine, David Cazier, Stephane Cotin
article
The 14th IEEE International Symposium on Mixed and Augmented Reality, Sep 2015, Fukuoka, Japan. pp.6
resume
Current methods dealing with non-rigid augmented reality only provide an augmented view when the topology of the tracked object is not modified, which is an important limitation. In this paper we solve this shortcoming by introducing a method for physics-based non-rigid augmented reality. Singularities caused by topological changes are detected by analyzing the displacement field of the underlying deformable model. These topological changes are then applied to the physics-based model to approximate the real cut. All these steps, from deformation to cutting simulation, are performed in real-time. This significantly improves the coherence between the actual view and the model, and provides added value.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01184495/file/2015ISMAR.pdf BibTex
Depth_Contour.jpg
titre
Improving depth perception during surgical augmented reality
auteur
Bruno Marques, Nazim Haouchine, Rosalie Plantefeve, Stephane Cotin
article
SIGGRAPH [Poster], Aug 2015, Los Angeles, United States. pp.Article No. 24, 2015, <10.1145/2787626.2792654>
resume
This study suggests a method to compensate the loss of depth perception while enhancing organ vessels and tumors to surgeons. This method relies on a combination of contour rendering technique and adaptive alpha blending to effectively perceive the vessels and tumors depth. In addition, this technique is designed to achieve real-time to satisfy the requirements of clinical routines, and has been tested on real human surgery.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01191101/file/template.pdf BibTex
teaser-hal.jpg
titre
Fracture in Augmented Reality
auteur
Nazim Haouchine, Alexandre Bilger, Jeremie Dequidt, Stephane Cotin
article
SIGGRAPH [Poster], Aug 2015, Los Angeles, United States. 2015
resume
We propose in this study an image-guided mesh cutting method to handle real-time augmentation of paper tearing. This method relies on the combination of visually-based fracture tracking algorithm and a physics-based model that is dynamically superimposed on the image.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01191090/file/template.pdf BibTex
thumb.png
titre
Surgery Training, Planning and Guidance Using the SOFA Framework
auteur
Hugo Talbot, Nazim Haouchine, Igor Peterlik, Jeremie Dequidt, Christian Duriez, Hervé Delingette, Stephane Cotin
article
Eurographics, May 2015, Zurich, Switzerland
resume
In recent years, an active development of novel technologies dealing with medical training, planning and guidance has become an increasingly important area of interest in both research and health-care manufacturing. A combination of advanced physical models, realistic human-computer interaction and growing computational power is bringing new solutions in order to help both medical students and experts to achieve a higher degree of accuracy and reliability in surgical interventions. In this paper, we present three different examples of medical physically-based simulations implemented in a common software platform called SOFA. Each example represents a different application: training for cardiac electrophysiology, pre-operative planning of cryosurgery and per-operative guidance for laparoscopy. The goal of this presentation is to evaluate the realism, accuracy and efficiency of the simulations, as well as to demonstrate the potential and flexibility of the SOFA platform.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01160297/file/Sofa-EG2015.pdf BibTex

Theses

haouchine_thesis.png
titre
Image-guided Simulation for Augmented Reality during Hepatic Surgery
auteur
Nazim Haouchine
article
Computer Science [cs]. Université de Lille1, 2015. English
resume
The main objective of this thesis is to provide surgeons with tools for pre and intra-operative decision support during minimally invasive hepatic surgery. These interventions are usually based on laparoscopic techniques or, more recently, flexible endoscopy. During such operations, the surgeon tries to remove a significant number of liver tumors while preserving the functional role of the liver. This involves defining an optimal hepatectomy, i.e. ensuring that the volume of post-operative liver is at least at 55% of the original liver and the preserving at hepatic vasculature. Although intervention planning can now be considered on the basis of preoperative patient-specific, significant movements of the liver and its deformations during surgery data make this very difficult to use planning in practice. The work proposed in this thesis aims to provide augmented reality tools to be used in intra-operative conditions in order to visualize the position of tumors and hepatic vascular networks at any time.
Accès au texte intégral et bibtex
https://hal.inria.fr/tel-01254439/file/Haouchine_Thesis.pdf BibTex

2014

Conference papers

liver-fem.png
titre
The Role of Ligaments: Patient-Specific or Scenario-Specific ?
auteur
Julien Bosman, Nazim Haouchine, Jérémie Dequidt, Igor Peterlik, Stéphane Cotin, Christian Duriez
article
International Symposium on Biomedical Simulation ISBMS, Oct 2014, Strasbourg, France. 2014
resume
In this paper, we present a preliminary study dealing with the importance of correct modeling of connective tissues such as ligaments in laparoscopic liver surgery simulation. We show that the model of these tissues has a significant impact on the overall results of the simulation. This is demonstrated numerically using two different scenarios from the laparoscopic liver surgery, both resulting in important deformation of the liver: insufflation of the abdominal cavity with gas (pneumoperitoneum) and manipulation with the liver lobe using a surgical instrument (grasping pincers). For each scenario, a series of simulations is performed with or without modeling the deformation of the ligaments (fixed constraints or biomechanical model with the parameter of the literature). The numerical comparison shows that modeling the ligament deformations can be at least as important as the correct selection of the patient-specific parameters, nevertheless this observation depends on the simulated scenario.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01068077/file/main.pdf BibTex
prob-statement2.png
titre
Automatic Alignment of pre and intraoperative Data using Anatomical Landmarks for Augmented Laparoscopic Liver Surgery
auteur
Rosalie Plantefeve, Nazim Haouchine, Jean Pierre Radoux, Stéphane Cotin
article
International Symposium on Biomedical Simulation ISBMS, Oct 2014, Strasbourg, France. 2014
resume
Each year in Europe 50,000 new liver cancer cases are diagnosed for which hepatic surgery combined to chemotherapy is the most common treatment. In particular the number of laparoscopic liver surgeries has increased significantly over the past years. This type of minimally invasive procedure which presents many benefits for the patient is challenging for the surgeons due to the limited field of view. Recently new augmented reality techniques which merge preoperative data and intraoperative images and permit to visualize internal structures have been proposed to help surgeons during this type of surgery. One of the difficulties is to align preoperative data with the intraoperative images. We propose in this paper a semi-automatic approach for solving the ill-posed problem of initial alignment for Augmented Reality systems during liver surgery. Our registration method relies on anatomical landmarks extracted from both the laparoscopic images and three-dimensional model, using an image-based soft-tissue reconstruction technique and an atlas-based approach, respectively. The registration evolves automatically from a quasi-rigid to a non-rigid registration. Furthermore, the surface-driven deformation is induced in the volume via a patient specific biomechanical model. The experiments conducted on both synthetic and in vivo data show promising results with a registration error of 2 mm when dealing with a visible surface of 30% of the whole liver.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01068246/file/article.pdf BibTex
teaser3.png
titre
Single View Augmentation of 3D Elastic Objects
auteur
Nazim Haouchine, Jérémie Dequidt, Marie-Odile Berger, Stéphane Cotin
article
International Symposium on Mixed and Augmented Reality – ISMAR, Sep 2014, Munich, Germany. 2014
resume
This paper proposes an efficient method to capture and augment highly elastic objects from a single view. 3D shape recovery from a monocular video sequence is an underconstrained problem and many approaches have been proposed to enforce constraints and resolve the ambiguities. State-of-the art solutions enforce smoothness or geometric constraints, consider specific deformation properties such as inextensibility or ressort to shading constraints. However, few of them can handle properly large elastic deformations. We propose in this paper a real-time method which makes use of a me chanical model and is able to handle highly elastic objects. Our method is formulated as a energy minimization problem accounting for a non-linear elastic model constrained by external image points acquired from a monocular camera. This method prevents us from formulating restrictive assumptions and specific constraint terms in the minimization. The only parameter involved in the method is the Young’s modulus where we show in experiments that a rough estimate of its value is sufficient to obtain a good reconstruction. Our method is compared to existing techniques with experiments conducted on computer-generated and real data that show the effectiveness of our approach. Experiments in the context of minimally invasive liver surgery are also provided.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01056323/file/template.pdf BibTex
titre
Réalité augmentée pour la chirurgie minimalement invasive du foie utilisant un modèle biomécanique guidé par l’image
auteur
Nazim Haouchine, Stéphane Cotin, Jérémie Dequidt, Erwan Kerrien, Marie-Odile Berger
article
Reconnaissance de Formes et Intelligence Artificielle (RFIA) 2014, Jun 2014, France. 2014
resume
Cet article présente une méthode de réalité augmentée pour la chirurgie minimalement invasive du foie. Le réseau vasculaire et les tumeurs internes reconstruites à partir des données pré-opératoires (IRM ou CT) peuvent ainsi être visualisées dans l’image laparoscopique afin de guider les gestes du chirurgien pendant l’opération. Cette méthode est capable de propager les déformations 3D de la surface du foie à ses structures internes grâce à un modèle biomécanique sous-jacent qui prend en compte l’anisotropie et l’hétérogénéité du tissu hépatique. Des résultats sont montrés sur une vidéo in-vivo d’un foie humain acquise pendant une opération et sur un foie en silicone.
Accès au texte intégral et bibtex
https://hal.archives-ouvertes.fr/hal-00988767/file/rfia2014_submission_89.pdf BibTex
titre
Towards an Accurate Tracking of Liver Tumors for Augmented Reality in Robotic Assisted Surgery
auteur
Nazim Haouchine, Jérémie Dequidt, Igor Peterlik, Erwan Kerrien, Marie-Odile Berger, Stéphane Cotin
article
International Conference on Robotics and Automation (ICRA), Jun 2014, Hong Kong, China. 2014
resume
This article introduces a method for tracking the internal structures of the liver during robot-assisted procedures. Vascular network, tumors and cut planes, computed from pre-operative data, can be overlaid onto the laparoscopic view for image-guidance, even in the case of large motion or deformation of the organ. Compared to current methods, our method is able to precisely propagate surface motion to the internal structures. This is made possible by relying on a fast yet accurate biomechanical model of the liver combined with a robust visual tracking approach designed to properly constrain the model. Augmentation results are demonstrated on in-vivo sequences of a human liver during robotic surgery, while quantitative validation is performed on an ex-vivo porcine liver experimentation. Validation results show that our approach gives an accurate surface registration with an error of less than 6mm on the position of the tumor.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-01003262/file/output.pdf BibTex

2013

Journal articles

titre
Deformation-based Augmented Reality for Hepatic Surgery
auteur
Nazim Haouchine, Jérémie Dequidt, Marie-Odile Berger, Stéphane Cotin
article
Studies in Health Technology and Informatics, IOS Press, 2013, 184
resume
In this paper we introduce a method for augmenting the laparoscopic view during hepatic tumor resection. Using augmented reality techniques, vessels, tumors and cutting planes computed from pre-operative data can be overlaid onto the laparoscopic video. Compared to current techniques, which are limited to a rigid registration of the pre-operative liver anatomy with the intra-operative image, we propose a real-time, physics-based, non-rigid registration. The main strength of our approach is that the deformable model can also be used to regularize the data extracted from the computer vision algorithms. We show preliminary results on a video sequence which clearly highlights the interest of using physics-based model for elastic registration.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-00768372/file/Haouchine_N.pdf BibTex

Conference papers

titre
Image-guided Simulation of Heterogeneous Tissue Deformation For Augmented Reality during Hepatic Surgery
auteur
Nazim Haouchine, Jérémie Dequidt, Igor Peterlik, Erwan Kerrien, Marie-Odile Berger, Stéphane Cotin
article
ISMAR – IEEE International Symposium on Mixed and Augmented Reality 2013, Oct 2013, Adelaide, Australia. 2013
resume
This paper presents a method for real-time augmentation of vas- cular network and tumors during minimally invasive liver surgery. Internal structures computed from pre-operative CT scans can be overlaid onto the laparoscopic view for surgery guidance. Com- pared to state-of-the-art methods, our method uses a real-time biomechanical model to compute a volumetric displacement field from partial three-dimensional liver surface motion. This permits to properly handle the motion of internal structures even in the case of anisotropic or heterogeneous tissues, as it is the case for the liver and many anatomical structures. Real-time augmentation results are presented on in vivo and ex vivo data and illustrate the benefits of such an approach for minimally invasive surgery.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-00842855/file/ISMAR13-Haouchine.pdf BibTex

2012

Conference papers

titre
Physics-based Augmented Reality for 3D Deformable Object
auteur
Nazim Haouchine, Jérémie Dequidt, Erwan Kerrien, Marie-Odile Berger, Stéphane Cotin
article
Eurographics Workshop on Virtual Reality Interaction and Physical Simulation, Dec 2012, Darmstadt, Germany. 2012
resume
This paper introduces an original method to perform augmented or mixed reality on deformable objects. Compared to state-of-the-art techniques, our method is able to track deformations of volumetric objects and not only surfacic objects. A flexible framework that relies on the combination of a 3D motion estimation and a physics-based deformable model used as a regularization and interpolation step allows to perform a non-rigid and robust registration. Results are exposed, based on computer-generated datasets and video sequences of real environments in order to assess the relevance of our approach.
Accès au texte intégral et bibtex
https://hal.inria.fr/hal-00768362/file/paper1028_final.pdf BibTex