Publications of Andrea Mendizabal

2020

Book sections

titre: Data-driven simulation for augmented surgery
auteur: Andrea Mendizabal, Eleonora Tagliabue, Tristan Hoellinger, Jean-Nicolas Brunet, Sergei Nikolaev, Stéphane Cotin
article: Bilen Emek Abali; Ivan Giorgio. Developments and Novel Approaches in Biomechanics and Metamaterials, 132, pp.71-96, 2020, 978-3-030-50464-9. ⟨10.1007/978-3-030-50464-9⟩
resume: To build an augmented view of an organ during surgery, it is essential to have a biomechanical model with appropriate material parameters and boundary conditions , able to match patient specific properties. Adaptation to the patient’s anatomy is obtained by exploiting the image-rich context specific to our application domain. While information about the organ shape, for instance, can be obtained preoper-atively, other patient-specific parameters can only be determined intraoperatively. To this end, we are developing data-driven simulations, which exploit information extracted from a stream of medical images. Such simulations need to run in real-time. To this end we have developed dedicated numerical methods, which allow for real-time computation of finite element simulations. The general principle consists in combining finite element approaches with Bayesian methods or deep learning techniques, that allow to keep control over the underlying computational model while allowing for inputs from the real world. Based on a priori knowledge of the mechanical behavior of the considered organ, we select a constitutive law to model its deformations. The predictive power of such constitutive law highly depends on the knowledge of the material parameters and A. Mendizabal
Accès au texte intégral et bibtex

Theses

titre: Machine Learning meets real-time Numerical Simulation – Application to surgical training, preoperative planning and surgical assistance
auteur: Andrea Mendizabal
article: Biomechanics [physics.med-ph]. Université de Strasbourg, 2020. English
resume: Many engineering applications require accurate numerical simulations of non-linear structures in real-time. Some important examples can be found in the field of medicine, in order to develop surgical training systems, or in the field of surgical navigation where augmented reality can bring significant improvements to the clinical gesture. To guarantee the accuracy of the simulations, patient-specific modeling must be pursued by taking into account personalized material parameters and boundary conditions. In the context of augmented surgery for instance, it is essential to perform an elastic registration between the preoperative and the intraoperative images. To this end, a patient-specific biomechanical model must be built to produce real-time finite-element simulations of the deformed organ. This is in practice very difficult to achieve as the problems to be solved are highly complex, in particular when non-linear deformations are considered. In this work, we propose a method combining finite-element simulations and deep neural networks in order to satisfy the rapidity and accuracy requirements of medical applications. In particular, we present the U-Mesh framework, capable of predicting in real-time the shape of a highly deformable organ like the liver in order to guide surgeons during interventions where following the organ’s deformation is crucial for the surgery to be successful.
Accès au texte intégral et bibtex

2019

Journal articles

titre: Force classification during robotic interventions through simulation-trained neural networks
auteur: Andrea Mendizabal, Raphael Sznitman, Stéphane Cotin
article: International Journal of Computer Assisted Radiology and Surgery, Springer Verlag, 2019, 14, pp.1601-1610. ⟨10.1007/s11548-019-02048-3⟩
resume: Intravitreal injection is among the most frequent treatment strategies for chronic ophthalmic diseases. The last decade has seen a serious increase in the number of intravitreal injections, and with it, adverse effects and drawbacks. To tackle these problems, medical assistive devices for robotized injections have been suggested and are projected to enhance delivery mechanisms for a new generation of pharmacological solutions. In this paper, we present a method aimed at improving the safety characteristics of upcoming robotic systems. Our vision-based method uses a combination of 2D OCT data, numerical simulation and machine learning to classify the range of the force applied by an injection needle on the sclera. We design a Neural Network (NN) to classify force ranges from Optical Coherence Tomography (OCT) images of the sclera directly. To avoid the need for large real data sets, the network is trained on images of simulated deformed sclera. This simulation is based on a finite element method and the model is parameterized using a Bayesian filter applied to observations of the deformation in OCT images. We validate our approach on real OCT data collected on five ex vivo porcine eyes using a robotically-guided needle. The thorough parameterization of the simulations leads to a very good agreement between the virtually generated samples used to train the network and the real OCT acquisitions. Results show that the applied force range on real data can be predicted with 93% accuracy. Through a simulation-trained neural network, our approach estimates the force range applied by a robotically guided needle on the sclera based solely on a single OCT slice of the deformed sclera. Being real-time, this solution can be integrated in the control loop of the system, permitting the prompt withdrawal of the needle for safety reasons.
Accès au texte intégral et bibtex

titre: Simulation of hyperelastic materials in real-time using deep learning
auteur: Andrea Mendizabal, Pablo Márquez-Neila, Stéphane Cotin
article: Medical Image Analysis, Elsevier, 2019, 59, pp.101569. ⟨10.1016/j.media.2019.101569⟩
resume: The finite element method (FEM) is among the most commonly used numerical methods for solving engineering problems. Due to its computational cost, various ideas have been introduced to reduce computation times, such as domain decomposition, parallel computing, adaptive meshing, and model order reduction. In this paper we present U-Mesh: a data-driven method based on a U-Net architecture that approximates the non-linear relation between a contact force and the displacement field computed by a FEM algorithm. We show that deep learning, one of the latest machine learning methods based on artificial neural networks, can enhance computational mechanics through its ability to encode highly non-linear models in a compact form. Our method is applied to two benchmark examples: a cantilever beam and an L-shape subject to moving punctual loads. A comparison between our method and proper orthogonal decomposition (POD) is done through the paper. The results show that U-Mesh can perform very fast simulations on various geometries, mesh resolutions and number of input forces with very small errors.
Accès au texte intégral et bibtex

Conference papers

titre: Physics-based Deep Neural Network for Real-Time Lesion Tracking in Ultrasound-guided Breast Biopsy
auteur: Andrea Mendizabal, Eleonora Tagliabue, Jean-Nicolas Brunet, Diego Dall’Alba, Paolo Fiorini, Stéphane Cotin
article: Computational Biomechanics for Medicine XIV, Oct 2019, Shenzhen, China
resume: In the context of ultrasound (US) guided breast biopsy, image fusion techniques can be employed to track the position of US-invisible lesions previously identified on a pre-operative image. Such methods have to account for the large anatomical deformations resulting from probe pressure during US scanning within the real-time constraint. Although biomechanical models based on the finite element (FE) method represent the preferred approach to model breast behavior, they cannot achieve real-time performances. In this paper we propose to use deep neural networks to learn large deformations occurring in ultrasound-guided breast biopsy and then to provide accurate prediction of lesion displacement in real-time. We train a U-Net architecture on a relatively small amount of synthetic data generated in an offline phase from FE simulations of probe-induced deformations on the breast anatomy of interest. Overall, both training data generation and network training are performed in less than 5 hours, which is clinically acceptable considering that the biopsy can be performed at most the day after the pre-operative scan. The method is tested both on synthetic and on real data acquired on a realistic breast phantom. Results show that our method correctly learns the deformable behavior modelled via FE simulations and is able to generalize to real data, achieving a target registration error comparable to that of FE models, while being about a hundred times faster.
Accès au texte intégral et bibtex

titre: Physics-based Deep Neural Network for Augmented Reality during Liver Surgery
auteur: Jean-Nicolas Brunet, Andrea Mendizabal, Antoine Petit, Nicolas Golse, Eric Vibert, Stéphane Cotin
article: MICCAI 2019 – 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, Oct 2019, Shenzhen, China. pp.8, ⟨10.1007/978-3-030-32254-0_16⟩
resume: In this paper we present an approach combining a finite element method and a deep neural network to learn complex elastic deformations with the objective of providing augmented reality during hep-atic surgery. Derived from the U-Net architecture, our network is built entirely from physically-based simulations of a preoperative segmenta-tion of the organ. These simulations are performed using an immersed-boundary method, which offers several numerical and practical benefits, such as not requiring boundary-conforming volume elements. We perform a quantitative assessment of the method using synthetic and ex vivo patient data. Results show that the network is capable of solving the deformed state of the organ using only a sparse partial surface displacement data and achieve similar accuracy as a FEM solution, while being about 100x faster. When applied to an ex vivo liver example, we achieve the registration in only 3 ms with a mean target registration error (TRE) of 2.9 mm.
Accès au texte intégral et bibtex

2018

Journal articles

titre: A Combined Simulation & Machine Learning Approach for Image-based Force Classification during Robotized Intravitreal Injections
auteur: Andrea Mendizabal, Tatiana Fountoukidou, Jan Hermann, Raphael Sznitman, Stéphane Cotin
article: Medical image computing and computer-assisted intervention : MICCAI .. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2018
resume: Intravitreal injection is one of the most common treatment strategies for chronic ophthalmic diseases. The last decade has seen the number of intravitreal injections dramatically increase, and with it, adverse effects and limitations. To overcome these issues, medical assistive devices for robotized injections have been proposed and are projected to improve delivery mechanisms for new generation of pharmacological solutions. In our work, we propose a method aimed at improving the safety features of such envisioned robotic systems. Our vision-based method uses a combination of 2D OCT data, numerical simulation and machine learning to estimate the range of the force applied by an injection needle on the sclera. We build a Neural Network (NN) to predict force ranges from Optical Coherence Tomography (OCT) images of the sclera directly. To avoid the need of large training data sets, the NN is trained on images of simulated deformed sclera. We validate our approach on real OCT images collected on five ex vivo porcine eyes using a robotically-controlled needle. Results show that the applied force range can be predicted with 94% accuracy. Being real-time, this solution can be integrated in the control loop of the system, allowing for in-time withdrawal of the needle.
Accès au texte intégral et bibtex

2017

Conference papers

titre: Face-based Smoothed Finite Element Method for Real-time Simulation of soft tissue
auteur: Andrea Mendizabal, Rémi Duparc, Huu Phuoc Bui, Christoph Paulus, Igor Peterlik, Stéphane Cotin
article: SPIE Medical Imaging, Feb 2017, Orlando, United States
resume: In soft tissue surgery, a tumor and other anatomical structures are usually located using the preoperative CT or MR images. However, due to the deformation of the concerned tissues, this information suffers from inaccuracy when employed directly during the surgery. In order to account for these deformations in the planning process, the use of a bio-mechanical model of the tissues is needed. Such models are often designed using the finite element method (FEM), which is, however, computationally expensive, in particular when a high accuracy of the simulation is required. In our work, we propose to use a smoothed finite element method (S-FEM) in the context of modeling of the soft tissue deformation. This numerical technique has been introduced recently to overcome the overly stiff behavior of the standard FEM and to improve the solution accuracy and the convergence rate in solid mechanics problems. In this paper, a face-based smoothed finite element method (FS-FEM) using 4-node tetrahedral elements is presented. We show that in some cases, the method allows for reducing the number of degrees of freedom, while preserving the accuracy of the discretization. The method is evaluated on a simulation of a cantilever beam loaded at the free end and on a simulation of a 3D cube under traction and compression forces. Further, it is applied to the simulation of the brain shift and of the kidney’s deformation. The results demonstrate that the method outperforms the standard FEM in a bending scenario and that has similar accuracy as the standard FEM in the simulations of brain shift and kidney deformation.
Accès au texte intégral et bibtex