We propose a deep learning approach to align preoperative 3D CT scans with intraoperative 2D X-ray images by predicting anatomical deformations.
While CT scans provide detailed 3D anatomical information, clinicians must often rely on 2D X-ray images during interventions.
Navigating the anatomy using only 2D, low contrast images, is made even more challenging by motion of the anatomy, due to breathing or other causes.
To help clinicians, we propose to augment 2D X-ray images with 3D information, updated in real-time to follow anatomical motion.
Our neural network-based solution makes this real-time augmentation possible by predicting the deformation between preoperative and intraoperative anatomies from a 2D X-ray image.
Because no suitable training dataset exists for this task, we generate synthetic training data by applying deformations to the preoperative 3D CT scan and simulating corresponding X-ray images.
Unlike previous methods, our approach does not rely on a predefined motion model, allowing it to generalize to complex, arbitrary deformations.
