HyperAIHyperAI

Command Palette

Search for a command to run...

AI model aligns surgical X-rays with 3D scans for precision

Researchers at MIT, in collaboration with Harvard Medical School and other leading medical institutions, have developed xvr, a novel artificial intelligence system that rapidly aligns real-time intraoperative X-rays with preoperative three-dimensional medical scans. Published recently in Nature, the technology addresses a longstanding challenge in minimally invasive surgery: the difficulty of accurately tracking surgical instruments using flat, two-dimensional fluoroscopic images against complex patient anatomy. Traditionally, surgeons manually register X-ray views with CT or MRI scans, a process that demands extensive training and slows procedure times. Existing AI-driven registration tools have struggled with robustness across diverse patient anatomies, largely due to insufficient high-quality annotated datasets and the physical variability of human bodies. xvr overcomes these limitations through a patient-specific machine learning approach combined with a pre-trained foundation model. Rather than attempting to build a universal algorithm, the system tailors its registration capabilities to individual patients without sacrificing speed. The framework utilizes a physics-based simulation to generate thousands of synthetic X-ray projections directly from a patient preoperative scan. This approach ensures anatomical accuracy and eliminates the hallucination risks common in generative AI. These synthetic images train an AI model to map two-dimensional X-ray data to three-dimensional spatial coordinates with sub-millimeter precision. While training such a model from scratch for each patient would require approximately twelve hours, the team optimized the process by pretraining a foundation model on over two thousand whole-body scans. This allows the system to adapt to a new patient anatomy in roughly five minutes, achieving accuracy equivalent to a freshly trained model. Validated across datasets from five hospitals, xvr significantly outperformed existing AI methods in both accuracy and robustness, covering a wide range of adult and pediatric procedures. Lead author Vivek Gopalakrishnan, a postdoctoral researcher in MIT Computer Science and Artificial Intelligence Laboratory, emphasized the clinical implications: many Americans live far from centers equipped to perform specialized interventions like emergency stroke treatments. By streamlining the registration process, the technology could democratize access to high-precision minimally invasive procedures, particularly in emergency settings where every minute counts. The research team, led by Gopalakrishnan alongside senior authors Polina Golland and Neel Dey, is now collaborating with surgical robotics manufacturers and clinical partners to integrate xvr into real-world navigation systems. Future development will focus on reducing adaptation time further to enable true real-time deployment, expanding the system to accommodate dynamic anatomical changes, and conducting additional clinical validation. Funded by the National Institutes of Health and several MIT initiatives, xvr represents a significant advance in medical image computing, bridging the gap between computational precision and practical surgical workflow.

Related Links