SAAR: Surgical Aid for Anatomical Reconstructions Using Projector-Based Augmented Reality and Computer Vision Systems

Overview

Accurate visualization of underlying anatomical structures is vital for precise surgical interventions, particularly in high-risk and minimally invasive procedures. Existing methods like fluoroscopy or camera-based systems divert the surgeon's attention to external displays, potentially increasing risk. This project proposes SAAR, a novel computer vision and projector-based navigational system that superimposes 3D anatomical reconstructions directly onto the patient's surgical site. SAAR employs a custom patient registration pipeline to convert preoperative CT scans into 3D reconstructions, which are then projected onto the surgical area using a calibrated projector system housed on a robotic frame mechanism. A splotch-based algorithm actively adjusts the frame to match the overlay to the patient's positioning, accounting for procedural movement. To augment surgical understanding, SAAR uses a custom convolutional neural network (CNN) with a U-net architecture for preoperative segmentation and identification of internal abnormalities. These regions of interest are highlighted via projection, guiding the surgeon's attention. Furthermore, SAAR leverages a custom-built CNN-LSTM (Long Short-Term Memory) architecture to predict 3D organ movement in the thoracic region. This feature utilizes biometric data, enabling dynamic visualization of the anatomy in real-time. The system was evaluated through extensive testing, involving training and validation on large datasets of publicly available CT and MRI scans. The accuracy of projection alignment was also rigorously assessed, demonstrating clinical viability. SAAR offers a pragmatic and cost-effective alternative to existing solutions that can potentially transform surgical visualization in a variety of environments.

Awards (1)

• Fourth Award of $500 $500

Competition history

• ISEF 2024 Biomedical Engineering · Entry ENBM051T · Los Angeles, California, United States

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.