A Multimodal Deep Learning Framework for a Novel Clinical Decision Support System in Retinal Disease Diagnosis and Management

Overview

Visual impairment (VI) affects 2.2 billion people worldwide, and 1 billion cases are preventable with early diagnosis. Retinal diseases are preventable causes of VI. Clinical Decision Support Systems (CDSSs) utilize AI to enhance diagnosis, but current CDSSs depend on heavy preprocessing and fail with speckle, a noise-like pattern that forms from light scattering off retinal microstructures. These systems also do not provide disease severity or interpretability. The Speckle-Aware Dynamic Vision Transformer framework leverages speckle as a diagnostic signal for feature extraction of hidden microstructures, as speckle is related to microstructures through physics of light. It achieved a 96.9% disease classification accuracy and a 0.99 Area Under the Curve value, outperforming FDA approved CDSSs. The Severity Estimation and Personalized Analysis framework quantifies disease severity by computing Euclidean distance between pathological and healthy anatomical feature vectors, providing a quantitative metric of severity for longitudinal monitoring with a 0.98 Spearman rank coefficient. The Clinical Reasoning and Analysis Framework for Trust provides interpretability across graphical, visual, and textual modalities using t-SNE dimensionality reduction, Swin-UNet retinal layer segmentation, and GPT-5 patient report generation. Dimensionality reduction mapped 768-dimensional feature vectors in 2D space for analyzing mathematical reasoning patterns. Layer segmentation achieved a 0.9393 Dice coefficient, highlighting biomarkers and layer deformations. Patient reports were generated with layer status, biomarkers, and clinical summaries and demonstrated 95% consistency. This system integrates diagnosis, severity quantification, and interpretability into a single end to end system.

Awards (2)

• Third Award of $1,200 $1,200
• Zydus Pharmaceuticals USA Inc.: First Place Translational Medical Science

Competition history

• ISEF 2026 Translational Medical Science · Entry TMED082

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.