Interpretable Multi-Plane Fusion Transformer for Accurate MRI-Based Knee Injury Detection and Severity Assessment

Overview

Knee injuries and osteoarthritis affect an estimated 50 million individuals worldwide every year. Many patients undergo unnecessary and invasive procedures even after diagnostic imaging, with arthroscopies for degenerative meniscal tears often providing no meaningful benefit to patients. Radiologists evaluate knee MRIs by reviewing axial, coronal, and sagittal views separately and mentally integrating findings, an expertise-dependent process with an inherent risk of overlooking subtle pathologies. This project introduces a novel interpretable multi-plane fusion transformer to assist clinicians in accurately detecting knee injuries in MRIs and assessing their severity for effective screening. Powered by a ViT encoder and slice-level attention mechanisms, the fusion transformer processes all three MRI planes. The model applies late fusion, learning plane-specific weights for ACL tears, meniscus tears, and general abnormalities, mirroring radiologist reasoning and improving subtle tear detection across views. The system provides a quantitative assessment of pathology severity, categorizing cases into low-, moderate-, or high-risk tiers to optimize triage and prioritize patient care. Additionally, the system incorporates 2D/3D heatmap visualizations highlighting suspicious injury regions in knee MRIs to provide model interpretability, supporting clinical trust in the system. Furthermore, it enhances clinical decision support through LLM-generated patient-readable explanations of model outputs. Trained on 1,130 knee MRI volumes, the model achieves accuracies of 93.8%, 90.3%, and 86.1% for general abnormalities, ACL tears, and meniscus tears, respectively. By combining multi-plane fusion, interpretability, and severity scoring, this work demonstrates the strong potential to improve the reliability of knee pathology screening and reduce unnecessary invasive procedures.

Competition history

CSEF 2026 Medicine & Physiology (Senior Division) · Entry S-15-23

Related projects

ISEF · 2025

An Analytical Study of Optimized EfficientNet CNNs, Vision Transformers, and Hybrid Transformers for State-of-the-Art MRI-Based Brain Tumor Diagnosis With Integrated Explainable AI

JSHS · 2024

FFTstack: Integrating Fourier Transform and Residual Learning for Arctic Sea Ice Forecasting

ISEF · 2026

A Multimodal Deep Learning Framework for a Novel Clinical Decision Support System in Retinal Disease Diagnosis and Management

CSEF · 2026

Multi-Scale Knowledge Transfer Convolutional Transformer: A Novel Deep Learning Framework for 3D Brain Vessel Segmentation

Closest projects by meaning, across every fair and year in the corpus.

Interpretable Multi-Plane Fusion Transformer for Accurate MRI-Based Knee Injury Detection and Severity Assessment

Overview

Competition history

Related projects

An Analytical Study of Optimized EfficientNet CNNs, Vision Transformers, and Hybrid Transformers for State-of-the-Art MRI-Based Brain Tumor Diagnosis With Integrated Explainable AI

FFTstack: Integrating Fourier Transform and Residual Learning for Arctic Sea Ice Forecasting

A Multimodal Deep Learning Framework for a Novel Clinical Decision Support System in Retinal Disease Diagnosis and Management

3D MRI Inpainting for Structural Reconstruction of Brain Tumor Regions via Composite Spatial–Frequency Optimization

Ensembling Convolutional and Transformer Architectures for Retinopathy of Prematurity Screening in Neonatal Fundus Image

WriVision: A Machine Learning Model for Quality Control of Wrist X-Rays

A Novel End-to-End Deep Learning Pipeline for Stereotactic Cranial Surgery Planning

Multi-Scale Knowledge Transfer Convolutional Transformer: A Novel Deep Learning Framework for 3D Brain Vessel Segmentation

Browse more like this