Rapid Screening for Major Skin Cancers with M-SCAHN: Multimodal Hierarchical CNN-Transformer Hybrid Networks with Advanced Interpretability, Lesion Evolution Tracking, Trait Identification, and Color Constancy for Improved Generalization in Diverse Populations
ISEF · 2024 Robotics and Intelligent Machines First Award
Overview
Skin cancer is the most prevalent cancer worldwide with 8 million incidences and 120,000 annual deaths. Visual inspection has a 40% misdiagnosis rate, causing treatment delays leading to increased mortality. Millions are referred for unnecessary, expensive, and invasive biopsies. This project introduces a point-of-care software application for accurate, immediate, and non-invasive skin cancer screening through innovative computational analysis of lesion images: Multimodal Skin Cancer Analysis with Hybrid Networks (M-SCAHN). Using 13,500 images, this research proposes novel hierarchical CNN-transformer hybrid deep-learning models. The multimodal classification model considers patient-specific factors such as age and anatomical location in its decisions. M-SCAHN provides interpretable diagnostic reasoning through GradCAM, improves screening in diverse populations with color constancy algorithms, and instantly identifies cancerous lesion traits. Physicians can upload an image, and in 18 seconds, M-SCAHN performs end-to-end analysis, detecting cancer with state-of-the-art accuracy: 98.9% sensitivity (99% CI: 97.6% - 100%) and 97.7% specificity (99% CI: 96.2% - 99.1%). The segmentation model detects lesion borders with 95.5% Dice score, a critical step for various dermatology applications. With zero localization failures, it is the only reliable skin lesion segmentation model, facilitating a novel lesion border evolution tracking algorithm. M-SCAHN is the first of its kind to detect all major skin cancers and pre-cancer with significant accuracy, interpretability, and a solution to aid clinician diagnosis of diverse skin. This research can revolutionize skin diagnostics by reducing mortality through early detection while minimizing unnecessary biopsies and costs.
Awards (1)
- First Award of $5,000 $5,000
Competition history
- ISEF 2024
Resources
Related projects
ISEF · 2024
Revolutionizing Non-Invasive Skin Cancer Detection Through a Novel Vision Transformer Application
ISEF · 2022
DermaTech: A Novel, Non-Invasive Technology To Detect Skin Cancer
ISEF · 2019
Developing a Novel, Accurate, and Rapid Computer Vision and Machine Learning Based Skin Disease Diagnosis Pipeline, Hardware Apparatus, and Mobile Application
ISEF · 2022
A Precise, Portable, Non-Invasive Melanoma Detection Device Using Image-Based Deep Learning Approach
Closest projects by meaning, across every fair and year in the corpus.
Source: Regeneron International Science and Engineering Fair