Multi-Scale Knowledge Transfer Convolutional Transformer: A Novel Deep Learning Framework for 3D Brain Vessel Segmentation

Overview

Recent studies show that small cerebrovascular abnormalities are the cause of many brain disorders. Early diagnosis and monitoring of the abnormalities in vivo at the micro-level facilitates improved understanding of the etiology of diseases and better treatment. Existing methods fall short in extracting complex 3D brain vessels from in-vivo Magnetic Resonance Imaging due to the lack of synergy in coordinated local and global pattern exploration, and their performance decreases significantly when handling lower-resolution images. Therefore, I propose a novel Multi-scale Knowledge Transfer Convolutional Transformer for 3D vessel segmentation. It uniquely integrates convolutional operation with Transformer in a U-net architecture, which responds to local receptive fields with convolution layers and global contexts with transformer encoders in a multi-scale fashion. It intrinsically enhances the synergy between local and global vessel feature extractions in the joint convolutional transformer embedding space, leading to a more accurate and reliable segmentation. Furthermore, to enable using relatively low-resolution images to segment fine-scale vessels, a novel knowledge transfer mechanism is designed to explore data interdependencies and automatically transfer knowledge gained from high-resolution data to the low-resolution handling network through multi-level loss functions to improve its fine vessel segmentation capability. Extensive experiments on benchmark image datasets have demonstrated that my proposed method outperforms all other state-of-the-art deep learning methods, and the integrated 3D brain vessel segmentation and visualization software allows accurate quantification of brain vessels for computer-aided diagnosis of brain diseases and scientific discoveries.

Competition history

ISEF 2024 Robotics and Intelligent Machines · Entry ROBO068 · Los Angeles, California, United States

Resources

ISEF abstract page

Related projects

ISEF · 2022

A Novel End-to-End Deep Learning Pipeline for Stereotactic Cranial Surgery Planning

ISEF · 2024

Integration of Deep Learning Into Automatic Volumetric Cardiovascular Dissection and Reconstruction in Simulated 3D Space for Medical Practice

ISEF · 2025

An Analytical Study of Optimized EfficientNet CNNs, Vision Transformers, and Hybrid Transformers for State-of-the-Art MRI-Based Brain Tumor Diagnosis With Integrated Explainable AI

ISEF · 2019

Segmenting CT Slices: Optimizing Lesion Detection through Mask Region-based Convolutional Neural Networks

Closest projects by meaning, across every fair and year in the corpus.