Beyond Accuracy: AI Brain Tumor Detection with GradCAM++ Interpretability & Clinical Deployment

▶ Click the next section for the video, Find IllumaDx at: https://illumadx.vercel.app/

In this one-minute video, I introduce IllumaDx: a clinical artificial intelligence model designed to address the question that is not usually asked about other medical artificial intelligence systems; what does the model look at when it tells you the correct result?

I trained 40 deep learning models using 4 different data augmentation techniques on brain MRI classification. Hypothesis-driven research identified which augmentation type was expected to perform best. That hypothesis was incorrect – and that is the discovery. Two augmentations were statistically indistinguishable in accuracy to 0.01%, but 33 times more loss and completely opposite attention from GradCAM++ for both accuracy values.

The rest of this presentation will explain the clinical challenge addressed (18 week median MRI wait times in Canada), the experimental methodology used, the statistical analysis used to validate the results, the interpretability analysis that led to discovery, and ultimately the deployed system itself (live demo of https://illumadx.vercel.app/).

All visualizations are my own unless cited or acknowledged.

Introduction

The use of artificial intelligence in medical image interpretation holds immense potential, as evidenced by deep learning achieving an accuracy rate equivalent to that of a trained radiologist for diagnosing brain tumours on MRIs. However, there is one major drawback: most clinical artificial intelligence algorithms are judged solely on their accuracy. Accuracy is an indicator of whether the algorithm has provided the right output, but it provides no clue on how the process took place. A computer could diagnose a tumour accurately by ignoring crucial information in parts of the brain, but its weakness will only become evident when it processes MRIs produced using equipment in a different location.

The Problem

Data augmentation is the process by which transformations are applied to models while training. While this methodology is well-established, there is no standardization for augmentations in medical imaging AI due to the lack of a clinically accepted method to create models that are reliable and accurate. Approximately 300,000 people every year go without timely specialist neuroradiology due to a lack of availability, highlighting the danger of implementing inaccurate models. The current body of research fails to address trustworthiness at all in its considerations of augmentation methodology.

Research Goal

This project explores the extent to which augmentation technique plays an important role in determining the accuracy and trustworthiness of AI models used for diagnosing brain tumours. Furthermore, the project presents IllumaDx, a deployable clinical artificial intelligence diagnostic application designed to prove that trust in AI is attainable.

Overview

To study the effect of augmentation strategy on the diagnostic performance and clinical credibility, a experiment was performed, where training was done for four different types of deep learning networks using the same data under the same circumstances, with augmentation strategy being the only independent variable.

Dataset Construction

An integrated database of 7,627 distinct brain MRI images was created using three databases: Sartaj Bhuvaji, Masoud Nickparvar, and Br35H. Cryptographic hashing with the MD5 algorithm was performed on all 11,964 images to eliminate repetition, reducing 4,337. The resultant database contained images of four tumour classes: glioma (2,301), meningioma (1,819), pituitary (1,776), and no-tumour (1,731). Weighted random sampling was applied to each mini-batch to correct class imbalance. An 80/20 train-validation split produced a validation set of 1,525 images.

Model Architecture & Training

The pre-trained ResNet-18 model on ImageNet was used. The choice of ResNet-18 was made due to its small network size, enabling real-time inference, wide application in medical imaging literature, and the possibility of overfitting with only 7,627 images. Ten separate models were trained per category, yielding 40 models total, spanning 50 epochs and over 380,000 effective training passes.

The 4 Augmentation Groups

Group A used no augmentations and acted as a control. Group B performed base augmentation using horizontal flip, 15-degree rotation, and brightness jitter. Group C performed severe augmentation, including vertical flip, 90-degree rotation, maximum colour jitter, and random erasing, without medical justification in actual MRI imaging. Group D used domain-specific augmentations; each transformation chosen based on real MRI imaging variations between scanner vendors.

Evaluation

All 40 models were tested against the unseen validation data. GradCAM++ highlighted model-specific attentional brain regions. Group-based statistical significance was calculated using one-way ANOVA, and effect sizes were estimated using Cohen's d.

Overview of Results

Among the 40 trained models, the four augmentation groups resulted in different levels of performance that altogether dispute the notion of accuracy as being adequate proof of clinical reliability. The highest level of accuracy was obtained by Group B, which recorded 99.69% ± 0.14%, closely followed by Group D with 99.68% ± 0.18%, Group A with 99.48% ± 0.33%, and lastly Group C with 98.66% ± 0.29%. The difference between the levels of accuracy recorded in all these four groups seems to be minimal, with only about 1% apart. But this would prove to be deceiving.

The Core Finding

While the most accurate prediction may seem like the main discovery of this project, the biggest one has been the divergence in losses during the training process. While Group B reached an end-of-epoch loss of about 0.003, Group C reached a final epoch loss of about 0.100. In other words, Group C had 33 times larger epoch loss compared to Group B, showing that the former never really understood the underlying structure of the dataset. It could remember patterns necessary to achieve good results on the validation set, but it was unstable internally.

Moreover, it was confirmed by another important criterion known as Expected Calibration Error (ECE), which is used to assess whether confidence scores of a classifier can be used to quantify uncertainty. The ECE value of Group C (0.0082) was 2.7 times larger compared to that of Group B (0.0030), confirming that Group C's confidence scores are unreliable and cannot be trusted in a clinical setting where uncertainty quantification matters.

Statistical Validation

A one-way ANOVA test conducted on all 40 models resulted in an F-value: 38.931 and a p-value: 0.000000, proving that the differences seen were not coincidental. The Cohen's d value when comparing Group B with Group C was found to be 4.750, which refers to practical importance when comparing two groups, with 0.8 being regarded as a large effect, therefore, this was a very large effect size. Importantly, when comparing Group B with Group D, p-value = 0.857 and d-value = 0.086, thus rejecting the original hypothesis: domain-specific augmentation provided no measurable benefit over basic augmentation.

GradCAM++ Analysis

The most convincing support for the hypothesis of the beyond accuracy was found by applying GradCAM++ interpretation analysis to the top-performing model of both groups on the same set of MRI images. In this case, Group B generated compact attention maps centered around the tumor, while Group C generated vague attention maps that attended on brain regions that had nothing to do with the task. An algorithm that pays attention to the wrong part of the brain is doomed to fail whenever it receives a scan from another hospital.

Literature Benchmarking

Group B's peak accuracy of 99.69% surpasses all previously published datasets, including Sartaj (91.38%), Cheng (91.28%), Pashaei (88.72%), and Abiwinanda (84.19%), establishing IllumaDx as a state-of-the-art.

Conclusion

This work demonstrates that accuracy is an inadequate measure for assessing AI systems in medicine, and constructs a real-world system that exemplifies the true measure of success. Even two classifiers with nearly equal accuracy can diverge dramatically in terms of training loss (33x), calibration error (2.7x), and, above all, the distinct brain regions targeted when making predictions. Contrary to the original hypothesis, the key contribution of this work is the discovery of the flaw itself.

IllumaDx : A Deployable Clinical AI Diagnostic System:

IllumaDx is an end-to-end AI diagnostic tool for medical settings designed to provide clinicians with reliable, explainable AI. The application can be found live at illumadx.vercel.app and includes live MRI inference for four types of tumors, GradCAM++ visualization that shows which areas of the brain contribute to each prediction, automatic PDF diagnostic report generation, mock EHR integration, a dual-gate safety system excluding non-MRI inputs and low-confidence predictions, a model comparison switch, a French/English interface option, and an ethics strategy matching Health Canada's SaMD pathway.

Clinical Significance

With 300,000 patients going without specialist neuroradiology annually, deploying a model without interpretable reasoning is not merely poor practice — it is ethically wrong. IllumaDx addresses this: every prediction includes an interpretability heatmap, a calibrated confidence score, and a clinical disclaimer. GradCAM++ interpretability and calibration metrics such as ECE should become standard components of clinical AI evaluation. Accuracy alone is the illusion of knowledge. IllumaDx is what knowledge actually looks like.

Future Directions

Further research will entail extending the application of IllumaDx from just brain tumors to other malignancies affecting various organs, such as lungs, colon, and livers. Clinical validation of the IllumaDx platform with Alberta Health Services will be the next important step in evaluating its actual diagnostic capabilities. Validation using BraTS 2020 clinical data will prove that the beyond accuracy paradigm applies in different hospitals and scanners. The research results are currently being compiled for publication in the Journal of Emerging Investigators. Federated learning in the augmented research is another long-term research goal.

Thanks!

I would like to thank the Harvard Moonshot MedScience Program and its faculty for their invaluable guidance and for deepening my understanding of the intersection between artificial intelligence and clinical medicine. They significantly shaped the ambition of this project, specifically sparking my interest and curiosity in using artificial intelligence in medicine. I would also like to thank my parents for their unwavering support throughout this entire process, from the earliest stages of research to the final deployment of IllumaDx. Finally, I would like to thank the open-source communities behind PyTorch, Hugging Face, and Vercel, whose tools made the development of IllumaDx possible.

REFERENCES

[1] Abiwinanda, N., Hanif, M., Hesaputra, S. T., Handayani, A., & Mengko, T. R. (2019). Brain tumor classification using convolutional neural network. In World Congress on Medical Physics and Biomedical Engineering 2018 (pp. 183–189). Springer. https://doi.org/10.1007/978-981-10-9035-6_33

[2] Bhuvaji, S., Kadam, A., Bhumkar, P., Dedge, S., & Kanchan, S. (2020). Brain tumor classification (MRI) [Dataset]. Kaggle. https://www.kaggle.com/datasets/sartajbhuvaji/brain-tumor-classification-mri

[3] BioRender. (2025). Figures created using BioRender. https://www.biorender.com

[4] Br35H. (2020). Br35H: Brain tumor detection 2020 [Dataset]. Kaggle. https://www.kaggle.com/datasets/ahmedhamada0/brain-tumor-detection

[5] Canva. (2025). Poster and figures designed using Canva Pro. https://www.canva.com

[6] Chattopadhyay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018). Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks. 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). https://doi.org/10.1109/WACV.2018.00097

[7] Cheng, J. (2017). Brain tumor dataset [Dataset]. Figshare.

https://doi.org/10.6084/m9.figshare.1512427

[8] Harvard Moonshot MedScience Program. (2025). Harvard Moonshot MedScience. Harvard University. https://www.harvardinnovationlabs.io

[9] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778. https://doi.org/10.1109/CVPR.2016.90

[10] Anthropic. (2025). Claude (claude-sonnet-4-6) [Large language model for hex code colours and overall help for poster designing, as well as website designing]. https://claude.ai

[11] Hugging Face. (2025). Hugging Face: The AI community building the future. https://huggingface.co

[12] Kaggle. (2025). Kaggle: Your machine learning and data science community. Google. https://www.kaggle.com

[13] Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR). https://arxiv.org/abs/1412.6980

[14] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 1097–1105.

[15] Nickparvar, M. (2021). Brain tumor MRI dataset [Dataset]. Kaggle. https://www.kaggle.com/datasets/masoudnickparvar/brain-tumor-mri-dataset

[16] Pashaei, A., Sajedi, H., & Jazayeri, N. (2018). Brain tumor classification via convolutional neural network and extreme learning machines. 2018 8th International Conference on Computer and Knowledge Engineering (ICCKE) (pp. 314–319). IEEE. https://doi.org/10.1109/ICCKE.2018.8566571

[17] PyTorch. (2025). PyTorch: An open source machine learning framework. Meta AI. https://pytorch.org

[18] Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. 2017 IEEE International Conference on Computer Vision (ICCV), 618–626. https://doi.org/10.1109/ICCV.2017.74

[19] Tan, M., & Le, Q. V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. International Conference on Machine Learning (ICML). https://arxiv.org/abs/1905.11946

[20] Vercel. (2025). Vercel: Develop. Preview. Ship. https://vercel.com

[21] World Health Organization. (2023). Cancer: Key facts. WHO. https://www.who.int/news-room/fact-sheets/detail/cancer

[22] Ruder, S. (2016). An overview of gradient descent optimization algorithms. arXiv. https://arxiv.org/abs/1609.04747World Health Organization. (n.d.). Diagnostic imaging.

https://www.who.int/health-topics/diagnostic-imaging

[23] Fraser Institute. (2025). Waiting Your Turn: Wait Times for Health Care

in Canada, 2025 Report. https://www.fraserinstitute.org/studies/

waiting-your-turn-wait-times-for-health-care-in-canada-2025

[24] Azem, G. (2023). Global Radiologist Distribution Analysis. Collective

Minds Health.

[25] Vadde, V., & Bukaita, W. (2025). Comparative evaluation of CNN and

ResNet18 architectures for MRI-based brain tumor classification

using deep learning. European Society of Medicine Journal.

[26] Vimala, B., et al. (2023). Detection and classification of brain

tumor using hybrid deep learning models. Scientific Reports, 13, 23029.

[27] Abdusalomov, A., et al. (2023). Brain tumor detection based on deep

learning approaches and magnetic resonance imaging. Cancers, 15(16), 4172.

[28] Anantharajan, S., et al. (2024). MRI brain tumor detection using deep

learning and machine learning approaches. Measurement: Sensors, 31, 101026.

[29] Singh, A. K., & Agarwal, R. (2025). Explainable AI-driven MRI-based

brain tumor classification. Frontiers in Artificial Intelligence.