How does AI fail at diagnosing Chest Scans?

In November 2025 I read an article about how AI could be used to help diagnose CT scans [1]. This got me wondering about the implications of AI being used in this way, and its potential short comings. I wanted to see potential drawbacks of using AI in this way, specifically I wanted to see how AI could improve and be used more safely in this field.

In this project I wanted to answer the question where and why would AI fail in diagnosing chest x-rays. I decided to choose chest x-rays because they were accessible and more familiar to me.

I made a few hypothesis's based on my prior knowledge and some reasearch. My first hypothesis was that AI would perform poorly on other datasets and better at diseases in the dataset that were more common [2]. My second hypothesis was that the Artificial Intelligence would rely on visual patterns/"tells" rather than analyzing a full chest scan as a human would, causing it to miss things [3]. Finally, my last hypothesis was that an AI model that was specialized in diagnosing just one disease would do better at diagnosing it then a model that diagnosed between many diseases.

I believe this project will help improve AI and at least identify where AI needs human oversight when being used in as crucial a space as the medical field. My findings can contribute to how AI is trained in the future to be safely integrated into the medical field.

In order to test the hypothesis's I made I decided to train an AI model to be able to diagnose chest x-rays. I used Google Colab and fast.ai in order to do this. First, I followed a basic tutorial [4] that taught me how to use my python coding to train a deep learning model on chest x-ray data. It diagnosed between pneumothorax and no pneumothorax only trained on 250 images.

I wanted my model to diagnose between 14 different diseases plus no finding cases, so I tweaked my code and used a large dataset of 112 000 chest x-rays off of Kaggle [5]. ChatGPT helped me with the debugging process [13].

Meanwhile, I researched on how AI is being used for medical imaging, specificlly some of its drawbacks in being fully incorporated in the field [2,3,6]. I learned about Grad-CAMs which I wanted to use to try and identify potential failure points in my model [7], as well as learning about AUC which is a way I could test the accuracy of my model [8]. Additionally, I did some background research on radiopaedia on the diseases my model was diagnosing [9].

Once my model was done training, I started with guaging its performance with confusion matrix's, AUC scores and having it diagnose hundred images of the set it was trained on to see roughly how it does. I then started running some experiments with it, first of which I compared my main model with a secondary model trained on a different Kaggle dataset [10]. Then I used the Grad-Cam to look for any patterns in how the dataset diagnoses of normal and altered images. I tested my model multiple times and had multiple versions of my models to ensure that the data I collected was trustworthy.

The main finding in my project was that despite AI being highly capable at diagnosing chest x-rays its reliability was dependent on the training dataset and was very limited.

I used Area under the Curve (AUC) to determine how well my model performed rather than a simple accuracy reading because my model was diagnosing multi-labelled images [8]. AUC was not limited by a threshold [which is a value that is used to determine whether a disease is present or not, for example if the threshold is 0.5 any disease that the model has a confidence above that if present], it focused on how accurately the model ranked the diseases in terms of confidence of its presence [Image 1].

We can gather from this that the model is very overconfident with rarer diseases that are not common in the dataset like hernia and more visible diseases like cardiomegaly, but less confident in more common diseases and no finding cases.

This shows my model has a tendency for false positives, when diseases are diagnosed when they aren't there, rather than false negatives, disease is not identified. Further shown by confusion matrixes in image 1.

I tested two models trained on different datasets against each other to see how they would compare to each others datasets. Not only did my main model do worse than my secondary model on the secondary model's dataset it's performance also dropped significantly from its own dataset. For example, my main dataset had an 0.972 AUC for cardiomegaly on its own dataset and a 0.668 on the secondary model's dataset [Image 2].

For my next test I had my model output grad cams of 100 images to observe patterns on how it diagnosed [Image 3]. The Grad Cam shows areas the AI deemed important in making its diagnosis [7]. I noticed a clear trend, when the AI model correctly diagnosed the image it deeming to correct area important. For example, when it diagnosed Cardiomegaly it looked at the correct area, the heart. However, when it diagnosed wrong it was usually deeming the wrong area important, such as the arm bones. I also attempted to see if blocking out medical devices and parts of the lung would effect the AI's response, but did not gather any meaningful information.

I also distorted images with noise, which is quite common thing to deal with when diagnosing x-rays. My model did not do well at all at diagnosing images with noise of high degree. It would consistently diagnose no finding and had very low confidence in its predictions [Image 4]. It also looked at areas of the image not relevant to the diagnosis, such as the edges, as the images had more and more noise in them.

For my final test, I compared the specialized model trained to diagnose pneumothorax only from the tutorial with my multilabel model trained to distinguish between multiple diseases. My findings showed that my multilabel model did better than the specialized one [Image 5].

I came to the conclusion that AI still has many improvements to make as it continues to be integrated as a potentially revolutionary tool in this part of the medical field.

Through my results we can see that the AI’s performance depended heavily on the data it was trained on.  This inherent bias makes it a lot harder to extrapolate the AI’s prediction to work for other hospitals [2].  Which proves part of my hypothesis that the AI would prefer data it has already dealt with and has trouble extrapolating its results onto unfamiliar territory.

Additionally, the way the neural network was shown in the Grad-Cam showed to me that the AI does not “understand” images the same way humans do and in many cases got its predictions from irrelevant areas in the image [3].  I think we can change this by training the AI with radiologists notes attached to every image directing the AI where to "look" for the disease. This could prevent the AI from making incorrect baseless patterns.

The low confidence when AI tried to diagnose images with noise showed its limitations, which is encountered commonly in this setting [11].  This is likely because the dataset that this model was trained on had relatively low noise and because I used no data augmentation.

Finally, AI's overconfidence is quite dangerous and I believe AI can at its current capacity serve to be a second opinion for radiologists but not work on its own.

Insights I have gained from this project is to give the AI more variety in its training. Specifically, I learned that an artificial intelligence will only be as good as its dataset.

Things I would have done differently are running more tests with models trained on pediatric datasets and spending more time developing my AI model so it could better compare to ones used.

In future projects, I would like to implement models with data augmentation, radiologist's notes in training data, and strategies like multiple AI models working together [12] to see how they would compare to my current model.

My parents helped me in my project by getting me Google Colab Pro and a lot of compute units in order for me to train my AI and run experiments with it. I want to thank them a great deal for their emotional and financial support, without them I could not have done this project at all.

Also, thank you to all the people working at the Thames Valley Science and Engineering Fair who helped me get prepared to go to the Canadian-Wide Science Fair.

I would also like to thank my teachers who in school taught me how to code and got me interested in coding, as well as my friends who sat through my long explainations before I went to present.

Thank you all very much!

AI detects hidden objects on chest scans better than radiologists. (2025). Southampton.ac.uk. https://www.southampton.ac.uk/news/2025/11/ai-detects-hidden-objects-on-chest-scans-better-than-radiologists.page

Burak Koçak, Ponsiglione, A., Stanzione, A., Bluethgen, C., João Santinha, Ugga, L., Huisman, M., Klontzas, M. E., Cannella, R., & Cuocolo, R. (2024). Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects. Diagnostic and Interventional Radiology, 31(2). https://doi.org/10.4274/dir.2024.242854

McLeod, G. A., Emma, Rosenal, T., & Forkert, N. D. (2025). Distinct visual biases affect humans and artificial intelligence in medical imaging diagnoses. Npj Digital Medicine. https://doi.org/10.1038/s41746-025-02226-5

Chest X-ray model – fastai. (2019). Fastai. https://docs.fast.ai/tutorial.medical_imaging.html

NIH Chest X-rays. (n.d.). Www.kaggle.com. https://www.kaggle.com/datasets/nih-chest-xrays/data

Chapman, S. R., Willner, L., Abouafech, A., Roberti, C., & Willner, C. (2026). Diagnostic Performance of Artificial Intelligence in Detecting COVID-19 Pneumonia on Chest Imaging. Cureus, 18(1), e101775. https://doi.org/10.7759/cureus.101775

Bhandari, M. (2024, March 8). Grad-CAM: A beginner’s Guide - Muskan Bhandari - Medium. Medium. https://medium.com/@bmuskan007/grad-cam-a-beginners-guide-adf68e80f4bb

Google Developers. (2019). Classification: ROC Curve and AUC | Machine Learning Crash Course. Google Developers. https://developers.google.com/machine-learning/crash-course/classification/roc-and-auc

Search | Radiopaedia.org. (2024). Radiopaedia. https://radiopaedia.org/search?scope=articles&sort=date_of_last_edit

Ashery. (2021). CheXpert-v1.0-small. Kaggle.com. https://www.kaggle.com/datasets/ashery/chexpert

‌Lin, C., Yoon, A. P., Wang, C.-W., Chao, T., Chung, K. C., & Kuo, C.-F. (2026). Effects of Image Degradation on Deep Neural Network Classification of Scaphoid Fracture Radiographs: Comparison Study of Different Noise Types. JMIR Medical Informatics, 14, e65596–e65596. https://doi.org/10.2196/65596

Khalifa, M., & Albadawy, M. (2024). AI in diagnostic imaging: Revolutionising accuracy and efficiency. Computer Methods and Programs in Biomedicine Update, 5(100146), 100146–100146. https://doi.org/10.1016/j.cmpbup.2024.100146

OpenAI. (2025). ChatGPT [Large language model]. https://chatgpt.openai.com/chat

Images used in video are from Canva's gallery