Operation Oracle: Multi-Modal, Explainable AI Systems for Accessible Cancer Detection

Transcript

Melanoma detected early has a 99% survival rate. After metastasis, however, that drops to 35%. So the difference isn’t treatment. It’s timing. To address this, I built Operation Oracle: a connected, low-cost cancer screening system designed to catch disease earlier.

The first system, NOMA AI, analyzes skin lesions and doesn’t just give a result. It shows why, using Grad-CAM heatmaps and ABCDE clinical criteria, while tracking changes over time. The second system, THORACIS AI, detects lung abnormalities using YAMNet-based acoustic analysis and microwave S21 measurements, capturing functional and structural changes traditional screening can miss.

In medicine, some lung cancers first appear as skin changes: paraneoplastic syndromes. So instead of treating detection as isolated, I designed both systems to share data through a unified platform, transforming two tools into a coordinated screening network. Because when detection is connected, timing becomes something we can control, and lives we can save.

Thank you.

Cancer is often deadly, not always because it cannot be treated, but because of its late discovery. In cancers like melanoma, progression from localized to life-threatening disease can occur in just weeks, with survival rates dropping significantly after metastasis (Figure 1). For lung cancer, long asymptomatic growth means many cases are detected at advanced stages, contributing to high mortality (Australian Institute of Health and Welfare, 2025).

Problem

This delay is not random, but systemic. In Alberta, patients can wait 12–16 weeks for a specialist consult (CityNews Calgary, 2024), while rural populations face diagnostic intervals 30-40% longer than urban areas (Canadian Partnership Against Cancer, 2022). At the same time, more recent audits of commercial medical AI systems have confirmed persistent performance disparities across skin tones, with darker-skinned patients consistently receiving less accurate predictions (Figure 2).

Approach

I was inspired to rethink early detection not as a single test, but as a connected system. In medicine, the body is not isolated, as some lung cancers first appear as skin changes, known as paraneoplastic syndromes (Cleveland Clinic, 2025). This revealed a key idea: detection should not rely on a single signal.

Solution

I developed Operation Oracle, a coordinated platform combining two systems: NOMA AI for skin analysis and THORACIS AI for lung assessment, designed as a supplementary assistive screening tool for primary care physicians, community health workers, or patients at home. In comparison to existing commercial screening systems,

my devices are connected through a shared interface that integrates results and prioritizes accessibility.

Research and Design Process

Figure 3 shows the overall architecture of Operation Oracle: NOMA AI for skin analysis, THORACIS AI for lung assessment, and a shared SQLite database for cross-system communication. This novel integrated design allows a finding in one system to trigger a follow-up recommendation in the other.

NOMA AI: Image-Based Skin Analysis

The skin cancer system uses a Raspberry Pi alongside an Arducam IMX519 camera and a 5-inch touchscreen, employing the ABCDE clinical criteria and Grad-CAM heatmap integration (Figure 4). Grad-CAM (Gradient-weighted Class Activation Mapping) generates heatmaps that highlight which regions of an image most influenced the model's prediction, transforming the AI into an explainable system.

To address dataset bias, I incorporated 10,480 real MILK10k images (222 dark skin) with 3x weighting and 3,000 GAN dark-skin training images, then externally validated on held-out MILK10k samples alongside my self-compiled dataset (n = 1200) (Figure 5).

The risk categories were based on the clinical urgency and malignancy potential of each of the 24 conditions, as established in dermatology literature. Malignant conditions (Melanoma, BCC, SCC) are classified as "High Risk." Pre-cancerous conditions (Actinic Keratosis, Sun Damage) are classified as "Medium Risk." Benign, infectious, and inflammatory conditions are classified as "Low Risk" or flagged for clinical correlation. This stratification ensures that the system's outputs are grounded in medical evidence.

THORACIS AI: Acoustic & Microwave System

The lung system utilizes feature-level concatenation of acoustic and microwave features (Figure 6). Lung sounds (n = 1211) were converted into embeddings using YAMNet (a pre-trained audio classifier), then classified using a neural network with dense layers.

For structural detection, a NanoVNA-F V2 measures S21 transmission parameters. A four-antenna array with RF switches provides 4 unique Tx-Rx paths for scanning tissue-mimicking phantoms made of agar-agar, NaCl, and distilled water. The "tumors" were made with graphite insertions (Figure 7).

AI Acknowledgement

I used DeepSeek for partial code generation and debugging, particularly for the novel microwave imaging subsystem, where existing reference code was unavailable. The skin lesion classification (NOMA AI) and acoustic analysis (THORACIS AI) subsystems relied primarily on existing open-source code structures and were adapted manually. All AI-generated code was reviewed, tested, and validated by me.

NOMA AI: Skin Cancer Detection

The model was trained on 26,838 images combining three sources: original clinical data (13,358 images, 24 classes), MILK10k (10,480 real images including 222 dark skin with 3x weighting), and GAN-generated data (3,000 images). Validation on held-out MILK10k samples alongside my self-compiled dataset (n= 1,200, 200 per Fitzpatrick category) achieved 86.67% overall accuracy, with stratified performance of 87% on dark skin (V-VI), 84% on medium skin (III-IV), and 89% on light skin (I-II). This demonstrates that intentional real-sample weighting with higher multipliers for underrepresented groups has the potential to reduce skin-tone bias in clinical AI algorithms (Figure 8).

Grad-CAM heatmaps confirmed that the model focuses on biologically meaningful features: in malignant cases, highlighted regions consistently aligned with asymmetry and border irregularity. Adding the ABCDE clinical framework improved classification of visually similar lesions, reducing false positives by requiring user confirmation of clinical criteria, a form of human-in-the-loop AI deployment (Figure 9).

Longitudinal tracking successfully flagged progression in test cases where artificial visual changes (enlargement, color variation) were introduced, showing potential for early change detection beyond single-time screening.

THORACIS AI: Lung Disease Detection

The acoustic subsystem achieved 86.4% accuracy across five respiratory conditions. Acoustic sensing captures functional, time-varying signals, such as airflow turbulence, wheezing, and crackle patterns, which in turn directly reflect respiratory mechanics. The microwave subsystem achieved 86.6% on 438 samples (3 conditions × 4 phantom compositions × 3 rotations × 4 paths, with synthetic augmentation).

The raw tumor signal (4.9 dB difference) was initially obscured by direct antenna coupling (air signal >40 dB stronger), a challenge inherent to electromagnetic sensing (Figure 10). Even after background subtraction, microwave-only accuracy across both datasets (audio and microwave) dropped to near-random (35.6%), while the acoustic model maintained 82%. This reflects a fundamental limitation: microwave data is primarily structural and near-binary, lacking multi-class discriminative power, whereas acoustic features encode richer functional patterns.

However, the modalities are complementary. Acoustic features provide temporal resolution, while microwave features capture tissue-specific dielectric properties. Fusing both yielded 99.3% accuracy, demonstrating that combining structural and functional signals resolves ambiguities neither modality can address independently (Figure 11).

Dielectric Analysis & Tissue Classification

Phantom experiments showed that different tissue-mimicking materials produced distinct electromagnetic signatures. Materials designed to represent tumor tissue consistently showed higher permittivity and conductivity compared to healthy analogs. Triple background subtraction isolated pure tumor signatures, with multi-path analysis localizing the tumor to coordinates on the 10×10 cm antenna grid (Figure 12). When used as features, these dielectric properties enabled the separation of multiple tissue types (tumour, healthy background), demonstrating that microwave data contains sufficient structural information to distinguish between biologically relevant conditions.

Discussion & Conclusion: What These Results Mean

Operation Oracle demonstrates that early disease detection becomes fundamentally more powerful when multiple biological signals are interpreted together. Together, they form a more complete picture of disease progression (Figure 13).

From NOMA AI, the most important finding was that explainability and calibration directly affect reliability. The system was not just classifying lesions, but showing why a region was considered high risk. The fact that performance varied across skin tones before calibration and dataset augmentation, then became more consistent afterward, shows that medical AI is highly sensitive to physiological, environmental, and database variation. This highlights that fairness in diagnostic tools must be engineered and validated.

From THORACIS AI, the key insight was that tumors are not just “visible” in imaging. They actively change how signals propagate through tissue. The measurable difference in microwave scattering confirmed that low-cost electromagnetic systems can detect structural differences in biological materials.

However, there are important limitations to note. Microwave experiments were conducted on tissue-mimicking phantoms rather than in vivo human tissue, which may not fully capture real biological complexity. Additionally, while synthetic data augmentation improved performance across skin tones, it may not fully represent the real-world diversity required for robust clinical validation.

Overall, this project shows that earlier and more equitable detection is possible when systems are multi-modal, explainable, and designed for real-world use. With low-cost hardware (NOMA AI ≈ $270; THORACIS AI ≈ $600), it demonstrates that advanced screening can be portable, accessible, and not limited to expensive clinical infrastructure.

Future Improvements & Extensions

Conduct a clinical pilot study with dermatologists and radiologists (n=50 patients) comparing NOMA AI to biopsy gold standard and THORACIS AI to low-dose CT.

Complete ex vivo validation on chicken breast tissue with injected tumor mimics.

Develop dynamic tissue-mimicking phantoms with inflated and deflated states through simulated respiration.

Extend longitudinal tracking to real clinical datasets (partnering with local dermatology clinics) to validate change detection on genuine disease progression.

Improve microwave imaging resolution using 8-antenna arrays with automated scanning stages.

Expand Operation Oracle into an integrated clinical decision support system that recommends specific treatments based on diagnosis.

I would like to express my gratitude to:

Mme. Erika Scholz, my supportive math teacher who aided me greatly during the mathematical research of my project,

Dr. Aniekan Udofia, PhD, my immensely patient and encouraging mother, who was always there to support me in every single step I took over the past year of research and development, even in moments of stress and difficulty,

Williams Udofia, P.Eng, my father, who generously aided in supplementing materials for my project,

Dr. Adekunbi Adetona, PhD, who connected me with professionals in the field of engineering,

Dr. Akolisa Ufodike, PhD, MBA, who encouraged me, believed in my vision, and willingly offered guidance and connections that strengthened the direction of this project,

And lastly, our CYSF delegates who provided impactful insight and support for Team Calgary!

References

American Cancer Society. (n.d.). Cancer in Indigenous populations. The Cancer Atlas. https://canceratlas.cancer.org/burden-of-cancer/cancer-in-indigenous-populations/

Arducam. (n.d.). 16MP IMX519. https://docs.arducam.com/Raspberry-Pi-Camera/Native-camera/16MP-IMX519/

Australian Institute of Health and Welfare. (2025). Cancer data in Australia: overview. https://www.aihw.gov.au/reports/cancer/cancer-data-in-australia/contents/overview

Bennet, D. (2010). Materials Science and Engineering: C. NaCl doping and the conductivity of agar phantoms. https://www.sciencedirect.com/science/article/pii/S0928493110002110?via%3Dihub

Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81, 1–15. http://proceedings.mlr.press/v81/buolamwini18a.html

Canadian Cancer Society. (n.d.). Survival statistics for melanoma skin cancer. https://cancer.ca/en/cancer-information/cancer-types/melanoma-skin/prognosis-and-survival/survival-statistics

Canadian Institute for Health Information. (2023). Physicians in Canada, 2023. https://www.cihi.ca/en/physicians-in-canada

Canadian Partnership Against Cancer. (2022). Road to recovery: Cancer system performance in the COVID-19 era. https://www.partnershipagainstcancer.ca/topics/cancer-in-covid-19-era/

CityNews Calgary. (2024, July 3). Alberta cancer patients facing long waits to see oncologists. https://calgary.citynews.ca/2024/07/03/alberta-oncologist-wait-cancer-diagnosis/

Cleveland Clinic. (2025, August 23). Paraneoplastic syndrome: Symptom, causes and treatment. https://my.clevelandclinic.org/health/diseases/17938-paraneoplastic-syndromes

Conor K McGarry et al. (2020). Physics in Medicine and Biology. Tissue mimicking materials for imaging and therapy phantoms: a review. https://doi.org/10.1088/1361-6560/abbd17

Daneshjou, R., Vodrahalli, K., Novoa, R. A., Jenkins, M., Liang, W., Rotemberg, V., Ko, J., Swetter, S. M., Bailey, E. E., Gevaert, O., Mukherjee, P., Phung, M., Yekrang, K., Fong, B., Sahasrabudhe, R., Allerup, J. A. C., Okata-Karigane, U., Zou, J., & Chiou, A. S. (2022). Disparities in dermatology AI performance on a diverse, curated clinical image set. Science advances, 8(32), eabq6147. https://doi.org/10.1126/sciadv.abq6147

DeepSeek. (2026). DeepSeek [Large language model]. https://chat.deepseek.com/

Díaz, D. (n.d.). MobileNet skin disease classification [Kaggle notebook]. Kaggle. https://www.kaggle.com/code/dhanidiazp/mobilenet-skin-deasese

DIYEngineers. (2022, April 21). 2N2222 pinout quick overview. https://www.diyengineers.com/2022/04/21/2n2222-pinout-quick-overview/

Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118. https://doi.org/10.1038/nature21056

Forefront Dermatology. (2021). How fast does melanoma spread? https://forefrontdermatology.com/how-fast-does-melanoma-spread/

GNU Octave Project. (n.d.). GNU Octave documentation. https://docs.octave.org/latest/

Gupta, A., Kumar Yadav, S., Durai C, A. D., Kuamr, V., Alsharif, M. H., Uthansakul, P., & Uthansakul, M. (2024). Enhanced breast tumor localization with DRA antenna backscattering and GPR algorithm in microwave imaging. Results in Engineering, 24, 103044. https://doi.org/10.1016/j.rineng.2024.103044

HAMTech RADIO SCANNER M0FXB CB DRONE HOBBY Diary. (2023, November 24). NanoVNA, antenna analyzer – First time set up [Video]. YouTube. https://www.youtube.com/watch?v=Ia8OlEp3Z58

Kim, Gyeonghoon; Kim, Doa; Hwang, Jiwon; Lee, Kanghoun; Park, Changhyun; Jegal, Seonggyu; Moon, Ik Jun; Kim, Namkug (2025), “Synthetic Dermatology Dataset for Racial Bias Mitigation in Tumor Classification”, Mendeley Data, V1, doi: https://10.17632/bb844rcjx5.1

Lazebnik, M., McCartney, L., Popovic, D., Watkins, C. B., Lindstrom, M. J., Harter, J., Sewall, S., Magliocco, A., Booske, J. H., Okoniewski, M., & Hagness, S. C. (2007). A large-scale study of the ultrawideband microwave dielectric properties of normal, benign and malignant breast tissues obtained from cancer surgeries. Physics in Medicine and Biology, 52(20), 6093–6115. https://doi.org/10.1088/0031-9155/52/20/002

Li, Z., & Gangwar, K. (2023). Microwave system for 2D and 3D imaging using RP2040 microcontroller [Undergraduate project, Cornell University]. https://ece4760.github.io/Projects/Fall2023/zl823_kg434/index.html

Martinos Center for Biomedical Imaging. (n.d.). Phantoms. https://phantoms.martinos.org/index.php?title=Main_Page

McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., Etemadi, M., Garcia-Vicente, F., Gilbert, F. J., Halling-Brown, M., Hassabis, D., Jansen, S., Karthikesalingam, A., Kelly, C. J., King, D., … Shetty, S. (2020). International evaluation of an AI system for breast cancer screening. Nature, 577(7788), 89–94. https://doi.org/10.1038/s41586-019-1799-6

MILK study team. (2025). MILK10k [Data set]. ISIC Archive. https://doi.org/10.34970/648456

Mini-Circuits. (n.d.). ZFSWA-2-46+ RF switch: Datasheet. https://www.minicircuits.com

MohammedTawfikMusaed. (n.d.). Asthma detection dataset version 2 [Data set]. Kaggle. https://www.kaggle.com/datasets/mohammedtawfikmusaed/asthma-detection-dataset-version-2

NanoRFE. (n.d.). NanoVNA V2 vector network analyzer. https://nanorfe.com/nanovna-v2.html

National Institutes of Health. (2025). Radiation from CT scans and cancer risks. https://www.nih.gov/news-events/nih-research-matters/radiation-ct-scans-cancer-risks

National Lung Screening Trial Research Team. (2011). Reduced lung-cancer mortality with low-dose computed tomographic screening. New England Journal of Medicine, 365(5), 395–409. https://doi.org/10.1056/NEJMoa1102873

openEMS Project. (n.d.). Simple patch antenna tutorial. https://openems.readthedocs.io/en/latest/python/openEMS/Tutorials/Simple_Patch_Antenna.html

Organisation for Economic Co-operation and Development. (2023). Health at a glance 2023: OECD indicators. https://doi.org/10.1787/7a7afb35-en

Rashidul0. (n.d.). Asthma detection from lung audio dataset [Notebook]. Kaggle. https://www.kaggle.com/code/rashidul0/asthma-detection-from-lung-audio-dataset

Raspberry Pi Foundation. (n.d.). Getting started with Raspberry Pi. https://www.raspberrypi.com/documentation/computers/getting-started.html

Samek, W., Wiegand, T., & Müller, K.-R. (2017). Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv. https://arxiv.org/abs/1708.08296

Silva, T. M. M., Conceição, R. C., & Godinho, D. M. (2025). Machine and deep learning applied to medical microwave imaging: A scoping review from reconstruction to classification. Progress in Biomedical Engineering. Advance online publication. https://doi.org/10.1088/2516-1091/ae0bd3

Singh, A., Paul, S., & Gayen, S. (2025). Design of AI-driven microwave imaging for lung tumor monitoring. Scientific Reports, 15, Article 34287. https://doi.org/10.1038/s41598-025-20566-w

Singh, R. (2024). Understanding and implementing MobileNetV3. Medium. https://medium.com/@RobuRishabh/understanding-and-implementing-mobilenetv3-422bd0bdfb5a

The Qt Company. (n.d.). Embedded Linux — Qt 6 documentation. https://doc.qt.io/qt-6/embedded-linux.html

Udofia, A. (n.d.). Skin diseases & cancer comprehensive dataset [Data set]. Kaggle. https://www.kaggle.com/datasets/anieetorudofia/skin-diseases-cancer-comprehensive-dataset

US Preventive Services Task Force. Screening for Skin Cancer: US Preventive Services Task Force Recommendation Statement. JAMA. 2023;329(15):1290–1295. https://doi:10.1001/jama.2023.4342

Images

Figure 1: Bergers, G., & Fendt, S. (2021). The metabolism of cancer cells during metastasis. Nature Reviews. Cancer, 21(3), 162–180. https://doi.org/10.1038/s41568-020-00320-2

Figure 4: UT Physicians. (n.d.). The ABCDEs of skin cancer: A guide to spot warning signs early. https://www.utphysicians.com/the-abcdes-of-skin-cancer-a-guide-to-spot-warning-signs-early/

Figure 5: Khan, A. T., Jensen, S. M., Khan, A. R., & Li, S. (2023). Plant disease detection model for edge computing devices. Frontiers in Plant Science, 14, 1308528. https://doi.org/10.3389/fpls.2023.1308528

Figure 8: Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81, 1–15. http://proceedings.mlr.press/v81/buolamwini18a.html