Early Frames to Full Picture: Deep Learning for Interpretable mRNA-LNP Delivery Prediction

Last summer, I took Anatomy and Physiology 12 and became fascinated by how lipid nanoparticles (LNPs) are used to carry mRNA in COVID vaccines. That curiosity grew into a literature review on drug delivery systems and eventually into this project, where I aim to accelerate the LNP drug-delivery testing using machine learning.

One of the challenges in LNP-based drug delivery is the slow testing process. Currently, determining whether a new LNP formulation successfully delivers its mRNA cargo requires about 12 hours of microscope observation per batch, as the green fluorescent reporter signal confirming successful delivery takes that long to be visible. This model takes advantage of the biological fact that the LNPs enter cells and begin intracellular trafficking in the first 3.5 hours, therefore, the eventual outcome can essentially be predicted in the first 3.5 hours.

In 2021, Harrison et al.[4] published a CNN-LSTM model for this task. However, I identified three key limitations: the LSTM may lose information across time steps, the model functioned as a black box with no spatial explanation, and there was no interactive tool for exploration. This project addresses all three issues.

This bottleneck slows down the development of mRNA vaccines, gene therapies, and personalized cancer treatments. If a model could predict the 12-hour outcome from only the first 3.5 hours of microscopy data, drug developers could screen 3 to 4 times more formulations per day on the same equipment, and accelerate research for the patients who need these therapies fastest.

I started with a literature review on drug-delivery systems and identified key challenges in testing LNP drug-delivery formulations. This led me to an open-source time-lapse microscopy dataset published by Harrison et al. [4] at Uppsala University and AstraZeneca, which gave me both a public dataset and a published benchmark to compare against.

The dataset contains images of 774 HepG2 human liver cancer cells. Each cell has 72 images taken every 10 minutes for 12h using a Yokogawa CV7000 robotic confocal microscope. Only the first 20 frames (0 to 3.5 hours) were used as model input.

To control for everything except architecture, I matched the setup of Harrison et al. exactly, i.e., the same train/test split (602 training cells, 172 test cells), the same target variable (GFP at hour 12), and the same evaluation metric (RMSE in log-centered space). This ensured that any difference in accuracy is attributed to my architectural choices, not different data conditions.

I made three architectural decisions to improve model accuracy compared to Harrison’s CNN-LSTM model. First, I replaced the LSTM with a Transformer which processes all frames in parallel instead of one at a time. Second, I added attention pooling so that the model weights informative time points more heavily. Third, I integrated Grad-CAM to produce a heatmap showing which pixels in each frame influenced the prediction.

The model was built in PyTorch using open-source packages, including torchvision for the pretrained VGG16 image encoder. The full list of tools is shown in Table 1.

Finally, I developed a dashboard using Streamlit, a python package, to enable exploration of all results without requiring programming expertise. The user can pick any test cell and see the prediction, the true value, all 20 Grad-CAM heatmaps. This makes the model usable as an actual scientific tool.

The CNN-Transformer hybrid model was evaluated on 172 held-out test cells that model had never seen during training. The model achieved a test RMSE of 0.5167 in log-centered GFP space, compared to the benchmark published by Harrison et al. [4] of approximately 0.60 using the same dataset, same test/train split and the same evaluation metric. This represents a 13.9% improvement, attributed specifically to the architectural changes since all other variables were held constant. The Pearson correlation coefficient (r) between predicted and true GFP across the 172 cells was 0.9054, and the R-squared was 0.8089 in the transformed space. This indicates that the model explains 80.9% of the variance in delivery efficiency between cells. The various test results are shown in Table 3.

The best training checkpoint was achieved at epoch 19, with a validation RMSE of 0.53 and a training RMSE of 0.55. Since validation error is lower than the training error, it indicates that the model has learned generalizable features rather than memorizing individual training cells. The test RMSE of 0.5167 on 172 held-out cells, drawn from the same fold structure Harrison et al. [4] used, further supports this conclusion.

Grad-CAM heatmaps revealed two clearly distinct spatial patterns. In high-GFP cells, where LNP delivery succeeded, activation concentrated near the nucleus, in the perinuclear region. This corresponds to LNP particles that successfully migrated inward from the cell periphery during the first 3.5 hours, which is the spatial signature of successful endosomal escape. In low-GFP cells, activation was more diffuse and spread across cell periphery, corresponding to LNP particles that remained trapped in peripheral endosomes and never reached the nucleus.

This is the most significant finding of the project. The model was trained only on raw microscopy images and a single GFP number per cell. It was never given any information about cell anatomy, endosomal biology, or where the LNP particles should be located.The Grad-CAM reveals it has independently learned to attend to perinuclear LNP localization, exactly what cell biologists have established through decades of endosomal escape research. The model is not just predicting accurately, it is also capturing the correct biological mechanism.

This project demonstrated that a CNN-Transformer hybrid architecture can predict mRNA-LNP delivery outcomes from only 3.5 hours of early cellular imaging data,  rather than the full 12-hours of image sequence. The model achieves a test RMSE of 0.5167, which is 13.9% below the Harrison et al. [4] benchmark of approximately 0.60. The results show that the parallel self-attention mechanism of the Transformer outperforms sequential LSTM processing for this temporal LNP prediction task.

During training, learned attention pooling was implemented and used in the final model, allowing the model to selectively weight the most informative temporal frames instead of treating all 20 frames equally. A Pearson correlation of 0.9054 and an R² of 0.8089 across 172 completely unseen test cells indicate that the model captures the large majority of biologically meaningful variation in per-cell delivery efficiency. This could speed up the development of mRNA vaccines, gene therapies, and cancer treatments.

Another significant outcome comes from Grad-CAM analysis. The Grad-CAM results show concentrated perinuclear activation in high-GFP cells and diffuse peripheral activation in low-GFP cells, consistent with endosomal escape biology. This suggests that the model has learned correct biological features from images alone, without any biological labels.

The entire project was completed independently using exclusively free, open-source tools, and a publicly available dataset.

Classification mode: Add a binary GFP positive/negative classification head to enable direct F1 score comparison with the Harrison et al. classification results.

Multi-cell-line validation: Evaluate the model on additional cell lines, including muscle (C2C12), immune, and neural cells, to determine whether the endosomal escape morphological signature generalizes beyond liver cells.

Real-time integration: Integrate the model with live robotic microscopy systems for immediate feedback during experiments, enabling truly adaptive LNP screening workflows.

Mechanism Discovery: Use interpretability insights to guide experimental biology

Uncertainty quantification: Add reliable confidence estimates for each prediction, enabling clinical decision support that knows when it is uncertain.

I would like to thank Harrison et al. at Uppsala University and AstraZeneca for making their LNP microscopy dataset and benchmark publicly available. Without their open data, this project would not have been possible, and their published RMSE of 0.60 gave me a meaningful benchmark for comparison. I am also grateful to my science teachers for their encouragement and support with the literature review. In addition, I would like to thank the mentors at Future Science Leaders at Science World for introducing me to machine learning and the scientific process. Finally, I would like to thank my parents for their support and for believing that this project was worth submitting to a science fair.

Work Cited

[1] U.S. Food and Drug Administration. (2018). New class of drugs fulfills promise of RNA-based medicine. Retrieved from https://www.fda.gov/drugs/spotlight-cder-science/new-class-drugs-fulfills-promise-rna-based-medicine

[2] Izon. (2023). Applications of lipid nanoparticles (LNPs) in nanomedicine. Retrieved from https://www.izon.com/videos/applications-of-lipid-nanoparticles-lnps-in-nanomedicine

[3] Xu, S., Hu, Z., Song, F., Xu, Y., & Han, X. (2025). Lipid nanoparticles: Composition, formulation, and application. Molecular Therapy - Methods & Clinical Development, 33(2), 101463. https://doi.org/10.1016/j.omtm.2025.101463

[4] Harrison, P. J., Wieslander, H., Sabirsh, A., Karlsson, J., Malmsjö, V., Hellander, A., Wählby, C., & Spjuth, O. (2021). Deep-learning models for lipid nanoparticle-based drug delivery. Nanomedicine, 16(13), 1097-1110. https://doi.org/10.2217/nnm-2020-0461

[5] Qian, Z., LaRochelle, J. R., Jiang, B., Lian, W., Hard, R. L., Selner, N. G., Luechapanichkul, R., Barrios, A. M., & Pei, D. (2014). Early endosomal escape of a cyclic cell-penetrating peptide allows effective cytosolic cargo delivery. Biochemistry, 53(24), 4034-4046.

[6] Chatterjee, S., Kon, E., Sharma, P., & Peer, D. (2024). Endosomal escape: A bottleneck for LNP-mediated therapeutics. Proceedings of the National Academy of Sciences, 121(11), e2307800120. https://doi.org/10.1073/pnas.2307800120

[7] Luther, D. C., Jeon, T., Goswami, R., Nagaraj, H., Kim, D., Lee, Y.-W., & Rotello, V. M. (2021). Protein delivery: If your GFP (or other small protein) is in the cytosol, it will also be in the nucleus. Bioconjugate Chemistry, 32(5), 891–896. https://doi.org/10.1021/acs.bioconjchem.1c00103

[8] Sato, Y. (2022). [Cross-section of an LNP-RNA showing mRNA encapsulated by lipids] [Diagram]. Asia Research News. Retrieved from https://www.asiaresearchnews.com/content/efficient-mrna-delivery-branched-lipids