Fragmented Faces, Whole Emotions: Advancing AI Emotion Detection With Partial Facial Input

Overview

Mental health disorders affect approximately 23% of American adults, highlighting the need for improved diagnostic tools (NIMH, 2024). Current diagnoses rely on self-reported surveys and clinical evaluations, which are subject to bias. Machine learning (ML)-based facial emotion recognition (FER) may support mental health diagnosis. However, training FER models currently relies on full face images that are restricted by data privacy regulations. Consequently, it remains unclear whether FER models can perform effectively when trained on non-personally identifiable data (non-PII). This project explores whether FER models can accurately classify emotions using isolated facial features rather than full faces. A public dataset was segmented to extract non-PII features, which were then used to train and evaluate an adapted model. Initial results showed that full faces achieved 60–80% accuracy, whereas nose and mouth regions and rotated images produced about 14% accuracy. Eye and eyebrow regions achieved under 50%, indicating a need for further model adaptation. A second experiment using a larger, higher-resolution dataset improved convergence. After modifying model layers, rotated images and nose-mouth regions matched full-face at 60–80% accuracy. These findings suggest that non-PII data can effectively train FER models. Future work will refine segmentation, improve accuracy, and implement a confusion matrix to analyze misclassification patterns. Training FER models on non-PII data could expand access to restricted datasets and support broader mental health research applications and clinical decision-making tools while preserving privacy protections for individuals across diverse populations and real-world healthcare settings.

Awards (1)

• Fourth Award of $600 $600

Competition history

• ISEF 2026 Behavioral and Social Sciences · Entry BEHA034

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.