HARMONI: Harmonically Adaptive Real-Time Multimodal Output for Neural Integration, an AI-Powered Music Therapy Platform for Global Mental Health

Overview

Over 1 Billion people globally live with mental health conditions driving depression & suicide crises. Music therapy is a proven nonpharmacological approach for emotional regulation, yet access remains limited. Most digital solutions rely on static playlists and self reported emotions, limiting effectiveness. Introducing HARMONI, a novel AI-driven audiovisual therapy platform that utilizes the Iso-principle for adaptive intervention through multimodal emotion detection to extend music beyond static composition using a reweighted hybrid architecture optimized by logistic regression. HARMONI integrates facial affect, vocal prosody, physiological signals & psychometric measures into a unified latent valence-arousal state representation. An ensemble of 11 AI models powers hybrid emotion detection, RAG personalization, conversational support, agentic orchestration for diffusion-based image synthesis & transformer-based music generation. 4 randomized controlled experiments (n=108) compared multimodal, personalized, adaptive & AI-generated experimental conditions against unimodal, generic, static and human composed control conditions using 70k datapoints. Weighted hybrid model exceeded unimodal models (acc=88%, ?=0.78, F1=86%) with higher cross-modal consistency. Experimental groups demonstrated significant improvements across outcome measures from baseline to post-therapy (?GAD7=-5.78; ?PSS10=-8.19; ?HR=-9.04; p<0.001) with moderate to large effect sizes (d=0.63–1.28). Engagement was significantly higher in two comparisons (?²=6.48–7.81, p<0.02). HARMONI provides a low cost scalable multilingual solution for accessible global mental health. At scale, a 5% improvement in therapeutic outcomes can benefit 50M+ people & save $5B+ in annual healthcare costs, advancing UNSDG 3 & 10

Awards (2)

• First Award of $6,000 $6,000
• Leighton and Bonnie Berger Family Award

Competition history

• ISEF 2026 Technology Enhances the Arts · Entry TECA008

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.