A Non-Invasive Ear-EEG Hearing Aid to Address the Cocktail Party Problem via Cloud-Based Deep Learning

Overview

The cocktail party problem renders conversation difficult for hard-of-hearing individuals in multi-speaker environments. Current hearing aids cannot amplify the primary speaker's voice alone due to their single-microphone design and lack of AI software. This project developed a breakthrough hearing aid that determines the primary speaker in multi-speaker environments by harnessing the brain's natural ability to differentiate speakers. In its second year, this project incorporated an upgraded tri-phase methodology and a novel ear-EEG hearing aid prototype. The first of its kind, this prototype can harness 5G to stream EEG and auditory data to a cloud-based server for processing, minimizing the computational load placed on the hearing aid's processor and reducing audio latency to under 10 ms. The software component modified a speaker-identifying AAD algorithm, which analyzed neural activity, in addition to an improved speech-separation algorithm. These algorithms' outputs were compared via a correlation analysis to determine the primary speaker, whose voice was amplified for maximum speech comprehensibility. Objective testing yielded an average cross-correlation (for the primary speaker) of 0.85, indicating strong differentiability between the primary and background speakers. For subjective testing, the test subject was tasked to correctly identify 70% of words spoken by the trial's primary speaker. Of the 40 trials, 35 met this threshold, producing an 87.5% subjective accuracy and demonstrating excellent audio comprehensibility. Altogether, with its compact ear-EEG and ability to stream data wirelessly to a cloud-based server for analysis, the developed technology shows potential in AI-powered hearing aids that can address the cocktail party problem.

Awards (3)

• Third Award of $1,000 $1,000
• Acoustical Society of America: Third Award of $600.00, plus students Mentor will be awarded $150. $600
• Air Force Research Laboratory on behalf of the United States Air Force: First Award of $750 in each Regeneron ISEF Category $750

Competition history

• ISEF 2021 Computational Biology and Bioinformatics · Entry CBIO096T

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.