Combating Stuttering via an Empowered Multi-modal Neural Network based on Facial and Audio Recognition Data

Overview

Stuttering, a communication disorder in which the flow of speech is broken by repetitions, prolongations, or abnormal stoppages of syllables, affects tens of millions of people. Stuttering causes people of all ages to have low self-esteem and trouble maintaining social relationships. I would like to use cutting-edge technology to improve their lives. Since the treatments for different degrees of stuttering are different, a crucial step in treating stuttering is accurate degree detection. Compared to current clinical practices, my anticipated research outcome is a low cost, accurate, and convenient approach in detecting the degree of stuttering. My key hypothesis is that erratic facial movements and abnormal verbal speech strongly indicate stuttering degree. My research methods include audio recognition in finding the degree of stuttering, facial recognition in finding action units, stuttering degree calculations for all training data, and a neural network model relating action units and stuttering degree. Additionally, a user-friendly software was designed and programmed to facilitate the functionalities such as video recording, video processing, and stuttering degree evaluation. To the best of my knowledge, this tool is the first of its kind. It is low cost (the current cost is $4,000 per person for speech therapy) and convenient (current treatments are limited by the availability of speech pathologists). With 350,000 data points involved in the training, validation, and testing of the software, the accuracy is high; therefore, there is a substantial relationship between verbal speech and facial movements and the degree of the occurrence of stuttering.

Awards (2)

• Fourth Award of $500 $500
• American Psychological Association: Third Award of $500 $500

Competition history

• ISEF 2019 Behavioral and Social Sciences · Entry BEHA001

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.