A Novel Approach to Recognize Emotion from Speech Using Machine Learning Algorithms to Aid Social Interaction of Kids with Autism

Overview

People with autism spectrum disorder (ASD) have difficulty in recognizing and in comprehending emotions from other's facial expressions. This impacts their development of empathy and affects their social interactions. Several studies have tried providing clues to people with autism about the emotion of the person interacting with them using facial recognition software as well as speech recognition software. These software use machine learning algorithms which learn from data and find patterns to classify the data. Most of these studies have used features of the sound waves of speech like the Mel Frequency Cepstral Coefficients (MFCC) to learn and predict the emotion associated with speech. In this study, a novel method was investigated that augments such learning with heuristic weights assigned to the words in the speech. This study experimented with different machine learning algorithms to determine emotion using this new technique and compared it to the traditional techniques. All the models were run in two modes; MFCC with the heuristic weights of the associated words and MFCC without the heuristic weights. The accuracy levels ranged from 69.7% to 90.9% when the models are run without heuristic weights but the accuracy increased by 3 to 12% when they were run using heuristic weights with total accuracies ranging from 81.8% to 93.9%. The multinomial logistic regression algorithm performed the best when used with the heuristic weights. The results indicate that using heuristic weights for words in the speech can help improve the software for emotion detection to help autistic people communicate more effectively.

Awards (2)

• Second Award of $2,000 $2,000
• Acoustical Society of America: First Award of $1,500, plus students School will be awarded $200, and Mentor awarded $500. $1,500

Competition history

• ISEF 2018 Robotics and Intelligent Machines · Entry ROBO012

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.