Silent Speech Decoding for Dysarthria Using sEMG

Verbal communication is a fundamental aspect of our daily lives, yet millions of people struggle to speak clearly due to speech disorders. Dysarthria is a motor speech disorder caused by weakness or poor coordination of the muscles used for speaking, which results in slurred, slow, or unclear speech. Individuals with Dysarthria often know what they want to say, but are unable to articulate words clearly due to reduced muscle control (Fig. 1).

Speech production relies on the coordinated activation of facial and jaw muscles (Fig. 2). When these muscles are weakened, speech becomes slurred, slow, or difficult to understand. Existing solutions such as speech therapy and assistive communication devices can help, but are often slow to use, limited or less effective for individuals with severe cases.

This project explores a new approach using surface electromyography (sEMG), a technique that measures electrical activity produced by muscles. By placing sEMG sensors on key areas of the face, it becomes possible to detect the muscle signals associated with silent speech, even when words are only weakly articulated, as is common in individuals with Dysarthria.

Design

The goal for this project was to develop a system that used sEMG signals and Machine Learning (ML) to recognize simple words such as "yes" and "no." The system consists of three sEMG sensors connected to an Arduino, which records muscle activity and sends the data to a computer for analysis (Fig. 3a). The recorded signals are then processed and used to train a machine learning model to identify patterns associated with specific words. This model is then used for real-time silent speech detection to aid patients with Dysarthria in expressing themselves (Fig. 3b, Fig. 3c).

Data Collection

To develop and test the system, I collected a total of 400 sEMG samples, with 200 samples from myself and 200 samples from five additional participants, for a total of six users. During data collection, participants silently mouthed the words "yes" and "no" while sEMG signals were recorded from facial muscles. To capture more variation and improve model accuracy, I also recorded both strong and weak signal version of each word. This dataset was then used to train and evaluate machine learning models, allowing me to test how well the system could recognize silent speech across different users.

Materials

Surface electromyography (sEMG) sensors (3 channels)

Disposable adhesive electrodes

Microcontroller (Arduino UNO)

Sensor link shields

Jumper wires/connection cables

Computer with Python installed

Arduino IDE

Machine learning libraries

CSV data storage files

Camera for consistent electrode placement

Participants (myself and additional test users)

Comparison of Machine Learning Models for Silent Speech Classification (Fig. 4)

Multiple machine learning models were evaluated for their ability to classify silent speech using sEMG signals. Logistic Regression achieved the highest accuracy (70%), followed closely by SVM and Random Forest (69%), while KNN and Gradient Boosting performed slightly lower (65%). Overall, no single model consistently outperformed the others across all tests, but simpler models such as Logistic Regression demonstrated strong and reliable performance. These results highlight the importance of model selection and also suggest that increased data and more detailed tests are needed to improve classification accuracy.

Impact of Training Data Quantity vs Model Accuracy (Fig. 5)

The impact of training data on model accuracy was clearly observed when comparing the “My Data vs My Data” experiment (200 recordings) to the “User_2 Data vs My Data” experiment (50 recordings). When the models were trained on the larger dataset, they achieved significantly higher accuracy because they were exposed to a wider range of signals, enabling them to better recognize sEMG patterns. On the contrary, when the models were trained on only 50 recordings, their accuracy and performance were almost cut in half. This highlights the importance of having enough training data and how it can strongly affect model performance.

Model Generalization to Unseen Data (Fig. 6)

The models were tested on data from users not included in the training set to evaluate generalization. Results showed a noticeable decrease in accuracy compared to the same-user testing. For example, while some models achieved up to 80% accuracy on certain users, performance dropped to as low as 40-50% on others. This variation means that sEMG signals differ significantly between individuals, making it challenging for models to accurately predict. These findings support the need for more diverse training data and user-specific calibration to improve accuracy and reliability.

ANOVA Validation Results

To validate the need for combined training data to improve model accuracy, an ANOVA test was conducted on all 150 features extracted from each participant’s sEMG data to determine whether differences between participants were statistically significant. The results showed that 134 out of 150 features (89%) had a p-value less than 0.05, indicating that the variation in sEMG signals between participants is unlikely to be due to chance. This finding helps explain the reduced cross-user accuracy observed during testing and supports the use of combined training data to improve the performance of the silent speech detection system.

Performance of Models Trained on Combined Multi-User Data (Fig. 7)

Models trained on combined data from multiple users showed significantly improved performance compared to cross-user testing. For example, Logistic Regression improved from 59% to 83%, SVM from 60% to 78%, and Random Forest and Gradient Boosting both increased drastically to 95% accuracy. This demonstrates that having diverse training data helps models identify sEMG signals across different users. These results support my ANOVA findings, which showed significant differences between users, and confirmed that multi-user training data is essential for improving model performance.

This project aimed to address the challenges of verbal communication for people with Dysarthria by developing an sEMG-based system that detects and generates intended speech. I successfully developed an early-stage prototype capable of detecting and classifying silent speech signals from facial muscle activity using surface electromyography (sEMG). While this prototype currently recognizes a limited vocabulary set (“yes” and “no”), I have successfully laid the groundwork for exploring the possibilities of sEMG-based silent speech recognition and have demonstrated that a correlation exists between sEMG signal patterns and specific words.

I came across a number of difficult challenges during development, such as inconsistent sEMG signals. Some of these were successfully resolved, while others (e.g., inconsistent readings for a user) will have to be explored in greater depth on a different timeline than what is available for this year’s science fair. In spite of all the setbacks I encountered, I am still proud of the prototype that I developed and consider it an amazing starting point for shifting the landscape for silent speech communication for people with Dysarthria in the near future.

Future work will shift from whole-word classification to phoneme-level detection, allowing recognition of individual sound units that can form words, sentences, and full conversations. This will improve scalability and flexibility, since a small set of phonemes can represent an entire language. Expanding the dataset to include more diverse users and additional training data will further improve performance. Future development will also explore wireless, wearable designs for practical real-world use. Overall, I will continue to refine the system with the aim to create a real-time, accessible communication tool for individuals with Dysarthria and support effective sEMG-based silent speech systems.

I would like to thank my dad, Idowu Oduntan, for his mentorship and guidance throughout this project. His advice on data collection, experimental design, and model selection was invaluable. I am also grateful to my brother, Opemipo Oduntan, for his early guidance, inspiration, and support, including funding the project.

I would like to thank my mom for her constant encouragement and motivation, which helped me stay focused during challenging moments. I am also thankful to my cousins for participating in data collection and contributing to the quality of my results.

I appreciate my parents for their continued support, and my friend Aadhi Chandrasekaran for helping me understand ANOVA and other aspects of statistical analysis. Finally, I would like to thank MSSS and the Manitoba science community for providing this opportunity to learn and grow.

Chowdhury, Adiba Tabassum, et al. “Decoding Silent Speech: A Machine Learning Perspective on Data, Methods, and Frameworks.” Neural Computing and Applications, 20 Feb. 2025, https://doi.org/10.1007/s00521-024-10456-z.

Cleveland Clinic. “Dysarthria & Speech | Cleveland Clinic.” Cleveland Clinic, 2023, my.clevelandclinic.org/health/diseases/17653-dysarthria.

Cleveland clinic. “Electromyography (EMG).” Cleveland Clinic, 10 Feb. 2023, my.clevelandclinic.org/health/diagnostics/4825-emg-electromyography.

“Diagnosis and Treatment - Mayo Clinic.” Mayoclinic.org, 2026, www.mayoclinic.org/diseases-conditions/dysarthria/diagnosis-treatment/drc-20371999?p=1. Accessed 6 Apr. 2026.

“Dysarthria -.” SpeechTherapy.org, 28 June 2025, speechtherapy.org/disorders/adults/dysarthria/.

GeeksforGeeks. “Gradient Boosting in ML.” GeeksforGeeks, 25 Aug. 2020, www.geeksforgeeks.org/machine-learning/ml-gradient-boosting/.

---. “KNearest Neighbor(KNN) Algorithm.” GeeksforGeeks, 14 Apr. 2017, www.geeksforgeeks.org/machine-learning/k-nearest-neighbours/.

---. “Logistic Regression in Machine Learning.” GeeksforGeeks, 9 May 2017, www.geeksforgeeks.org/machine-learning/understanding-logistic-regression/.

---. “Random Forest Algorithm in Machine Learning.” GeeksforGeeks, 22 Feb. 2024, www.geeksforgeeks.org/machine-learning/random-forest-algorithm-in-machine-learning/.

---. “Support Vector Machine (SVM) Algorithm.” GeeksforGeeks, 20 Jan. 2021, www.geeksforgeeks.org/machine-learning/support-vector-machine-algorithm/.

“How Is Speech Produced? From Airflow to Articulation.” Biology Insights, 13 Aug. 2025, biologyinsights.com/how-is-speech-produced-from-airflow-to-articulation/.

Huang, Juebin. “Dysarthria.” Merck Manual Consumer Version, Merck Manuals, 2 Aug. 2023, www.merckmanuals.com/home/brain-spinal-cord-and-nerve-disorders/brain-dysfunction/dysarthria.

Mayo Clinic. “Dysarthria - Symptoms and Causes.” Mayo Clinic, 2024, www.mayoclinic.org/diseases-conditions/dysarthria/symptoms-causes/syc-20371994.

---. “Electromyography (EMG) - Mayo Clinic.” Mayoclinic.org, 21 May 2019, www.mayoclinic.org/tests-procedures/emg/about/pac-20393913.

“AlterEgo.” Alterego.io, 2025, www.alterego.io/.

R., Christopher S, MdRasedul I, M Assad-Uz Z, Mohammad H. “A Comprehensive Study on EMG Feature Extraction and Classifiers.” Open Access Journal of Biomedical Engineering and Its Applications, vol. 1, no. 1, 7 Feb. 2018, lupinepublishers.com/biomedical-sciences-journal/fulltext/a-comprehensive-study-on-eMG-feature-extraction-and-classifiers.ID.000104.php.

Satterlee, Nicholas, et al. “Sentence-Level Silent Speech Recognition Using a Wearable EMG/EEG Sensor System with AI-Driven Sensor Fusion and Language Model.” Sensors, vol. 25, no. 19, 5 Oct. 2025, pp. 6168–6168, www.mdpi.com/3530330, https://doi.org/10.3390/s25196168.

Simonyan, Kristina, and Barry Horwitz. “Laryngeal Motor Cortex and Control of Speech in Humans.” The Neuroscientist, vol. 17, no. 2, 28 Feb. 2011, pp. 197–208, https://doi.org/10.1177/1073858410386727.

Tang, Chenyu, et al. “Sensing Technologies for Silent Speech Interfaces.” Nature Sensors, vol. 1, no. 1, 15 Jan. 2026, pp. 16–26, https://doi.org/10.1038/s44460-025-00010-2.