Personalized Cloud-Based sEMG-to-Haptic System to Enable Long-Distance Communication for Usher Syndrome

Overview

Usher Syndrome is a congenital condition that causes severe hearing and vision loss. Despite 140 million deafblind individuals worldwide, research and support remain insufficient. Existing assistive technologies are not hands free, are expensive, bulky, and difficult to use. Current methods of communication for the deafblind are limited to tactile communication that cannot operate beyond close-proximity interactions – further isolating the deafblind from society. Hence, there is a need for a method of long-distance communication. This study presents a highly personalized real-time long-distance wearable communication device for Usher Syndrome and the broader deafblind population. Using surface electromyography (sEMG) sensors, the system classifies muscle activation patterns and translates them into haptic feedback for a remote recipient. A novel, rapidly customizable gesture classification system creates small, personalized models for each user. These small models account for the variations in muscle signal characteristics, making the system robust to user-specific differences. A cloud-based architecture reduces computational complexity and enables cost-effective hardware. The wearable glove has embedded vibration motors that stimulate fingers corresponding to tactile sign language, effectively conveying messages. Human testing, approved by a UC Berkeley IRB, conducted on a small sample of human subjects. It demonstrated that personalized models outperform cross-user models in classification accuracy. Additionally, real-time testing confirmed end-to-end latency of under one second, highlighting the system's potential for long-distance, wearable, personalized deafblind communication.

Competition history

• ISEF 2025 Biomedical Engineering · Entry ENBM043

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.