CO-Sign Language

Overview

Individuals with conditions such as Amyotrophic Lateral Sclerosis (ALS) or paralysis often face significant barriers in communicating speech and mobility limitations. These challenges necessitate innovative solutions to restore their ability to interact effectively with others. This project proposes CO-sign Language, a revolutionary communication system for individuals with limited or impossible speech and mobility. A new language was entirely developed based on patterns of eye movements, and an AI model was trained to convert those patterns into speech in real time. A second model that can distinguish between intentional and unintentional eye movements was also created to prevent false detections, enhancing accuracy. Features such as autocomplete and autocorrect were added to achieve faster communication, predict what the user is planning to type, and suggest complete words as they type with their eyes. A third model is being developed to construct full sentences based upon context. To facilitate the project usage, a wearable prototype was built, resembling a pair of smart glasses, which contains an interior camera, a Raspberry Pi board, and a speaker. In addition to being supported with an Augmented Reality display. CO-sign Language was tested on 15 deaf individuals, 15 hearing individuals, and 1 Advanced ALS user to assess word per minute (WPM), real-life accuracy, and user experience. The system achieved 20 WPM with 98.8% accuracy. Moreover, users demonstrated satisfaction with the system, being easy to use. CO-sign Language is a leap forward towards assistive technology, providing a voice to individuals through only their eyes.

Competition history

• ISEF 2025 Systems Software · Entry SOFT052T

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.