Tonguage: A Unified Computational Human-Machine Interface Framework Expanding Digital Access for Individuals With Severe Motor Disabilities

Overview

Severe motor disabilities create a critical interaction gap, yet existing Human-Machine Interfaces (HMIs) suffer from signal instability, environmental sensitivity, and reliance on invasive surgery or costly sensors. This research introduces Tonguage, a unified, non-invasive computational framework that transforms tongue motion into a precise and intuitive control language for human–machine interaction. Tonguage is defined across four computational dimensions: Position, Gesture, Movement, and Multi-modality. To ensure reliability, the framework integrates error prevention mechanisms, including multi-face filtering and visual focus detection, reducing unintended activations while preserving user intent. The technical core is a multi-stage computer vision pipeline optimized through iterative refinement. It employs custom algorithms for adaptive tongue segmentation, orientation estimation, tongue-tip localization, and mouth-state detection. This design ensures real-time stability using standard cameras, bypassing traditional HMI limitations. Tonguage has been validated across an integrated interaction ecosystem spanning digital access (typing and mouse), mobility (wheelchair navigation), physical assistance (robotic arms), and entertainment, demonstrating broad real-world coverage with potential for further expansion. Experimental results validate Tonguage's reliability, achieving 92.5% recognition accuracy, 97% target selection accuracy, and under 1% collision error in wheelchair trials. These results demonstrate a robust, low-cost framework ready for real-world deployment, with the potential to restore independence and transform how individuals with motor impairments interact with the world.

Awards (3)

• Fourth Award of $600 $600
• International Council on Systems Engineering - INCOSE: Certificate of Honorable Mention, a 1-year free student membership to the INCOSE, and free virtual admission to the 2022 International Symposium of the INCOSE
• IEEE Foundation: The IEEE Foundation Presidents' Scholarship Award of $10,000 $10,000

Competition history

• ISEF 2026 Software Design · Entry SFTD046

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.