“Minimally Calibrated High Performance Communication Interfaces for the Neurologically Impaired”: New Directions Using Language Models

Overview

Amyotrophic lateral sclerosis is a progressive neurodegenerative disease involving motor neurons in the cerebral cortex severely impairing patient’s lives. P300 speller-based brain computer interfaces (BCI) provide an alternate communication medium based on subject’ EEG response to characters on a highlighted flashboard. Alternatively, in paralyzed patients, implanted electrodes capture brain neural activity which are then algorithmically interpreted with machine learning techniques. However, in all these BCI’s, communication speed is severely affected due to poor system design choices. Further, they also require extensive calibration on a subject-by-subject basis. Drawing from diverse areas such as speech/language processing and data compression, this cross-disciplinary research presents multiple high- efficiency state-of-the-art BCI systems along with low-cost therapeutic gaming applications. Redundancy in human communication is exploited with powerful multi-level language models along with a smoothing technique to account for predicting out of vocabulary characters and words. Probabilistically optimized BCI flashboards and scanning including a new Huffman scanning scheme inspired by data compression are designed and analyzed. Using extensive simulations with multi-subject EEG data, speed improvements of almost 40% using word prediction based on partial and prior words are demonstrated in noninvasive BCI systems. Subsequently, a state-of-the-art invasive “typing without hands” BCI system is demonstrated. More specifically, a paralyzed subject’s imagined handwriting is deciphered directly from neural signals from implanted sensors using AI algorithms and converted to text on screen. Further, calibration requirements are minimized by augmenting training with public handwriting datasets and a very high accuracy of almost 87% is obtained.