WAV: A Novel Subvocalization Interpretation Device

Overview

The purpose of this project was to develop a cost-effective and accurate subvocal interpreter using a novel machine learning algorithm paired with a dual EMG/EEG hardware solution. The solution developed, known as WAV, utilizes an algorithmic analysis of waves collected from electrodes to translate muscle movements and brain waves into speech. The machine learning algorithm developed was trained on two global subvocalization datasets known as the EMG and EEG-UKA databases. A novel machine learning algorithm was developed to translate electrode subvocalization signals from EEG/EMG electrode response to English. The signals are preprocessed and sent to a convolutional neural network architecture to train subject-dependent models to classify the silently spoken sentences. With mel-frequency cepstral coefficient based representations, discrete cosine transformation, and periodogram estimates, these signals could be interpreted and analyzed. Using a microcontroller paired with electrodes placed in strategic locations, data was transmitted to an AWS Cloud server. The algorithm has an accuracy of 96.7% across the entirety of the dataset. Overall, the implications of WAV stretch to the interpretation of vocally disabled patients, tactical military communication, and all around convenient communication. This device also steps into the field of IoT communication, proposing a future of silent speech. With promising accuracy and a lightweight design, WAV has the potential to be normalized as a communication medium. With an average speed of 1.92 seconds per translated word, WAV breaks boundaries in speed. The device costs less than $50, pushing a new form of safe communication into the market.

Competition history

• ISEF 2021 Biomedical Engineering · Entry ENBM052

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.