Machine-Learning-Based Identification of Cognitive Engagement States In EEG Data Driven By Visual Stimulation

Overview

Monitoring cognitive engagement and comprehension may prevent accidents and save lives, especially with drivers of autonomous vehicles, air traffic controllers, and in other attention-critical situations. However, there is currently no passive monitoring capable of determining a person’s cognitive engagement with the world around them. This project assesses the possibility of detecting the cognitive engagement state (comprehension) of a person based on the frequency following response of the brain to motion. Electroencephalography (EEG) data of deaf singers and hearing non-signers watching two sets of videos was used. Set one showed a signer producing sentences (normal human motion, comprehensible to native signers); set two contained the same videos played in reverse (abnormal motion, incomprehensible to both groups). The peak correlation between the subject’s EEG signal and the video’s motion (amplitude, timing, brain-region) was passed into a variety of machine learning (ML) algorithms which were trained to identify the subject (signer versus non-signer) and their cognitive state (understanding/normal versus abnormal). Feature analysis was used to optimize performance and to characterize the neurophysiological response. Identification of signers observing sign language (2-state classification) was 100% accurate. 4-state classification (2 groups x 2 sets) yielded above-chance performance (77%). Based on feature importance, both populations engage in predictive processing of human motion, but signer’s alpha frequency engages earlier with executive and language regions of the brain than with non-signers. These findings demonstrate that ML analysis using only the EEG-based frequency-following-response to visual stimuli can be effectively used to identify higher cognitive engagement and to monitor operator understanding.

Competition history

• JSHS 2023 Category not listed

Resources

• National JSHS abstract book ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.