NEXUS: A Multimodal Device for Deafblind Communication using Hybrid Dynamic Speaker Diarization

Overview

Deafblindness is a combined disability of vision and hearing impairments that create significant challenges to communication and information access. Many deafblind individuals rely on intervenors to access spoken information, but access to these services is highly limited due to shortages, uneven availability, and funding restrictions. To address this gap, NEXUS was developed as a wearable assistive device that enables real-time, independent communication. NEXUS captures surrounding speech, identifies who is speaking, and converts conversations into accessible formats like braille and enlarged text, allowing users to follow group conversations without relying on human intermediaries. Notably, the device uses speaker diarization that combines spatial awareness with voice pattern analysis, making speaker identification robust even in complex, multi-speaker environments. This approach outperforms current state-of-the-art models for real-time, far-field diarization. Overall, NEXUS provides an accessible and scalable solution that enhances communication and supports greater autonomy among deafblind communities.

Awards (5)

• Young Scientist Award
• Challenge Award
• Special Award
• Silver Medal
• Selected for CWSF 2026

Competition history

• CWSF 2026 Digital Technology

Related projects

Closest projects by meaning, across every fair and year in the corpus.