Development of Assistive Technologies for the Visually Impaired Using AI

Overview

According to the World Health Organization, about 43 million people are blind worldwide, emphasizing the importance of assistive technologies to combat their everyday challenges. Our project attempts to partially replace lost sight with wearable, offline-capable devices that enable users to orient themselves indoors and outdoors, navigate safely, avoid hazards, recognize and locate objects, and guide their hand toward a grasp – all in real time. To meet these goals, we created two devices: one that focuses on locating target objects and subsequently guides the user’s hand to a grasp, and one that provides spatial orientation and navigation. The first device combines a Luxonis OAK-D stereo camera running YOLOv8 on-chip for object detection and accurate depth estimation connected to a Radxa Zero single-board computer, local speech recognition using Vosk, speech synthesis using eSpeak, a Bluetooth headset for two-way communication, and four vibrotactile motors for haptic guidance. Tests showed that after 45 minutes of training, a user is capable of repeatedly locating and grasping an object 80 centimeters away in 17 seconds with acoustic feedback and in 21 seconds with haptic feedback. We also found a linear relationship between object distance and grasp time. To our knowledge this device is completely unique as it combines off-line operation with real-time feedback, while being compact and wearable. The second device, now at the design stage, is a backpack unit that incorporates point cloud data from a 3D LiDAR and object detection for indoor/outdoor orientation and navigation. Testing of this device is planned next.

Competition history

• ISEF 2025 Embedded Systems · Entry EBED045T

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.