GIVS: A Novel, Generative Artificial Intelligence Vision System for the Visually Impaired
ISEF · 2024 Embedded Systems Fourth Award
Overview
Vision is one of the most crucial senses. However, visual impairment - the inability to properly correct eyesight - has grown to affect more than 253 million people worldwide, with lifelong consequences. Despite the growing need to address visual impairment, existing electronic travel aids (ETAs) are limited in versatility and highly cost-prohibitive. The researcher devised a solution to this problem by developing a novel Generative Artificial Intelligence Vision System (GIVS). GIVS employs a multimodal generative AI model to provide vivid vision-to-text-to-speech descriptions of the user's surroundings, mimicking the natural language communication of a human assistant. The model architecture consists of an image encoder, querying transformer, linear projection layer, and large language model, which the user interacts with through voice commands on a smartphone. The researcher designed a wearable eyeglass frame with 2 cameras, an inertial measurement unit (IMU), LiDAR, and GPS sensors to provide accurate, real-time data, at a total cost of less than $90. Voice recognition was further developed to activate optical character recognition (OCR) text recognition and GPS/IMU-based navigation. Across multiple testing environments, GIVS exhibited a state-of-the-art mean detection rate and bounding box IoU for everyday objects. Furthermore, the generative AI achieved an A-rated BLEU-score, the OCR was accurate, and the navigation system showed minimal localization error in testing. The prototype was intuitive, user-friendly, and met all criteria and constraints, demonstrating the great potential generative AI offers in assisting the visually impaired.
Awards (2)
- Fourth Award of $500 $500
- International Council on Systems Engineering - INCOSE: Certificate of Honorable Mention, a 1-year free student membership to the INCOSE, and free virtual admission to the 2022 International Symposium of the INCOSE
Competition history
- ISEF 2024
Resources
Related projects
ISEF · 2023
IVY: Intelligent Vision System for the Visually Impaired
ISEF · 2022
IVY - Intelligent Vision System for the Visually Impaired
ISEF · 2024
ViABL: Visual Assistant for the Blind With VLMs
CWSF · 2026
VisionIQ - A Novel Multi-Model AI Approach for Visual Impairment
Closest projects by meaning, across every fair and year in the corpus.
Source: Regeneron International Science and Engineering Fair