fairfolio
Saved Projects

Feel The Ball: Convert Ball Motion to Touch for Vision/Hearing - Impaired Sport Audiences

ISEF · 2024 Systems Software

Zhang, Amy · Lakewood High School

Overview

My research aims to develop an innovative system that makes sporting events more accessible and engaging to sports enthusiasts with vision or hearing impairments. Unlike existing captioning-based (i.e., see the real-time commentary) or audio-based (i.e., hear the ball) assistance methods, my system enables users to feel the spatial, dynamic ball movements. The architecture of this system includes extracting real-time ball motion information from broadcast sports videos using computer vision and artificial intelligence technologies and transforming it into dynamic touch feeling through a force feedback device (so-called haptics techniques). Specifically, a novel Swin Transformer combined with DeconvNet and Long Short-Term Memory techniques are developed to achieve reliable tracking of small, blurred, fast-moving balls from broadcast videos. Then a novel psychology-guided haptic technique is developed to enable the user to feel the spatiotemporal ball motion through a force-feedback joystick. This system helps visual or hearing-impaired spectators have a more immersive accessibility to live sports and thus experience the excitement accompanying a sports match. Additionally, this system will heighten people’s sense of reality and spatial awareness and potentially change the way people interact with the real world, advancing future immersive technologies in various fields requiring sensory augmentation. Besides personal use, the fundamental technology innovation in reliable tracking of small and fast-moving objects and intuitive haptics of real- world object motion has broader impact on other applications such as video surveillance and airport safety.

Competition history

Resources

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

Source: Regeneron International Science and Engineering Fair

Save projects to your library

Sign in with Google to keep track of projects you find interesting, organized into folders. An account also raises your daily allowance for “Has this been done?”, and lets you create a key for the MCP server with a much higher limit than anonymous use. Browsing stays public.

Continue with Google