Using Computer Vision and Augmented Reality To Utilize Pupil Movement as an Alternative to Traditional Tactile Input Systems

Overview

In recent years, significant advances in the fields of assistive robotics and AI have been made to help those with disabilities experience the world with more autonomy than ever before. However, many of these innovations, such as assistive computer software, care robots, and advanced wheelchairs, while life changing for many, leave behind one demographic: those who lack fine motor control in their hands. Whether they are suffering from paralysis, cerebral palsy, or a variety of other disabilities, people who cannot manipulate their hands with a sufficient degree of precision are unable to profit from the benefits and freedom offered by many assistive technologies. Our project seeks to change this. By using computer vision, augmented reality, and other rapidly developing technologies, our project is able to use the motion and position of the human pupil as input for applications that would traditionally require a physical input such as a joystick or mouse, opening a wide variety of possibilities for those who are unable to take advantage of traditional assistive solutions. While this system has many limitations, the variety of distinct inputs offered by the eye are enough to replicate many assistive input methods that currently exist, and further development will only improve and expand the capability of eye-based input. Our project demonstrates a novel but versatile and effective system that allows people suffering from disabilities as extreme and debilitating as quadriplegia to reap the benefits of decades of progress in assistive robotics that has thus far been inaccessible to them.

Competition history

• ISEF 2022 Robotics and Intelligent Machines · Entry ROBO057T · Atlanta, Georgia, United States

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.