Versatile and Customizable Gesture Recognition-Based Computer Control Using Open Source, Inexpensive Computer Vision Technology
ISEF · 2022 Systems Software
Overview
As technology increasingly impacts daily life, technological interaction is critical to the future of equity and health. In an effort to advance this interaction, the researcher sought to design a versatile and customizable system for computer control, powered by webcam based gesture recognition. Replacing or augmenting traditional mouse control with a flexible computer vision system could offer more versatile and natural control. A non-contact system could also reduce repetitive motion injuries, promoting equity for those with hand injuries or disabilities. Python was selected as the language for the system due to available gesture recognition and computer control libraries. Open Computer Vision (OpenCV) was a critical library for the system’s base recognition, predicting hand position via landmarks. Open source python libraries enabled creation of a user interface for customization of control features. The final prototype proved promising but unrefined. The system succeeded in detecting hand position changes and executing computer control within 10 milliseconds. This was the case for most functions under normal lighting conditions, and proved functional for real-time computer control. However, some sensitive landmark positions and rigid gesture recognition caused misinterpretation of gestures. The prototype also required specific hand movements which harmed its ergonomic potential, failing to remove hand stresses. This project revealed that camera-powered gesture recognition with current technology is promising but requires refinement to resolve recognition errors and improve natural interaction. The findings support future research and improvement, anticipating future benefits for versatility, customization, ergonomics, and equity of computer control.
Competition history
- ISEF 2022
Resources
Related projects
ISEF · 2016
Eye Connect: Using Computer Vision to Create Low-Cost Assistive Technology
ISEF · 2017
3D Computer Vision: Human/Machine Interaction via Kinect
ISEF · 2021
Utilizing Computer Vision and Machine Learning Systems to Develop an Algorithm Helping Physically Disabled People to Use Computer
ISEF · 2024
Mouseless Mouse: Enhancing Digital Interaction for Individuals With Limited Hand Mobility via Amplify and Smooth Extracted Data From Head Movement Using Real-Time Facial Tracking and Deep Learning
Closest projects by meaning, across every fair and year in the corpus.
Source: Regeneron International Science and Engineering Fair