AirTouch: A Gesture-Based Cross-Platform App that Enables Touch-Free Input for Human Computer Interaction

Overview

AirTouch is an application aimed at providing hand hover user interaction on a webcam-enabled device using hand and face detection algorithm. I created this cross-platform, sensor-free application using HTML5/JS language to overcome limitations like platform-restriction and dependence on specialized sensors in current solutions for touch-free user interaction. I applied the Viola-Jones algorithm to the live input of the camera. The algorithm works by identifying HAAR-features – small rectangles inside which the pixel values are added, then passing them through Adaboost to create strong HAAR-features by weighing the weak ones. The output is the position of the fist or eyes. This enables the user to interact with the screen by hovering user’s fist over the device rather than touching it. To enhance the accuracy of the detection, I added automatic calibration based on the ambient light reading. In AirTouch, the behavior of a gesture changes based on context. For example, I have used the change in the y-direction (Gesture) for scrolling (Behavior) a 2D page(Context). However, the same gesture when used in a new context – 3D Model, exhibits a new behavior – that is a rotation of the 3D Model. I have applied AirTouch to scrolling and zooming a lab report, to show how researchers can use devices without contaminating samples in laboratories, and a recipe, to show how chefs can refer to it without transferring germs to the food. I have also applied AirTouch to 3D rotation and zooming of VRML/OBJ-3D models. This app has numerous potential real life applications to provide an interactive, hygienic, and convenient user experience. AirTouch can also detect eyes. This may aid quadriplegics to use laptops, tablets and smartphones, without any specialized equipment or sensors.

Competition history

• ISEF 2016 Category not listed · Entry SOFT002

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.