Feature Selection From Virtual Reality Hand Tracking Data for Robot Gripper Teleoperation Control

Overview

Teleoperation of robots is crucial for maintaining a safe distance for unpredictable or dangerous situations. However, precise control of robotic grippers is necessary to carry out human tasks well. Natural human hand movement has not been napped to robotic grippers that allows for full functionality and embodiment, especially because of the many different forms of robotic hands available today. Additionally, in current robotic grippers, thumb movement is limited to one axis, and is often a large contributor to prosthetic ineffectiveness. I used a virtual reality hand tracking program to collect joint angle data for different human hand movements: 3 axes for each joint on each finger, along with rotational and positional data. I tested twelve different dynamic hand movements that simulated everyday motion, including pinches, grasps, and thumb movements. Furthermore, I collected steady hand position data from the start and end points of each dynamic motion to identify different categories of movement to further understand the differences in dynamic movements, focusing specifically on thumb movement. I trained a Random Forest Classifier on a small percentage of the steady state hand data , and it predicted each position with 100% accuracy each time. My next steps in this project include assessing the accuracy of predictions for dynamic movement data passed into the algorithm trained on the steady state data. Eventually, this will allow me to map these dynamic hand movements onto the joints of a robotic gripper to facilitate natural hand movements and functionality, allowing for prosthetic embodiment and comfort in social settings, while also allowing for more real world applications of robotic grippers.

Competition history

• ISEF 2025 Robotics and Intelligent Machines · Entry ROBO042

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.