Project SERSI: Smartphone-Enabled Robotic Sign Language Interpreter

Overview

Deaf and hard-of-hearing people often use American Sign Language (ASL) to communicate. Since ASL interpreters are frequently inaccessible, affordable robotic interpreters are a promising alternative or supplemental means of communication. Project SERSI (Smartphone Enabled Robotic Sign-Language Interpreter) is an inexpensive 3D printed servo-actuated robotic hand that can perform most ASL letter signs. A robotic hand was 3D printed and constructed. A second iteration of the robotic hand was designed and constructed with finger mobility improvements. Initially, a Bluetooth-based smartphone app was used to communicate speech and text to the robot. This was replaced by a superior WiFi-enabled app. Software was developed to translate the app’s text output into motor instructions. The motors control the robot’s fingers to create appropriate signs, spelling out phrases letter by letter. The first version of Project SERSI is able to execute two-thirds of the letters in the alphabet and is able to perform half of the letters with high accuracy to a human hand performing the same position. The second version creates two additional letters and improves others. Advantages of Project SERSI over existing robotic solutions include its integration of speech and text translation in a practical interface, high speed of 40 signs per minute, and total consistency replicating signs. Project SERSI is also easy to build and costs less than $150.

Awards (1)

• Third Award of $1,000 $1,000

Competition history

• ISEF 2022 Robotics and Intelligent Machines · Entry ROBO015T · Atlanta, Georgia, United States

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.