TALK: An American Sign Language Computer Program

Overview

Around 1-1.5 million people became deaf in the United States later in life. How can they be taught American Sign Language (ASL)? To provide a solution to this problem, the researcher created a computer application written in Python that can detect different handshapes, allowing anyone to learn ASL at their own pace. Firstly, the researcher created two programs: One that uses the researcher's dataset to create a trained model and one that uses that trained model to recognize hand shapes and create new datasets. Using the latter program, the researcher created their own dataset, allowing them to create a model for the main program to use. The researcher then tested each sign individually and analyzed the program's ability to recognize each hand shape. Most of the signs that got tested were able to be detected. However, the signs R and U get mixed up frequently, which makes sense; from the computer's perspective, R and U are only slightly different. Other than that, most of the signs work very well. In conclusion, the researcher concluded with a program that could accurately read 93-95% of common signs in ASL. Combining sign and hand detection needs to be more seamless, but in the future, the researcher plans to improve and refine their program to help counteract this.

Awards (1)

• Fourth Award of $500 $500

Competition history

• ISEF 2024 Technology Enhances the Arts · Entry TECA021 · Los Angeles, California, United States

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.