Real-Time Sign Language Detector Using TensorFlow SSD Model and Python

Overview

Communication is a means of passing on information to other people through verbal, written, or visual means. It constructs our social world and allows us to create bonds with others. People who are deaf or hard of hearing speak in sign language. Despite this, there is a linguistic barrier amongst deaf individuals because those with normal hearing lack knowledge of sign language. Technology-based solutions can be used to overcome this problem. This project builds a real-time sign language recognition system. The gestures for the dataset are collected using a Python program and the Webcam and labeled in the LabelImg package. A label map is created for each gesture and TensorFlow records are created. The TensorFlow object detection pipeline is set up and updated for training and trained on 20000 steps. The trained model is loaded from the latest checkpoint which was created during the training of the model. The real-time detection is then done using the Webcam, OpenCV , and Numpy. The confidence score is recorded and analyzed. An evaluation was done on the trained model to find the recall and mean average precision (mAP) to determine the performance of the model. The highest confidence score was the letter Lwith a 94% confidence score while the lowest confidence score was the letter Owith only a 51% confidence score. The mAP was 0.714 and the recall was 0.747. These results show that the system was able to successfully train and can detect sign language gestures in real time.

Competition history

• JSHS 2023 Category not listed

Resources

• National JSHS abstract book ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.