Happy Friend or Angry Stranger? A Neural Network Based Smart and Affordable Assistant System for Visually Impaired Peopl

Overview

Objectives/Goals There is a need for a computer vision based assistant system for visually impaired people to help them recognize people, their emotion and maybe to navigate them through the streets. I want to build a Raspberry Pi based camera system running state of the art artificial intelligence face and emotion recognition software written in python. Methods/Materials First, finding efficient open source python library to find face locations and recognize faces. Study and implement feed forward neural network in python to predict emotions. Attach camera, touch screen and speaker to Raspberry Pi, implement the code to take picture when instructed, locate faces in the picture, run face recognition software to identify faces, recognize people, and their emotions, combine all information and output to speaker using text to speech library. System will say [Happy|Sad|Neutral|Angry|Disgust|Fear|Contempt|Surprise][<Name of the person>|Stranger]. Results I was successfully able to implement and integrate the system, which takes the picture, locate faces and identify people and their emotions. It displays the picture with additional information added and speak the information. It recognizes people with 94% accuracy and predicted emotions with 70% accuracy. Conclusions/Discussion My project successfully demonstrated the concept of such an assistant device. Accuracy of emotion recognition can be improved further by the use of CNN. Additional computer vision functions can be added to identify obstacles in the path of people and to navigate them.

Competition history

• CSEF 2018 Mathematical Sciences · Entry J1506

Resources

• CSEF project summary ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

• All Mathematics projects
• CSEF Mathematics projects
• Every CSEF 2018 project