fairfolio
Saved Projects

Deep Learning Real-Time Object Detection through Convolutional Neural Networks Using OpenCV for the Visually Impaired

CSEF · 2018 Computational Systems & Analysis

Prachi Bhagavatha, Jasmine Ngo · Silver Creek High School, San Jose, Santa Clara County

Overview

Objectives/Goals The objective of our BerriVest device is to accurately do object detection by alerting the user of frontal obstacles and uneven surfaces and image processing, by specifically naming the obstacle(s) in an arbitrary environment. Methods/Materials Raspberry Pi 2 (Python 3.4.2), OpenCV 3.4, numPy software libraries, external power supply (5V battery), pi NoIR camera (8 megapixels), 2 HC-SR04 ultrasonic sensors, 6 M/F premium jumper wires, 2 1kohm resistors, 2 2kohm resistors, soldering gun, lead, headphones/earbuds, protective case for Raspberry Pi, LiDAR sensor, IR sensor, Adafruit Ultimate GPS Tracker, and Caffe model. Tested fifteen people in arbitrary environments, such as the living room, kitchen, garage, and outside in the neighborhood, where the BerriVest device detected objects to investigate the accuracy of the implementation of the convolutional neural network models we used: ImageNet and GoogleNet. Results With only the ImageNet model, we had each of the 15 test subjects conduct three trials for each of the nine obstacle detections for a bicycle, chair, car, person, dining table, sofa, TV monitor, stop sign, and fire hydrant. Then, after adding the GoogleNet model in parallel with the ImageNet model, we had our test subjects walk around arbitrary environments, such as their living room, kitchen, garage, and outside in the neighborhood. Results depicted that for 11 out of the 15 subjects, there was a high percentage of the actual object accuracy, which proves the consistency of the BerriVest in detecting various obstacles. Conclusions/Discussion We embedded artificial intelligence by experimenting with two pre-trained models to train the neural network into our program, so the BerriVest can efficiently do image processing and name exactly what the obstruction is in front of the user. We first successfully experimented with the MobileNet model and then added, in parallel, the GoogleNet model. With the GoogleNet model, the BerriVest can now detect objects under thousands of more detailed classifications due to its large database. We concluded that implementing supervised learning on Caffe yielded accurate and faster image processing -- reducing the lag time -- and object detection that enhanced our final product.

Summary statement

We created a hands-free device and programmed it using Python to implement multiple optical flow algorithms using convolutional neural networks that detects frontal obstacles and uneven surfaces in live feed, aiding the visually-impaired.

Help received

My partner and I designed the components of the BerriVest device and built it ourselves. We received help and supervision from Mr. Raghavendra Bhagavatha with the coding and understanding of the concepts of machine learning.

Competition history

  • CSEF 2018 Computational Systems & Analysis · Entry S0804

Resources

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

Source: California Science & Engineering Fair public projects

Save projects to your library

Sign in with Google to keep track of projects you find interesting, organized into folders. An account also raises your daily allowance for “Has this been done?”, and lets you create a key for the MCP server with a much higher limit than anonymous use. Browsing stays public.

Continue with Google