Improved Machine Perception During Autonomous Driving via Semisupervised Deep Learning and Multisensor Datastream Fusion

Overview

Objectives/Goals The objective of this project was to build a machine perception system capable of fusing sensor data from disparate sources to give an autonomous car situational awareness. Traditional computer vision systems perform poorly with multiple sensor streams and 3D reconstructions because relatively little work has been done in the field of sensory fusion, especially with regards to extracting information from the overlap of feeds. Methods/Materials This project proposed a method of tensor stacking, processing a stream of fused sensory input in conjunction with a new method of generating initial weights in a semi-supervised custom convolutional neural network. This architecture generated semantic identification and tracking tags of objects in the car's surroundings in real-time. Google's TensorFlow library was utilized in the execution of some elements of the cnn. Results The proposed model was tested with the KITTI benchmark dataset and compared to scores generated with individual sensor feed processing in traditional convolutional neural networks, the current standard in autonomous vehicle detection. The accuracy of pedestrian detection saw an improvement from 64.70% to 79.01%, and the oblique cyclist detection accuracy increased from 61.37% to 68.92%. The semi-supervised method of weight initialization proved to support the speed of the algorithm as data size increased. Conclusions/Discussion The results validate the efficacy of such tensor stacking and sensory fusion routes and faster detection of pedestrians and cyclist at the most oblique orientation. Additionally, semi-supervised weight initialization can be implemented in many other applications of neural networks, increasing the speed of each processing layer as the network learns. As a result, the proposed methods show potential in improving the situational awareness of autonomous vehicles.

Competition history

• CSEF 2016 Mathematics & Computer Science · Entry S1419

Resources

• CSEF project summary ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

• All Computational Sciences projects
• CSEF Computational Sciences projects
• Every CSEF 2016 project