An Intelligent Driver Assist System Based on Multimodal Sensor Fusion

Overview

Objectives/Goals This research is aimed at developing a driver-assistance system based on multimodal sensor fusion to help drivers in navigating their cars safely. It includes elderly drivers and those with disabilities including poor reflex, eyesight and hearing. Methods/Materials I have developed a computer-based system for a car, integrated with an array of digital cameras, omnidirectional microphones, accelerometer and gyroscope. In the first phase of this project, I have focused on the problem of changing lanes and merging on-ramp on a highway. By analyzing video data from these sensors using optical flow and morphological techniques, this system computes motion vectors of objects in the scene. Next, it performs iterative feature extraction on this vector space, and applies a camera handoff algorithm to identify cars in motion. It also uses array-processing techniques for audio source-localization and tracking, and analyzes power spectrum of audio signals using spectrograms based on Short-Time Fourier Transforms to identify surrounding vehicles. Data from all the sensors are combined to give the driver a final Go / No-Go signal for lane change. Results I collected 300+ data-samples under different traffic, lighting and road conditions. My overall system correctly provided alerts in 92.8% of these test-cases. Conclusions/Discussion Unlike previous researchers who used expensive lasers, LIDARs or time-of-flight cameras, I have applied cost-effective audio and video sensors to address this lane change problem, just as humans utilize their eyes and ears while driving. My system could also detect if the driver was distracted due to phone conversation or weaving in and out of lanes while driving.

Awards (1)

• Honorable Mention

Competition history

• CSEF 2012 Mathematics & Software · Entry S1420

Resources

• CSEF project summary ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

• All Computational Sciences projects
• CSEF Computational Sciences projects
• Every CSEF 2012 project