Deep Learning Control System Using an Air-Gapped Optical Input Stream

Overview

The problem of minimizing risk in industrial plant settings is of high interest for research. Long hours in industrial control rooms induce fatigue, drowsiness, and ease of distraction, posing a risk of failure or catastrophe. Considering the critical importance industrial plants have on the modern-day economy, the minimization of risk in such settings should be prioritized. Recently, deep learning has proven successful on a host of applications, including online recommendation systems, online language translation, image recognition, robotics, and medical diagnosis. This research aimed to develop a deep learning control system that learned to match human performance on a simulated task (a 1990’s Microsoft DOS racing game called Car & Driver) with only optical input, serving as a preliminary prototype for controlling a power plant from only optical feed of dials and gauges. A supervised learning approach was taken; a convolutional neural network (CNN) was trained on 6 hours of pre-recorded human expert play data to return the optimal keystroke given a still frame image of the task. When evaluated offline on a test dataset of still image frames of the same task, the CNN achieved ~90% accuracy. To evaluate the CNN online on the task itself, a novel hardware interface consisting of two connected Arduino boards was developed to facilitate CNN-task interaction. When all components were incorporated into a cohesive control system, the CNN achieved near-human performance on the in-game metrics “Average Speed,” “Top Speed,” and “Lap Time.” This demonstrates that this system can achieve human-performance when applied to an actual power plant scenario.

Competition history

• AJAS 2019 Category not listed

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

• All Robotics and Intelligent Machines projects
• AJAS Robotics and Intelligent Machines projects
• Every AJAS 2019 project