Direct Control of Autonomous Mobile Robots via Deep Reinforcement Learning

Overview

Objectives/Goals When designing robot software, some of the most important components are the controllers, which produce actuator commands based on sensor input. While many different control algorithms exist, they all require human work to perform calculations and tuning. Developments in machine learning, however, make it possible to produce low level commands that can challenge human control by training neural networks with an technique called deep reinforcement learning. By shifting the work of learning an optimal control policy from humans to computers, this project brings the possibility of robots that can teach themselves how to control different systems. Furthermore, by exploring different robot control responses, neural networks open up the capability to control complex systems without the limitations of traditional control algorithms. If deep neural networks can be trained to produce low-level motor commands for mobile robots based on sensor inputs, then they can be used as alternatives to traditional control techniques. Methods/Materials In this project, I used the Deep Deterministic Policy Gradient technique to develop an algorithm that can learn to control different types of mobile robots to perform low level tasks such as moving to a specific position. My algorithm employs machine learning optimization techniques such as a replay history and batch normalization to quickly learn an optimal controller for many different control tasks. The algorithm was implemented using the Scala programming language and TensorFlow machine learning library. Results The algorithm developed in this project has stellar results on a wide variety of robot types. The algorithm was tested to learn positional control in both simple environments such as a point-robot in a 1-dimensional space and complex ones such as a differential drive robot in a 2-dimensional space. In all environments, the algorithm was able to learn efficient control policies with less than 500 training episodes. Conclusions/Discussion This project demonstrates the potential for deep reinforcement learning to be applied to low-level robot control tasks. By using deep neural networks to learn nonlinear policies, reinforcement learning has the ability to learn policies that compete with hand-tuned PID and state-space controllers without the need for manual tuning.