Obstacle Avoiding Robot: A Reinforcement Learning Approach

Overview

Objectives Artificial intelligence is the wave of our future. We wanted to undertake a science project involving AI with real life applicability. Our goal was to program a robot that could detect and avoid obstacles using a Q learning algorithm. This capability could be applied in many helpful ways, such as in delivery systems or self-driving wheelchairs. We hypothesized that when the Q learning algorithm records more states, it learns an obstacle free path that covers a larger area of the enclosed space. Methods For this project, we obtained a LEGO Mindstorms Ev3 robot and attached an ultrasonic sensor to detect obstacles and a gyro sensor to make precise turns. We programmed the robot with a Q learning algorithm to avoid obstacles in an enclosed area by selectively moving forward or rotating to the right. In Q learning there are states, actions possible with each state, and a reward associated with an action. When the reward is high, the robot knows it performed the best action. The robot learns to choose moves that maximize the value of the reward. We first simulated the experiment on a computer using a virtual 11x11 coordinate plane. Results We performed over 300 trials with different parameters to make the Q learning algorithm run successfully. We tested the influence of various parameters on our algorithm and determined that a discount factor less than 0.75 makes the robot learn in fewer than thirty moves and the probability of random moves must be less than 0.1 for the algorithm to learn in a consistent manner. The effectiveness of the algorithm is influenced by how the states are encoded. Our state function remembers the last n moves made by the robot. We demonstrated that the area of the enclosed space covered by the robot increased as n was increased from 2 to 10. Conclusions Our experiments confirmed the hypothesis, that increasing the number of states recorded by the robot enabled it to cover a larger area. Once we programmed the Ev3 robot with our Q learning algorithm, it was remarkable to see it adapt to new obstacles and surroundings as the algorithm learned and taught itself.

Competition history

• CSEF 2019 Mathematical Sciences · Entry J1403

Resources

• CSEF project summary ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.

Browse more like this

• All Mathematics projects
• CSEF Mathematics projects
• Every CSEF 2019 project