A Q-learning approach for controlling a robotic goalkeeper during penalty procedure

Penalty kick procedure is one of the most challenging tasks in the robotic soccer games. This study was aimed to solve the problem of penalty kick goalkeeping for RoboCup Small Size soccer robots. Two different approaches were presented and compared to each other. The first approach was a simple hard-coded algorithm and the second one was based on reinforcement learning. In reinforcement learning (RL) in addition to the common reward function, a novel rewarding function was applied based on epoch value. The results extracting from both simulation and real experiments showed that the RL approaches were more successful in solving the problem in comparison with the hard-coded algorithm. Moreover, in the RL approach, the epoch based reward function was more suitable compared to the common reward function to solve the problem. In conclusion, RL based approaches with the epoch based reward function can be a proper choice for the problem compare to the hard-coded methods.

Keywords:

Reinforcement Learning , Q-learning , Small Size Soccer , Robotic goalkeeping , Robocup