Fuzzy Radial Basis Function Least Square Policy Iteration: A Novel Critic-Only Reinforcement Learning Framework

In this paper, a new form of critic-only Reinforcement Learning algorithm for continuous state spaces control problems is proposed. Our approach, called Fuzzy-RBF Least Square Policy Iteration (FRLSPI), tunes the weight parameters of the fuzzy-RBF network (a hybrid model constituted by combining Takagi-Sugeno fuzzy rule inference system with RBF network) online and is acquired through combining Least Squares Policy Iteration (LSPI) with fuzzy-RBF network as a function approximator. In FRLSPI, based on the basis functions defined in the fuzzy-RBF network, a solution has been provided for the challenge of determining the state-action basis functions in LSPI. We also provide positive theoretical results concerning an error bound between the optimal and the approximated Action Value Function (AVF) for FRLSPI. Our proposed method has suitable features such as positive mathematical analysis, learning rate independency and, comparatively good convergence properties. Simulation studies regarding the mountain-car control task and acrobat problem demonstrate the applicability and performance of our learning framework. The overall results indicate that the proposed idea can outperform previously known reinforcement learning algorithms.

Keywords:

Fuzzy reinforcement learning , fuzzy-RBF network , generalization , Least Square Policy Iteration