Humanoid Robots in Rehabilitation: A Comparative Study of SAC, TD۳, and A۲C Algorithms

In this research, a comprehensive analysis of three advanced reinforcement learning algorithms – Soft Actor-Critic (SAC), Twin Delayed DDPG (TD۳), and Advantage Actor-Critic (A۲C) – specifically applied to humanoid robots in the context of rehabilitation, using the Gym library's Humanoid model. The primary purpose was to identify the most effective algorithm for facilitating complex rehabilitative tasks such as standing and walking, which are crucial functionalities in rehabilitation robotics. Our investigation revealed that while the TD۳ algorithm showed potential, it was prone to converging to local minima, a significant limitation in the nuanced realm of rehabilitation. Similarly, the A۲C algorithm struggled with convergence issues in our specific use case, suggesting its limited applicability in the precise and adaptive control required for rehabilitation robots. This led to an in-depth exploration of the SAC algorithm. The SAC algorithm stood out for its exceptional performance in the rehabilitation scenario, attributed to its robustness and adaptability in continuous action spaces – a critical feature for the complex movements required in therapeutic settings. This algorithm demonstrated superior ability in handling the intricacies of bipedal locomotion, a key aspect in robotic rehabilitation. This study makes a substantial contribution to the field of rehabilitation robotics. It provides valuable insights into the application of advanced reinforcement learning algorithms in enhancing the functionality and effectiveness of humanoid rehabilitation robots. The findings from this research not only highlight the importance of choosing the right algorithm for specific rehabilitation tasks but also open avenues for future advancements in the development of more efficient and responsive rehabilitation robots.

Keywords:

Advantage Actor-Critic (A۲C) , Humanoid Robot Control , Rehabilitation Robotics , Reinforcement Learning Algorithms , Soft Actor-Critic (SAC) , Twin Delayed DDPG (TD۳)