Obstacle-Avoidance Motion Planning Method of Manipulator Based on Deep Reinforcement Learning

In recent years, driven by the rapid advancement of industrial automation and intelligent manufacturing, the demand for manipulators to perform tasks in complex environments has significantly increased. Traditional motion planning methods face challenges of computational complexity and lack of adaptability when handling robots with redundant degrees of freedom. To address these issues, this paper proposes a novel obstacle avoidance motion planning method for manipulators based on deep reinforcement learning (DRL), which is especially important when unmanned systems operate in a GPS-free environment. The primary contributions are twofold: (ⅰ) we propose an innovative algorithm, termed PPO-HER, which incorporates hindsight experience replay (HER) into proximal policy optimization (PPO) during training, effectively leveraging failed experiences to enhance policy learning efficiency; (ⅱ) an adaptive action exploration strategy is designed to accelerate training convergence and avoid local optima. Experimental results demonstrate that the PPO-HER algorithm and the adaptive action exploration strategy perform exceptionally well across three environments of varying difficulty, significantly improving network training speed and success rates, shortening planned paths, and enhancing autonomous obstacle avoidance capabilities. These findings provide new ideas for target location and tracking of unmanned systems, and lay the foundation for future learning and data-driven approaches in complex environments.