A Robot Grasping Policy Based on Viewpoint Selection Experience Enhancement Algorithm

To solve the problem of the low success rate of robot vision grasping using fixed environment camera in the scene of cluttered and stacked objects, an eye-hand follow-up camera viewpoint selection policy based on deep reinforcement learning is proposed to improve the accuracy and speed of vision-based grasping. Firstly, a Markov decision process model is constructed for robot active vision-based grasping task, then the problem of viewpoint selection is transformed into a problem of solving the viewpoint value function. A deconvolution network with encoderdecoder structure is used to approximate the viewpoint action value function, and the reinforcement learning is carried out based on the deep Q-network framework. Then, to resolve the problem of sparse reward existing in reinforcement learning, a novel viewpoint experience enhancement algorithm is proposed. The different enhancement methods between the successful and failed grasping process are designed respectively. And the reward region can be expanded from a single point to a circular region for improving the convergence speed of the approximation network. The preliminary experiment is deployed on the simulation platform, and the robot model and the grasping environment are simultaneously built in the simulation platform to implement the offline reinforcement learning. In the process, the proposed viewpoint experience enhancement algorithm can effectively improve the sample utilization rate and speed up the convergence of training. Based on the proposed viewpoint experience enhancement algorithm, the viewpoint action value function approximation network can converge within 2 h. To obtain the results from the verification with application, the proposed viewpoint selection policy is applied to the real-world scenes with robot for grasping experiments. The result shows that the viewpoint optimization based on this policy can effectively promote the accuracy and speed of robot grasping. Compared with the general grasping methods, the proposed viewpoint selection policy needs only one viewpoint selection in real-world robot grasping to find the focus region with high grasping success rate. And the method can also promote the processing efficiency of the best viewpoint selection. The grasping success rate in cluttered scenes is increased by 22.8% against the single-view method, and the mean picks per hour can reach 294 units. As whole, it shows that the proposed policy has the capacity of industrial application.