Synchronous detection method for litchi fruits and picking points of a litchi-picking robot based on improved YOLOv8-pose

In the unstructured litchi orchard, precise identification and localization of litchi fruits and picking points are crucial for litchi-picking robots. Most studies adopt multi-step methods to detect fruit and locate picking points, which are slow and struggle to cope with complex environments. This study proposes a YOLOv8-iGR model based on YOLOv8n-pose improvement, integrating end-to-end network for both object detection and key point detection. Specifically, this study considers the influence of auxiliary points on picking point and designs four litchi key point strategies. Secondly, the architecture named iSaE is proposed, which combines the capabilities of CNN and attention mechanism. Subsequently, C2f is replaced by Generalized Efficient Layer Aggregation Network (GELAN) to reduce model redundancy and improve detection accuracy. Finally, based on RFAConv, RFAPoseHead is designed to address the issue of parameter sharing in large convolutional kernels, thereby more effectively extracting feature information. Experimental results demonstrate that YOLOv8-iGR achieves an AP of 95.7% in litchi fruit detection, and the Euclidean distance error of picking points is less than 8 pixels across different scenes, meeting the requirements of litchi picking. Additionally, the GFLOPs of the model are reduced by 10.71%. The accuracy of the model’s localization for picking points was tested through field picking experiments. In conclusion, YOLOv8-iGR exhibits outstanding detection performance along with lower model complexity, making it more feasible for implementation on robots. This will provide technical support for the vision system of the litchi-picking robot.