Pre-harvest apple recognition and location using improved YOLOv11

Apple detection and localization can serve as the visual perception in intelligent picking robots. Its detection accuracy, lightweight performance and real-time inference can dominated the efficiency and picking success rate of picking equipment in orchard. However, conventional object detection models are seriously degraded under variable lighting conditions, random leaf shading, dense fruit overlap and complex background interference, particularly in complex natural environments. It is often required for the high recognition and localization accuracy to reduce redundant parameters and excessive computational complexity. Visual algorithms can be expected to deploy into engineering application of picking robots in modern agriculture. In this study, an improved YOLOv11 model was proposed for apple recognition and localization for complex scenarios in orchard. An intelligent robot of apple picking was developed to capture fruit image samples in real unstructured environments. The original images were collected to proportionally divide into training, validation and test set. A high-quality dataset was constructed after image preprocessing. Single sample features of original images were removed to enrich the feature diversity of apple targets under different light intensities and shooting postures. Generalization performance and environmental adaptability were improved after data preprocessing and enhancement, including brightness adjustment, contrast stretching and posture transformation. The entire dataset was standardized and optimized to solve the large parameter volume and high computational complexity of the original YOLOv11 model. Feature extraction was improved for multi-scale apple targets. A lightweight GhostConv module was introduced to replace the standard convolution layers in the Backbone feature extraction network and Neck feature fusion network. Structural optimization was effectively reduced the overall model parameters and computational consumption to capture the fine-grained feature, particularly for small, medium and large apples in complex background interference. Furthermore, spatial attention perception was enhanced in the width and height dimensions. Coordinate Attention (CA) mechanism was also embedded into the Neck network. Different from the conventional single-dimensional channel attention, this mechanism decomposed the channel feature encoding into two independent one-dimensional feature encoding operations along horizontal and vertical spatial dimensions. The network was effectively captured with long-range feature in orchard images. Accurate spatial location information of apple targets was retained to significantly improve the detection robustness for occluded and overlapping fruits. In addition, an optimal bounding box localization was adopted to combine non-maximum suppression and confidence sorting strategy in the detection head. The optimal confidence prediction probability and the spatial coordinate information were intelligently screened for each apple target, leading to the most accurate and reliable bounding box. The optimal target pose coordinates were provided real-time and high-precision visual feedback for the end-effector of the picking robot, indicating the grasping planning and picking accuracy of partially blocked and overlapping apples. The experimental results show that the recognition accuracy of the improved YOLOv11 reached 94.7%, which was 4.1 percentage points higher than the original model (90.6%). The parameter quantity and computational complexity (GFLOPs) of the improved model were 2.19 M and 5.6 G, with a reduction of 0.39 M and 1.2 G, respectively. The loss value decreased by 0.08, while the inference speed (FPS) dropped by 9 frames per second. The slight decrease in FPS was offset by the significant improvement in accuracy, while the reduction of parameters and computational load. When deployed on the self-developed picking robot for field apple picking tests, the overall picking success rate reached 91.3%, indicating the excellent detection performance. Theoretical support can also offer for apple recognition and localization of picking robots.