Multi-target segmentation, recognition, and localization for an asparagus-harvesting robot based on YOLO11n

Asparagus harvesting can be confined to the efficacy of robotic vision in recent years. Asparagus spears are characterized by a slender morphology in their natural growth state. These tender stems are highly prone to mutual occlusion and overlapping when growing densely in field conditions. Furthermore, the stout mother stems can simultaneously present as the complex background interference. Collectively, it is often required for the high accuracy of the multi-target segmentation and recognition using machine vision. In this study, the lightweight instance segmentation model (YOLO11n-seg) was adopted as a baseline, in order to improve the precise positioning and harvesting performance of the robotic end-effector. Consequently, an optimized model named YOLO11n-SAL was also proposed to specifically tailor the slender, occluded targets with high fidelity. Two modules were introduced to enhance the feature extraction and attention mechanisms in the architectural framework. Firstly, the multi-scale edge enhancement Module (MEEM) was conceptually designed and integrated in order to mitigate the challenge wherein the edge features of the slender asparagus targets were inherently weak and easily lost during convolutional operations. Multi-scale decomposition was performed on the convolutional feature maps. The MEEM effectively extracted and intensified the edge and contour information before feature fusion. The sensitivity to the target boundaries was significantly elevated for the high segmentation precision, thereby enhancing the perceptual capability of the targets with the slender morphological structures. Secondly, the separated and enhancement attention module (SEAM) was introduced to rectify the feature confusion and data incompleteness caused by inter-target occlusion. Attention separation over both channel and spatial dimensions was also utilized to adaptively perceive the local and global features of the occluded asparagus at the varying scales. These features were selectively enhanced and effectively fused to better position the visible subjects of the partially masked targets, while suppressing the background noise and distractor information. The robust performance of the detection and recognition was maintained even within the complex and cluttered environments. A series of experiments was conducted to verify the effectiveness of the improved model. Quantitative evaluation results indicate that the improved YOLO11n-SAL model achieved significant gains over all key performance indicators, compared with the baseline model. In the detection task of the target bounding box, the superior performance was achieved with a detection precision of 94.2%, a recall rate of 83.1%, a mean average precision at IoU threshold 0.5 (mAP_0.5) of 91.2%, and a mean average precision at IoU threshold 0.5-0.95(mAP_0.5-0.95) of 76.2%. In the more granular instance mask segmentation, the model also performed impressively. The segmentation precision, recall, mAP_0.5 and mAP_0.5-0.95 reached 93.4%, 77.9%, 90.7%, and 62.7%, respectively. Furthermore, the heatmap analysis demonstrated that the YOLO11n-SAL model was markedly improved to perceive the asparagus edge features over different scenarios, with the superior multi-target segmentation and recognition under occluded conditions. The high accuracy of the segmentation and recognition was achieved to reduce the interference in the complex multi-scenario environments, compared with the baseline. Finally, a series of asparagus recognition, positioning, harvesting, and grasping trials were carried out using depth cameras and mechanical arms, in order to validate the cognition and position performance in the actual deployment scenarios. The empirical results showed that a positioning success rate of not less than 90% was accompanied by effective harvesting and grasping performance. These findings can provide reliable technical support for the advancement of robotic harvesting in precision agriculture.