Manual picking cannot fully meet the large-scale production in China at present. Robotic picking has been an inevitable trend, particularly with the shortage of labor resources and the rapid development of mechanical automation. It is very necessary to accurately identify and position the apples in the complex environments. Fruit attitude fusion acquisition can be synchronously realized and then classified the apple information. Sometimes, only a small portion of target fruit is covered from the orchard environment, including the leaves, branches, and fruits. There are the small differences among the fruit growth patterns. The convolutional neural network is easy to cause the deep feature map, and then lose the key information of fruit covering parts after multiple convolution operations, resulting in the misrecognition of the fruit growth pattern. At the same time, the detection network can easily identify two apples as one for the overlapping fruits in the natural environment, thus causing the omission of the occluded fruits. In this study, an improved YOLOv7 model was proposed to recognize the apple posture from the growth morphologies. Firstly, the multi-scale feature fusion network was improved to add a 160×160 feature scale layer in the backbone network. The sensitivity of the model was enhanced to identify the tiny local features; Secondly, CBAM attention mechanism was introduced to improve the target region of interest; Finally, the Soft-NMS was used to effectively avoid the high-density overlapping targets being suppressed at one time, thus reducing the missed detection. The experimental results show that the recognition accuracy, recall and average recognition precision of DCS-YOLOv7 were 86.9%, 80.5% and 87.1%, respectively, which were 4.2%, 2.2% and 3.7% higher than the original YOLOv7 model. The average accuracy and speed were greatly improved to fully meet the requirements of picking robot. In addition, an apple gesture recognition was proposed using semantic segmentation and the minimum outer join features. Firstly, comparison tests showed that the Unet model exhibited the best performance in apple image segmentation. The average pixel accuracies were 0.7 and 0.2 percentage points higher than those of DeepLabv3+ and PSPNet. The average intersection and merger ratios were 1.6 and 1.1 percentage points higher as well. The average speed of segmentation also outperformed the rest. As such, the UNet instance segmentation network was chosen as the apple segmentation model. The apple image was segmented using UNet semantic segmentation network. The apple and calyx contour features were obtained by the contour extraction , and then the pose of unobstructed apple was obtained using the apple minimum external feature. The accuracy was 94% to detect the apple pose. The average processing time for each image was 15.7ms, indicating the better acquisition for the pose of apple target. The validity and correctness of recognition model were verified with the high detection accuracy to integrate the recognition of fruit growth pattern and posture. The recognition of fruit posture was considered to classify the growth pattern of apples. The end-effector can rapidly and accurately pick the fruits in a suitable way. The finding can lay the foundation for the non-destructive and efficient picking of apples.

Real-time detection of kiwifruits in natural environments is essential for automated kiwifruit harvesting. In this study, a lightweight convolutional neural network called the YOLOv4-GS algorithm was proposed for kiwifruit detection. The backbone network CSPDarknet-53 of YOLOv4 was replaced with GhostNet to improve accuracy and reduce network computation. To improve the detection accuracy of small targets, the upsampling of feature map fusion was performed for network layers 151 and 154, and the spatial pyramid pooling network was removed to reduce redundant computation. A total of 2766 kiwifruit images from different environments were used as the dataset for training and testing. The experiment results showed that the F1-score, average accuracy, and Intersection over Union (IoU) of YOLOv4-GS were 98.00%, 99.22%, and 88.92%, respectively. The average time taken to detect a 416×416 kiwifruit image was 11.95 ms, and the model’s weight was 28.8 MB. The average detection time of GhostNet was 31.44 ms less than that of CSPDarknet-53. In addition, the model weight of GhostNet was 227.2 MB less than that of CSPDarknet-53. YOLOv4-GS improved the detection accuracy by 8.39% over Faster R-CNN and 8.36% over SSD-300. The detection speed of YOLOv4-GS was 11.3 times and 2.6 times higher than Faster R-CNN and SSD-300, respectively. In the indoor picking experiment and the orchard picking experiment, the average speed of the YOLOv4-GS processing video was 28.4 fps. The recognition accuracy was above 90%. The average time spent for recognition and positioning was 6.09 s, accounting for about 29.03% of the total picking time. The overall results showed that the YOLOv4-GS proposed in this study can be applied for kiwifruit detection in natural environments because it improves the detection speed without compromising detection accuracy.