Camellia oleifera fruits are required for the high harvesting efficiency during the non-flowering and fruiting period. In this study, the key points of lateral branches was located to reduce the tree damage for the efficient harvesting. Firstly, the dataset was constructed to label the lateral branches in the segments. The labelling was then divided into five categories, including cof (camellia oleifera fruits), br_con (coniferous branch), danger (vibration danger area, the area of forked stems), br_pro (vibration-prioritised picking area), and br_m (the area of transitional branches, in which harvesting was not considered). Segmented labelling of side branches was reconstructed from the side branches, particularly for the subsequent priority detection of key points. UNet network was selected as the segmentation network. The CloFormer attention mechanism was then added into the UNet, named Clo-UNet. As such, the high-precision segmentation was realized on the lateral branches. The field experiment was carried out to verify the detection. The mIoU values of the continuous fruiting branch br_con, the vibration hazardous region danger, and the vibration priority region br_pro reached 85.36%, 86.37% and 81.29%, respectively, while the mPA values were 94.97%, 96.17% and 89.48%, respectively. The bifurcation points of lateral branch were avoided to reduce the end of the branch during the actual harvesting. The actuator and robotic arm were reduced the possibility of damage at the branch bifurcation point. The Clo-UNet was further designed for the vibration picking priority of key points, named as the Clo-UNet-Point. The center of mass was selected in the br_pro region as the vibration picking point, in order to ensure the maximum transmission of the actuator's excitation force during harvesting. If the br_pro was absent, the detection was then determined to remove the br_con region. The reason was that the br_con was the continuous fruiting branch, thus bending into a bow shape under the gravity of the camellia oleifera fruits. The convexity was firstly used to determine the shape of the region, if the convexity was greater than 500. A bow shape was set as the rectangle for the bow shaped region, whereas, the straight line was fitted to the region. After that, the pixel points of the region to the straight line were calculated as the harvesting point. The minimum distance from the pixel points of the region to the straight line were taken as the harvesting point. The harvesting point of the rectangular region was calculated in the same way as br_pro. Finally, all the images were observed on the validation set. The key point was detected in the case of segmented branches. The key point was 100% on the branches. The pickup point recognition was summarized to determine the harvesting priority of YOLOv8n-Point, compared with the Clo-UNet-Point. The whole detection of YOLOv8n-Point was taken about 1.94 s in the traditional image processing, such as the skeleton extraction. By contrast, the Clo-UNet-Point was 0.15 s. The harvesting point was beyond the branch, while the key point was detected at the end of the branch unsuitable for the transmission of force during vibration. The finding can lay the theoretical foundation on the automatic vibration harvesting for the non-flowering and fruiting contemporaneous classes of camellia oleifera fruits.

High-precision recognition is limited to multiple occlusions and small sizes of camellia oleifera fruits in natural environments. In this study, COF-YOLOv5s was proposed to accurately and rapidly locate the camellia oleifera fruits using YOLOv5s. Three aspects were used to improve the model. Specifically, a small target detection layer was first added. The lightweight module Faster Block from FasterNet was then embedded into the C3 module. Biformer attention mechanism was finally added. Experimental results show that only Faster-C3 to YOLOv5s increased the mAP, R and P by 1.8, 5.5 and 1.6 percentage points, respectively, compared with the original YOLOv5s,inference time decreased by 0.5 percentage points and 3.6 ms, indicating that the Faster-C3 was balanced the detection accuracy and speed. The small target detection layer significantly improved the mAP, R, and P, which increased by 1.8, 4.2, and 3.2 percentage points, respectively, compared with the original one. There was an increase in the inference time of 2.3 ms. After that, Faster-C3 was incorporated into the network with Biformer. The small target detection layer reduced both the inference time and parameter count. FasterBlock embedded into C3 mitigated the increase in the parameter count and memory access, due to the addition of the attention mechanism and small target detection layer. After all three were incorporated into the network, the mAP, R, and P increased by 4.4, 7.5, and 5.0 percentage points, respectively, compared with the original network. The highest increase was observed in R. Therefore, the network reduced the miss rate, and the inference time was only 1.8 ms longer than that of the original ones, indicating the effectiveness of this model. The improved network was achieved in P, R, and mAP of 97.6%, 97.8%, and 99.1%, respectively, on the test set, which were 5.0, 7.5, and 4.4 percentage points higher than before. The inference time was 10.3 ms, and the model weight file was only 16.1 MB. Finally, the improved model was deployed on the Jetson Xavier NX, and then combined with the ZED mini camera. The identification and positioning experiments were carried out on the camellia oleifera fruits. The recall rate of COF-YOLOv5s was 91.7% in indoor experiments, which was 47.3 percentage points higher than before. The recall rate of green camellia oleifera fruits was 68.8% in outdoor experiments. Furthermore, the recall rate was 64.3% for the small red camellia oleifera fruits under weak light conditions. The feasible theoretical support was provided to upgrade the agricultural equipment, in order to realize the intelligence and scale of the crop industry. Both indoor and outdoor experiments showed that there was some deviation in the detection on the test set. The main reason was that the camera was close to the target with a distance of about 0.2-0.4 m, resulting in the captured images being close-up shots. By contrast, the camera was about 1.2 m away from the fruit in the indoor/outdoor harvesting, which was equivalent to a long-shot picture. Identification errors then resulted in lower recall rates in indoor/outdoor experiments, compared with the test set.

Real-time detection of kiwifruits in natural environments is essential for automated kiwifruit harvesting. In this study, a lightweight convolutional neural network called the YOLOv4-GS algorithm was proposed for kiwifruit detection. The backbone network CSPDarknet-53 of YOLOv4 was replaced with GhostNet to improve accuracy and reduce network computation. To improve the detection accuracy of small targets, the upsampling of feature map fusion was performed for network layers 151 and 154, and the spatial pyramid pooling network was removed to reduce redundant computation. A total of 2766 kiwifruit images from different environments were used as the dataset for training and testing. The experiment results showed that the F1-score, average accuracy, and Intersection over Union (IoU) of YOLOv4-GS were 98.00%, 99.22%, and 88.92%, respectively. The average time taken to detect a 416×416 kiwifruit image was 11.95 ms, and the model’s weight was 28.8 MB. The average detection time of GhostNet was 31.44 ms less than that of CSPDarknet-53. In addition, the model weight of GhostNet was 227.2 MB less than that of CSPDarknet-53. YOLOv4-GS improved the detection accuracy by 8.39% over Faster R-CNN and 8.36% over SSD-300. The detection speed of YOLOv4-GS was 11.3 times and 2.6 times higher than Faster R-CNN and SSD-300, respectively. In the indoor picking experiment and the orchard picking experiment, the average speed of the YOLOv4-GS processing video was 28.4 fps. The recognition accuracy was above 90%. The average time spent for recognition and positioning was 6.09 s, accounting for about 29.03% of the total picking time. The overall results showed that the YOLOv4-GS proposed in this study can be applied for kiwifruit detection in natural environments because it improves the detection speed without compromising detection accuracy.