Current research status and development trends of fruit and vegetable harvesting robots

Fruit and vegetable harvesting robots have been the cutting-edge research hotspots in the important branches of agricultural machinery. The automation level can also be enhanced for labor-saving in the agricultural industry. In this study, a systematic review was presented on the current status and trends of the fruit and vegetable harvesting robots in the world. The technological breakthroughs and engineering practices were analyzed in the evolution of the typical machine from laboratory prototypes to commercial products. The current progress has focused on the top international journals and the industry over the past five years. Key mechanisms and core technologies were summarized for the fruit and vegetable harvesting robots. The technical path of the visual perception was then examined from the two-dimensional imaging to three-dimensional multimodal perception. A comparison was also made on the differences in the rate range, accuracy, and environmental adaptability between passive stereo vision and active structured light, as well as the time-of-flight methods. The matching failure was determined after the texture loss in the unstructured environment of the farmland. In the walking mechanism section, there was the terrain adaptation of the wheeled, tracked, and hybrid drive modes in different scenarios, such as flat land, hills, and greenhouses. The performance was also compared on their obstacle crossing, turning radius, energy consumption efficiency, and the impact of the vibration on the end precision. Among them, the end effector was the core working component. Three mainstream schemes were then classified: negative pressure adsorption, tool shearing, and flexible grasping type. Grasping-picking coordination strategies were evaluated as suitable for the different fruit and vegetable scenarios, such as apples, strawberries, and tomatoes. Each scheme was explored on the damage rate, single fruit operation time, success rate, and crop characteristic constraints. In the core technology, the navigation and positioning evolved from the single GNSS to the multi-sensor fusion SLAM. Some strategies were proposed for the satellite signal occlusion under dense planting environments in orchards, such as the tightly coupled vision-inertial-wheel speed. Furthermore, the target recognition was also summarized from conventional image processing to deep learning. A trade-off optimization was conducted on the Faster R-CNN, Mask R-CNN, and YOLO series models in terms of detection accuracy and real-time performance. The performance was degraded under occlusion, drastic changes in lighting, and interference from similar color tones. In mechanical arm path planning, the swarm intelligence algorithms were evaluated to rapidly expand the random trees, graph search, and artificial potential field in the static obstacle avoidance and dynamic response. Motion redundancy and trajectory smoothing were observed in the multi-arm collaboration and mobile grasping. The current technical bottlenecks were attributed to the five aspects: low flexibility of the end effector leading to mechanical damage rate; low dynamic adaptability of planning, with the lagging responses to dynamic obstacles, such as the swaying branches and leaves; severe degradation of the perception accuracy in the complex environments; lack of the multi-machine collaboration resulting in the repeated or missed operations; all-weather operation with the dust- and water-proof grades and battery life unable to meet the continuous operation requirements of the large fields. Finally, a perception-decision-control system was integrated to break through the existing technical barriers in smart agriculture. Some recommendations were then proposed, including the collaborative design of the software and hardware, multi-modal data fusion, agricultural large models training, and swarm intelligence scheduling, picking robots towards high efficiency, low damage, and generalization. The finding can also provide the core equipment support for the future smart agriculture, particularly from single-machine autonomy to swarm intelligence.