The increasingly widespread application of autonomous field mowers in agriculture has significantly heightened the demand for precise and reliable tree trunk detection technologies, particularly in complex and challenging operational environments. To overcome the inherent limitations of single-sensor systems, such as the sparse point cloud resolution in Light Detection and Ranging (LiDAR), photometric sensitivity in camera-based methods, and persistent occlusion interference, this study proposes a multi-sensor fusion framework that integrates data from multi-line LiDAR and a monocular camera for robust tree trunk detection. First, a spatio-temporal calibration framework was developed to ensure accurate alignment of multi-source data. Subsequently, the PointPillars network was utilized for efficient extraction of 3D point cloud features, while an improved You Only Look Once Version 8 Nano (YOLOv8n) model was integrated to enable precise 2D image feature extraction. Additionally, the Complete Intersection over Union (CIoU) fusion strategy was adopted to enable effective cross-modal bounding box matching. Experimental results demonstrate that the proposed fusion approach achieves average positioning errors of 0.0619 m in the horizontal direction and 0.0583 m in the vertical direction, along with a tree trunk detection accuracy of 93.68%. This method effectively resolves the false detection issues typically encountered with traditional point cloud clustering algorithms in complex environments, while also mitigating performance degradation in vision-based detection under complex texture conditions. The proposed framework presents an innovative approach to environment-aware perception for autonomous mowing operations.
- Article type
- Year
- Co-author
Open Access
Issue
High-precision localization technology serves as the crucial foundation in enabling the autonomous navigation operations of intelligent agricultural robots. However, the traditional global navigation satellite system (GNSS) localization method faces numerous limitations, such as tree shadow, electromagnetic interference, and other factors in the agricultural environment brings challenges to the accuracy and reliability of localization technology. To address the deficiencies and achieve precise localization of agricultural robots independent of GNSS, a localization method was proposed based on the fusion of three-dimensional light detection and ranging (LiDAR) data and inertial measurement unit (IMU) information to enhance localization accuracy and reliability.
LiDAR was used to obtain point cloud data in the agricultural environment and realize self-localization via point cloud matching. By integrating real-time motion parameter measurements from the IMU with LiDAR data, a high-precision localization solution for agricultural robots was achieved through a specific fusion algorithm. Firstly, the LiDAR-obtained point cloud data was preprocessed and the depth map was used to save the data. This approach could reduce the dimensionality of the original LiDAR point cloud, and eliminate the disorder of the original LiDAR point cloud arrangement, facilitating traversal and clustering through graph search. Given the presence of numerous distinct crops like trees in the agricultural environment, an angle-based clustering method was adopted. Specific angle-based clustering criteria were set to group the point cloud data, leading to the segmentation of different clusters of points, and obvious crops in the agricultural environment was effectively perceived. Furthermore, to improve the accuracy and stability of positioning, an improved three-dimensional normal distribution transform (3D-NDT) localization algorithm was proposed. This algorithm operated by matching the LiDAR-scanned point cloud data in real time with the pre-existing down sampled point cloud map to achieve real-time localization. Considering that direct down sampling of LiDAR point clouds in the agricultural environment could result in the loss of crucial environmental data, a point cloud clustering operation was used in place of down sampling operation, thereby improving matching accuracy and positioning precision. Secondly, to address potential constraints and shortcomings of using a single sensor for robot localization, a multi-sensor information fusion strategy was deployed to improve the localization accuracy. Specifically, the extended Kalman filter algorithm (EKF) was chosen to fuse the localization data from LiDAR point cloud and the IMU odometer information. The IMU provided essential motion parameters such as acceleration and angular velocity of the agricultural robot, and by combining with the LiDAR-derived localization information, the localization of the agricultural robot could be more accurately estimated. This fusion approach maximized the advantages of different sensors, compensated for their individual limitations, and improved the overall localization accuracy of the agricultural robot.
A series of experimental results in the Gazebo simulation environment of the robot operating system (ROS) and real operation scenarios showed that the fusion localization method proposed had significant advantages. In the simulation environment, the average localization errors of the proposed multi-sensor data fusion localization method were 1.7 and 1.8 cm, respectively, while in the experimental scenario, these errors were 3.3 and 3.3 cm, respectively, which were significantly better than the traditional 3D-NDT localization algorithm. These findings showed that the localization method proposed in this study could achieve high-precision localization in the complex agricultural environment, and provide reliable localization assistance for the autonomous functioning of agricultural robots.
The proposed localization method based on the fusion of LiDAR data and IMU information provided a novel localization solution for the autonomous operation of agricultural robots in areas with limited GNSS reception. Through the comprehensive utilization of multi-sensor information and adopting advanced data processing and fusion algorithms, the localization accuracy of agricultural robots could be significantly improved, which could provide a new reference for the intelligence and automation of agricultural production.
京公网安备11010802044758号