The root is one of the most crucial parts of the plant to affect the overall healthy plant. The leaf can be the corresponding indicator at various stages of growth, even in the ultimate crop. However, some significant challenges still remain in the in-situ detection of soil roots. In this study, an in-situ measurement approach was presented for the tomato root phenotypes using the array ESP32-camera (CAM). Tomato root images were also captured for the in-situ measurement of root phenotypic parameters. A 4×4 array ESP32-CAM combined with 4×4 OV2640 lens module was used for the in-situ automatic wireless acquisition of soil root images. The camera was calibrated and corrected for the aberrations using the Zhang Zhengyou calibration. The image alignment was achieved using scale-invariant feature transform and K-nearest neighbor feature detection matching. The image stitching was obtained for the inter-camera transformation matrix using offline calibration. The semantic segmentation of root images was improved to introduce the efficient multi-head self-attention mechanism. The U-Net model was improved to mix the dice loss and cross-entropy loss. In-situ image acquisition experiments of tomato root systems were also conducted to obtain the images of periodic root changes. The root system was measured manually, i.e., the root length was measured in sections by a soft ruler, the root diameter was obtained by averaging the points, the root area was approximated by the product of length and average diameter, and the root depth and width were obtained by measuring the vertical longitudinal depth and horizontal longitudinal width. Morphological processing and skeleton extraction were used to measure the root length, root area, and root mean diameter using pixel point scanning. By contrast, the root phenotypic parameters (such as the root depth and root width) were measured using the root convexity package. The results showed that the root mean square error (RMSE) of standard shape image stitching iterations of 4×4 camera array was less than 1.11 mm, and the subjective quality scores of the structural similarity index and difference of edge map of global stitched images were above 0.85, the peak signal-to-noise ratio (PSNR) was greater than 32 dB. The improved U-Net model shared the greater improvement in the search for root system completeness and accuracy. Specifically, the precision, recall, intersection over the union, and F1 value of the improved U-Net model on the tomato root segmentation were 86.06%, 78.98%, 71.41%, and 82.37%, respectively, which were 18.97 percentage points, 13.21 percentage points, 21.67 percentage points, and 16.30 percentage points higher than those of the original U-Net model training, respectively. Furthermore, the mean absolute percentage errors of calculated area, length, mean diameter, root depth, and root width of root systems were 7.78%, 5.66%, 8.48%, 2.40%, and 2.23%, respectively, and the coefficients of determination were 0.91, 0.93, 0.84, 0.98, and 0.99, respectively, compared with the manual measurements. In-situ measurement of the root phenotype of tomato plants can also be extended to the other plants or fruit trees.

Apple surface defects have posed a serious threat to the grading efficiency and market value of the fruits. An accurate and rapid detection can be expected for the post-harvest quality of the fruits in recent years. However, the conventional single-view imaging has been widely adopted to only capture the partial surface information of the fruits, due to the irregular spherical shape of the apples. The missed detection of the defects can often occur in the uncollected areas. It is then required to detect the large-scale blind areas for the high-precision quality inspection in the intelligent upgrade of the apple industry. In this study, a three-view imaging method was proposed to detect the apple surface defects. The high redundancy in the multi-view imaging was also reduced to detect the features of the surface defects under the complex backgrounds. A reliable detection was realized using three key technical procedures. Firstly, an imaging dataset was collected for the apple surface defects. Three Intel RealSense D415 depth cameras were deployed to accurately obtain the imaging area. The hardware synchronization function of the cameras was also used to acquire the simultaneous data in real time. The RGB and depth images after data acquisition were converted into three-dimensional point clouds after internal parameter calibration. Then a series of point cloud processing steps were performed on the image data, including the preprocessing, coarse registration, fine registration, downsampling, and surface reconstruction. The 3D reconstruction of the apples was realized to calculate the area of the joint imaging region in the three-view system. Secondly, a region segmentation was proposed, suitable for the three-view apple images using a standard sphere model. The redundant background and overlapping regions were removed from the three-view images. Thirdly, the basic You Only Look Once version 11 (YOLOv11) model was improved to enhance the performance of the detection. Specifically, the C3k module in the Neck part of the original model was replaced with the Non-local Attention Residual Multi-Layer Perceptron (NARM) module. The NARM-YOLOv11 model was constructed to capture the long-range feature dependencies and then identify the small-scale defects. A series of experiments was carried out to verify the effectiveness of the improved model. The results showed that the three-view imaging was realized to fuse the multi-angle surface information of the apples after the precise point cloud registration and reconstruction, in terms of the imaging area performance. There was an increase from 34.6% of the single-view imaging to 74.3% in the average proportion of the apple surface imaging area. Among them, the detection blind area was significantly reduced to basically cover most of the apple surface. In the image segmentation with the standard sphere model, the redundant regions were effectively removed from the three-view images, with an average redundant region removal rate of 20.5%. Furthermore, the average defect detection repetition rate caused by overlapping imaging areas was reduced from 26.0% of the original images to 7.6% after segmentation. The average missed detection rate was controlled at 3.6%. The high redundancy was avoided in the multi-view imaging for the high accuracy of the subsequent defect identification. In the test of the improved NARM-YOLOv11 model, the precision, recall, and mean average precision (mAP) of the NARM-YOLOv11 model increased by 2.7, 2.5, and 3.4 percentage points, respectively, compared with the basic YOLOv11 model. As such, the NARM module was introduced to enhance the feature extraction, especially for the small-scale and low-contrast surface defects of the apples. The frame rate decreased only by 1.7 frames per second, thereby fully meeting the requirements of real-time detection in practical applications. The reason was that the model complexity shared a slight increase due to the addition of the attention mechanism and multi-layer perceptron structure. The overall performance of the detection was achieved for the detection of the apple surface defects. The average precision reached 89.7%, and the average defect recognition rate was 88.1%, indicating the high reliability and practicality of the integrated system. The three-view imaging, image segmentation, and improved NARM-YOLOv11 model were combined to detect the features under complex backgrounds, in order to avoid the large blind area of the single-view imaging and the high redundancy of the three-view imaging. The full-surface defect detection of the spherical fruits can provide a feasible technical scheme for the intelligent upgrading of the post-harvest quality inspection in the apple industry. The finding can also offer solid support to combine the multi-view imaging and deep learning in modern agriculture.

Navigation challenges have been posed on the conventional systems in greenhouses, such as significant light variations, narrow crop row spacing, failure to receive satellite signals, and rigid travel paths. In this study, autonomous navigation was proposed to integrate laser vision with 3D SLAM (simultaneous localization and mapping). Environmental data was collected from 3D LIDAR VLP-16 (Velodyne LiDAR) and an IMU (inertial measurement unit). The LIO-SAM (tightly-coupled lidar inertial odometry via smoothing and mapping) was employed to generate 3D point cloud maps, which were subsequently downscaled to the raster maps. This integration included the data from wheeled odometers and visual odometers using an Extended Kalman filter. Visual odometers provided the positional information to correct and update the state prediction of the mobile platform, functioning as a local localization tool. Additionally, the adaptive Monte Carlo localization data introduced the weights to the ndt-matching (normal distributions transform matching), in order to enhance the accuracy of global localization. Moreover, the autonomous walking system was utilized as the A* algorithm and dynamic window algorithm for the path creation and autonomous navigation. The navigation system of autonomous walking was composed of a remote monitoring platform and an on-board system. Specifically, the remote monitoring platform was responsible for selecting the working mode of the onboard system, then releasing the instruction of target points, and finally displaying the location. The on-board system was the executor of the instructions, in order to receive and execute the task instructions ordered by the monitoring platform. The remote monitoring and onboard systems were combined to realize the autonomous navigation task of the greenhouse transportation robot, according to real-time communication through a wireless network. Experimental results showed that the maximum relative error, the maximum absolute error, and the root mean square error of the greenhouse navigation map constructed by the LIO-SAM algorithm reached 9.9%, 0.081 and 0.063 m, respectively. The improved localization algorithm reduced the horizontal and vertical deviations in the autonomous walking system (less than 0.020 and 0.090 m, respectively). The new system maintained mean values of horizontal deviation, longitudinal deviation, and heading declination below 0.120 m, 0.100 m, and 8.5°, respectively, with standard deviations of less than 0.070 m, 0.140 m, and 6.6°, respectively. The approach had significantly improved the accuracy of positioning and navigation. This navigation scheme can fulfill the need for high-precision localization and navigation in the autonomous walking systems within the greenhouse. The findings can provide theoretical and technical support to autonomous mobile platforms.