Accurate segmentation of non-traversable regions at crop row ends is often required for autonomous navigation, row-changing path planning, and safe operation of agricultural machinery in hilly farmland. Row-end scenes can also present more complex visual features in hilly farmland, such as weed-covered bunds, irregular terrain, blurred boundaries, weak texture features, slopes, and complex soil–vegetation. Missed or false segmentation of non-traversable regions can easily occur in the field, especially for the boundary between the row-end area and the adjacent field. In addition, the perception model is required to maintain high segmentation accuracy and lightweight performance in agricultural robots, particularly with edge computing under limited computational resources. In this study, a real-time segmentation was proposed for non-traversable regions at crop row ends in hilly farmland using an improved BiSeNetV2 network. A row-end image dataset was constructed under various regions and scenes using data acquisition platform. 1280 valid images were retained after image screening, duplicate removal, and pixel-level annotation. 784 images were selected for model training, 196 validation images, and 300 testing images. To avoid potential data leakage due to high similarity between adjacent frames in continuous videos, the dataset was divided at the original level, according to video segments as the basic unit. Data augmentation was applied only into the training set, while the validation and testing sets remained as original images. The number of training samples was expanded from 784 to 5100 after augmentation. The diversity of training samples was improved to preserve the independence of evaluation. Two improvements were introduced for the lightweight architecture of BiSeNetV2: (1) An efficient channel attention (ECA) module was embedded to enhance local cross-channel feature interaction for the representation of fine spatial information, such as boundary and texture gradients. (2) A guided multi-scale aggregation (GMSA) branch was designed to fuse semantic and spatial features at 1/2, 1/4, and 1/8 scales. The network was strengthened to learn semantic structure and geometric boundary features from non-traversable regions after GMSA and auxiliary supervision. Thereby segmentation performance was improved under weed coverage, blurred boundaries, and irregular targets. The experimental results showed that the improved BiSeNetV2 achieved an mIoU of 92.58%, mPA of 93.18%, Precision of 93.05%, Recall of 94.25%, and F1 Score of 93.64%, which were 3.62, 3.93, 4.02, 4.23, and 4.12 percentage points higher than those of the baseline BiSeNetV2, respectively. The model size was only 2.5 MB, the computational cost was 9.8 G FLOPs, and the inference speed reached 31.78 frames/s. The better overall performance was achieved in segmentation accuracy, model compactness, computational cost, and inference efficiency, compared with PSPNet, HRNet, SegNet, U-Net, and DeepLabv3+. Visual comparison further indicated that the more complete and continuous segmentation masks were produced under complex row-end scenes. The model was then deployed on a Jetson Orin NX edge device mounted on a collaborative harvesting robot. A real-time perception was developed for non-traversable regions at crop row ends. An average processing speed of 14.68 frames/s was obtained in the online field test. Stable segmentation was maintained for the real-time perception during dynamic operation, especially when the scale and position of non-traversable regions changed in the camera view. The findings can provide the technical support to path planning for the row-switching and turning of agricultural machinery in the hilly farmland.

Wheat harvesting robots can frequently confine the key challenges during actual field operations, such as the missed or incomplete harvesting in the lodging areas, and header blockages. In this study, an online detection was developed to monitor the wheat lodging areas using an improved DeepLabv3+ semantic segmentation. The environmental perception and recognition of harvesting robots were enhanced in the complex and variable farmland, thereby supporting highly efficient and low-loss mechanized harvesting. The dataset was constructed with 4 250 high-resolution images under the lodging scenarios in wheat fields using a ZED2i vehicle-mounted binocular RGB camera. Data collection was conducted in June 2023 in the main winter wheat production zone of Xincao Farm, Yancheng, Jiangsu Province, China. Data augmentation was applied to improve the generalization of the model under diverse field conditions, including random color adjustment, image rotation, and contrast enhancement, simulation in ambient lighting, and motion blur induced by machinery vibration. Each image was annotated at the pixel level using the LabelMe tool, allowing for the precise semantic segmentation training and evaluation. Furthermore, a lightweight and high-performance segmentation model, named MVDC-DeepLabv3+, was developed to fully meet the real-time field deployment. Three architectural enhancements were introduced as follows. Firstly, the original Xception backbone of DeepLabv3+ was replaced by MobileViT, which was a hybrid lightweight architecture that combined convolutional neural networks and Transformer components. The model complexity was significantly reduced to capture the local texture features and global semantic information, in order to recognize the small-scale and irregular lodging areas in cluttered environments. Secondly, the atrous spatial pyramid pooling (ASPP) module was restructured using depthwise separable Atrous convolutions and an optimized dilation rate scheme. Furthermore, the multi-scale contextual features were fused to reduce the computational cost. Thirdly, a convolutional block attention module (CBAM) was embedded in the following shallow feature extraction on the subtle edge and boundary information using channel and spatial attention mechanisms. Experimental results demonstrated that the MVDC-DeepLabv3+ achieved excellent segmentation accuracy on the constructed dataset, with the mean intersection over union (mIoU) of 94.10%, a mean pixel accuracy (mPA) of 97.44%, an F1-score of 96.91%, a precision of 96.40%, and a recall of 97.45%. The performance was improved by 3.70, 2.79, 2.09, 3.05, and 1.00 percentage points, respectively, compared with the original DeepLabv3+. The total model size was reduced to 5.47 MB, which was 75.6% and 97.4% smaller than the MobileNetV2 and Xception backbones, respectively, suitable for deployment on the embedded and resource-constrained platforms. A comparative analysis was carried out with the mainstream semantic segmentation models, such as UNet, SegNet, BiSeNet, and PSPNet. The superior performance was achieved to accurately segment the small lodging areas and then delineate the complex boundaries. Additional tests were also conducted under low-light nighttime conditions and dusty environments. The model’s robustness was further validated after the test. The pixel errors of the lodging detection were below 1.50%, indicating the reliability under challenging visual conditions. Finally, the segmentation model was integrated into an online detection system with PyQt5 and a vehicle-mounted camera interface. The system was then achieved in an average inference speed of 14.23 frames per second, where the relative pixel error was maintained below 1%. The high segmentation accuracy, strong robustness, and real-time performance can offer an effective technical solution to the intelligent wheat harvesting in precision agriculture.

Complex environments featuring variable lighting and backgrounds similar in color to the target objects present challenges for the rapid and accurate detection of tobacco leaves, which is critical for the development of automated tobacco leaf harvesting robots. This study introduces a depth filtering approach to filter out complex regions based on distance information, thereby simplifying the detection task, and proposes a lightweight detection method based on an enhanced YOLOv5s model. Initially, the YOLOv5s backbone network is substituted with a more lightweight MobileNetV2 to reduce the model size. Subsequently, sparse model training combined with the scaling factor distribution rules of batch normalization layers is utilized to identify and eliminate inconsequential neural network channels. Finally, fine-tuning and knowledge distillation techniques are employed to achieve a model accuracy close to the YOLOv5s baseline. Experimental results indicate that the depth filtering method can improve the model’s precision, recall, and mean Average Precision (mAP) by 11.2%, 29.6%, and 17.1%, respectively. The optimized lightweight model achieves a precision of 91.1%, a recall of 90.8%, and an mAP of 91.6%, with a memory footprint of only 1.4MB. It delivers a detection frame rate of 112 fps on desktop computers and 21 fps on mobile devices, which is approximately 3.5 and 4 times faster, respectively, compared to the baseline YOLOv5s tobacco leaf detection model. The precision, recall, and mAP experience a marginal decrease of 3.8, 1.6, and 2.8 percentage points, respectively, while the memory consumption is merely 10% of the pre-optimization amount. In summary, the proposed method enables the accurate detection of tobacco leaves against near-color backgrounds. Simultaneously, it achieves effective lightweighting of the model without compromising its performance, thereby providing technical support for deploying tobacco leaf detection on mobile platforms.