Real-time segmentation of non-traversable regions at crop row ends in hilly farmland using improved BiSeNetV2

Accurate segmentation of non-traversable regions at crop row ends is often required for autonomous navigation, row-changing path planning, and safe operation of agricultural machinery in hilly farmland. Row-end scenes can also present more complex visual features in hilly farmland, such as weed-covered bunds, irregular terrain, blurred boundaries, weak texture features, slopes, and complex soil–vegetation. Missed or false segmentation of non-traversable regions can easily occur in the field, especially for the boundary between the row-end area and the adjacent field. In addition, the perception model is required to maintain high segmentation accuracy and lightweight performance in agricultural robots, particularly with edge computing under limited computational resources. In this study, a real-time segmentation was proposed for non-traversable regions at crop row ends in hilly farmland using an improved BiSeNetV2 network. A row-end image dataset was constructed under various regions and scenes using data acquisition platform. 1280 valid images were retained after image screening, duplicate removal, and pixel-level annotation. 784 images were selected for model training, 196 validation images, and 300 testing images. To avoid potential data leakage due to high similarity between adjacent frames in continuous videos, the dataset was divided at the original level, according to video segments as the basic unit. Data augmentation was applied only into the training set, while the validation and testing sets remained as original images. The number of training samples was expanded from 784 to 5100 after augmentation. The diversity of training samples was improved to preserve the independence of evaluation. Two improvements were introduced for the lightweight architecture of BiSeNetV2: (1) An efficient channel attention (ECA) module was embedded to enhance local cross-channel feature interaction for the representation of fine spatial information, such as boundary and texture gradients. (2) A guided multi-scale aggregation (GMSA) branch was designed to fuse semantic and spatial features at 1/2, 1/4, and 1/8 scales. The network was strengthened to learn semantic structure and geometric boundary features from non-traversable regions after GMSA and auxiliary supervision. Thereby segmentation performance was improved under weed coverage, blurred boundaries, and irregular targets. The experimental results showed that the improved BiSeNetV2 achieved an mIoU of 92.58%, mPA of 93.18%, Precision of 93.05%, Recall of 94.25%, and F1 Score of 93.64%, which were 3.62, 3.93, 4.02, 4.23, and 4.12 percentage points higher than those of the baseline BiSeNetV2, respectively. The model size was only 2.5 MB, the computational cost was 9.8 G FLOPs, and the inference speed reached 31.78 frames/s. The better overall performance was achieved in segmentation accuracy, model compactness, computational cost, and inference efficiency, compared with PSPNet, HRNet, SegNet, U-Net, and DeepLabv3+. Visual comparison further indicated that the more complete and continuous segmentation masks were produced under complex row-end scenes. The model was then deployed on a Jetson Orin NX edge device mounted on a collaborative harvesting robot. A real-time perception was developed for non-traversable regions at crop row ends. An average processing speed of 14.68 frames/s was obtained in the online field test. Stable segmentation was maintained for the real-time perception during dynamic operation, especially when the scale and position of non-traversable regions changed in the camera view. The findings can provide the technical support to path planning for the row-switching and turning of agricultural machinery in the hilly farmland.