Segmentation of vegetation remote sensing images can minimize the interference of background, thus achieving efficient monitoring and analysis for vegetation information. The segmentation of vegetation poses a significant challenge due to the inherently complex environmental conditions. Currently, there is a growing trend of using spectral sensing combined with deep learning for field vegetation segmentation to cope with complex environments. However, two major constraints remain: the high cost of equipment required for field spectral data collection; the availability of field datasets is limited and data annotation is time-consuming and labor-intensive. To address these challenges, we propose a weakly supervised approach for field vegetation segmentation by using spectral reconstruction (SR) techniques as the foundation and drawing on the theory of vegetation index (VI). Specifically, to reduce the cost of data acquisition, we propose SRCNet and SRANet based on convolution and attention structure to reconstruct multispectral images of fields, respectively. Then, borrowing from the VI principle, we aggregate the reconstructed data to establish the connection of spectral bands, obtaining more salient vegetation information. Finally, we employ the adaptation strategy to segment the fused feature map using a weakly supervised method, which does not require manual labeling to obtain a field vegetation segmentation result. Our segmentation method can achieve a Mean Intersection over Union (MIoU) of 0.853 on real field datasets, which outperforms the existing methods. In addition, we have open-sourced a dataset of unmanned aerial vehicle (UAV) RGB-multispectral images, comprising 2358 pairs of samples, to improve the richness of remote sensing agricultural data. The code and data are available at ​https://github.com/GZU-SAMLab/VegSegment_SR, and ​http://sr-seg.samlab.cn/.

Plant diseases are a critical driver of the global food crisis. The integration of advanced artificial intelligence technologies can substantially enhance plant disease diagnostics. However, current methods for early and complex detection remain challenging. Employing multimodal technologies, akin to medical artificial intelligence diagnostics that combine diverse data types, may offer a more effective solution. Presently, the reliance on single-modal data predominates in plant disease research, which limits the scope for early and detailed diagnosis. Consequently, developing text modality generation techniques is essential for overcoming the limitations in plant disease recognition. To this end, we propose a method for aligning plant phenotypes with trait descriptions, which diagnoses text by progressively masking disease images. First, for training and validation, we annotate 5,728 disease phenotype images with expert diagnostic text and provide annotated text and trait labels for 210,000 disease images. Then, we propose a PhenoTrait text description model, which consists of global and heterogeneous feature encoders as well as switching-attention decoders, for accurate context-aware output. Next, to generate a more phenotypically appropriate description, we adopt 3 stages of embedding image features into semantic structures, which generate characterizations that preserve trait features. Finally, our experimental results show that our model outperforms several frontier models in multiple trait descriptions, including the larger models GPT-4 and GPT-4o. Our code and dataset are available at https://plantext.samlab.cn/.

Plant disease diagnosis in time can inhibit the spread of the disease and prevent a large-scale drop in production, which benefits food production. Object detection-based plant disease diagnosis methods have attracted widespread attention due to their accuracy in classifying and locating diseases. However, existing methods are still limited to single crop disease diagnosis. More importantly, the existing model has a large number of parameters, which is not conducive to deploying it to agricultural mobile devices. Nonetheless, reducing the number of model parameters tends to cause a decrease in model accuracy. To solve these problems, we propose a plant disease detection method based on knowledge distillation to achieve a lightweight and efficient diagnosis of multiple diseases across multiple crops. In detail, we design 2 strategies to build 4 different lightweight models as student models: the YOLOR-Light-v1, YOLOR-Light-v2, Mobile-YOLOR-v1, and Mobile-YOLOR-v2 models, and adopt the YOLOR model as the teacher model. We develop a multistage knowledge distillation method to improve lightweight model performance, achieving 60.4% mAP@.5 in the PlantDoc dataset with small model parameters, outperforming existing methods. Overall, the multistage knowledge distillation technique can make the model lighter while maintaining high accuracy. Not only that, the technique can be extended to other tasks, such as image classification and image segmentation, to obtain automated plant disease diagnostic models with a wider range of lightweight applicability in smart agriculture. Our code is available at https://github.com/QDH/MSKD.