In the era of social media, the evolution of news has diversified its format, incorporating texts, images, and videos. However, the majority of news recommendation methods focus solely on text data, overlooking the substantial role of news images. This paper introduces a news recommendation method based on CLIP (Contrastive Language-Image Pretraining), NR-CLIP, which is a CLIP-guided algorithm with enhanced multimodal news recommendation via multi-view learning. Specifically, our method employs a CLIP encoder to embed textual and visual information into the same feature space of neural networks, where a unified news textual representation is learned by treating titles, categories, subcategories, and bodies as different views of news. In addition, feature enhancements are applied to fully fuse textual and visual features. Finally, click history and user representations are embedded to predict the click probability of candidate news. Extensive comparisons and in-depth analyses with state-of-the-art news recommendation methods have been presented on V-MIND (Visual Microsoft News Dataset), which provides the visual information on the basis of the classic MIND (Microsoft News Dataset), demonstrating that our proposed method effectively improves the performance of news recommendation.

Training software models for crop disease diagnosis requires large image datasets to achieve high accuracy. We describe a lesion information transfer diffusion model, LesionDiff, for generating image data that augments a real-world disease lesion image dataset. An information preprocessing module identifies lesion areas on leaves, an enhancement module captures diverse visual and semantic lesion features, and a generation module fills missing regions in masked disease images by synthesizing lesion phenotypes. This augmentation increased the average diagnostic accuracy of a test dataset by more than 3%.

Leaves are central indicators of photosynthesis and plant growth status, and their precise monitoring is crucial for smart agriculture. Dense leaf detection, as a foundation for leaf morphology analysis, must address challenges such as occlusion and overlap, directly enabling key tasks including phenotypic trait extraction, disease identification, and yield estimation. Leaves are the most important plant organs, and monitoring leaves is a crucial aspect of crop surveillance. Dense leaf detection plays an important role as a fundamental technology for leaf monitoring. Existing dense leaf detection methods rely on traditional modular detectors and generic feature extraction, lacking designs tailored to real-world dense leaf scenarios. The methods for dense leaf detection generally use traditional modular detectors and general feature extraction techniques, without designing methods specifically for dense leaves in reality. In detail, in complex field scenarios, it still faces challenges like incomplete individual feature extraction due to high leaf overlap and difficult network convergence caused by excessive leaf density. To this end, we propose the Leaf-DETR framework, which effectively addresses these challenges through the Progressive Feature Fusion Pyramid Network (P-FPN) and the Crowded Query Refinement Strategy (CQR). First, we construct the largest dense leaf detection dataset to date, containing 1696 images and 85,375 annotation boxes. Second, P-FPN alleviates the feature confusion problem of overlapping leaves through the multi-stage fusion of features and the Adaptive Feature Aggregation module (AFA), enhancing the interaction between low-level details and high-level semantics. Third, the CQR strategy significantly reduces the matching cost of crowded candidate boxes and improves the network convergence efficiency by culling a crowded query method and introducing a one-to-many matching mechanism. Finally, experimental results show that Leaf-DETR improves mAP@50 by 1% and AR@300 by 1.4% over the baseline model on our self-constructed dataset, outperforming existing detection methods. Furthermore, the model exhibits extremely fast training convergence and demonstrates strong generalization capability on both field-collected monitoring images and other staple crops, fully highlighting its practical value in complex agricultural scenarios. Finally, experiments show that Leaf-DETR outperforms existing detection methods on the self-built dataset and demonstrates good performance generalization in monitoring collected images, as well as for other staple food crops, which verifies its practicality in complex agricultural scenarios.

The code and detailed information are available at http://leafdetr.samlab.cn.

Staple crops are the cornerstone of the food supply but are frequently threatened by plant diseases. Effective disease management, including disease identification and severity assessment, helps to better address these challenges. Currently, methods for disease severity assessment typically rely on calculating the area proportion of disease segmentation regions or using classification networks for severity assessment. However, these methods require large amounts of labeled data and fail to quantify lesion proportions when using classification networks, leading to inaccurate evaluations. To address these issues, we propose an automated framework for disease severity assessment that combines multi-task learning and knowledge-driven large-model segmentation techniques. This framework includes an image information processor, a lesion and leaf segmentation module, and a disease severity assessment module. First, the image information processor utilizes a multi-task learning strategy to analyze input images comprehensively, ensuring a deep understanding of disease characteristics. Second, the lesion and leaf segmentation module employ prompt-driven large-model technology to accurately segment diseased areas and entire leaves, providing detailed visual analysis. Finally, the disease severity assessment module objectively evaluates the severity of the disease based on professional grading standards by calculating lesion area proportions. Additionally, we have developed a comprehensive database of diseased leaf images from major crops, including several task-specific datasets. Experimental results demonstrate that our framework can accurately identify and assess the types and severity of crop diseases, even without extensive labeled data. Codes and data are available at http://dkp-ads.samlab.cn/.

Plant diseases remain a major constraint on crop productivity, requiring timely and accurate diagnostic approaches to secure agricultural yields. While existing automated diagnosis methods primarily rely on image data and achieve notable results, their performance often declines in complex field environments with noise and interference. Multimodal learning provides a promising solution by integrating complementary cues from various data sources. However, the heterogeneity between plant phenotypes and other modalities, such as textual descriptions, poses a significant challenge for effective fusion. To address this issue, we propose PlantIF, a multimodal feature interactive fusion model for plant disease diagnosis based on graph learning. PlantIF comprises three key components: image and text feature extractors, semantic space encoders, and a multimodal feature fusion module. Specifically, we employ pre-trained image and text feature extractors to extract visual and textual features enriched with prior knowledge of plant diseases. Semantic space encoders then map these features into both shared and modality-specific spaces, enabling the capture of cross-modal and unique semantic information. To enhance context understanding, we design a multimodal feature fusion module to process and fuse different modal semantic information, and then extract the spatial dependency between plant phenotype and text semantics through the self-attention graph convolution network. We evaluate PlantIF on a multimodal plant disease dataset with 205,007 images and 410,014 texts, achieving 96.95 % accuracy, 1.49 % higher than existing models. These results demonstrate the potential of multimodal learning in plant disease diagnosis and highlight PlantIF's value in precision agriculture. Codes are available at https://github.com/GZU-SAMLab/PlantIF.

The growth of the global population has increased the demand for fruits and vegetables, while high harvesting labor costs severely constrain industry development. Currently, relevant personnel primarily utilize 2D object detection technology to facilitate automated harvesting, aiming to reduce labor costs. However, 2D detection technology is limited to providing planar information and cannot meet the requirements of scenarios that need 3D spatial data, whereas 3D object detection technology can effectively address these needs, including point cloud-based methods and monocular-based methods. Since point cloud-based object detection methods require expensive equipment, they are not suitable for low-cost agricultural harvesting scenarios. In contrast, monocular 3D object detection methods have the advantage of only requiring a camera and being easy to deploy. However, there is a lack of specialized monocular 3D object detection datasets and algorithms suited for natural scenes in the agricultural field, which limits the application and development of this technology in agricultural automation. To address this, we construct a 3D object detection dataset for wax gourds and propose a network called TPDNet, which aims to capture the 3D information of objects from a single RGB image for fruits and vegetables in fields. Specifically, we construct a depth estimation and enhance module that introduces depth information into the model with the help of depth auxiliary labels, and improves the representation of depth information by utilizing weight information across spatial and channel dimensions. Meanwhile, since depth features and image features are heterogeneous, we design the phenotype aggregation and phenotype intensify module to capture the correspondence between image and depth features, promoting the effective fusion of image and depth information. The experimental results show that our method significantly outperforms others in terms of mAP_3D and mAP_BEV metrics, demonstrating the effectiveness and validity of our proposed method. We open our code and dataset at: http://tpdnet.samlab.cn.

Plant sensors are commonly used in agricultural production, landscaping, and other fields to monitor plant growth and environmental parameters. As an important basic parameter in plant monitoring, leaf inclination angle (LIA) not only influences light absorption and pesticide loss but also contributes to genetic analysis and other plant phenotypic data collection. The measurements of LIA provide a basis for crop research as well as agricultural management, such as water loss, pesticide absorption, and illumination radiation. On the one hand, existing efficient solutions, represented by light detection and ranging (LiDAR), can provide the average leaf angle distribution of a plot. On the other hand, the labor-intensive schemes represented by hand measurements can show high accuracy. However, the existing methods suffer from low automation and weak leaf–plant correlation, limiting the application of individual plant leaf phenotypes. To improve the efficiency of LIA measurement and provide the correlation between leaf and plant, we design an image-phenotype-based noninvasive and efficient optical sensor measurement system, which combines multi-processes implemented via computer vision technologies and RGB images collected by physical sensing devices. Specifically, we utilize object detection to associate leaves with plants and adopt 3-dimensional reconstruction techniques to recover the spatial information of leaves in computational space. Then, we propose a spatial continuity-based segmentation algorithm combined with a graphical operation to implement the extraction of leaf key points. Finally, we seek the connection between the computational space and the actual physical space and put forward a method of leaf transformation to realize the localization and recovery of the LIA in physical space. Overall, our solution is characterized by noninvasiveness, full-process automation, and strong leaf–plant correlation, which enables efficient measurements at low cost. In this study, we validate Auto-LIA for practicality and compare the accuracy with the best solution that is acquired with an expensive and invasive LiDAR device. Our solution demonstrates its competitiveness and usability at a much lower equipment cost, with an accuracy of only 2. 5° less than that of the widely used LiDAR. As an intelligent processing system for plant sensor signals, Auto-LIA provides fully automated measurement of LIA, improving the monitoring of plant physiological information for plant protection. We make our code and data publicly available at http://autolia.samlab.cn.

Wheat is the most widely grown crop in the world, and its yield is closely related to global food security. The number of ears is important for wheat breeding and yield estimation. Therefore, automated wheat ear counting techniques are essential for breeding high-yield varieties and increasing grain yield. However, all existing methods require position-level annotation for training, implying that a large amount of labor is required for annotation, limiting the application and development of deep learning technology in the agricultural field. To address this problem, we propose a count-supervised multiscale perceptive wheat counting network (CSNet, count-supervised network), which aims to achieve accurate counting of wheat ears using quantity information. In particular, in the absence of location information, CSNet adopts MLP-Mixer to construct a multiscale perception module with a global receptive field that implements the learning of small target attention maps between wheat ear features. We conduct comparative experiments on a publicly available global wheat head detection dataset, showing that the proposed count-supervised strategy outperforms existing position-supervised methods in terms of mean absolute error (MAE) and root mean square error (RMSE). This superior performance indicates that the proposed approach has a positive impact on improving ear counts and reducing labeling costs, demonstrating its great potential for agricultural counting tasks. The code is available at http://csnet.samlab.cn.

Plant diseases threaten global food security by reducing crop yield; thus, diagnosing plant diseases is critical to agricultural production. Artificial intelligence technologies gradually replace traditional plant disease diagnosis methods due to their time-consuming, costly, inefficient, and subjective disadvantages. As a mainstream AI method, deep learning has substantially improved plant disease detection and diagnosis for precision agriculture. In the meantime, most of the existing plant disease diagnosis methods usually adopt a pre-trained deep learning model to support diagnosing diseased leaves. However, the commonly used pre-trained models are from the computer vision dataset, not the botany dataset, which barely provides the pre-trained models sufficient domain knowledge about plant disease. Furthermore, this pre-trained way makes the final diagnosis model more difficult to distinguish between different plant diseases and lowers the diagnostic precision. To address this issue, we propose a series of commonly used pre-trained models based on plant disease images to promote the performance of disease diagnosis. In addition, we have experimented with the plant disease pre-trained model on plant disease diagnosis tasks such as plant disease identification, plant disease detection, plant disease segmentation, and other subtasks. The extended experiments prove that the plant disease pre-trained model can achieve higher accuracy than the existing pre-trained model with less training time, thereby supporting the better diagnosis of plant diseases. In addition, our pre-trained models will be open-sourced at https://pd.samlab.cn/ and Zenodo platform https://doi.org/10.5281/zenodo.7856293.