Accurate 3D phenotyping of agricultural produce remains challenging due to the trade-off between reconstruction quality and acquisition throughput in existing sensing technologies. While RGB-D cameras enable high-throughput scanning in operational settings like harvesting conveyors, they produce incomplete, low-quality 3D models. Conversely, close-range Structure-from-Motion (SfM) produces high-quality reconstructions but is not suitable for high-throughput field application. This study bridges this gap through 3DPotatoTwin, a paired dataset containing 339 tuber samples across three cultivars collected in Hokkaido, Japan. Our dataset uniquely combines: (1) conveyor-acquired RGB-D point clouds, (2) ground measurement, (3) SfM reconstructions under indoor controlled environment, and (4) aligned model pairs with transformation matrices. The multi-sensory alignment employs an semi-supervised pin-guided pipeline incorporating single-pin extraction and referencing, cross-strip matching, and binary-color-enhanced ICP, achieving 0.59 ​± ​0.11 ​mm registration accuracy. Beyond serving as a benchmark for 3D phenotyping algorithms, the dataset enables training of 3D completion networks to reconstruct high-quality 3D models from partial RGB-D point clouds. Meanwhile, the proposed semi-automated annotation pipeline has the potential to accelerate 3D dataset generation for similar studies. The presented methodology demonstrates broader applicability for multi-sensor data fusion across crop phenotyping applications. The dataset and pipeline source code are publicly available at HuggingFace and GitHub, respectively.

The development of computer vision-based rice phenotyping techniques is crucial for precision field management and accelerated breeding, which facilitate continuously advancing rice production. Among phenotyping tasks, distinguishing image components is a key prerequisite for characterizing plant growth and development at the organ scale, enabling deeper insights into ecophysiological processes. However, owing to the fine structure of rice organs and complex illumination within the canopy, this task remains highly challenging, underscoring the need for a high-quality training dataset. Such datasets are scarce, both because of a lack of large, representative collections of rice field images and because of the time-intensive nature of the annotation. To address this gap, we created the first comprehensive multiclass rice semantic segmentation dataset, RiceSEG. We gathered nearly 50,000 high-resolution, ground-based images from five major rice-growing countries (China, Japan, India, the Philippines, and Tanzania), encompassing more than 6000 genotypes across all growth stages. From these original images, 3078 representative samples were selected and annotated with six classes (background, green vegetation, senescent vegetation, panicle, weeds, and duckweed) to form the RiceSEG dataset. Notably, the subdataset from China spans all major genotypes and rice-growing environments from northeastern to southern regions. Both state-of-the-art convolutional neural networks and transformer-based semantic segmentation models were used as baselines. While these models perform reasonably well in segmenting background and green vegetation, they face difficulties during the reproductive stage, when canopy structures are more complex and when multiple classes are involved. These findings highlight the importance of our dataset for developing specialized segmentation models for rice and other crops. The RiceSEG dataset is publicly available at www.global-rice.com.

The increase in the global population is leading to a doubling of the demand for protein. Soybean (Glycine max), a key contributor to global plant-based protein supplies, requires ongoing yield enhancements to keep pace with increasing demand. Precise, on-plant seed counting and localization may catalyze breeding selection of shoot architectures and seed localization patterns related to superior performance in high planting density and contribute to increased yield. Traditional manual counting and localization methods are labor-intensive and prone to error, necessitating more efficient approaches for yield prediction and seed distribution analysis. To solve this, we propose MSANet: a novel deep learning framework tailored for counting and localization of soybean seeds on mature field-grown soy plants. A multi-scale attention map mechanism was applied to maximize model performance in seed counting and localization in soybean breeding fields. We compared our model with a previous state-of-the-art model using the benchmark dataset and an enlarged dataset, including various soybean genotypes. Our model outperforms previous state-of-the-art methods on all datasets across various soybean genotypes on both counting and localization tasks. Furthermore, our model also performed well on in-canopy 360° video, dramatically increasing data collection efficiency. We also propose a technique that enables previously inaccessible insights into the phenotypic and genetic diversity of single plant vertical seed distribution, which may accelerate the breeding process. To accelerate further research in this domain, we have made our dataset and software publicly available: https://github.com/UTokyo-FieldPhenomics-Lab/MSANet.

Weed is a major biological factor causing declines in crop yield. However, widespread herbicide application and indiscriminate weeding with soil disturbance are of great concern because of their environmental impacts. Site-specific weed management (SSWM) refers to a weed management strategy for digital agriculture that results in low energy loss. Deep learning is crucial for developing SSWM, as it distinguishes crops from weeds and identifies weed species. However, this technique requires substantial annotated data, which necessitates expertise in weed science and agronomy. In this study, we present a channel attention mechanism-driven generative adversarial network (CA-GAN) that can generate realistic synthetic weed data. The performance of the model was evaluated using two datasets: the public segmented Plant Seedling Dataset (sPSD), featuring nine common broadleaf weeds from arable land, and the Institute for Sustainable Agro-ecosystem Services (ISAS) dataset, which includes five common summer weeds in Japan. Consequently, the synthetic dataset generated by the proposed CA-GAN obtained an 82.63% recognition accuracy on the sPSD and 93.46% on the ISAS dataset. The Fréchet inception distance (FID) score test measures the similarity between a synthetic and real dataset, and it has been shown to correlate well with human judgments of the quality of synthetic samples. The synthetic dataset achieved a low FID score (20.95 on the sPSD and 24.31 on the ISAS dataset). Overall, the experimental results demonstrated that the proposed method outperformed previous state-of-the-art GAN models in terms of image quality, diversity, and discriminability, making it a promising approach for synthetic agricultural data generation.

Recently, deep learning-based fruit detection applications have been widely used in the modern fruit industry; however, the training data labeling process remains a time-consuming and labor-intensive process. Auto labeling can provide a convenient and efficient data source for constructing smart orchards based on deep-learning technology. In our previous study, based on a labeled source domain fruit dataset, we used a generative adversarial network and a fruit detection model to achieve auto labeling of unlabeled target domain fruit images. However, since the current method uses one species source domain fruit to label multiple species target domain fruits, there is a problem of the domain gap in both the foreground and the background between the training data (retaining the source domain fruit label information) and the application data (target domain fruit images) of the fruit detection model. Therefore, we propose a domain-adaptive anchor-free fruit detection model, DomAda-FruitDet, and apply it to the previously proposed fruit labeling method to further improve the accuracy. It consists of 2 design aspects: (a) With a foreground domain-adaptive structure based on double prediction layers, an anchor-free method with multiscale detection capability is constructed to generate adaptive bounding boxes that overcome the foreground domain gap; (b) with a background domain-adaptive strategy based on sample allocation, we enhance the ability of the model to extract foreground object features to overcome the background domain gap. As a result, the proposed method can label actual apple, tomato, pitaya, and mango datasets, with an average precision of 90.9%, 90.8%, 88.3%, and 94.0%, respectively. In conclusion, the proposed DomAda-FruitDet effectively addressed the problem of the domain gap and improved effective auto labeling for fruit detection tasks.

On-farm food loss (i.e., grade-out vegetables) is a difficult challenge in sustainable agricultural systems. The simplest method to reduce the number of grade-out vegetables is to monitor and predict the size of all individuals in the vegetable field and determine the optimal harvest date with the smallest grade-out number and highest profit, which is not cost-effective by conventional methods. Here, we developed a full pipeline to accurately estimate and predict every broccoli head size (n > 3,000) automatically and nondestructively using drone remote sensing and image analysis. The individual sizes were fed to the temperature-based growth model and predicted the optimal harvesting date. Two years of field experiments revealed that our pipeline successfully estimated and predicted the head size of all broccolis with high accuracy. We also found that a deviation of only 1 to 2 days from the optimal date can considerably increase grade-out and reduce farmer's profits. This is an unequivocal demonstration of the utility of these approaches to economic crop optimization and minimization of food losses.

Although deep learning-based fruit detection techniques are becoming popular, they require a large number of labeled datasets to support model training. Moreover, the manual labeling process is time-consuming and labor-intensive. We previously implemented a generative adversarial network-based method to reduce labeling costs. However, it does not consider fitness among more species. Methods of selecting the most suitable source domain dataset based on the fruit datasets of the target domain remain to be investigated. Moreover, current automatic labeling technology still requires manual labeling of the source domain dataset and cannot completely eliminate manual processes. Therefore, an improved EasyDAM_V3 model was proposed in this study as an automatic labeling method for additional classes of fruit. This study proposes both an optimal source domain establishment method based on a multidimensional spatial feature model to select the most suitable source domain, and a high-volume dataset construction method based on transparent background fruit image translation by constructing a knowledge graph of orchard scene hierarchy component synthesis rules. The EasyDAM_V3 model can automatically obtain fruit label information from the dataset, thereby eliminating manual labeling. To test the proposed method, pear was used as the selected optimal source domain, followed by orange, apple, and tomato as the target domain datasets. The results showed that the average precision of annotation reached 90.94%, 89.78%, and 90.84% for the target datasets, respectively. The EasyDAM_V3 model can obtain the optimal source domain in automatic labeling tasks, thus eliminating the manual labeling process and reducing associated costs and labor.

Developing automated soybean seed counting tools will help automate yield prediction before harvesting and improving selection efficiency in breeding programs. An integrated approach for counting and localization is ideal for subsequent analysis. The traditional method of object counting is labor-intensive and error-prone and has low localization accuracy. To quantify soybean seed directly rather than sequentially, we propose a P2PNet-Soy method. Several strategies were considered to adjust the architecture and subsequent postprocessing to maximize model performance in seed counting and localization. First, unsupervised clustering was applied to merge closely located overcounts. Second, low-level features were included with high-level features to provide more information. Third, atrous convolution with different kernel sizes was applied to low- and high-level features to extract scale-invariant features to factor in soybean size variation. Fourth, channel and spatial attention effectively separated the foreground and background for easier soybean seed counting and localization. At last, the input image was added to these extracted features to improve model performance. Using 24 soybean accessions as experimental materials, we trained the model on field images of individual soybean plants obtained from one side and tested them on images obtained from the opposite side, with all the above strategies. The superiority of the proposed P2PNet-Soy in soybean seed counting and localization over the original P2PNet was confirmed by a reduction in the value of the mean absolute error, from 105.55 to 12.94. Furthermore, the trained model worked effectively on images obtained directly from the field without background interference.