Corn is a globally important economic crop. Certain trait parameters of corn ears kernels per ear are essential indicators for corn breeding. However, acquiring these parameters faces two challenges: i) manual measurement is labor-intensive and error-prone, and ii) vision-based corn phenotyping machines require fixed image capturing environment and are cost-prohibitive. To address these limitations, we introduce CornPheno, a user-friendly, low-end, smartphone-based approach capable of executing corn ear phenotyping in the wild. CornPheno highlights three corn ear parameters: kernels per ear, rows per ear, and kernels per row. Technically, inspired by crowd localization in computer vision, we first extract kernels per ear based on a Corn data-trained Point quEry Transformer (CornPET). CornPET generates interpretable per-kernel point predictions and supports subsequent row detection. To detect rows, we introduce a novel point-based corn row detection approach, termed unicorn, featured by sqUeezed clusteriNg and bI-direCtional pOint seaRchiNg, to phenotype rows per ear and kernels per row. With adaptive geometric modeling, our approach is robust to partial rows, curved rows, and missing kernels. To promote the use of CornPheno, we have integrated it into OpenPheno, a WeChat-based mini-program, and made it open-access for corn breeders. We hope our approach can provide the community with a user-friendly and cost-effective way to facilitate corn breeding.

We present Depth-Informed Crop Segmentation (DepthCropSeg), an almost unsupervised crop segmentation approach without manual pixel-level annotations. Crop segmentation is a fundamental vision task in agriculture, which benefits a number of downstream applications such as crop growth monitoring and yield estimation. Over the past decade, image-based crop segmentation approaches have shifted from classic color-based paradigms to recent deep learning-based ones. The latter, however, rely heavily on large amounts of data with high-quality manual annotation such that considerable human labor and time are spent. In this work, we leverage Depth Anything V2, a vision foundation model, to produce high-quality pseudo crop masks for training segmentation models. We compile a dataset of 17,199 images from six public plant segmentation sources, generating pseudo masks from depth maps after normalization and thresholding. After a coarse-to-fine manual screening, 1378 images with reliable masks are selected. We compare four semantic segmentation models and enhance the top-performing one with depth-informed two-stage self-training and depth-informed post-processing. To evaluate the feasibility and robustness of DepthCropSeg, we benchmark the segmentation performance on 10 public crop segmentation testing sets and a self-collect dataset covering in-field, laboratory, and unmanned aerial vehicle (UAV) scenarios. Experimental results show that our DepthCropSeg approach can achieve crop segmentation performance comparable to the fully supervised model trained with manually annotated data (86.91 vs. 87.10). For the first time, we demonstrate almost unsupervised, close-to-full-supervision crop segmentation successfully.

The number of leaves at a given time is important to characterize plant growth and development. In this work, we developed a high-throughput method to count the number of leaves by detecting leaf tips in RGB images. The digital plant phenotyping platform was used to simulate a large and diverse dataset of RGB images and corresponding leaf tip labels of wheat plants at seedling stages (150,000 images with over 2 million labels). The realism of the images was then improved using domain adaptation methods before training deep learning models. The results demonstrate the efficiency of the proposed method evaluated on a diverse test dataset, collecting measurements from 5 countries obtained under different environments, growth stages, and lighting conditions with different cameras (450 images with over 2,162 labels). Among the 6 combinations of deep learning models and domain adaptation techniques, the Faster-RCNN model with cycle-consistent generative adversarial network adaptation technique provided the best performance (R² = 0.94, root mean square error = 8.7). Complementary studies show that it is essential to simulate images with sufficient realism (background, leaf texture, and lighting conditions) before applying domain adaptation techniques. Furthermore, the spatial resolution should be better than 0.6 mm per pixel to identify leaf tips. The method is claimed to be self-supervised since no manual labeling is required for model training. The self-supervised phenotyping approach developed here offers great potential for addressing a wide range of plant phenotyping problems. The trained networks are available at https://github.com/YinglunLi/Wheat-leaf-tip-detection.