Deep learning for sorghum yield forecasting using uncrewed aerial systems and lab-derived imagery

The AI revolution, advanced Graphics Processing Units (GPUs), and open-source platforms have enabled Machine Learning (ML) and Deep Learning (DL) algorithms to rapidly and accurately extract phenotypic features from imagery. Such advancements have led to phenotypic digitization and made rapid yield forecasting possible. Yield predictions are critical to assess the merit of genotypes to propel cultivar development. This trial followed a three-replicated Randomized Complete Block Design (RCBD) with 36 diverse sorghum genotypes in 2023 at Ashland Bottoms, Kansas. The field images were captured 6 m above using a DJI M300 drone at 90° nadir and 45° oblique angles. This research trained YOLO and Faster R-CNN (Detectron2) models to harness yield attributes from UAS field and lab images. The YOLO models outperformed the Faster R-CNN in detecting sorghum panicles, achieving a mean average precision at 50 % IoU (mAP@0.50) scores of 0.92–0.98, compared to 0.61–0.89 for Faster R-CNN. Panicle detection from field imagery showed a linear correlation of 0.86 with ground truth field panicle counts. Lab imagery analyses measured panicle area, seed counts, and seed area with correlation coefficients of 0.79, 0.94, and 0.25 with respective ground truth observations. Support Vector Regression (SVR), Random Forest Regression (RFR), and Decision Tree Regression (DTR) were used to predict yield with correlation coefficients of 0.74, 0.71, and 0.78, respectively, and SHapley Additive exPlanation (SHAP) analysis revealed panicle seed count as the primary driver of yield prediction. We observed YOLO models are well-suited for extracting yield-predictive features from pertinent images. Such features can then be incorporated into ML regression models to predict yield per se performance with greater accuracy. The GitHub link is provided in the Data availability section.