Inverting rice nitrogen content with multimodal data fusion of unmanned aerial vehicle remote sensing and ground observations

Nitrogen is a key nutrient for crop growth. Excessive or insufficient nitrogen affects crop growth, yield, and quality. Additionally, excessive nitrogen fertilizer can cause soil and water pollution. Applying panicle fertilizer during the late jointing stage can promote rice panicle growth. Therefore, accurately and timely monitoring of nitrogen status in rice fields during the late jointing stage and timely optimizing fertilization strategies is crucial for ensuring rice yield and environmental protection. This paper integrates multimodal data from unmanned aerial vehicle (UAV) remote sensing and ground observations to construct inversion models for leaf nitrogen content (LNC) and plant nitrogen content (PNC) of rice at the late jointing stage. The research was conducted at the Shapu Experimental Base of the Agricultural Science Research Institute in Zhaoqing City, Guangdong Province, with two field experiments carried out during the late rice seasons of 2021 and 2022. Each of Experiment 1 (2021) and Experiment 2 (2022) included 30 experimental plots, designed with 5 nitrogen fertilizer gradients, 2 planting densities, and 3 replications. Phosphorus and potassium fertilizers were applied uniformly across all plots. UAVs equipped with multispectral and RGB cameras were used to acquire remote sensing images of rice canopies during the late jointing stage. Vegetation indices (VIs) and texture feature values (TFVs) were extracted from the multispectral images, with TFVs derived using the gray level co-occurrence matrix (GLCM) method. Texture indices (TIs) were then constructed by combining TFVs. RGB images were used to generate digital surface models (DSM) for bare ground (pre-transplant) and rice fields (late jointing stage). These DSMs, combined with ground reference methods, were used to construct crop surface models (CSM) to derive estimated canopy heights (ECH) for each plot. Manually collected data included measured canopy height (MCH) and field nitrogen management data (FN) used as ground observations. For each experimental plot, three representative rice plants were selected as samples. After removing the roots, the leaves and stems were separated and dried at 85 ℃ to a constant weight, which was recorded as the aboveground biomass of the leaves and stems. The true values of leaf nitrogen content and stem nitrogen content were obtained using the Kjeldahl method. Combining these values with the dry weight data, the true values of plant nitrogen content were calculated. The maximal information coefficient (MIC) was used as an evaluation metric for feature assessment and selection. Random forest regression algorithms were employed to construct inversion models for rice LNC and PNC, respectively, using the coefficient of determination (R²), root mean squared error (RMSE), and mean absolute error (MAE) as model evaluation metrics. The analysis and experimental results indicate: TIs constructed using combinations of TFVs significantly enhanced the correlation between texture information and LNC and PNC. When the UAV flight height was 100 m, the Ratio Texture Index constructed using a 9×9 sliding window size in the GLCM method showed the best performance, improving the MIC value by 11.48% compared to the best TFV. For conventional machine-transplanted rice planting density, the correlation between TIs and LNC and PNC was best when the GLCM sliding window size was set to 9×9 or 11×11 at a UAV flight height of 100 m. The ECH derived from the CSM showed a high correlation with the manual MCH in the field. Including canopy height (MCH or ECH) as an input feature in the random forest regression model significantly improved the inversion accuracy of rice nitrogen content. The ECH extracted from the CSM showed high estimation accuracy (R² = 0.77, RMSE = 3.4 cm, MAE = 2.8 cm). The inclusion of canopy height (MCH or ECH) in the model construction improved the inversion accuracy for PNC more significantly compared to LNC. Integrating UAV remote sensing and ground observation multimodal data, the random forest regression algorithm significantly improved the inversion accuracy of rice LNC and PNC at the late jointing stage. Considering both inversion accuracy and operational convenience, it is recommended to use a feature combination of VI+TI+ECH+FN in field production. The results demonstrate that constructing random forest regression models by integrating UAV remote sensing and ground observation multimodal data can accurately detect rice LNC and PNC, providing a scientific basis for rice field management and fertilization decision-making.