Deep learning for three-dimensional (3D) plant phenomics

Plant phenomics, the comprehensive study of plant phenotypes, has gained prominence as a vital tool for understanding the intricate relationships between genotypes and the environment. Image-based plant phenomics has progressed rapidly, and three-dimensional (3D) phenotyping is a valuable extension of traditional 2D phenomics. However, the increased data dimensionality poses challenges to feature extraction and phenotyping. In recent decades, deep learning has led to remarkable progress in revolutionizing 3D phenotyping. Therefore, this review highlights the importance of using deep learning in 3D plant phenomics. It systematically overviews the capabilities of deep learning for 3D computer vision, covering 3D representation, classification, detection and tracking, semantic segmentation, instance segmentation, and generation. Additionally, deep learning techniques for 3D point preprocessing (e.g., annotation, downsampling, and dataset organization) and various plant phenotyping tasks are discussed. Finally, the challenges and perspectives associated with deep learning in 3D plant phenomics are summarized, including (1) benchmark dataset construction by using synthetic datasets and methods such as generative artificial intelligence and unsupervised or weakly supervised learning; (2) accurate and efficient 3D point cloud analysis by leveraging multitask learning, lightweight models, and self-supervised learning; and (3) deep learning for 3D plant phenomics by exploring interpretability, extensibility, and multimodal data utilization. The exploration of deep learning in 3D plant phenomics is poised to spur breakthroughs in a new dimension of plant science.