Three dimensional (3D) instance segmentation is essential for precision characterization of tree architecture at the branch level, which supports both tree fruit crop breeding and the development of robotic systems for orchard management. Existing methods usually use sparse convolution-based operation, which requires a coordinate quantization preprocess to generate sparse tensors, risking the loss of geometric details for fine-grained downstream phenotyping tasks. To overcome this challenge, we developed the dynamic sparse point-voxel transformer (DSPVFormer) model for the efficient and accurate 3D instance segmentation of high-resolution point clouds for dormant apple trees. Our hybrid DSPVFormer architecture maximizes the use of raw point features by dynamically mapping and aggregating the raw point features into the sparse voxel embeddings, capturing strong geometric features that may be discarded during quantization. Evaluations demonstrate that DSPVFormer achieved statistically significant improvements over baseline models on most instance segmentation metrics, which are further translated into more accurate phenotyping evaluation including branch counting and pruning map generation. These advances directly benefit downstream applications in plant phenotyping and robotic pruning for tree crops such as apples. Meanwhile, experimental results on phenotyping tasks suggested that phenotyping-specific evaluation metrics should be prioritized over upstream computer vision performance metrics to realize the full potential of high-throughput phenotyping for real-world applications.
- Article type
- Year
Open Access
Research Article
Issue
Open Access
Research Article
Issue
The architecture of apple trees plays a pivotal role in shaping their growth and fruit-bearing potential, forming the foundation for precision apple management. Traditionally, 2D imaging technologies were employed to delineate the architectural traits of apple trees, but their accuracy was hampered by occlusion and perspective ambiguities. This study aimed to surmount these constraints by devising a 3D geometry-based processing pipeline for apple tree structure segmentation and architectural trait characterization, utilizing point clouds collected by a terrestrial laser scanner (TLS). The pipeline consisted of four modules: (a) data preprocessing module, (b) tree instance segmentation module, (c) tree structure segmentation module, and (d) architectural trait extraction module. The developed pipeline was used to analyze 84 trees of two representative apple cultivars, characterizing architectural traits such as tree height, trunk diameter, branch count, branch diameter, and branch angle. Experimental results indicated that the established pipeline attained an R2 of 0.92 and 0.83, and a mean absolute error (MAE) of 6.1 cm and 4.71 mm for tree height and trunk diameter at the tree level, respectively. Additionally, at the branch level, it achieved an R2 of 0.77 and 0.69, and a MAE of 6.86 mm and 7.48° for branch diameter and angle, respectively. The accurate measurement of these architectural traits can enable precision management in high-density apple orchards and bolster phenotyping endeavors in breeding programs. Moreover, bottlenecks of 3D tree characterization in general were comprehensively analyzed to reveal future development.
京公网安备11010802044758号