Dynamic sparse point voxel transformer for 3D point cloud instance segmentation of dormant apple trees

Three dimensional (3D) instance segmentation is essential for precision characterization of tree architecture at the branch level, which supports both tree fruit crop breeding and the development of robotic systems for orchard management. Existing methods usually use sparse convolution-based operation, which requires a coordinate quantization preprocess to generate sparse tensors, risking the loss of geometric details for fine-grained downstream phenotyping tasks. To overcome this challenge, we developed the dynamic sparse point-voxel transformer (DSPVFormer) model for the efficient and accurate 3D instance segmentation of high-resolution point clouds for dormant apple trees. Our hybrid DSPVFormer architecture maximizes the use of raw point features by dynamically mapping and aggregating the raw point features into the sparse voxel embeddings, capturing strong geometric features that may be discarded during quantization. Evaluations demonstrate that DSPVFormer achieved statistically significant improvements over baseline models on most instance segmentation metrics, which are further translated into more accurate phenotyping evaluation including branch counting and pruning map generation. These advances directly benefit downstream applications in plant phenotyping and robotic pruning for tree crops such as apples. Meanwhile, experimental results on phenotyping tasks suggested that phenotyping-specific evaluation metrics should be prioritized over upstream computer vision performance metrics to realize the full potential of high-throughput phenotyping for real-world applications.