AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Article Link
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Point Cloud Completion of Plant Leaves under Occlusion Conditions Based on Deep Learning

Haibo Chen1,2,Shengbo Liu2,3,Congyue Wang2,3Chaofeng Wang2,3Kangye Gong4Yuanhong Li2,3,( )Yubin Lan2,3,5,( )
Experimental Basis and Practical Training Center, South China Agricultural University, Guangzhou, China
National Center for International Collaboration Research on Precision Agricultural Aviation Pesticides Spraying Technology, Guangzhou, China
College of Electronics Engineering (College of Artificial Intelligence), South China Agricultural University, Guangzhou, China
College of Engineering, South China Agricultural University, Guangzhou, China
Department of Biological and Agricultural Engineering, Texas A&M University, College Station, TX, United States

†These authors contributed equally to this work.

Show Author Information

Abstract

The utilization of 3-dimensional point cloud technology for non-invasive measurement of plant phenotypic parameters can furnish important data for plant breeding, agricultural production, and diverse research applications. Nevertheless, the utilization of depth sensors and other tools for capturing plant point clouds often results in missing and incomplete data due to the limitations of 2.5D imaging features and leaf occlusion. This drawback obstructed the accurate extraction of phenotypic parameters. Hence, this study presented a solution for incomplete flowering Chinese Cabbage point clouds using Point Fractal Network-based techniques. The study performed experiments on flowering Chinese Cabbage by constructing a point cloud dataset of their leaves and training the network. The findings demonstrated that our network is stable and robust, as it can effectively complete diverse leaf point cloud morphologies, missing ratios, and multi-missing scenarios. A novel framework is presented for 3D plant reconstruction using a single-view RGB-D (Red, Green, Blue and Depth) image. This method leveraged deep learning to complete localized incomplete leaf point clouds acquired by RGB-D cameras under occlusion conditions. Additionally, the extracted leaf area parameters, based on triangular mesh, were compared with the measured values. The outcomes revealed that prior to the point cloud completion, the R2 value of the flowering Chinese Cabbage’s estimated leaf area (in comparison to the standard reference value) was 0.9162. The root mean square error (RMSE) was 15.88 cm2, and the average relative error was 22.11%. However, post-completion, the estimated value of leaf area witnessed a significant improvement, with an R2 of 0.9637, an RMSE of 6.79 cm2, and average relative error of 8.82%. The accuracy of estimating the phenotypic parameters has been enhanced significantly, enabling efficient retrieval of such parameters. This development offers a fresh perspective for non-destructive identification of plant phenotypes.

References

1

Zhao C. Big data of plant phenomics and its research progress. J Agric Big Data. 2019;1(2):5–14.

2

Li Z, Guo R, Li M, Chen Y, Li G. A review of computer vision technologies for plant phenotyping. Comput Electron Agric. 2020;176:105672.

3

Hu Y, Wang L, Xiang L, Wu Q, Jiang H. Automatic nondestructive growth measurement of leafy vegetables based on kinect. Sensors. 18(3):806.

4

Hui F, Zhu J, Hu P, Meng L, Zhu B, Guo Y, Li B, Ma Y. Image-based dynamic quantification and high-accuracy 3D evaluation of canopy structure of plant populations. Ann Bot. 2018;121(5):1079–1088.

5

Li Y, Liu J, Zhang B, Wang Y, Yao J, Zhang X, Fan B, Li X, Hai Y, Fan X. Three dimensional reconstruction and phenotype measurement of maize seedlings based on multi-view image sequences. Front Plant Sci. 2022;13:974339.

6

Wu S, Wen W, Gou W, Lu X, Zhang W, Zheng C, Xiang Z, Chen L, Guo X. A miniaturized phenotyping platform for individual plants using multi-view stereo 3D reconstruction. Front Plant Sci. 2022;13:897746.

7

Mónica HH, Roderik L, Wolfgang G. Leaf movements of indoor plants monitored by terrestrial LiDAR. Front Plant Sci. 2018;9:189.

8

Li J, Tang L. Developing a low-cost 3D plant morphological traits characterization system. Comput Electron Agric. 2017;143:1–13.

9

Sun G, Wang X. Three-dimensional morphological measurement method for a fruit tree canopy based on Kinect sensor self-calibration. Agronomy. 2019;9(11):741.

10

Li Y, Dai A, Guibas LJ, Nießner M. Database-assisted object retrieval for real-time 3D reconstruction. Comput Graph Forum. 2015;34(2):435–446.

11

Lu J, Lan Y, Wu Z, Liang X, Chang X, Deng X, Wu Z, Tang Y. Optimization of ICP point cloud registration in plants 3D modeling. Trans Chin Soc Agric Eng (Trans CSAE). 2022;38(2):183–191.

12

Chen H, Liu S, Wang L, Wang C, Xiang X, Zhao Y, Lan Y. Automatic 3D reconstruction and verification of an individual crop using Kinect V3. Trans Chin Soc Agric Eng (Trans CSAE). 2022;38(16):215–223.

13

Gené-Mola J, Sanz-Cortiella R, Rosell-Polo JR, Escolà A, Gregorio E. In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions. Comput Electron Agric. 2021;188(4):Article 106343.

14

Lou M, Lu J, Wang L, Jiang H, Zhou M. Growth parameter acquisition and geometric point cloud completion of lettuce. Front Plant Sci. 2022;13:947690.

15
Qi CR, Su H, Kaichun M, Guibas LJ. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Paper presented at: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017 Jul 21–26; Honolulu, HI.
16
Yuan W, Khot T, Held D, Mertz C, Hebert M. PCN: Point completion network. Paper presented at: 2018 International Conference on 3D Vision (3DV); 2018 Sep 5–8; Verona, Italy.
17
Pan L, Chen X, Cai Z, Zhang J, Zhao H, Yi S, Liu Z. Variational relational point completion network. Paper presented at: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2021 Jun 20–25; Nashville, TN.
18
Zhang J, Chen X, Cai Z, Pan L, Zhao H, Yi S. Unsupervised 3d shape completion through gan inversion. ArXiv. 2021. https://doi.org/10.48550/arXiv.2104.13366
19
Muhammad S, Hyunjoo JL, Young MK. RL-GAN-Net: A Reinforcement Learning Agent Controlled GAN Network for Real-Time Point Cloud Shape Completion. Paper presented at: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2019 Jun 15–20; Long Beach, CA.
20
Wang Y, Wu S, Huang H, Cohen-Or D, Sorkine-Hornung O. Patch-based progressive 3D point set upsampling. Paper presented at: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2019 Jun 15–20; Long Beach, CA.
21
Huang Z, Yu Y, Xu J. PF-Net: point fractal network for 3D point cloud completion. Paper presented at: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2020 Jun 13–19; Seattle, WA.
22
Li X, Zhou Z, Xu Z, Jiang H, Zhao H. Plant leaf point cloud completion based on deep learning. Paper presented at: Sixth Symposium on Novel Optoelectronic Detection Technology and Applications; 2019 Dec 3–5; Beijing, China.
23

Xiao H, Huanliang X, Shihang M, Ling C, Jiangbo W, Haoyun W. Point cloud complementation method of Epipremnum aureum leaves under occlusion conditions based on MSF-PPD network. Trans Chin Soc Agric Mach. 2021;52(9):141–148.

24

Zeng A, Jiewei P, Chang L, Dan P, Yanrong J, Xiaobo Z. Plant point cloud completion network based on multi-scale geometry-aware point transformer. Trans Chin Soc Agric Eng (Trans CSAE). 2022;38(4):198–205.

25

Hung H, Zhang H, Zheng Y, Liu Z, Guo P, Li X. A new flowering Chinese cabbage—‘Youlv 501 caixin’. Chin Veg. 2011;1(10):86–88.

26

Rabbani T, Van Den Heuvel F, Vosselmann G. Segmentation of point clouds using smoothness constraint. Int Arch Photogramm Remote Sens Spatial Inf Sci. 2006;36:248–253.

27
Rusu RB, Cousins S. 3D is here: Point Cloud Library (PCL). Paper presented at: 2011 IEEE International Conference on Robotics and Automation; 2011 May 9–13; Shanghai, China.
28
Zhou Q, Park J, Koltun V. Open3D: A modern library for 3D data processing. ArXiv. 2018. https://doi.org/10.48550/arXiv.1801.09847.
29
Qi CR, Yi L, Su H, Guibas LJ. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. ArXiv. 2017. https://doi.org/10.48550/arXiv.1706.02413
30
Aspert N, Santa-Cruz D, Ebrahimi T. MESH: Measuring errors between surfaces using the Hausdorff distance. Paper presented at: Proceedings of the IEEE International Conference on Multimedia and Expo; 2002 Aug 26–29; Lausanne, Switzerland.
31

Callieri M, Ranzuglia G, Dellepiane M, Cignoni P, Scopigno R. Meshlab as a complete tool for the integration of photos and color with high resolution 3d geometry data. Comput Appl Quant Methods Archaeol. 2012; Mar:406–416.

32
You Y, Lou Y, Liu Q, Tai Y, Ma L, Lu C, Wang W. Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution. Paper presented at: Proceedings of the AAAI Conference on Artificial Intelligence; 2020 Feb 7–12; New York, USA.
33

Yi L, Kim VG, Ceylan D, Shen I, Yan M, Su H, Lu C, Huang Q, Sheffer A, Guibas LJ. A scalable active framework for region annotation in 3D shape collections. ACM Trans on Graph. 2016;35(6):2101–21012.

34
Pang X, Li F, Ding N, Zhong X. Upright-net: Learning upright orientation for 3d point cloud. Paper presented at: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2022 Jun 18–24; New Orleans, LA.
Plant Phenomics
Article number: 0117
Cite this article:
Chen H, Liu S, Wang C, et al. Point Cloud Completion of Plant Leaves under Occlusion Conditions Based on Deep Learning. Plant Phenomics, 2023, 5: 0117. https://doi.org/10.34133/plantphenomics.0117

222

Views

13

Crossref

14

Web of Science

15

Scopus

0

CSCD

Altmetrics

Received: 25 April 2023
Accepted: 01 November 2023
Published: 15 November 2023
© 2023 Haibo Chen et al. Exclusive licensee Nanjing Agricultural University. No claim to original U.S. Government Works.

Distributed under a Creative Commons Attribution License 4.0 (CC BY 4.0).

Return