AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Article Link
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Self-Supervised Plant Phenotyping by Combining Domain Adaptation with 3D Plant Model Simulations: Application to Wheat Leaf Counting at Seedling Stage

Yinglun Li1Xiaohai Zhan1Shouyang Liu1( )Hao Lu2( )Ruibo Jiang1Wei Guo3Scott Chapman4Yufeng Ge5Benoit de Solan6,7Yanfeng Ding1Frédéric Baret1,6
Plant Phenomics Research Centre, Academy for Advanced Interdisciplinary Studies, Jiangsu Collaborative Innovation Center for Modern Crop Production, Nanjing Agricultural University, Nanjing, China
Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China
Graduate School of Agricultural and Life Sciences, The University of Tokyo, 1-1-1 Midori-cho, Nishitokyo City, Tokyo, Japan
School of Agriculture and Food Sciences, The University of Queensland, St. Lucia, Queensland 4072, Australia
Department of Biological Systems Engineering, University of Nebraska-Lincoln, Lincoln, Nebraska 68583, United States
INRAE, Avignon Université, UMR EMMAH, UMT CAPTE, 228, route de l’aérodrome - CS 40509, 84914 Avignon Cedex 9, France
ARVALIS Institut du végétal, 3 rue Joseph et Marie Hackin, 75116 Paris, France
Show Author Information

Abstract

The number of leaves at a given time is important to characterize plant growth and development. In this work, we developed a high-throughput method to count the number of leaves by detecting leaf tips in RGB images. The digital plant phenotyping platform was used to simulate a large and diverse dataset of RGB images and corresponding leaf tip labels of wheat plants at seedling stages (150,000 images with over 2 million labels). The realism of the images was then improved using domain adaptation methods before training deep learning models. The results demonstrate the efficiency of the proposed method evaluated on a diverse test dataset, collecting measurements from 5 countries obtained under different environments, growth stages, and lighting conditions with different cameras (450 images with over 2,162 labels). Among the 6 combinations of deep learning models and domain adaptation techniques, the Faster-RCNN model with cycle-consistent generative adversarial network adaptation technique provided the best performance (R2 = 0.94, root mean square error = 8.7). Complementary studies show that it is essential to simulate images with sufficient realism (background, leaf texture, and lighting conditions) before applying domain adaptation techniques. Furthermore, the spatial resolution should be better than 0.6 mm per pixel to identify leaf tips. The method is claimed to be self-supervised since no manual labeling is required for model training. The self-supervised phenotyping approach developed here offers great potential for addressing a wide range of plant phenotyping problems. The trained networks are available at https://github.com/YinglunLi/Wheat-leaf-tip-detection.

References

1

Lóopez-Castañeda C, Richards RA, Farquhar GD. Variation in early vigor between wheat and barley. Crop Sci. 1995;35(2):472–479.

2

Lóopez-Castañeda C, Richards RA, Farquhar GD, Williamson RE. Seed and seedling characteristics contributing to variation in early vigor among temperate cereals. Crop Sci. 1996;36(5):1257–1266.

3

Richards RA, Lukacs Z. Seedling vigour in wheat-sources of variation for genetic and agronomic improvement. Aust J Agric Res. 2002;53(1):41–50.

4

McMaster GS, Wilhelm WW. Accuracy of equations predicting the phyllochron of wheat. Crop Sci. 1995;35(1):30–36.

5

Gauthier M, Barillot R, Andrieu B. Simulating grass phenotypic plasticity as an emergent property of growth zone responses to carbon and nitrogen metabolites. in silico Plants. 2021;3(2):diab034.

6

Ninomiya S, Baret F, Cheng Z-M. Plant Phenomics: Emerging transdisciplinary science. Plant Phenomics. 2019;2019:2765120.

7

Pasala R, Pandey BB. Plant phenomics: High-throughput technology for accelerating genomics. J Biosci. 2020;45(1).

8

McMaster GS. Phytomers, phyllochrons, phenology and temperate cereal development. J Agric Sci. 2005;143(2-3):137–150.

9

Cao W, Moss DN. Temperature effect on leaf emergence and phyllochron in wheat and Barley. Crop Sci. 1989;29:1018–1021.

10

Watt M, Fiorani F, Usadel B, Rascher U, Muller O, Schurr U. Phenotyping: New windows into the plant for breeders. Annu Rev Plant Biol. 2020;71:689–712.

11

Tardieu F, Cabrera-Bosquet L, Pridmore T, Bennett M. Plant Phenomics, from sensors to knowledge. Curr Biol. 2017;27(15):R770–R783.

12

Yang W, Feng H, Zhang X, Zhang J, Doonan JH, Batchelor WD, Xiong L, Yan J. Crop phenomics and high-throughput phenotyping: Past decades, current challenges, and future perspectives. Mol Plant. 2020;13(2):187–214.

13

Zhao C, Zhang Y, Du J, Guo X, Wen W, Gu S, Wang J, Fan J. Crop phenomics: Current status and perspectives. Front Plant Sci. 2019;10:714.

14

Hamuda E, Glavin M, Jones E. A survey of image processing techniques for plant extraction and segmentation in the field. Comput Electron Agric. 2016;125:184–199.

15

Kamilaris A, Prenafeta-Boldú FX. Deep learning in agriculture: A survey. Comput Electron Agric. 2018;147:70–90.

16

Kamilaris A, Prenafeta-Boldú FX. A review of the use of convolutional neural networks in agriculture. J Agric Sci. 2018;156(3):312–322.

17

Chandra AL, Desai SV, Balasubramanian VN, Ninomiya S, Guo W. Active learning with point supervision for cost-effective panicle detection in cereal crops. Plant Methods. 2020;16(1).

18

David E, Serouart M, Smith D, Madec S, Velumani K, Liu SY, Wang X, Pinto F, Shafiee S, Tahir ISA, et al. Global wheat head detection 2021: An improved dataset for benchmarking wheat head detection methods. Plant Phenomics. 2021;2021:9846158.

19
Wang Y, Cao ZG, Bai XD, Yu ZH, Li YA. An automatic detection method to the field wheat based on image processing. Paper presented at:8th Symposium on Multispectral Image Processing and Pattern Recognition (MIPPR) - Automatic Target Recognition and Navigation; 2013 Oct 26–27; Wuhan, China.
20

Lu H, Cao Z. TasselNetV2+: A fast implementation for high-throughput plant counting from high-resolution RGB imagery. Front Plant Sci. 2020;11:541960.

21

Dalagnol R, Wagner FH, Emilio T, Streher AS, Galvao LS, Ometto J, Aragao L. Canopy palm cover across the Brazilian Amazon forests mapped with airborne LiDAR data and deep learning. Remote Sens Ecol Conserv. 2022;8(5):601–614.

22

Vong CN, Conway LS, Feng AJ, Zhou JF, Kitchen NR, Sudduth KA. Corn emergence uniformity estimation and mapping using UAV imagery and deep learning. Comput Electron Agric. 2022;198.

23
Vishal MK, Banerjee B, Saluja R, Raju D, Chinnusamy V, Kumar S, Sahoo RN, Adinarayana J. Leaf counting in rice (Oryza Sativa L.) using object detection: A deep learning approach. Paper presented at: IGARSS 2020. IEEE International Geoscience and Remote Sensing Symposium; 2020 Sep 26–Oct 02; Waikoloa, HI, USA.
24
Muzammul M, Li X. A survey on deep domain adaptation and tiny object detection challenges, techniques and datasets. arXiv. 2021. https://doi.org/10.48550/arXiv.2107.07927.
25
Miao C, Guo A, Thompson AM, Yang J, Ge Y, Schnable JC. Automation of leaf counting in maize and sorghum using deep learning. Plant Phenome J. 2021.
26

Wang DY, Zhang DY, Yang GJ, Xu B, Luo YW, Yang XD. SSRNet: In-field counting wheat ears using multi-stage convolutional neural network. IEEE Trans Geosci Remote Sens. 2021;60.

27

Espejo-Garcia B, Mylonas N, Athanasakos L, Fountas S. Improving weeds identification with a repository of agricultural pre-trained deep neural networks. Comput Electron Agric. 2020;175.

28

Shin HC, Roth HR, Gao MC, Lu L, Xu ZY, Nogues I, Yao JH, Mollura D, Summers RM. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging. 2016;35(5):1285–1298.

29

Zheng YY, Kong JL, Jin XB, Wang XY, Su TL, Zuo M. CropDeep: The crop vision dataset for deep-learning-based classification and detection in precision agriculture. Sensors. 2019;19(5):1058.

30

Baek JY, Yoo YS, Bae SH. Generative adversarial ensemble learning for face forensics. IEEE Access. 2020;8:45421–45431.

31
Ekbatani HK, Pujol O, Segui S. Synthetic data generation for deep learning in counting pedestrians. Paper presented at: ICPRAM 2017. Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods (ICPRAM); 2017 Feb 24–26; Porto, Portugal. p. 318–323.
32

Wang Y, Deng WW, Liu ZY, Wang JS. Deep learning-based vehicle detection with synthetic image data. IET Intell Transp Syst. 2019;13(7):1097–1105.

33

Gibbs JA, Pound M, French AP, Wells DM, Murchie E, Pridmore T. Approaches to three-dimensional reconstruction of plant shoot topology and geometry. Funct Plant Biol. 2017;44(1):62–75.

34

Abbas A, Jain S, Gour M, Vankudothu S. Tomato plant disease detection using transfer learning with C-GAN synthetic images. Comput Electron Agric. 2021;187.

35

Barth R, Ijsselmuiden J, Hemming J, Van Henten EJ. Synthetic bootstrapping of convolutional neural networks for semantic plant part segmentation. Comput Electron Agric. 2019;161:291–304.

36

Zhang K, Wu Q, Chen Y. Detecting soybean leaf disease from synthetic image using multi-feature fusion faster R-CNN. Comput Electron Agric. 2021;183.

37
Tremblay J, Prakash A, Acuna D, Brophy M, Jampani V, Anil C, To T, Cameracci E, Boochoon S, Birchfield S. Training deep networks with synthetic databridging the reality gap by domain randomization. arXiv. 2018. https://doi.org/10.48550/arXiv.1804.06516.
38
Giuffrida MV, Dobrescu A, Doerner P, Tsaftaris SA. Leaf counting without annotations using adversarial unsupervised domain adaptation. Paper presented at:2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW); 2019 Jun 16–17; Long Beach, CA, USA.
39
Tobin J, Fong R, Ray A, Schneider J, Zaremba W, Abbeel P. Domain randomization for transferring deep neural networks from simulation to the real world. Paper presented at:2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); 2017 Sep 24–28; Vancouver, BC, Canada.
40
Hsu H-K, Yao C-H, Tsai Y-H, Hung W-C, Tseng H-Y, Singh M, Yang M-H. Progressive domain adaptation for object detection. arXiv. 2019. https://doi.org/10.48550/arXiv.1910.11319.
41
Li W, Li F, Luo Y, Wang P, Sun J. Deep domain adaptive object detection: A survey. Paer presented at:2020 IEEE Symposium Series on Computational Intelligence (SSCI); 2020 Dec 01–04; Canberra, ACT, Australia.
42

Hartley ZKJ, French AP. Domain adaptation of synthetic images for wheat head detection. Plants. 2021;10(12).

43
Scheck T, Grassi AP, Hirtz G. Unsupervised domain adaptation from synthetic to real images for anchorless object detection. arXiv. 2020. https://doi.org/10.48550/arXiv.2012.08205.
44

Liu S, Martre P, Buis S, Abichou M, Andrieu B, Baret F. Estimation of plant and canopy architectural traits using the digital plant phenotyping platform. Plant Physiol. 2019;181(3):881–890.

45
Fournier C, Andrieu B, Ljutovac S, Saint-Jean S. ADEL-wheat: A 3D architectural model of wheat development. In: Hu B-G, Jaeger M, editors. Plant growth modeling, simulation, visualization and applications. Springer; 2003. p. 54–63.
46

Zhang WL, Chen KZ, Wang JQ, Shi Y, Guo W. Easy domain adaptation method for filling the species gap in deep learning-based fruit detection. Hortic Res. 2021;8(1).

47
Deng WJ, Zheng L, Ye QX, Kang GL, Yang Y, Jiao JB. Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. Paper presented at: IEEE 2018. Proceedings of the 31st IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2018 Jun 18–22; Salt Lake City, UT. p. 994–1003.
48

Pradal C, Dufour-Kowalski S, Boudon F, Fournier C, Godin C. OpenAlea: A visual programming and component-based software platform for plant modelling. Funct Plant Biol. 2008;35(10):751–760.

49

Liu S, Baret F, Allard D, Jin X, Andrieu B, Burger P, Hemmerlé M, Comar A. A method to estimate plant density and plant spacing heterogeneity: Application to wheat crops. Plant Methods. 2017;13(1):38.

50
Zhu JY, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. Paper presented at:16th IEEE International Conference on Computer Vision (ICCV); 2017 Oct 22–29; Venice, Italy. p. 2242–2251.
51
Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. Paper presented at:2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016; Seattle, WA. p. 779–788.
52
Ren SQ, He KM, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. Paper presented at:29th Annual Conference on Neural Information Processing Systems (NIPS); 2015; Montreal, Canada.
53
Song Q, Jiang Z, Wang Y. Rethinking counting and localization in crowds: A purely point-based framework. Paper presented at: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2021.
54
Chen Y, Li W, Sakaridis C, Dai DX, Van Gool L. Domain adaptive faster R-CNN for object detection in the wild. Paper presented at: IEEE 2018. Proceedings of the 31st IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2018 Jun 18–22; Salt Lake City, UT. p. 3339–3348.
55
Velumani K. Deep learning algorithms for high-throughput cereal plant and organ identification. Avignon: Université d'Avignon; 2021.
56
Wang W, Liao S, Zhao F, Kang C, Shao L. DomainMix: Learning generalizable person re-identification without human annotations. arXiv. 2020. https://doi.org/10.48550/arXiv.2011.11953
57

Bourgault M, Webber HA, Chenu K, O'Leary GJ, Gaiser T, Siebert S, Dreccer F, Huth N, Fitzgerald GJ, Tausz M, et al. Early vigour in wheat: Could it lead to more severe terminal drought stress under elevated atmospheric [CO2] and semi-arid conditions? Glob Chang Biol. 2020;26(7):4079–4093.

58

Duan T, Chapman SC, Holland E, Rebetzke GJ, Guo Y, Zheng B. Dynamic quantification of canopy structure to characterize early plant vigour in wheat genotypes. J Exp Bot. 2016;67(15):4523–4534.

59

Baret F, Solan BD, Lopez-Lozano R, Kai M, Weiss M. GAI estimates of row crops from downward looking digital photos taken perpendicular to rows at 57.5° zenith angle: Theoretical considerations based on 3D architecture models and application to wheat crops. Agric For Meteorol. 2010;150(11):1393–1401.

Plant Phenomics
Article number: 0041
Cite this article:
Li Y, Zhan X, Liu S, et al. Self-Supervised Plant Phenotyping by Combining Domain Adaptation with 3D Plant Model Simulations: Application to Wheat Leaf Counting at Seedling Stage. Plant Phenomics, 2023, 5: 0041. https://doi.org/10.34133/plantphenomics.0041

170

Views

23

Crossref

21

Web of Science

21

Scopus

0

CSCD

Altmetrics

Received: 12 January 2023
Accepted: 17 March 2023
Published: 11 April 2023
© 2023 Yinglun Li et al. Exclusive Licensee Nanjing Agricultural University. No claim to original U.S. Government Works.

Distributed under a Creative Commons Attribution License (CC BY 4.0).

Return