AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Plant Phenomics Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Process-Based Crop Modeling for High Applicability with Attention Mechanism and Multitask Decoders

Taewon Moon1,2, Dongpil Kim3, Sungmin Kwon1, Jung Eek Son1,2( )
Department of Agriculture, Forestry and Bioresources, Seoul National University, Seoul 08826, Republic of Korea
Research Institute of Agriculture and Life Sciences, Seoul National University, Seoul 08826, Republic of Korea
Protected Horticulture Research Institute, National Institute of Horticultural & Herbal Science, Rural Development Administration, Haman 52054, Republic of Korea
Show Author Information

Abstract

Crop models have been developed for wide research purposes and scales, but they have low compatibility due to the diversity of current modeling studies. Improving model adaptability can lead to model integration. Since deep neural networks have no conventional modeling parameters, diverse input and output combinations are possible depending on model training. Despite these advantages, no process-based crop model has been tested in full deep neural network complexes. The objective of this study was to develop a process-based deep learning model for hydroponic sweet peppers. Attention mechanism and multitask learning were selected to process distinct growth factors from the environment sequence. The algorithms were modified to be suitable for the regression task of growth simulation. Cultivations were conducted twice a year for 2 years in greenhouses. The developed crop model, DeepCrop, recorded the highest modeling efficiency (= 0.76) and the lowest normalized mean squared error (= 0.18) compared to accessible crop models in the evaluation with unseen data. The t-distributed stochastic neighbor embedding distribution and the attention weights supported that DeepCrop could be analyzed in terms of cognitive ability. With the high adaptability of DeepCrop, the developed model can replace the existing crop models as a versatile tool that would reveal entangled agricultural systems with analysis of complicated information.

References

1

Gijzen H, Heuvelink E, Challa H, Marcelis LFM, Dayan E, Cohen S, Fuchs M. HORTISIM: A model for greenhouse crops and greenhouse climate. Acta Hortic. 1998;456:441–450.
Crossref Google Scholar
2

Jones JW, Antle JM, Basso B, Boote KJ, Conant RT, Foster I, Godfray HCJ, Herrero M, Howitt RE, Janssen S, et al. Brief history of agricultural systems modeling. Agric Syst. 2017;155:240–254.
Crossref Google Scholar
3

Peng B, Guan K, Tang J, Ainsworth EA, Asseng S, Bernacchi CJ, Cooper M, Delucia EH, Elliott JW, Ewert F, et al. Towards a multiscale crop modelling framework for climate change adaptation assessment. Nat Plants. 2020;6:338–348.
Crossref Google Scholar
4

Katzin D, van Henten EJ, van Mourik S. Process-based greenhouse climate models: Genealogy, current status, and future directions. Agric Syst. 2022;198:103388.
Crossref Google Scholar
5

Newbery F, Qi A, Fitt BD. Modelling impacts of climate change on arable crop diseases: Progress, challenges and applications. Curr Opin Plant Biol. 2016;32:101–109.
Crossref Google Scholar
6

Chenu K, Porter JR, Martre P, Basso B, Chapman SC, Ewert F, Bindi M, Asseng S. Contribution of crop models to adaptation in wheat. Trends Plant Sci. 2017;22:472–490.
Crossref Google Scholar
7

Wang E, Martre P, Zhao Z, Ewert F, Maiorano A, Rötter RP, Kimball BA, Ottman MJ, Wall GW, White JW, et al. The uncertainty of crop yield projections is reduced by improved temperature response functions. Nat Plants. 2017;3:Article 17102.
Crossref Google Scholar
8

Holzworth D, Huth NI, Fainges J, Brown H, Zurcher E, Cichota R, Verrall S, Herrmann NI, Zheng B, Snow V. APSIM next generation: Overcoming challenges in modernising a farming systems model. Environ Model Softw. 2018;103:43–51.
Crossref Google Scholar
9

de Wit A, Boogaard H, Fumagalli D, Janssen S, Knapen R, van Kraalingen D, Supit I, van der Wijngaart R, van Diepen K. 25 years of the WOFOST cropping systems model. Agric Syst. 2019;168:154–167.
Crossref Google Scholar
10

Chapagain R, Remenyi TA, Harris RMB, Mohammed CL, Huth N, Wallach D, Rezaei EE, Ojeda JJ. Decomposing crop model uncertainty: A systematic review. Field Crops Res. 2022;279:108448.
Crossref Google Scholar
11

Antle J, Basso B, Conant RT, Godfray HCJ, Jones JW, Herrero M, Howitt RE, Keating BA, Munoz-Carpena R, Rosenzweig C, et al. Towards a new generation of agricultural system data, models and knowledge products: Design and improvement. Agric Syst. 2017;155:255–268.
Crossref Google Scholar
12
Altes-Buch Q, Quoilin S, Lemort V. Greenhouses: A Modelica library for the simulation of greenhouse climate and energy systems. Paper presented at: Proceedings of the 13th International Modelica Conference; 2019 Mar 4–6; Regensburg, Germany.
Crossref
13

Li T, Hasegawa T, Yin X, Zhu Y, Boote K, Adam M, Bregaglio S, Buis S, Confalonieri R, Fumoto T, et al. Uncertainties in predicting rice yield by current crop models under a wide range of climatic conditions. Glob Change Biol. 2015;21:1328–1341.
Crossref Google Scholar
14

Rötter R, Hoffmann M, Koch M, Müller C. Progress in modelling agricultural impacts of and adaptations to climate change. Curr Opin Plant Biol. 2018;45:255–261.
Crossref Google Scholar
15

Hammer G, Messina C, Wu A, Cooper M. Biological reality and parsimony in crop models—Why we need both in crop improvement! in silico Plants. 2019;1: diz010.
Crossref Google Scholar
16

Roberts MJ, Braun NO, Sinclair TR, Lobell DB, Schlenker W. Comparing and combining process-based crop models and statistical models with some implications for climate change. Environ Res Lett. 2017;12:Article 095010.
Crossref Google Scholar
17

Müller C, Franke J, Jägermeyr J, Ruane AC, Elliott J, Moyer E, Heinke J, Falloon PD, Folberth C, Francois L, et al. Exploring uncertainties in global crop yield projections in a large ensemble of crop models and CMIP5 and CMIP6 climate scenarios. Environ Res Lett. 2021;16:Article 034040.
Crossref Google Scholar
18

Schierhorn F, Hofmann M, Gagalyuk T, Ostapchuk I, Müller D. Machine learning reveals complex effects of climatic means and weather extremes on wheat yields during different plant developmental stages. Clim Chang. 2021;169:39.
Crossref Google Scholar
19

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–444.
Crossref Google Scholar
20

Koirala A, Walsh KB, Wang Z, McCarthy C. Deep learning—Method overview and review of use for fruit detection and yield estimation. Comput Electron Agric. 2019;162:219–234.
Crossref Google Scholar
21

Yang B, Xu Y. Applications of deep-learning approaches in horticultural research: A review. Hortic Res. 2021;8:Article 123.
Crossref Google Scholar
22
Tan X, Qin T, Soong F, Liu T-Y. A survey on neural speech synthesis. arXiv. 2021. http://arxiv.org/abs/2106.15561
23
Chorowski JK, Bahdanau D, Serdyuk D, Cho K, Bengio Y. Attention-based models for speech recognition. In: Cortes C, Lawrence N, Lee D, Sugiyama M, Garnett R, editors. Advances in neural information processing systems. Montreal, Canada: Curran Associates, Inc.; 2015 [accessed 11 Apr 2022]. https://proceedings.neurips.cc/paper/2015/hash/1068c6e4c8051cfd4e9ea8072e3189e2-Abstract.html
24
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. In: Advances in neural information processing systems. Long Beach (CA): Curran Associates, Inc.; 2017.
25
Zhang H, Goodfellow I, Metaxas D, Odena A. Self-attention generative adversarial networks. arXiv. 2019. http://arxiv.org/abs/1805.08318
26
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Paper presented at: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016 Jun 27–30; Las Vegas, NV.
Crossref
27

Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A, et al. Mastering the game of go without human knowledge. Nature. 2017;550:354–359.
Crossref Google Scholar
28

Kamilaris A, Prenafeta-Boldú FX. A review of the use of convolutional neural networks in agriculture. J Agric Sci Technol. 2018;156:312–322.
Crossref Google Scholar
29

Benos L, Tagarakis AC, Dolias G, Berruto R, Kateris D, Bochtis D. Machine learning in agriculture: A comprehensive updated review. Sensors. 2021;21:3758.
Crossref Google Scholar
30

Osinga SA, Paudel D, Mouzakitis SA, Athanasiadis IN. Big data in agriculture: Between opportunity and solution. Agric Syst. 2022;195:103298.
Crossref Google Scholar
31

Easlon HM, Bloom AJ. Easy leaf area: Automated digital image analysis for rapid and accurate measurement of leaf area. Appl Plant Sci. 2014;2:1400033.
Crossref Google Scholar
32

Xu R, Dai J, Luo W, Yin X, Li Y, Tai X, Han L, Chen Y, Lin L, Li G. A photothermal model of leaf area index for greenhouse crops. Agric For Meteorol. 2010;150:541–552.
Crossref Google Scholar
33

Niu Z, Zhong G, Yu H. A review on the attention mechanism of deep learning. Neurocomputing. 2021;452:48–62.
Crossref Google Scholar
34

Han K, Wang Y, Chen H, Chen X, Guo J, Liu Z, Tang Y, Xiao A, Xu C, Xu Y, et al. A survey on vision transformer. IEEE Trans Pattern Anal Mach Intell, 2022:1-1.
Crossref Google Scholar
35
Rong X. Word2Vec parameter learning explained. arXiv. 2016. http://arxiv.org/abs/1411.2738
36
Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. arXiv. 2013. http://arxiv.org/abs/1301.3781
37

Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al. Human-level control through deep reinforcement learning. Nature. 2015;518:529–533.
Crossref Google Scholar
38
Medina JR, Kalita J. Parallel attention mechanisms in neural machine translation. Paper presented at: 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA); 2018 Dec 17–20; Orlando, FL.
Crossref
39

Zhao C, Liu B, Xiao L, Hoogenboom G, Boote KJ, Kassie BT, Pavan W, Shelia V, Kim KS, Hernandez-Ochoa IM, et al. A SIMPLE crop model. Eur J Agron. 2019;104:97–106.
Crossref Google Scholar
40

Sánchez-Molina JA, Pérez N, Rodríguez F, Guzmán JL, López JC. Support system for decision making in the management of the greenhouse environmental based on growth model for sweet pepper. Agric Syst. 2015;139:144–152.
Crossref Google Scholar
41
Hoogenboom G, Porter CH, Boote KJ, Shelia V, Wilkens PW, Singh U, White JW, Lizaso JI, Patricia Moreno L, Pavan W, et al. The DSSAT crop modeling ecosystem. In: Boote KJ, editor. Advances in crop modeling for a sustainable agriculture. Cambridge (UK): Burleigh Dodds Science Publishing; 2019. p. 173–216.
Crossref
42

Scaife MA, Jones D. The relationship between crop yield (or mean plant weight) of lettuce and plant density, length of growing period, and initial plant weight. J Agric Sci. 1976;86:83–91.
Crossref Google Scholar
43

Nash JE, Sutcliffe JV. River flow forecasting through conceptual models part I—A discussion of principles. J Hydrol. 1970;10:282–290.
Crossref Google Scholar
44

He J, Jones JW, Graham WD, Dukes MD. Influence of likelihood function choice for estimating crop model parameters using the generalized likelihood uncertainty estimation method. Agric Syst. 2010;103:256–264.
Crossref Google Scholar
45

Lee J-W, Moon T, Son J-E. Development of growth estimation algorithms for hydroponic bell peppers using recurrent neural networks. Horticulturae. 2021;7:284.
Crossref Google Scholar
46

Bergstra J, Komer B, Eliasmith C, Yamins D, Cox DD. HyperOpt: A python library for model selection and hyperparameter optimization. Comput Sci Discov. 2015;8:Article 014008.
Crossref Google Scholar
47
Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv. 2017. https://arxiv.org/abs/1412.6980
48
Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv. 2015. https://doi.org/10.48550/arXiv.1502.03167
49
Ba JL, Kiros JR, Hinton GE. Layer normalization. arXiv. 2016. https://arxiv.org/abs/1607.06450
50
Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, et al. TensorFlow: A system for large-scale machine learning. arXiv. 2016. https://doi.org/10.48550/arXiv.1605.08695
51

Caruana R. Multitask learning. Mach Learn. 1997;28:41–75.
Crossref Google Scholar
52
Ruder S. An overview of multi-task learning in deep neural networks. arXiv. 2017. http://arxiv.org/abs/1706.05098
53

Hornik K. Approximation capabilities of multilayer feedforward networks. Neural Netw. 1991;4(2):251–257.
Crossref Google Scholar
54

Van Diepen C van, Wolf J, Van Keulen H, Rappoldt C. WOFOST: A simulation model of crop production. Soil Use Manag. 1989;5:16–24.
Crossref Google Scholar
55

Kimm H, Guan K, Jiang C, Miao G, Wu G, Suyker AE, Ainsworth EA, Bernacchi CJ, Montes CM, Berry JA, et al. A physiological signal derived from sun-induced chlorophyll fluorescence quantifies crop physiological response to environmental stresses in the U.S. Corn Belt. Environ Res Lett. 2021;16:124051.
Crossref Google Scholar
56

Hsiao J, Yun K, Moon KH, Kim S-H. A process-based model for leaf development and growth in hardneck garlic (Allium sativum). Ann Bot. 2019;124:1143–1160.
Crossref Google Scholar
57
Zhuang F, Qi Z, Duan K, Xi D, Zhu Y, Zhu H, Xiong H, He Q. A comprehensive survey on transfer learning. arXiv. 2020. http://arxiv.org/abs/1911.02685
58

Qiu X, Sun T, Xu Y, Shao Y, Dai N, Huang X. Pre-trained models for natural language processing: A survey. Sci China Technol Sci. 2020;63:1872–1897.
Crossref Google Scholar
59

Wang H, Yeung D-Y. Towards Bayesian deep learning: A framework and some existing methods. IEEE Trans Knowl Data Eng. 2016;28:3395–3408.
Crossref Google Scholar
60

Angelov P, Soares E. Towards explainable deep neural networks (xDNN). Neural Netw. 2020;130:185–194.
Crossref Google Scholar
61

Newman SJ, Furbank RT. Explainable machine learning models of major crop traits from satellite-monitored continent-wide field trial data. Nat Plants. 2021;7:1354–1363.
Crossref Google Scholar
62

Lischeid G, Webber H, Sommer M, Nendel C, Ewert F. Machine learning in crop yield modelling: A powerful tool, but no surrogate for science. Agric For Meteorol. 2022;312:108698.
Crossref Google Scholar
63
Ruder S. An overview of gradient descent optimization algorithms. arXiv. 2017. http://arxiv.org/abs/1609.04747
64
Panageas I, Piliouras G. Gradient descent only converges to minimizers: Non-isolated critical points and invariant regions. arXiv. 2016. http://arxiv.org/abs/1605.00405
Plant Phenomics
Volume 5,
2023
Article number: 0035
DOI: 10.34133/plantphenomics.0035
Cite this article:
Moon T, Kim D, Kwon S, et al. Process-Based Crop Modeling for High Applicability with Attention Mechanism and Multitask Decoders. Plant Phenomics, 2023, 5: 0035. https://doi.org/10.34133/plantphenomics.0035

157

Views

6

Crossref

5

Web of Science

5

Scopus

0

CSCD

Altmetrics
Received: 14 July 2022
Accepted: 26 February 2023
Published: 13 April 2023
© 2023 Taewon Moon et al. Exclusive licensee Nanjing Agricultural University. No claim to original U.S. Government Works.

Distributed under a Creative Commons Attribution License (CC BY 4.0).
Return