AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Plant Phenomics Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Instance Segmentation and Berry Counting of Table Grape before Thinning Based on AS-SwinT

Wensheng Du1,2, Ping Liu1( )
Shandong Agricultural Equipment Intelligent Engineering Laboratory; Shandong Provincial Key Laboratory of Horticultural, Machinery and Equipment; College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai’an 271000, China
School of Construction Machinery, Shandong Jiaotong University, Jinan 250357, China
Show Author Information

Abstract

Berry thinning is one of the most important tasks in the management of high-quality table grapes. Farmers often thin the berries per cluster to a standard number by counting. With an aging population, it is hard to find adequate skilled farmers to work during thinning season. It is urgent to design an intelligent berry-thinning machine to avoid exhaustive repetitive labor. A machine vision system that can determine the number of berries removed and locate the berries removed is a challenge for the thinning machine. A method for instance segmentation of berries and berry counting in a single bunch is proposed based on AS-SwinT. In AS-SwinT, Swin Transformer is performed as the backbone to extract the rich characteristics of grape berries. An adaptive feature fusion is introduced to the neck network to sufficiently preserve the underlying features and enhance the detection of small berries. The size of berries in the dataset is statistically analyzed to optimize the anchor scale, and Soft-NMS is used to filter the candidate frames to reduce the missed detection of densely shaded berries. Finally, the proposed method could achieve 65.7 APbox, 95.0 AP0.5box, 57 APsbox, 62.8 APmask, 94.3 AP0.5mask, 48 APsmask, which is markedly superior to Mask R-CNN, Mask Scoring R-CNN, and Cascade Mask R-CNN. Linear regressions between predicted numbers and actual numbers are also developed to verify the precision of the proposed model. RMSE and R2 values are 7.13 and 0.95, respectively, which are substantially higher than other models, showing the advantage of the AS-SwinT model in berry counting estimation.

References

1

Roberto SR, Borges WFS, Colombo RC, Koyama R, Hussain I, de Souza RT. Berry-cluster thinning to prevent bunch compactness of ‘BRS Vitoria’, a new black seedless grape. Sci Hortic. 2015;197:297–303.
Crossref Google Scholar
2

Silvestre JP, Roberto SR, Colombo RC, Azeredo Gonçalves LS, Koyama R, Shahab M, Ahmed S, de Souza RT. Bunch sizing of ‘BRS Nubia’ table grape by inflorescence management, shoot tipping and berry thinning. Sci Hortic. 2017;225:764–770.
Crossref Google Scholar
3

Hyun WS, Gi HK, Cheol C. Effects of plant growth regulators and floral cluster thinning on fruit quality of ‘Shine Muscat’ grape. Hortic Sci Technol. 2019;37(6):678–686.
Crossref Google Scholar
4

Cubero S, Diago MP, Blasco J, Tardaguila J, Prats-Montalbán JM, Ibáñez J, Tello J, Aleixos N. A new method for assessment of bunch compactness using automated image analysis: Bunch compactness assessment using image analysis. Aust J Grape Wine Res. 2015;21(1):101–109.
Crossref Google Scholar
5

Du W, Wang C, Zhu Y, Liu L, Liu P. Fruit stem clamping points location for table grape thinning using improved mask r-cnn. Trans Chin Soc Agric Eng. 2022;38:169.
Google Scholar
6

Du W, Zhu Y, Li S, Liu P. Spikelets detection of table grape before thinning based on improved YOLOV5s and Kmeans under the complex environment. Comput Electron Agric. 2022;203:107432.
Crossref Google Scholar
7
Buayai P, Yok-In K, Inoue D, Leow CS, Nishizaki H, Makino K, Mao X. End-to-end inflorescence measurement for supporting table grape trimming with augmented reality. Paper presented at: International Conference on Cyberworlds (CW); 2021 Sep 28–30; Caen, France.
Crossref
8

Buayai P, Saikaew KR, Mao X. End-to-end automatic berry counting for table grape thinning. IEEE Access. 2021;9:4829–4842.
Crossref Google Scholar
9

Nuske S, Wilshusen S, Achar L, Yoder SN, Singh S. Automated visual yield estimation in vineyards: Automated visual yield estimation. J Field Robotic. 2014;31(5):837–860.
Crossref Google Scholar
10

Aquino A, Millan B, Diago M-P, Tardaguila J. Automated early yield prediction in vineyards from on-the-go image acquisition. Comput Electron Agric. 2018;144:26–36.
Crossref Google Scholar
11

Pérez-Zavala R, Torres-Torriti M, Cheein FA, Troni G. A pattern recognition strategy for visual grape bunch detection in vineyards. Comput Electron Agric. 2018;151:136–149.
Crossref Google Scholar
12

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–444.
Crossref Google Scholar
13

Palacios F, Bueno G, Salido J, Diago MP, Hernández I, Tardaguila J, Hernandez I, Tardaguila J. Automated grapevine flower detection and quantification method based on computer vision and deep learning from on-the-go imaging using a mobile sensing platform under field conditions. Comput Electron Agric. 2020;178:105796.
Crossref Google Scholar
14

Coviello L, Cristoforetti M, Jurman G, Furlanello C. GBCNet: In-field grape berries counting for yield estimation by dilated CNNs. Appl Sci. 2020;10(14):4870.
Crossref Google Scholar
15
Zabawa L, Kicherer A, Klingbeil L, Töpfer R, Kuhlmann H, Roscher R, Topfer R, Kuhlmann H, Roscher R. Counting of grapevine berries in images via semantic segmentation using convolutional neural networks. ISPRS J Photogramm Remote Sens, 2020;164:73–83.
Crossref
16
Deng G, Geng T, He C, Wang X, He B, Duan L. TSGYE: Two-stage grape yield estimation. Neural information processing. Cham (Switzerland):Springer International Publishing; 2020. Vol. 1332 of Communications in Computer and Information Science. p. 580–588.
Crossref
17

Aquino A, Diago MP, Millán B, Tardáguila J. A new methodology for estimating the grapevine-berry number per cluster using image analysis. Biosyst Eng. 2017;156:80–95.
Crossref Google Scholar
18

Liu S, Zeng X, Whitty M. 3DBunch: A novel iOS-smartphone application to evaluate the number of grape berries per bunch using image analysis techniques. IEEE Access. 2020;8:114663–114674.
Crossref Google Scholar
19

Luo L, Liu W, Lu Q, Wang J, Wen W, Yan D, Tang Y. Grape berry detection and size measurement based on edge image processing and geometric morphology. Mach Des. 2021;9(10):233.
Crossref Google Scholar
20

Sun M, Xu L, Chen X, Ji Z, Zheng Y, Jia W. BFP net: Balanced feature pyramid network for small apple detection in complex orchard environment. Plant Phenomics. 2022;2022:9892464.
Crossref Google Scholar
21

Gai R, Chen N, Yuan H. A detection algorithm for cherry fruits based on the improved YOLO-v4 model. Neural Comput Applic. 2021;35:13895–13906.
Crossref Google Scholar
22

Zheng T, Jiang M, Li Y, Feng M. Research on tomato detection in natural environment based on RC-YOLOv4. Comput Electron Agric. 2022;198:107029.
Crossref Google Scholar
23

Tu S, Pang J, Liu H, Zhuang N, Chen Y, Zheng C, Wan H,Xue Y. Passion fruit detection and counting based on multiple scale faster R-CNN using RGB-D images. Precis Agric. 2020;21(5):1072–1091.
Crossref Google Scholar
24

Shen L, Su J, Huang R, Quan W, Song Y, Fang Y, Su B. Fusing attention mechanism with mask R-CNN for instance segmentation of grape cluster in the field. Front Plant Sci. 2022;13:934450.
Crossref Google Scholar
25
He K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. Paper presented at: Proceedings of the IEEE International Conference on Computer Vision (ICCV); 2017 Oct 22–29; Venice, Italy.
Crossref
26
Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B. Swin Transformer: Hierarchical vision Transformer using Shifted Windows. Paper presented at: IEEE/CVF International Conference on Computer Vision (ICCV); 2021 Oct 10–17; Montreal, Canada.
Crossref
27
Liu S, Huang D, Wang Y. Learning spatial fusion for single-shot object detection. ArXiv. 2019. https://doi.org/10.48550/arXiv.1911.09516
28

Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell. 2017;39(6):1137–1149.
Crossref Google Scholar
29

Guo Y, Lan Y, Chen X. CST: Convolutional Swin Transformer for detecting the degree and types of plant diseases. Comput Electron Agric. 2022;202:107407.
Crossref Google Scholar
30

Wang F, Rao Y, Luo Q, Jin X, Jiang Z, Zhang W, Li S. Practical cucumber leaf disease recognition using improved Swin transformer and small sample size. Comput Electron Agric. 2022;199:107163.
Crossref Google Scholar
31

Bi C, Hu N, Zou Y, Zhang S, Xu S, Yu H. Development of deep learning methodology for maize seed variety recognition based on improved Swin transformer. Agronomy. 2022;12(8):1843.
Crossref Google Scholar
32
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Paper presented at: IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016 Jun 27–30; Las Vegas, NV.
Crossref
33
Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S.Feature pyramid networks for object detection. Paper presented at: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017 Jul 21–26; Honolulu, HI.
Crossref
34
Chen K, Wang J, Pang J, Cao Y, Xiong Y, Li X, Sun S, Feng W, Liu Z, Xu X, etal. MMDetection: Open mmlab detection toolbox and benchmark. ArXiv. 2019. https://doi.org/10.48550/arXiv.1906.07155
35
Neubeck A, Van Gool L. Efficient non-maximum suppression. Paper presented at: 18th International Conference on Pattern Recognition (ICPR’06); 2006 Aug 20–24; Hong Kong, China.
Crossref
36
Bodla N, Singh B, Chellappa R, Davis LS. Soft-NMS – Improving object detection with one line of code. ArXiv. 2017. https://doi.org/10.48550/arXiv.1704.04503
Crossref
37
Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL. Microsoft COCO: Common Objects in Context. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T, editors. Computer Vision – ECCV 2014. Cham (Switzerland):Springer International Publishing; 2014; p. 740–755.
Crossref
38
Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated residual transformations for deep neural networks. Paper presented at: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017 Jul 21–26; Honolulu, HI.
Crossref
39
Huang Z, Huang L, Gong Y, Huang C, Wang X. Mask Scoring R-CNN. Paper presented at: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2019 Jun 15–20; Long Beach, CA.
Crossref
40
Cai Z, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection. Paper presented at: IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2018 Jun 18–22; Salt Lake City, UT.
Crossref
Plant Phenomics
Volume 5,
2023
Article number: 0085
DOI: 10.34133/plantphenomics.0085
Cite this article:
Du W, Liu P. Instance Segmentation and Berry Counting of Table Grape before Thinning Based on AS-SwinT. Plant Phenomics, 2023, 5: 0085. https://doi.org/10.34133/plantphenomics.0085

170

Views

9

Crossref

8

Web of Science

10

Scopus

0

CSCD

Altmetrics
Received: 16 January 2023
Accepted: 08 August 2023
Published: 29 August 2023
© 2023 Wensheng Du and Ping Liu Exclusive licensee Nanjing Agricultural University. No claim to original U.S. Government Works.

Distributed under a Creative Commons Attribution License 4.0 (CC BY 4.0).
Return