Journal Home > Volume 7 , Issue 1
Big Data Mining and Analytics 2024, 7(1): 1-11 https://doi.org/10.26599/BDMA.2022.9020048
Open Access | Issue | Published: 25 December 2023

Multi-Scale Feature Fusion Model for Bridge Appearance Defect Detection

Show Author's Information Rong Pang1,  Yan Yang2( ),  Aiguo Huang2,  Yan Liu2,  Peng Zhang3,  Guangwu Tang4
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China, and with China Merchants Chongqing Road Engineering Inspection Center Co., Ltd., Chongqing 400067, China, and also with State Key Laboratory of Bridge Engineering Structural Dynamics, Chongqing 400067, China
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China
China Merchants Chongqing Road Engineering Inspection Center Co., Ltd., Chongqing 400067, China, and with State Key Laboratory of Bridge Engineering Structural Dynamics, Chongqing 400067, China
State Key Laboratory of Bridge Engineering Structural Dynamics, Chongqing 400067, China
Keywords:
lightweight network, defect detection, Multi-scale Feature Fusion (MFF), Region Of Interest (ROI) alignment
Cite this article:
Pang R, Yang Y, Huang A, et al. Multi-Scale Feature Fusion Model for Bridge Appearance Defect Detection. Big Data Mining and Analytics, 2024, 7(1): 1-11. https://doi.org/10.26599/BDMA.2022.9020048
Download citation
EndNote(RIS)
BibTeX

1297

Views

454

Downloads

Citations

1

Crossref

0

WoS

0

Scopus

0

CSCD

Abstract Full text About this article

Although the Faster Region-based Convolutional Neural Network (Faster R-CNN) model has obvious advantages in defect recognition, it still cannot overcome challenging problems, such as time-consuming, small targets, irregular shapes, and strong noise interference in bridge defect detection. To deal with these issues, this paper proposes a novel Multi-scale Feature Fusion (MFF) model for bridge appearance disease detection. First, the Faster R-CNN model adopts Region Of Interest (ROI) pooling, which omits the edge information of the target area, resulting in some missed detections and inaccuracies in both detecting and localizing bridge defects. Therefore, this paper proposes an MFF based on regional feature Aggregation (MFF-A), which reduces the missed detection rate of bridge defect detection and improves the positioning accuracy of the target area. Second, the Faster R-CNN model is insensitive to small targets, irregular shapes, and strong noises in bridge defect detection, which results in a long training time and low recognition accuracy. Accordingly, a novel Lightweight MFF (namely MFF-L）model for bridge appearance defect detection using a lightweight network EfficientNetV2 and a feature pyramid network is proposed, which fuses multi-scale features to shorten the training speed and improve recognition accuracy. Finally, the effectiveness of the proposed method is evaluated on the bridge disease dataset and public computational fluid dynamic dataset.


menu
Abstract
Full text
Outline
About this article

Multi-Scale Feature Fusion Model for Bridge Appearance Defect Detection

Show Author's information Rong Pang1, Yan Yang2( ), Aiguo Huang2, Yan Liu2, Peng Zhang3, Guangwu Tang4
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China, and with China Merchants Chongqing Road Engineering Inspection Center Co., Ltd., Chongqing 400067, China, and also with State Key Laboratory of Bridge Engineering Structural Dynamics, Chongqing 400067, China
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China
China Merchants Chongqing Road Engineering Inspection Center Co., Ltd., Chongqing 400067, China, and with State Key Laboratory of Bridge Engineering Structural Dynamics, Chongqing 400067, China
State Key Laboratory of Bridge Engineering Structural Dynamics, Chongqing 400067, China

Abstract

Although the Faster Region-based Convolutional Neural Network (Faster R-CNN) model has obvious advantages in defect recognition, it still cannot overcome challenging problems, such as time-consuming, small targets, irregular shapes, and strong noise interference in bridge defect detection. To deal with these issues, this paper proposes a novel Multi-scale Feature Fusion (MFF) model for bridge appearance disease detection. First, the Faster R-CNN model adopts Region Of Interest (ROI) pooling, which omits the edge information of the target area, resulting in some missed detections and inaccuracies in both detecting and localizing bridge defects. Therefore, this paper proposes an MFF based on regional feature Aggregation (MFF-A), which reduces the missed detection rate of bridge defect detection and improves the positioning accuracy of the target area. Second, the Faster R-CNN model is insensitive to small targets, irregular shapes, and strong noises in bridge defect detection, which results in a long training time and low recognition accuracy. Accordingly, a novel Lightweight MFF (namely MFF-L）model for bridge appearance defect detection using a lightweight network EfficientNetV2 and a feature pyramid network is proposed, which fuses multi-scale features to shorten the training speed and improve recognition accuracy. Finally, the effectiveness of the proposed method is evaluated on the bridge disease dataset and public computational fluid dynamic dataset.

Keywords: lightweight network, defect detection, Multi-scale Feature Fusion (MFF), Region Of Interest (ROI) alignment

References(23)

[1]
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg, SSD: single shot MultiBox detector, in Proc. 14th European Conf. Computer Vision, Amsterdam, The Netherlands, 2016, pp. 21−37.
DOI
[2]
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, You only look once: Unified, real-time object detection, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA. 2016, pp. 779−788.
DOI
[3]
R. Girshick, J. Donahue, T. Darrell, and J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, in Proc. 2014 IEEE Conf. Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014, pp. 580−587.
DOI
[4]

L. H. Liu, Selective random forest optimization algorithm based on improved immunogenetics and its application, (in Chinese), Electr. Technol. Software Eng., no. 24, pp. 148–149, 2020.
Google Scholar
[5]

A. Malini, P. Priyadharshini, and S. Sabeena, An automatic assessment of road condition from aerial imagery using modified VGG architecture in faster-RCNN framework, J. Intell. Fuzzy Systems Appl. Eng. Technol., vol. 40, no. 6, pp. 11411–11422, 2021.
DOI Google Scholar
[6]

K. He, X. Zhang, S. Ren, and J. Sun, Spatial pyramid pooling in deep convolutional networks for visual recognition, IEEE Trans. Patt. Anal. Mach. Intell., vol. 37, no. 9, pp. 1904–1916, 2015.
DOI Google Scholar
[7]

S. Ren, K. He, R. Girshick, and J. Sun, Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Trans. Patt. Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, 2017.
DOI Google Scholar
[8]

Z. Y. Sun, L. L. Pei, W. Li, X. L. Hao, and Y. Chen, Pavement sealed crack detection method based on improved faster R-CNN, (in Chinese), J. South China Univ. Technol. (Nat. Sci. Ed.), vol. 48, no. 2, pp. 84–93, 2020.
Google Scholar
[9]
C. Gou, B. Peng, T. Li, and Z. Gao, Pavement crack detection based on the improved Faster-RCNN, in Proc. 2019 IEEE 14th Int. Conf. Intelligent Systems and Knowledge Engineering (ISKE), Dalian, China, 2019, pp. 962–967.
DOI
[10]
Department of Comprehensive Planning, Ministry of Transport of the People’s Republic of China, Statistical bulletin on the development of the transport industry in 2021, (in Chinese), http://www.gov.cn/xinwen/2022-05/25/content_5692174.htm, 2022.
[11]

S. Lu, Cause analysis of bridge quality problems, (in Chinese), Heilongjiang Sci. Technol. Inform., no. 6, p. 231, 2016.
Google Scholar
[12]
American Association of State High-way and Transportation Officials, Bridging the gap: Restoring and rebuilding the nation’s bridges, https://ntrl.ntis.gov/NTRL/dashboard/searchResults/titleDetail/PB2009101207.xhtml,2008.
[13]
H. Adeli and X. Jiang, Intelligent Infrastructure : Neural, Networks, Wavelets and Chaos Theory for Intelligent Transportation Systems and Smart Structures. Boca Raton, FL, USA: CRC Press, 2018.
[14]

J. Valenca, D. Dias-da-Costa, E. Jlio, and H. Costa, Automatic crack monitoring using photogrammetry and image processing, Measurement, vol. 46, no. 1, pp. 433–441, 2013.
DOI Google Scholar
[15]

S. Yang, L. T. Shao, X. X. Guo, X. Liu, and J. Zhang, Skeleton and fractal law based image recognition algorithm for concrete crack, (in Chinese), Chin. J. Sci. Instr., vol. 33, no. 8, pp. 1850–1855, 2012.
Google Scholar
[16]
P. R. Muduli and U. C. Pati, A novel technique for wall crack detection using image fusion, in Proc. 2013 Int. Conf. Computer Communication and Informatics, Coimbatore, India, 2013, pp. 1–6.
DOI
[17]

Y. J. Cha, W. Choi, and O. Büyüközürk, Deep learning-based crack damage detection using convolutional neural net-works, Comput.-Aided Civil Infrastruct. Eng., vol. 32, no. 5, pp. 361–378, 2017.
DOI Google Scholar
[18]

S. Wang, X. Wu, Y. H. Zhang, and Q. Chen, Image crack detection with fully convolutional network based on deep learning, J. Computer-Aided Des. Comput. Graph., vol. 30, no. 5, pp. 859–867, 2018.
Google Scholar
[19]

F. H. Cai, Y. X. Zhang, and J. Huang, Bridge surface crack detection algorithm based on YOLOv3 and attention mechanism, (in Chinese), Patt. Recognit. Artif. Intell., vol. 33, no. 10, pp. 926–933, 2020.
Google Scholar
[20]
T. Y. Lin, P. Dollär, R. Girshick, K. He, B. Hariharan, and S. Belongie, Feature pyramid networks for object detection, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 936-944.
DOI
[21]
M. Tan and Q. V. Le, EfficientNet: Rethinking model scaling for convolutional neural networks, in Proc. 36th Int. Conf. on Machine Learning, Long Beach, CA, USA, 2019, pp. 6105−6114.
[22]
M. Tan and Q. V. Le, Efficientnetv2: Smaller models and faster training, in Proc. 38th Int. Conf. Machine Learning, Virtual Event, 2021, pp. 10096−10106.
[23]

K. He, G. Gkioxari, P. Dollár, and R. Girshick, , Mask R-CNN, in Proc. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 2980–2988.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 31 August 2022
Revised: 29 October 2022
Accepted: 26 November 2022
Published: 25 December 2023
Issue date: March 2024

Copyright

© The author(s) 2023.

Acknowledgements

Acknowledgment

This work was supported by the National Natural Science Foundation of China (No. 61976247) and the Major R&D Programs of China (No. 2019YFB-1310400).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return