Discover the SciOpen Platform and Achieve Your Research Goals with Ease.
Search articles, authors, keywords, DOl and etc.
Remote sensing is of great importance for analyzing and studying various phenomena occurrence and development on Earth. Today is possible to extract features specific to various fields of application with the application of modern machine learning techniques, such as Convolutional Neural Networks (CNN) on MultiSpectral Images (MSI). This systematic review examines the application of 1D-, 2D-, 3D-, and 4D-CNNs to MSI, following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. This review addresses three Research Questions (RQ): RQ1: “In which application domains different CNN models have been successfully applied for processing MSI data?”, RQ2: “What are the commonly utilized MSI datasets for training CNN models in the context of processing multispectral satellite imagery?”, and RQ3: “How does the degree of CNN complexity impact the performance of classification, regression or segmentation tasks for multispectral satellite imagery?”. Publications are selected from three databases, Web of Science, IEEE Xplore, and Scopus. Based on the obtained results, the main conclusions are: (1) The majority of studies are applied in the field of agriculture and are using Sentinel-2 satellite data; (2) Publications implementing 1D-, 2D-, and 3D-CNNs mostly utilize classification. For 4D-CNN, there are limited number of studies, and all of them use segmentation; (3) This study shows that 2D-CNNs prevail in all application domains, but 3D-CNNs prove to be better for spatio-temporal pattern recognition, more specifically in agricultural and environmental monitoring applications. 1D-CNNs are less common compared to 2D-CNNs and 3D-CNNs, but they show good performance in spectral analysis tasks. 4D-CNNs are more complex and still underutilized, but they have potential for complex data analysis. More details about metrics according to each CNN are provided in the text and supplementary files, offering a comprehensive overview of the evaluation metrics for each type of machine learning technique applied.
Z. Adiri, R. Lhissou, A. El Harti, A. Jellouli, and M. Chakouri, Recent advances in the use of public domain satellite imagery for mineral exploration: A review of landsat-8 and sentinel-2 applications, Ore Geol. Rev., vol. 117, p. 103332, 2020.
A. Jamali and M. Mahdianpari, A cloud-based framework for large-scale monitoring of ocean plastics using multi-spectral satellite imagery and generative adversarial network, Water, vol. 13, no. 18, p. 2553, 2021.
D. Walshe, D. McInerney, R. van de Kerchove, C. Goyens, P. Balaji, and K. A. Byrne, Detecting nutrient deficiency in spruce forests using multispectral satellite imagery, Int. J. Appl. Earth Obs. Geoinform., vol. 86, p. 101975, 2020.
F. Marchese, N. Genzano, M. Neri, A. Falconieri, G. Mazzeo, and N. Pergola, A multi-channel algorithm for mapping volcanic thermal anomalies by means of sentinel-2 MSI and landsat-8 OLI data, Remote Sens., vol. 11, no. 23, p. 2876, 2019.
Z. Yi, L. Jia, and Q. Chen, Crop classification using multi-temporal sentinel-2 data in the Shiyang river basin of China, Remote Sens., vol. 12, no. 24, p. 4052, 2020.
C. Sun, Y. Bian, T. Zhou, and J. Pan, Using of multi-source and multi-temporal remote sensing data improves crop-type mapping in the subtropical agriculture region, Sensors, vol. 19, no. 10, p. 2401, 2019.
P. Lynch, L. Blesius, and E. Hines, Classification of urban area using multispectral indices for urban planning, Remote Sens., vol. 12, no. 15, p. 2503, 2020.
Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature, vol. 521, no. 7553, pp. 436–444, 2015.
L. Li, C. Solana, F. Canters, and M. Kervyn, Testing random forest classification for identifying lava flows and mapping age groups on a single Landsat 8 image, J. Volcanol. Geotherm. Res., vol. 345, pp. 109–124, 2017.
X. Huang and L. Zhang, An SVM ensemble approach combining spectral, structural, and semantic features for the classification of high-resolution remotely sensed imagery, IEEE Trans. Geosci. Remote Sens., vol. 51, no. 1, pp. 257–272, 2013.
A. da Penha Pacheco, J. A. da Silva Junior, A. M. Ruiz-Armenteros, and R. F. F. Henriques, Assessment of k-nearest neighbor and random forest classifiers for mapping forest fire areas in central Portugal using landsat-8, sentinel-2, and terra imagery, Remote Sens., vol. 13, no. 7, p. 1345, 2021.
A. Ivanda, L. Šerić, M. Bugarić, and M. Braović, Mapping chlorophyll-a concentrations in the Kaštela bay and Brač channel using ridge regression and sentinel-2 satellite images, Electronics, vol. 10, no. 23, p. 3004, 2021.
P. Das and V. Pandey, Use of logistic regression in land-cover classification with moderate-resolution multispectral data, J. Indian Soc. Remote Sens., vol. 47, no. 8, pp. 1443–1454, 2019.
L. Ma, Y. Liu, X. Zhang, Y. Ye, G. Yin, and B. A. Johnson, Deep learning in remote sensing applications: A meta-analysis and review, ISPRS J. Photogramm. Remote Sens., vol. 152, pp. 166–177, 2019.
X. X. Zhu, D. Tuia, L. Mou, G. S. Xia, L. Zhang, F. Xu, and F. Fraundorfer, Deep learning in remote sensing: A comprehensive review and list of resources, IEEE Geosci. Remote Sens. Mag., vol. 5, no. 4, pp. 8–36, 2017.
L. Mou, L. Bruzzone, and X. X. Zhu, Learning spectral-spatial-temporal features via a recurrent convolutional neural network for change detection in multispectral imagery, IEEE Trans. Geosci. Remote Sens., vol. 57, no. 2, pp. 924–935, 2019.
S. M. Borzov and O. I. Potaturkin, Spectral-spatial methods for hyperspectral image classification. review, Optoelectron. Instrum. Data Process., vol. 54, no. 6, pp. 582–599, 2018.
S. Jia, S. Jiang, Z. Lin, N. Li, M. Xu, and S. Yu, A survey: Deep learning for hyperspectral image classification with few labeled samples, Neurocomputing, vol. 448, pp. 179–204, 2021.
M. J. Khan, H. S. Khan, A. Yousaf, K. Khurshid, and A. Abbas, Modern trends in hyperspectral image analysis: A review, IEEE Access, vol. 6, pp. 14118–14129, 2018.
X. Yuan, J. Shi, and L. Gu, A review of deep learning methods for semantic segmentation of remote sensing imagery, Expert Syst. Appl., vol. 169, p. 114417, 2021.
S. Bera, V. K. Shrivastava, and S. C. Satapathy, Advances in hyperspectral image classification based on convolutional neural networks: A review, CMES Comput. Model. Eng. Sci., vol. 133, no. 2, pp. 219–250, 2022.
Y. Li, H. Zhang, X. Xue, Y. Jiang, and Q. Shen, Deep learning for remote sensing image classification: A survey, WIREs Data Min. Knowledge Discovery, vol. 8, no. 6, p. e1264, 2018.
T. Kattenborn, J. Leitloff, F. Schiefer, and S. Hinz, Review on convolutional neural networks (CNN) in vegetation remote sensing, ISPRS J. Photogramm. Remote Sens., vol. 173, pp. 24–49, 2021.
L. Zhang, L. Zhang, and B. Du, Deep learning for remote sensing data: A technical tutorial on the state of the art, IEEE Geosci. Remote Sens. Mag., vol. 4, no. 2, pp. 22–40, 2016.
J. E. Ball, D. T. Anderson, and C. S. C. Sr, Comprehensive survey of deep learning in remote sensing: Theories, tools, and challenges for the community, J. Appl. Remote Sens., vol. 11, no. 4, p. 042609, 2017.
M. J. Page, J. E. McKenzie, P. M. Bossuyt, I. Boutron, T. C. Hoffmann, C. D. Mulrow, L. Shamseer, J. M. Tetzlaff, E. A. Akl, S. E. Brennan, et al., The PRISMA 2020 statement: An updated guideline for reporting systematic reviews, Int. J. Surg., vol. 88, p. 105906, 2021.
H. Gu, Y. Wang, S. Hong, and G. Gui, Blind channel identification aided generalized automatic modulation recognition based on deep learning, IEEE Access, vol. 7, pp. 110722–110729, 2019.
L. Alzubaidi, J. Zhang, A. J. Humaidi, A. Al-Dujaili, Y. Duan, O. Al-Shamma, J. Santamaría, M. A. Fadhel, M. Al-Amidie, and L. Farhan, Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions, J. Big Data, vol. 8, no. 1, p. 53, 2021.
W. Hu, Y. Huang, L. Wei, F. Zhang, and H. Li, Deep convolutional neural networks for hyperspectral image classification, J. Sens., vol. 2015, p. 258619, 2015.
S. Ojaghi, Y. Bouroubi, S. Foucher, M. Bergeron, and C. Seynat, Deep learning-based emulation of radiative transfer models for top-of-atmosphere BRDF modelling using sentinel-3 OLCI, Remote Sens., vol. 15, no. 3, p. 835, 2023.
L. P. Osco, Q. Wu, E. L. de Lemos, W. N. Gonçalves, A. P. M. Ramos, J. Li, and J. Marcato Junior, The segment anything model (SAM) for remote sensing applications: From zero to one shot, Int. J. Appl. Earth Obs. Geoinform., vol. 124, p. 103540, 2023.
S. M. M. Nejad, D. Abbasi-Moghadam, A. Sharifi, N. Farmonov, K. Amankulova, and M. Lászlź, Multispectral crop yield prediction using 3D-convolutional neural networks and attention convolutional LSTM approaches, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 16, pp. 254–266, 2023.
M. Weiss, F. Jacob, and G. Duveiller, Remote sensing for agricultural applications: A meta-review, Remote Sens. Environ., vol. 236, p. 111402, 2020.
D. Phiri, M. Simwanda, S. Salekin, V. R. Nyirenda, Y. Murayama, and M. Ranagalage, Sentinel-2 data for land cover/use mapping: A review, Remote Sens., vol. 12, no. 14, p. 2291, 2020.
D. P. Roy, M. A. Wulder, T. R. Loveland, C. E. Woodcock, R. G. Allen, M. C. Anderson, D. Helder, J. R. Irons, D. M. Johnson, R. Kennedy, et al., Landsat-8: Science and product vision for terrestrial global change research, Remote Sens. Environ., vol. 145, pp. 154–172, 2014.
A. Savtchenko, D. Ouzounov, S. Ahmad, J. Acker, G. Leptoukh, J. Koziana, and D. Nickless, Terra and aqua MODIS products available from NASA GES DAAC, Adv. Space Res., vol. 34, no. 4, pp. 710–714, 2004.
Y. Chen, L. Tang, Z. Kan, A. Latif, X. Yang, M. Bilal, and Q. Li, Cloud and cloud shadow detection based on multiscale 3D-CNN for high resolution multispectral imagery, IEEE Access, vol. 8, pp. 16505–16516, 2020.
Q. Zhang, Q. Yuan, Z. Li, F. Sun, and L. Zhang, Combined deep prior with low-rank tensor SVD for thick cloud removal in multitemporal images, ISPRS J. Photogramm. Remote Sens., vol. 177, pp. 161–173, 2021.
H. Jiang and N. Lu, Multi-scale residual convolutional neural network for haze removal of remote sensing images, Remote Sens., vol. 10, no. 6, p. 945, 2018.
F. Kong, K. Hu, Y. Li, D. Li, and S. Zhao, Spectral-spatial feature partitioned extraction based on CNN for multispectral image compression, Remote Sens., vol. 13, no. 1, p. 9, 2021.
X. Zhao, Y. Ma, Y. Xiao, J. Liu, J. Ding, X. Ye, and R. Liu, Atmospheric correction algorithm based on deep learning with spatial-spectral feature constraints for broadband optical satellites: Examples from the HY-1C coastal zone imager, ISPRS J. Photogramm. Remote Sens., vol. 205, pp. 147–162, 2023.
F. Salvetti, V. Mazzia, A. Khaliq, and M. Chiaberge, Multi-image super resolution of remotely sensed images using residual attention deep neural networks, Remote Sens., vol. 12, no. 14, p. 2207, 2020.
F. Dorr, Satellite image multi-frame super resolution using 3D wide-activation neural networks, Remote Sens., vol. 12, no. 22, p. 3812, 2020.
S. He, R. Zhou, S. Li, S. Jiang, and W. Jiang, Disparity estimation of high-resolution remote sensing images with dual-scale matching network, Remote Sens., vol. 13, no. 24, p. 5050, 2021.
L. Wang, X. Wang, Z. Zhao, Y. Wu, J. Xu, H. Zhang, J. Yu, Q. Sun, and Y. Bai, Multi-factor status prediction by 4D fractal CNN based on remote sensing images, Fractals, vol. 30, no. 2, p. 2240101, 2022.
M. Hossin and M. N. Sulaiman, A review on evaluation metrics for data classification evaluations, Int. J. Data Min. Knowl. Manage. Process, vol. 5, no. 2, pp. 1–11, 2015.
A. V. Tatachar, Comparative assessment of regression models based on model evaluation metrics, Int. Res. J. Eng. Technol., vol. 8, no. 9, pp. 853–860, 2021.
S. Jeong, J. Ko, and J. M. Yeom, Predicting rice yield at pixel scale through synthetic use of crop and deep learning models with satellite data in south and North Korea, Sci. Total Environ., vol. 802, p. 149726, 2022.
F. Sabo, M. Meroni, F. Waldner, and F. Rembold, Is deeper always better? Evaluating deep learning models for yield forecasting with small data, Environ. Monit. Assess., vol. 195, no. 10, p. 1153, 2023.
Z. Xu, H. Sun, T. Zhang, H. Xu, D. Wu, and J. Gao, Evaluating established deep learning methods in constructing integrated remote sensing drought index: A case study in China, Agric. Water Manage., vol. 286, p. 108405, 2023.
S. Vulova, F. Meier, A. D. Rocha, J. Quanz, H. Nouri, and B. Kleinschmit, Modeling urban evapotranspiration using remote sensing, flux footprints, and artificial intelligence, Sci. Total Environ., vol. 786, p. 147293, 2021.
X. Zhou, Q. Xin, Y. Dai, and W. Li, A deep-learning-based experiment for benchmarking the performance of global terrestrial vegetation phenology models, Global Ecol. Biogeogr., vol. 30, no. 11, pp. 2178–2199, 2021.
P. M. Maier, S. Keller, and S. Hinz, Deep learning with WASI simulation data for estimating chlorophyll a concentration of inland water bodies, Remote Sens., vol. 13, no. 4, p. 718, 2021.
S. S. Mukonza and J. L. Chiang, Micro-climate computed machine and deep learning models for prediction of surface water temperature using satellite data in Mundan water reservoir, Water, vol. 14, no. 18, p. 2935, 2022.
A. Ivanda, L. Šerića, D. Žagar, and K. Oštir, An application of 1D convolution and deep learning to remote sensing modelling of secchi depth in the northern Adriatic sea, Big Earth Data, vol. 8, no. 1, pp. 82–114, 2024.
Y. Zeng, T. Liang, D. Fan, and H. He, A novel algorithm for the retrieval of chlorophyll a in marine environments using deep learning, Water, vol. 15, no. 21, p. 3864, 2023.
V. Sagan, M. Maimaitijiang, S. Bhadra, M. Maimaitiyiming, D. R. Brown, P. Sidike, and F. B. Fritschi, Field-scale crop yield prediction using multi-temporal worldview-3 and planetscope satellite data and deep learning, ISPRS J. Photogramm. Remote Sens., vol. 174, pp. 265–281, 2021.
J. Lee, J. Im, D. H. Cha, H. Park, and S. Sim, Tropical cyclone intensity estimation using multi-dimensional convolutional neural networks from geostationary satellite data, Remote Sens., vol. 12, no. 1, p. 108, 2020.
J. Zhong, J. Sun, Z. Lai, and Y. Song, Nearshore bathymetry from icesat-2 lidar and sentinel-2 imagery datasets using deep learning approach, Remote Sens., vol. 14, no. 17, p. 4229, 2022.
M. Papadomanolaki, S. Christodoulidis, K. Karantzalos, and M. Vakalopoulou, Unsupervised multistep deformable registration of remote sensing imagery based on deep learning, Remote Sens., vol. 13, no. 7, p. 1294, 2021.
Z. Wang, S. Fang, and J. Zhang, Spatiotemporal fusion model of remote sensing images combining single-band and multi-band prediction, Remote Sens., vol. 15, no. 20, p. 4936, 2023.
M. Qiao, X. He, X. Cheng, P. Li, H. Luo, L. Zhang, and Z. Tian, Crop yield prediction from multi-spectral, multi-temporal remotely sensed imagery using recurrent 3D convolutional neural networks, Int. J. Appl. Earth Obs. Geoinform., vol. 102, p. 102436, 2021.
R. Fernandez-Beltran, T. Baidar, J. Kang, and F. Pla, Rice-yield prediction with multi-temporal sentinel-2 data and 3D CNN: A case study in Nepal, Remote Sens., vol. 13, no. 7, p. 1391, 2021.
T. Fei, B. Huang, X. Wang, J. Zhu, Y. Chen, H. Wang, and W. Zhang, A hybrid deep learning model for the bias correction of SST numerical forecast products using satellite data, Remote Sens., vol. 14, no. 6, p. 1339, 2022.
L. Wang, W. Li, X. Wang, and J. Xu, Remote sensing image analysis and prediction based on improved Pix2Pix model for water environment protection of smart cities, PeerJ Comput. Sci., vol. 9, p. e1292, 2023.
G. Chen, Q. Pei, and M. Kamruzzaman, Remote sensing image quality evaluation based on deep support value learning networks, Signal Process Image Commun., vol. 83, p. 115783, 2020.
L. Zhang, P. Liu, L. Wang, J. Liu, B. Song, Y. Zhang, G. He, and H. Zhang, Improved 1-km-resolution hourly estimates of aerosol optical depth using conditional generative adversarial networks, Remote Sens., vol. 13, no. 19, p. 3834, 2021.
D. Kaur and Y. Kaur, Various image segmentation techniques: A review, Int. J. Comput. Sci. Mobile Comput., vol. 3, no. 5, pp. 809–814, 2014.
H. Ghandorh, W. Boulila, S. Masood, A. Koubaa, F. Ahmed, and J. Ahmad, Semantic segmentation and edge detection—approach to road detection in very high resolution satellite images, Remote Sens., vol. 14, no. 3, p. 613, 2022.
R. Li, S. Zheng, C. Duan, L. Wang, and C. Zhang, Land cover classification from remote sensing images based on multi-scale fully convolutional network, Geo-Spat. Inf. Sci., vol. 25, no. 2, pp. 278–294, 2022.
E. Saralioglu and O. Gungor, Semantic segmentation of land cover from high resolution multispectral satellite images by spectral-spatial convolutional neural network, Geocarto Int., vol. 37, no. 2, pp. 657–677, 2022.
I. Gallo, L. Ranghetti, N. Landro, R. L. Grassa, and M. Boschetti, In-season and dynamic crop mapping using 3D convolution neural networks and sentinel-2 time series, ISPRS J. Photogramm. Remote Sens., vol. 195, pp. 335–352, 2023.
S. Mohammadi, M. Belgiu, and A. Stein, Improvement in crop mapping from satellite image time series by effectively supervising deep neural networks, J. Photogramm. Remote Sens., vol. 198, pp. 272–283, 2023.
E. Kalinicheva, J. Sublime, and M. Trocan, Unsupervised satellite image time series clustering using object-based approaches and 3D convolutional autoencoder, Remote Sens., vol. 12, no. 11, p. 1816, 2020.
M. R. Rahman and S. K. Saha, Multi-resolution segmentation for object-based classification and accuracy assessment of land use/land cover classification using remotely sensed data, J. Indian Soc. Remote Sens., vol. 36, no. 2, pp. 189–201, 2008.
S. Park and N. W. Park, Effects of class purity of training patch on classification performance of crop classification with convolutional neural network, Appl. Sci., vol. 10, no. 11, p. 3773, 2020.
413
Views
43
Downloads
0
Crossref
0
Web of Science
0
Scopus
0
CSCD
Altmetrics
The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).