Journal Home > Volume 28 , Issue 2

Recently, several well-performing deep convolutional neural networks were proposed for remote sensing image super-resolution (SR). However, these methods rarely consider that remote sensing images are corruptible by additional noise, blurring, and other factors. Therefore, to eliminate the interference of these factors, especially the noise, we propose a novel information purification network (IPN) for remote sensing image SR. The proposed information purification block (IPB) can process channel-wise features differently by channel separation and rescale spatial-wise features adaptively through the proposed multi-scale spatial attention mechanism. We further design an information group to explore a more powerful expressive combination of IPBs. Moreover, long and short skip connections can transmit abundant low-frequency information, making IPBs pay more attention to high-frequency information. We mix the images under various degradation models as training data in the training phase. In this way, the network can directly reconstruct various degraded images. Experiments on AID and UC Merced Land-Use datasets under multiple degradation models demonstrate that the proposed IPN performs better than state-of-the-art methods.


menu
Abstract
Full text
Outline
About this article

Information Purification Network for Remote Sensing Image Super-Resolution

Show Author's information Zheyuan Wang1Liangliang Li2Linxin Xing1Jiawen Wang3Kaipeng Sun3Hongbing Ma2 ( )
College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China
Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
Shanghai Institute of Satellite Engineering, Shanghai 201109, China

Abstract

Recently, several well-performing deep convolutional neural networks were proposed for remote sensing image super-resolution (SR). However, these methods rarely consider that remote sensing images are corruptible by additional noise, blurring, and other factors. Therefore, to eliminate the interference of these factors, especially the noise, we propose a novel information purification network (IPN) for remote sensing image SR. The proposed information purification block (IPB) can process channel-wise features differently by channel separation and rescale spatial-wise features adaptively through the proposed multi-scale spatial attention mechanism. We further design an information group to explore a more powerful expressive combination of IPBs. Moreover, long and short skip connections can transmit abundant low-frequency information, making IPBs pay more attention to high-frequency information. We mix the images under various degradation models as training data in the training phase. In this way, the network can directly reconstruct various degraded images. Experiments on AID and UC Merced Land-Use datasets under multiple degradation models demonstrate that the proposed IPN performs better than state-of-the-art methods.

Keywords: deep convolutional neural networks, remote sensing image, super-resolution, information purification network

References(36)

[1]
L. Zhang and X. Wu, An edge-guided image interpolation algorithm via directional filtering and data fusion, IEEE Trans. Image Process., vol. 15, no. 8, pp. 2226–2238, 2006.
[2]
K. Zhang, X. Gao, D. Tao, and X. Li, Single image super-resolution with non-local means and steering kernel regression, IEEE Trans. Image Process., vol. 21, no. 11, pp. 4544–4556, 2012.
[3]
R. Timofte, V. De, and L. Van Gool, Anchored neighborhood regression for fast example-based super-resolution, in Proc. 2013 IEEE Int. Conf. Computer Vision, Sydney, Australia, 2013, pp. 1920–1927.
[4]
W. He, N. Yokoya, L. Yuan, and Q. Zhao, Remote sensing image reconstruction using tensor ring completion and total variation, IEEE Trans. Geosci. Remote Sens., vol. 57, no. 11, pp. 8998–9009, 2019.
[5]
Z. Shao, L. Wang, Z. Wang, and J. Deng, Remote sensing image super-resolution using sparse representation and coupled sparse autoencoder, IEEE J. Selected Topics Appl. Earth Obs. Remote Sens., vol. 12, no. 8, pp. 2663–2674, 2019.
[6]
K. Jiang, Z. Wang, P. Yi, G. Wang, T. Lu, and J. Jiang, Edge-enhanced GAN for remote sensing image superresolution, IEEE Trans. Geosci. Remote Sens., vol. 57, no. 8, pp. 5799–5812, 2019.
[7]
W. Xu, G. Xu, Y. Wang, X. Sun, D. Lin, and Y. Wu, High quality remote sensing image super-resolution using deep memory connected network, in Proc. 2018 IEEE Int. Geoscience and Remote Sensing Symp., Valencia, Spain, 2018, pp. 8889–8892.
[8]
C. Dong, C. C. Loy, K. He, and X. Tang, Learning a deep convolutional network for image super-resolution, in Proc. 13th European Conf. Computer Vision, Zurich, Switzerland, 2014, pp. 184–199.
[9]
T. Tong, G. Li, X. Liu, and Q. Gao, Image super-resolution using dense skip connections, in Proc. 2017 IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 4809–4817.
[10]
J. Kim, J. K. Lee, and K. M. Lee, Deeply-recursive convolutional network for image super-resolution, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 1637–1645.
[11]
S. Liu, R. Gang, C. Li, and R. Song, Adaptive deep residual network for single image super-resolution, Comp. Visual Media, vol. 5, no. 4, pp. 391–401, 2019.
[12]
M. Shen, P. Yu, R. Wang, J. Yang, and L. Xue, Image super-resolution via multi-path recursive convolutional network, (in Chinese), Opto-Electron. Eng., vol. 46, no. 11, p. 180489, 2019.
[13]
S. Lei, Z. Shi, and Z. Zou, Super-resolution for remote sensing images via local–global combined network, IEEE Geosci. Remote Sens. Lett., vol. 14, no. 8, pp. 1243–1247, 2017.
[14]
X. Dong, X. Sun, X. Jia, Z. Xi, L. Gao, and B. Zhang, Remote sensing image super-resolution using novel dense-sampling networks, IEEE Trans. Geosci. Remote Sens., vol. 59, no. 2, pp. 1618–1633, 2021.
[15]
Z. Pan, W. Ma, J. Guo, and B. Lei, Super-resolution of single remote sensing image based on residual dense backprojection networks, IEEE Trans. Geosci. Remote Sens., vol. 57, no. 10, pp. 7918–7933, 2019.
[16]
X. Dong, L. Wang, X. Sun, X. Jia, L. Gao, and B. Zhang, Remote sensing image super-resolution using second-order multi-scale networks, IEEE Trans. Geosci. Remote Sens., vol. 59, no. 4, pp. 3473–3485, 2021.
[17]
S. Wang, T. Zhou, Y. Lu, and H. Di, Contextual transformation network for lightweight remote-sensing image super-resolution, IEEE Trans. Geosci. Remote Sens., vol. 60, p. 5615313, 2022.
[18]
W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang, Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 1874–1883.
[19]
J. Kim, J. K. Lee, and K. M. Lee, Accurate image super-resolution using very deep convolutional networks, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 1646–1654.
[20]
B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, Enhanced deep residual networks for single image super-resolution, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, 2017, pp. 1132–1140.
[21]
Z. Hui, X. Wang, and X. Gao, Fast and accurate single image super-resolution via information distillation network, in Proc. 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 723–731.
[22]
Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, Residual dense network for image super-resolution, in Proc. 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 2472–2481.
[23]
Y. Qiu, R. Wang, D. Tao, and J. Cheng, Embedded block residual network: A recursive restoration model for single-image super-resolution, in Proc. 2019 IEEE/CVF Int. Conf. Computer Vision (ICCV), Seoul, Republic of Korea, 2019, pp. 4179–4188.
[24]
J. Liu, W. Zhang, Y. Tang, J. Tang, and G. Wu, Residual feature aggregation network for image super-resolution, in Proc. 2020 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 2356–2365.
[25]
J. Hu, L. Shen, S. Albanie, G. Sun, and E. Wu, Squeeze-and-excitation networks, IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 8, pp. 2011–2023, 2020.
[26]
S. Woo, J. Park, J. Y. Lee, and I. S. Kweon, CBAM: Convolutional block attention module, in Proc. 15th European Conf. Computer Vision, Munich, Germany, 2018, pp. 3–19.
[27]
Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, Image super-resolution using very deep residual channel attention networks, in Proc. 15th European Conf. Computer Vision, Munich, Germany, 2018, pp. 294–310.
[28]
T. Dai, J. Cai, Y. Zhang, S. T. Xia, and L. Zhang, Second-order attention network for single image super-resolution, in Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 11057–11066.
[29]
Z. Hui, X. Gao, Y. Yang, and X. Wang, Lightweight image super-resolution with information multi-distillation network, in Proc. 27th ACM Int. Conf. Multimedia, Nice, France, 2019, pp. 2024–2032.
[30]
L. Courtrai, M. T. Pham, C. Friguet, and S. Lefèvre, Small object detection from remote sensing images with the help of object-focused super-resolution using wasserstein GANs, in Proc. 2020 IEEE Int. Geoscience and Remote Sensing Symp., Waikoloa, HI, USA, 2020, pp. 260–263.
[31]
A. Ma, J. Wang, Y. Zhong, and Z. Zheng, FactSeg: Foreground activation-driven small object semantic segmentation in large-scale remote sensing imagery, IEEE Trans. Geosci. Remote Sens., vol. 60, p. 5606216, 2021.
[32]
D. Zhang, J. Shao, X. Li, and H. T. Shen, Remote sensing image super-resolution via mixed high-order attention network, IEEE Trans. Geosci. Remote Sens., vol. 59, no. 6, pp. 5183–5196, 2021.
[33]
H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, Pyramid scene parsing network, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 6230–6239.
[34]
G. S. Xia, J. Hu, F. Hu, B. Shi, X. Bai, Y. Zhong, L. Zhang, and X. Lu, AID: A benchmark data set for performance evaluation of aerial scene classification, IEEE Trans. Geosci. Remote Sens., vol. 55, no. 7, pp. 3965–3981, 2017.
[35]
Y. Yang and S. Newsam, Bag-of-visual-words and spatial extensions for land-use classification, in Proc. 18th SIGSPATIAL Int. Conf. Advances in Geographic Information Systems, San Jose, CA, USA, 2010, pp. 270–279.
[36]
J. Liu, J. Tang, and G. Wu, Residual feature distillation network for lightweight image super-resolution, in Proc. European Conf. Computer Vision, Glasgow, UK, 2020, pp. 41–55.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 18 October 2021
Revised: 02 February 2022
Accepted: 07 February 2022
Published: 29 September 2022
Issue date: April 2023

Copyright

© The author(s) 2023.

Acknowledgements

This work was supported by the Shanghai Aerospace Science and Technology Innovation Fund (No. SAST2019-048) and the Cross-Media Intelligent Technology Project of Beijing National Research Center for Information Science and Technology (BNRist) (No. BNR2019TD01022).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return