Journal Home > Volume 5 , Issue 2
Big Data Mining and Analytics 2022, 5(2): 110-119 https://doi.org/10.26599/BDMA.2021.9020020
Open Access | Issue | Published: 25 January 2022

MAGAN: Unsupervised Low-Light Image Enhancement Guided by Mixed-Attention

Show Author's Information Renjun Wang,  Bin Jiang( ),  Chao Yang,  Qiao Li,  Bolin Zhang
College of Computer Science and Electronic Engineering, Hunan University, Changsha 410006, China
Keywords:
unsupervised learning, Generative Adversarial Network (GAN), low-light image enhancement, mixed-attention
Cite this article:
Wang R, Jiang B, Yang C, et al. MAGAN: Unsupervised Low-Light Image Enhancement Guided by Mixed-Attention. Big Data Mining and Analytics, 2022, 5(2): 110-119. https://doi.org/10.26599/BDMA.2021.9020020
Download citation
EndNote(RIS)
BibTeX

1401

Views

841

Downloads

Citations

22

Crossref

11

WoS

20

Scopus

0

CSCD

Abstract Full text About this article

Most learning-based low-light image enhancement methods typically suffer from two problems. First, they require a large amount of paired data for training, which are difficult to acquire in most cases. Second, in the process of enhancement, image noise is difficult to be removed and may even be amplified. In other words, performing denoising and illumination enhancement at the same time is difficult. As an alternative to supervised learning strategies that use a large amount of paired data, as presented in previous work, this paper presents an mixed-attention guided generative adversarial network called MAGAN for low-light image enhancement in a fully unsupervised fashion. We introduce a mixed-attention module layer, which can model the relationship between each pixel and feature of the image. In this way, our network can enhance a low-light image and remove its noise simultaneously. In addition, we conduct extensive experiments on paired and no-reference datasets to show the superiority of our method in enhancing low-light images.


menu
Abstract
Full text
Outline
About this article

MAGAN: Unsupervised Low-Light Image Enhancement Guided by Mixed-Attention

Show Author's information Renjun Wang, Bin Jiang( ), Chao Yang, Qiao Li, Bolin Zhang
College of Computer Science and Electronic Engineering, Hunan University, Changsha 410006, China

Abstract

Most learning-based low-light image enhancement methods typically suffer from two problems. First, they require a large amount of paired data for training, which are difficult to acquire in most cases. Second, in the process of enhancement, image noise is difficult to be removed and may even be amplified. In other words, performing denoising and illumination enhancement at the same time is difficult. As an alternative to supervised learning strategies that use a large amount of paired data, as presented in previous work, this paper presents an mixed-attention guided generative adversarial network called MAGAN for low-light image enhancement in a fully unsupervised fashion. We introduce a mixed-attention module layer, which can model the relationship between each pixel and feature of the image. In this way, our network can enhance a low-light image and remove its noise simultaneously. In addition, we conduct extensive experiments on paired and no-reference datasets to show the superiority of our method in enhancing low-light images.

Keywords: unsupervised learning, Generative Adversarial Network (GAN), low-light image enhancement, mixed-attention

References(46)

[1]
M. Abdullah-Al-Wadud, M. H. Kabir, M. A. A. Dewan, and O. Chae, A dynamic histogram equalization for image contrast enhancement, IEEE Trans. Consum. Electron., vol. 53, no. 2, pp. 593-600, 2007.
DOI Google Scholar
[2]
H. Ibrahim and N. S. P. Kong, Brightness preserving dynamic histogram equalization for image contrast enhancement, IEEE Trans. Consum. Electron., vol. 53, no. 4, pp. 1752-1758, 2007.
DOI Google Scholar
[3]
C. Lee, C. Lee, and C. S. Kim, Contrast enhancement based on layered difference representation of 2D histograms, IEEE Trans. Image Process., vol. 22, no. 12, pp. 5372-5384, 2013.
DOI Google Scholar
[4]
K. Nakai, Y. Hoshi, and A. Taguchi, Color image contrast enhacement method based on differential intensity/saturation gray-levels histograms, in Proc. of the 2013 Int. Symp. Intelligent Signal Processing and Communication Systems, Naha, Japan, 2013, pp. 445-449.
DOI Google Scholar
[5]
S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, T. Greer, B. ter Haar Romeny, J. B. Zimmerman, and K. Zuiderveld, Adaptive histogram equalization and its variations, Comput. Vis. Graph. Image Process., vol. 39, no. 3, pp. 355-368, 1987.
DOI Google Scholar
[6]
X. J. Guo, Y. Li, and H. B. Ling, LIME: Low-light image enhancement via illumination map estimation, IEEE Trans. Image Process., vol. 26, no. 2, pp. 982-993, 2017.
DOI Google Scholar
[7]
D. J. Jobson, Z. Rahman, and G. A. Woodell, A multiscale retinex for bridging the gap between color images and the human observation of scenes, IEEE Trans. Image Process., vol. 6, no. 7, pp. 965-976, 1997.
DOI Google Scholar
[8]
M. D. Li, J. Y. Liu, W. H. Yang, X. Y. Sun, and Z. M. Guo, Structure-revealing low-light image enhancement via robust retinex model, IEEE Trans. Image Process., vol. 27, no. 6, pp. 2828-2841, 2018.
DOI Google Scholar
[9]
S. Park, S. Yu, B. Moon, S. Ko, and J. Paik, Low-light image enhancement using variational optimization-based retinex model, IEEE Trans. Consum. Electron., vol. 63, no. 2, pp. 178-184, 2017.
DOI Google Scholar
[10]
X. T. Ren, W. H. Yang, W. H. Cheng, and J. Y. Liu, LR3M: Robust low-light enhancement via low-rank regularized retinex model, IEEE Trans. Image Process., vol. 29, pp. 5862-5876, 2020.
DOI Google Scholar
[11]
S. H. Wang, J. Zheng, H. M. Hu, and B. Li, Naturalness preserved enhancement algorithm for non-uniform illumination images, IEEE Trans. Image Process., vol. 22, no. 9, pp. 3538-3548, 2013.
DOI Google Scholar
[12]
Z. Q. Ying, G. Li, and W. Gao, A bio-inspired multi-exposure fusion framework for low-light image enhancement, arXiv preprint arXiv: 1711.00591, 2017.
Google Scholar
[13]
Y. S. Chen, Y. C. Wang, M. H. Kao, and Y. Y. Chuang, Deep photo enhancer: Unpaired learning for image enhancement from photographs with GANs, in Proc. of the the 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 6306-6314.
DOI Google Scholar
[14]
A. Ignatov, N. Kobyshev, R. Timofte, and K. Vanhoey, DSLR-quality photos on mobile devices with deep convolutional networks, in Proc. of the 2017 IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 3297-3305.
DOI Google Scholar
[15]
A. Ignatov, N. Kobyshev, R. Timofte, K. Vanhoey, and L. Van Gool, WESPE: Weakly supervised photo enhancer for digital cameras, in Proc. of the 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA, 2018, pp. 691-700.
DOI Google Scholar
[16]
R. X. Wang, Q. Zhang, C. W. Fu, X. Y. Shen, W. S. Zheng, and J. Y. Jia, Underexposed photo enhancement using deep illumination estimation, in Proc. of the 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019, pp. 6842-6850.
DOI Google Scholar
[17]
K. G. Lore, A. Akintayo, and S. Sarkar, LLNet: A deep autoencoder approach to natural low-light image enhancement, Pattern Recogn., vol. 61, pp. 650-662, 2017.
DOI Google Scholar
[18]
K. Lu and L. H. Zhang, TBEFN: A two-branch exposure-fusion network for low-light image enhancement, IEEE Trans. Multimed., vol. 23, pp. 4093-4105, 2020.
DOI Google Scholar
[19]
F. F. Lv, Y. Li, and F. Lu, Attention-guided low-light image enhancement, arXiv preprint arXiv: 1908.00682, 2019.
Google Scholar
[20]
F. F. Lv, F. Lu, J. H. Wu, and C. Lim, MBLLEN: Low-light image/video enhancement using CNNs, in British Machine Vision Conf. (BMVC), Northumbria, UK, 2018, p. 220.
Google Scholar
[21]
W. Q. Ren, S. F. Liu, L. Ma, Q. Q. Xu, X. Y. Xu, X. C. Cao, J. P. Du, and M. H. Yang, Low-light image enhancement via a deep hybrid network, IEEE Trans. Image Process., vol. 28, no. 9, pp. 4364-4375, 2019.
DOI Google Scholar
[22]
Y. Wang, Y. Cao, Z. J. Zha, J. Zhang, Z. W. Xiong, W. Zhang, and F. Wu, Progressive retinex: Mutually reinforced illumination-noise perception network for low-light image enhancement, in Proc. 27th ACM Int. Conf. Multimedia, Nice, France, 2019, pp. 2015-2023.
DOI Google Scholar
[23]
C. Wei, W. J. Wang, W. H. Yang, and J. Y. Liu, Deep retinex decomposition for low-light enhancement, arXiv preprint arXiv: 1808.04560, 2018.
Google Scholar
[24]
K. Xu, X. Yang, B. C. Yin, and R. W. H. Lau, Learning to restore low-light images via decomposition-and-enhancement, in Proc. of the 2020 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Seattle, WA, USA, 2020, pp. 2278-2287.
DOI Google Scholar
[25]
Y. H. Zhang, J. W. Zhang, and X. J. Guo, Kindling the darkness: A practical low-light image enhancer, in Proc. 27th ACM Int. Conf. Multimedia, Nice, France, 2019, pp. 1632-1640.
DOI Google Scholar
[26]
M. F. Zhu, P. B. Pan, W. Chen, and Y. Yang, EEMEFN: Low-light image enhancement via edge-enhanced multi-exposure fusion network, Proc. AAAI Conf. Artif. Intell., vol. 34, no. 7, pp. 13106-13113, 2020.
DOI Google Scholar
[27]
Y. F. Jiang, X. Y. Gong, D. Liu, Y. Cheng, C. Fang, X. H. Shen, J. C. Yang, P. Zhou, and Z. Y. Wang, EnlightenGAN: Deep light enhancement without paired supervision, IEEE Trans. Image Process., vol. 30, pp. 2340-2349, 2021.
DOI Google Scholar
[28]
W. Xiong, D. Liu, X. H. Shen, C. Fang, and J. B. Luo, Unsupervised real-world low-light image enhancement with decoupled networks, arXiv preprint arXiv: 2005.02818, 2020.
Google Scholar
[29]
S. Anwar and N. Barnes, Real image denoising with feature attention, in Proc. of the 2019 IEEE/CVF Int. Conf. Computer Vision, Seoul, Republic of Korea, 2019, pp. 3155-3164.
DOI Google Scholar
[30]
J. Hu, L. Shen, and G. Sun, Squeeze-and-excitation networks, in Proc. of the 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 7132-7141.
DOI Google Scholar
[31]
X. L. Wang, R. Girshick, A. Gupta, and K. M. He, Non-local neural networks, in Proc. of the 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 7794-7803.
DOI Google Scholar
[32]
S. Woo, J. Park, J. Y. Lee, and I. S. Kweon, CBAM: Convolutional block attention module, in Proc. 15th European Conf. Computer Vision, Munich, Germany, 2018, pp. 3-19.
DOI Google Scholar
[33]
J. Y. Zhu, T. Park, P. Isola, and A. A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, in Proc. of the 2017 IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 2242-2251.
DOI Google Scholar
[34]
M. H. Fan, W. J. Wang, W. H. Yang, and J. Y. Liu, Integrating semantic segmentation and retinex model for low-light image enhancement, in Proc. 28th ACM Int. Conf. Multimedia, Seattle, WA, USA, 2020, pp. 2317-2325.
DOI Google Scholar
[35]
K. C. K. Chan, X. T. Wang, X. Y. Xu, J. W. Gu, and C. C. Loy, GLEAN: Generative latent bank for large-factor image super-resolution, in Proc. of the 2021 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Nashville, TN, USA, 2021, pp. 14240-14249.
DOI Google Scholar
[36]
X. T. Wang, K. Yu, C. Dong, and C. C. Loy, Recovering realistic texture in image super-resolution by deep spatial feature transform, in Proc. of the 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 606-615.
DOI Google Scholar
[37]
X. M. Li, C. F. Chen, S. C. Zhou, X. H. Lin, W. M. Zuo, and L. Zhang, Blind face restoration via deep multi-scale component dictionaries, in Proc. 16th European Conf. Computer Vision, Glasgow, UK, 2020, pp. 399-415.
DOI Google Scholar
[38]
X. Lin, Z. J. Wang, L. Z. Ma, and X. B. Wu, Saliency detection via multi-scale global cues, IEEE Trans. Multimed., vol. 21, no. 7, pp. 1646-1659, 2019.
DOI Google Scholar
[39]
Z. J. Wang, L. Z. Ma, X. Lin, and H. Zhong, Saliency detection via multi-center convex hull prior, in Proc. of the 2018 IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), Calgary, Canada, 2018, pp. 1867-1871.
DOI Google Scholar
[40]
Z. X. Wang, Z. Quan, Z. J. Wang, X. J. Hu, and Y. Y. Chen, Text to image synthesis with bidirectional generative adversarial network, in Proc. of the 2020 IEEE Int. Conf. Multimedia and Expo (ICME), London, UK, 2020, pp. 1-6.
DOI Google Scholar
[41]
W. Liu, Z. J. Wang, B. Yao, and J. Yin, Geo-ALM: Poi recommendation by fusing geographical information and adversarial learning mechanism, in Proc. 28th Int. Joint Conf. Artificial Intelligence, Macao, China, 2019, pp. 1807-1813.
DOI Google Scholar
[42]
X. Lin, L. Z. Ma, B. Sheng, Z. J. Wang, and W. S. Chen, Utilizing two-phase processing with FBLS for single image deraining, IEEE Trans. Multimed., vol. 23, pp. 664-676, 2020.
DOI Google Scholar
[43]
V. Mnih, N. Heess, A. Graves, and K. Kavukcuoglu, Recurrent models of visual attention, arXiv preprint arXiv: 1406.6247, 2014.
Google Scholar
[44]
Q. L. Wang, B. G. Wu, P. F. Zhu, P. H. Li, W. M. Zuo, and Q. H. Hu, ECA-Net: Efficient channel attention for deep convolutional neural networks, in Proc. of the 2020 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 11531-11539.
DOI Google Scholar
[45]
W. Z. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. H. Wang, Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, in Proc. of the 2016 IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 1874-1883.
DOI Google Scholar
[46]
X. D. Mao, Q. Li, H. R. Xie, R. Y. K. Lau, Z. Wang, and S. P. Smolley, Least squares generative adversarial networks, in Proc. of the 2017 IEEE Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 2813-2821.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 15 October 2021
Accepted: 03 November 2021
Published: 25 January 2022
Issue date: June 2022

Copyright

© The author(s) 2022.

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (No. 62072169) and Changsha Science and Technology Research Plan (No. KQ2004005).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return