AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home CAAI Artificial Intelligence Research Article
PDF (4.6 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Self-Sparse Generative Adversarial Networks

Wenliang Qian1,2, Yang Xu1,2, Wangmeng Zuo3, Hui Li1,2( )
Labortoray of Artificial Intelligence, Harbin Institute of Technology, Harbin 150001, China
School of Civil Engineering, Harbin Institute of Technology, Harbin 150090, China
School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China
Show Author Information

Abstract

Generative adversarial networks (GANs) are an unsupervised generative model that learns data distribution through adversarial training. However, recent experiments indicated that GANs are difficult to train due to the requirement of optimization in the high dimensional parameter space and the zero gradient problem. In this work, we propose a self-sparse generative adversarial network (Self-Sparse GAN) that reduces the parameter space and alleviates the zero gradient problem. In the Self-Sparse GAN, we design a self-adaptive sparse transform module (SASTM) comprising the sparsity decomposition and feature-map recombination, which can be applied on multi-channel feature maps to obtain sparse feature maps. The key idea of Self-Sparse GAN is to add the SASTM following every deconvolution layer in the generator, which can adaptively reduce the parameter space by utilizing the sparsity in multi-channel feature maps. We theoretically prove that the SASTM can not only reduce the search space of the convolution kernel weight of the generator but also alleviate the zero gradient problem by maintaining meaningful features in the batch normalization layer and driving the weight of deconvolution layers away from being negative. The experimental results show that our method achieves the best Fréchet inception distance (FID) scores for image generation compared with Wasserstein GAN with gradient penalty (WGAN-GP) on MNIST, Fashion-MNIST, CIFAR-10, STL-10, mini-ImageNet, CELEBA-HQ, and LSUN bedrooms datasets, and the relative decrease of FID is 4.76%–21.84%. Meanwhile, an architectural sketch dataset (Sketch) is also used to validate the superiority of the proposed method.

Keywords

generative adversarial networksself-adaptive sparse transform moduleself-sparse generative adversarial network (Self-Sparse GAN)

References

1
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, Generative adversarial nets, in Proc. 28th Annu. Conf. Neural Information Processing Systems, Montreal, Canada, 2014, pp. 2672–2680.
2
A. Radford, L. Metz, and S. Chintala, Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv preprint arXiv: 1511.06434, 2015.
3
L. Mescheder, A. Geiger, and S. Nowozin, Which Training Methods for GANs do actually converge, in Proc. 35th Int. Conf. Machine Learning, Stockholm, Sweden, 2018, pp. 3481–3490.
4
T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, Improved techniques for training GANs, in Proc. 30th Annu. Conf. Neural Information Processing Systems, Barcelona, Spain, 2016, pp. 2226–2234.
5
M. Arjovsky and L. Bottou, Towards principled methods for training generative adversarial networks, in Proc. 5th Int. Conf. Learning Representations, Toulon, France, 2017.
6
S. Jenni and P. Favaro, On stabilizing generative adversarial training with noise, in Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019, pp. 12137–12145.
Crossref
7
T. Karras, T. Aila, S. Laine, and J. Lehtinen, Progressive growing of GANs for improved quality, stability, and variation, arXiv preprint arXiv: 1710.10196, 2017.
8
H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena, Self-attention generative adversarial networks, in Proc. 36th Int. Conf. Machine Learning, Long Beach, CA, USA, 2019, pp. 7354–7363.
9
M. Arjovsky, S. Chintala, and L. Bottou, Wasserstein GAN, arXiv preprint arXiv: 1701.07875, 2017.
10
X. Mao, Q. Li, H. Xie, R. Y. K. Lau, Z. Wang, and S. P. Smolley, Least squares generative adversarial networks, in Proc. 2017 IEEE Int. Conf. on Computer Vision, Venice, Italy, 2017, pp. 2813–2821.
Crossref
11
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville, Improved training of Wasserstein GANs, in Proc. 31st Annu. Conf. Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 5767–5777.
12
T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, Spectral normalization for generative adversarial networks, in Proc. 6th Int. Conf. Learning Representations, Vancouver, Canada, 2018.
13
B. Liu, M. Wang, H. Foroosh, M. Tappen, and M. Penksy, Sparse convolutional neural networks, in Proc. 2015 IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, USA, 2015, pp. 806–814.
14
C. Louizos, M. Welling, and D. P. Kingma, Learning sparse neural networks through L_0 regularization, in Proc. 6th Int. Conf. Learning Representations, Vancouver, Canada, 2018.
15

L. Deng, The MNIST database of handwritten digit images for machine learning research [best of the web], IEEE Signal Process. Mag., vol. 29, no. 6, pp. 141–142, 2012.
Crossref Google Scholar
16
H. Xiao, K. Rasul, and R. Vollgraf, Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms, arXiv preprint arXiv: 1708.07747, 2017.
17
A. Krizhevsky, Learning multiple layers of features from tiny images, https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf, 2009.
18
A. Coates, A. Ng, and H. Lee, An analysis of single-layer networks in unsupervised feature learning, in Proc. 14th Int. Conf. Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, 2011, 215–223.
19
O. Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and D. Wierstra. Matching networks for one shot learning, in Proc. 30th Annu. Conf. Neural Information Processing Systems, Barcelona, Spain, 2016, pp. 3630–3638.
20
F. Yu, Y. Zhang, S. Song, A. Seff, and J. Xiao, LSUN: Construction of a large-scale image dataset using deep learning with humans in the Loop, arXiv preprint arXiv: 1506.03365, 2015
21
M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, GANs trained by a two time-scale update rule converge to a local Nash equilibrium, in Proc. 31st Annu. Conf. Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 6626–6637.
22
A. Brock, J. Donahue, and K. Simonyan, Large scale GAN training for high fidelity natural image synthesis, arXiv preprint arXiv: 1809.11096, 2018.
23
Y. Zhou and T. L. Berg, Learning temporal transformations from time-lapse videos, in Proc. 14th Eur. Conf. Computer Vision, Amsterdam, The Netherlands, 2016, pp. 262–277.
Crossref
24
P. Isola, J. Y. Zhu, T. Zhou, and Alexei A. Efros, Image-to-image translation with conditional adversarial networks, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 5967–5976.
Crossref
25
O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, and J. Matas, Deblurgan: Blind motion deblurring using conditional adversarial networks, in Proc. 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 8183–8192.
Crossref
26
M. Huh, S. H. Sun, and N. Zhang, Feedback adversarial learning: Spatial feedback for improving generative adversarial networks, in Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019, pp. 1476–1485.
Crossref
27
T. Karras, S. Laine, and T. Aila, A style-based generator architecture for generative adversarial networks, in Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019, pp. 4396–4405.
Crossref
28
T. Chen, M. Lucic, N. Houlsby, and S. Gelly, On self modulation for generative adversarial networks, arXiv preprint arXiv: 1810.01365, 2018.
29
S. Mahdizadehaghdam, A. Panahi, and H. Krim, Sparse generative adversarial network, in Proc. 2019 IEEE/CVF Int. Conf. Computer Vision Workshop, Seoul, Republic of Korea, 2019, pp. 3063-3071.
Crossref
30

F. Liu, L. Jiao, and X. Tang, Task-oriented GAN for PolSAR image classification and clustering, IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 9, pp. 2707–2719, 2019.
Crossref Google Scholar
31
A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet classification with deep convolutional neural networks, in Proc. 26th Annu. Conf. Neural Information Processing Systems, Lake Tahoe, NV, USA, 2012, pp. 1106–1114.
32
K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv: 1409.1556, 2014.
33

R. Shang, W. Zhang, M. Lu, L. Jiao, and Y. Li, Feature selection based on non-negative spectral feature learning and adaptive rank constraint, Knowl. -Based Syst., vol. 236, p. 107749, 2022.
Crossref Google Scholar
34

R. Shang, X. Zhang, J. Feng, Y. Li, and L. Jiao, Sparse and low-dimensional representation with maximum entropy adaptive graph for feature selection, Neurocomputing, vol. 485, pp. 57–73, 2022.
Crossref Google Scholar
35
J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang and, H. Lu, Dual attention network for scene segmentation, in Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019, pp. 3141–3149.
Crossref
36
Y. Wang, Z. Chen, F. Wu, and G. Wang, Person re-identification with cascaded pairwise convolutions, in Proc. 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2019, pp. 1470-1478.
Crossref
37
D. P. Kingma and J. Ba, Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
38

W. Qian, Y. Xu, and H. Li, A self-sparse generative adversarial network for autonomous early-stage design of architectural sketches, Comput. -Aided Civ. Infrastruct. Eng., vol. 37, no. 5, pp. 612–628, 2021.
Crossref Google Scholar
CAAI Artificial Intelligence Research
Volume 1 Issue 1,
September 2022
Pages 68-78
DOI: 10.26599/AIR.2022.9150005
Cite this article:
Qian W, Xu Y, Zuo W, et al. Self-Sparse Generative Adversarial Networks. CAAI Artificial Intelligence Research, 2022, 1(1): 68-78. https://doi.org/10.26599/AIR.2022.9150005

2031

Views

212

Downloads

4

Crossref

Altmetrics
Received: 26 May 2022
Revised: 07 August 2022
Accepted: 12 August 2022
Published: 28 August 2022
© The author(s) 2022

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return