Journal Home > Volume 7 , Issue 4
Computational Visual Media 2021, 7(4): 483-497 https://doi.org/10.1007/s41095-021-0220-1
Research Article | Open Access | Issue | Published: 17 May 2021

Image smoothing based on global sparsity decomposition and a variable parameter

Show Author's Information Xiang Ma1,  Xuemei Li1,2,  Yuanfeng Zhou1,  Caiming Zhang1,2,3( )
Shandong University, Jinan 250101, China
Shandong Co-Innovation Center of Future Intelligent Computing, Yantai 264025, China
Digital Media Technology Key Lab of Shandong Province, Jinan 250014, China
Keywords:
image smoothing, texture removal, global sparse decomposition, Bessel method
Cite this article:
Ma X, Li X, Zhou Y, et al. Image smoothing based on global sparsity decomposition and a variable parameter. Computational Visual Media, 2021, 7(4): 483-497. https://doi.org/10.1007/s41095-021-0220-1
Download citation
EndNote(RIS)
BibTeX

625

Views

42

Downloads

Citations

15

Crossref

15

WoS

14

Scopus

0

CSCD

Abstract Full text About this article

Smoothing images, especially with rich texture, is an important problem in computer vision. Obtaining an ideal result is difficult due to complexity, irregularity, and anisotropicity of the texture. Besides, some properties are shared by the texture and the structure in an image. It is a hard compromise to retain structure and simultaneously remove texture. To create an ideal algorithm for image smoothing, we face three problems. For images with rich textures, the smoothing effect should be enhanced. We should overcome inconsistency of smoothing results in different parts of the image. It is necessary to create a method to evaluate the smoothing effect. We apply texture pre-removal based on global sparse decomposition with a variable smoothing parameter to solve the first two problems. A parametric surface constructed by an improved Bessel method is used to determine the smoothing parameter. Three evaluation measures: edge integrity rate, texture removal rate, and gradient value distribution are proposed to cope with the third problem. We use the alternating direction method of multipliers to complete the whole algorithm and obtain the results. Experiments show that our algorithm is better than existing algorithms both visually and quantitatively. We also demonstrate our method’s ability in other applications such as clip-art compression artifact removal and content-aware image manipulation.


menu
Abstract
Full text
Outline
About this article

Image smoothing based on global sparsity decomposition and a variable parameter

Show Author's information Xiang Ma1, Xuemei Li1,2, Yuanfeng Zhou1, Caiming Zhang1,2,3( )
Shandong University, Jinan 250101, China
Shandong Co-Innovation Center of Future Intelligent Computing, Yantai 264025, China
Digital Media Technology Key Lab of Shandong Province, Jinan 250014, China

Abstract

Smoothing images, especially with rich texture, is an important problem in computer vision. Obtaining an ideal result is difficult due to complexity, irregularity, and anisotropicity of the texture. Besides, some properties are shared by the texture and the structure in an image. It is a hard compromise to retain structure and simultaneously remove texture. To create an ideal algorithm for image smoothing, we face three problems. For images with rich textures, the smoothing effect should be enhanced. We should overcome inconsistency of smoothing results in different parts of the image. It is necessary to create a method to evaluate the smoothing effect. We apply texture pre-removal based on global sparse decomposition with a variable smoothing parameter to solve the first two problems. A parametric surface constructed by an improved Bessel method is used to determine the smoothing parameter. Three evaluation measures: edge integrity rate, texture removal rate, and gradient value distribution are proposed to cope with the third problem. We use the alternating direction method of multipliers to complete the whole algorithm and obtain the results. Experiments show that our algorithm is better than existing algorithms both visually and quantitatively. We also demonstrate our method’s ability in other applications such as clip-art compression artifact removal and content-aware image manipulation.

Keywords: image smoothing, texture removal, global sparse decomposition, Bessel method

References(34)

[1]
Sun, Y. J.; Schaefer, S.; Wang, W. P. Image structure retrieval via L minimization. IEEE Transactions on Visualization and Computer Graphics Vol. 24, No. 7, 2129-2139, 2018.
DOI Google Scholar
[2]
Tomasi, C.; Manduchi, R. Bilateral filtering for gray and color images. In: Proceedings of the International Conference on Computer Vision, 839-846, 1998.
[3]
Chen, J. W.; Paris, S.; Durand, F. Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics Vol. 26, No. 3, 103, 2007.
DOI Google Scholar
[4]
Weiss, B. Fast median and bilateral filtering. ACM Transactions on Graphics Vol. 25, No. 3, 519-526, 2006.
DOI Google Scholar
[5]
Cho, H.; Lee, H.; Kang, H.; Lee, S. Bilateral texture filtering. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 128, 2014.
DOI Google Scholar
[6]
Bao, L. C.; Song, Y. B.; Yang, Q. X.; Yuan, H.; Wang, G. Tree filtering: Efficient structure-preserving smoothing with a minimum spanning tree. IEEE Transactions on Image Processing Vol. 23, No. 2, 555-569, 2014.
DOI Google Scholar
[7]
Perona, P.; Malik, J. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 12, No. 7, 629-639, 1990.
DOI Google Scholar
[8]
He, K. M.; Sun, J.; Tang, X. O. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 6, 1397-1409, 2013.
DOI Google Scholar
[9]
Subr, K.; Soler, C.; Durand, F. Edge-preserving multiscale image decomposition based on local extrema. ACM Transactions on Graphics Vol. 28, No. 5, Article No. 147, 2009.
DOI Google Scholar
[10]
Farbman, Z.; Fattal, R.; Lischinski, D.; Szeliski, R. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics Vol. 27, No. 3, Article No. 67, 2008.
DOI Google Scholar
[11]
Lindeberg, T. Scale-space theory: A basic tool for analyzing structures at different scales. Journal of Applied Statistics Vol. 21, Nos. 1-2, 225-270, 1994.
DOI Google Scholar
[12]
Mikolajczyk, K.; Schmid, C. An affine invariant interest point detector. In: Proceedings of the 7th European Conference on Computer Vision-Part I, 128-142, 2002.
DOI
[13]
Cai, B. L.; Xing, X. F.; Xu, X. M. Edge/structure preserving smoothing via relativity-of-Gaussian. In: Proceedings of the IEEE International Conference on Image Processing, 250-254, 2017.
DOI
[14]
Rudin, L. I.; Osher, S.; Fatemi, E. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena Vol. 60, Nos. 1-4, 259-268, 1992.
DOI Google Scholar
[15]
Xu, L.; Lu, C. W.; Xu, Y.; Jia, J. Y. Imagesmoothing via L gradient minimization. ACM Transactions on Graphics Vol. 30, No. 6, Article No. 174, 2011.
DOI Google Scholar
[16]
Ono, S. L gradient projection. IEEE Transactions on Image Processing Vol. 26, No. 4, 1554-1564, 2017.
DOI Google Scholar
[17]
Ono, S. Edge-preserving filtering by projection onto L gradient constraint. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 1492-1496, 2017.
DOI
[18]
Xu, L.; Yan, Q.; Xia, Y.; Jia, J. Y. Structure extraction from texture via relative total variation. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 139, 2012.
DOI Google Scholar
[19]
Chen, Q. F.; Xu, J.; Koltun, V. Fast image processing with fully-convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, 2516-2525, 2017.
DOI
[20]
Xu, L.; Ren, J.; Yan, Q.; Liao, R.; Jia, J. Deep edge-aware filters. In: Proceedings of the 32nd International Conference on Machine Learning, Vol. 37, 1669-1678, 2015.
[21]
Kim, Y.; Ham, B.; Do, M. N.; Sohn, K. Structure-texture image decomposition using deep variational priors. IEEE Transactions on Image Processing Vol. 28, No. 6, 2692-2704, 2019.
DOI Google Scholar
[22]
Zhao, M.; Zhang, H. X.; de Sun, J. A novel image retrieval method based on multi-trend structure descriptor. Journal of Visual Communication and Image Representation Vol. 38, 73-81, 2016.
DOI Google Scholar
[23]
Zhang, F.; Li, J. J.; Liu, P. Q.; Fan, H. Computing knots by quadratic and cubic polynomial curves. Computational Visual Media Vol. 6, No. 4, 417-430, 2020.
DOI Google Scholar
[24]
Liu, X. X.; Zhang, Y. F.; Bao, F. X.; Shao, K.; Sun, Z. Y.; Zhang, C. M. Kernel-blending connection approximated by a neural network for image classification. Computational Visual Media Vol. 6, No. 4, 467-476, 2020.
DOI Google Scholar
[25]
Wang, Y. L.; Yang, J. F.; Yin, W. T.; Zhang, Y. A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences Vol. 1, No. 3, 248-272, 2008.
DOI Google Scholar
[26]
Gu, S. H.; Xie, Q.; Meng, D. Y.; Zuo, W. M.; Feng, X. C.; Zhang, L. Weighted nuclear norm minimization and its applications to low level vision. International Journal of Computer Vision Vol. 121, No. 2, 183-208, 2017.
DOI Google Scholar
[27]
Eckstein, J.; Bertsekas, D. P. On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Mathematical Programming Vol. 55, Nos. 1-3, 293-318, 1992.
DOI Google Scholar
[28]
Boyd, S.; Parikh, N.; Chu, E.; Peleato, B.; Eckstein, J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends®in Machine Learning Vol. 3, No. 1, 1-122, 2011.
DOI Google Scholar
[29]
Zhang, R.; Kwok, J. Asynchronous distributed ADMM for consensus optimization. In: Proceedings of the 31st International Conference on International Conference on Machine Learning, Vol. 32, II-1701-II-1709, 2014.
[30]
Sun, D. L.; Févotte, C. Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 6201-6205, 2014.
DOI
[31]
Zhang, M. L.; Desrosiers, C. High-quality image restoration using low-rank patch regularization and global structure sparsity. IEEE Transactions on Image Processing Vol. 28, No. 2, 868-879, 2019.
DOI Google Scholar
[32]
Ma, G. H.; Zhang, M. L.; Li, X. M.; Zhang, C. M. Image smoothing based on image decomposition and sparse high frequency gradient. Journal of Computer Science and Technology Vol. 33, No. 3, 502-510, 2018.
DOI Google Scholar
[33]
Zhu, F. D.; Liang, Z. T.; Jia, X. X.; Zhang, L.; Yu, Y. Z. A benchmark for edge-preserving image smoothing. IEEE Transactions on Image Processing Vol. 28, No. 7,3556-3570, 2019.
DOI Google Scholar
[34]
Jian, M. W.; Zhang, W. Y.; Yu, H.; Cui, C. R.; Nie, X. S.; Zhang, H. X.; Yin, Y. Saliency detection based on directional patches extraction and principal local color contrast. Journal of Visual Communication and Image Representation Vol. 57, 1-11, 2018.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 25 January 2021
Accepted: 01 March 2021
Published: 17 May 2021
Issue date: December 2021

Copyright

© The Author(s) 2021

Acknowledgements

This work was supported by NSFC Joint Fund with Zhejiang Integration of Informatization and Industrialization under Key Project (U1609218).

Rights and permissions

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.

Return