Non-dominated sorting based multi-page photo collage

Yu Song; Fan Tang; Weiming Dong; Changsheng Xu

doi:10.1007/s41095-021-0221-0

AI Chat Paper

Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.

Chat more with AI

| Sign up

Browse by Subject

Search for peer-reviewed journals with full access.

Journals A - Z

About Us

Discover the SciOpen Platform and Achieve Your Research Goals with Ease.

About Us

Publish with Us

Support

Search articles, authors, keywords, DOl and etc.

Published Date

Reset Search

{{expandStatus?'Exit ':''}}Advanced Search

Journals A - Z

About Us

Publish with Us

Support

PDF (5.2 MB)

Cite

EndNote(RIS) BibTeX

Collect

Submit Manuscript

AI Chat Paper

Show Outline

Outline

Show full outline

Hide outline

Outline

Show full outline

Hide outline

Research Article | Open Access

Non-dominated sorting based multi-page photo collage

Yu Song^{¹^,²^,⁴}, Fan Tang^³, Weiming Dong^{¹^,²^,⁴}(

), Changsheng Xu^{¹^,²^,⁴}

1NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing 100040, China

2School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100190, China

3School of Artificial Intelligence, Jilin University, Changchun 130012, China

4CASIA-LLvision Joint Lab, Beijing 100190, China

Show Author Information

Graphical Abstract

Abstract

The development of social networking services (SNSs) revealed a surge in image sharing. The sharing mode of multi-page photo collage (MPC), which posts several image collages at a time, can often be observed on many social network platforms, which enables uploading images and arrangement in a logical order. This study focuses on the construction of MPC for an image collection and its formulation as an issue of joint optimization, which involves not only the arrangement in a single collage but also the arrangement among different collages. Novel balance-aware measurements, which merge graphic features and psy-chological achievements, are introduced. Non-dominated sorting genetic algorithm is adopted to optimize the MPC guided by the measurements. Experiments demonstrate that the proposed method can lead to diverse, visually pleasant, and logically clear MPC results, which are comparable to manually designed MPC results.

Keywords

multi-page photo collage balance-aware measurements non-dominated sorting genetic algorithm

References

[1]

Yu,

Z. Q.

; Lu,

; Guo,

Y. W.

; Fan,

R. F.

; Liu,

M. M.

; Wang,

W. P.

Content-aware photo collage using circle packing. IEEE Transactions on Visualization and Computer Graphics Vol. 20, No. 2, 182-195, 2014.

Crossref Google Scholar

[2]

Liu,

L. J.

; Zhang,

H. J.

; Jing,

G. M.

; Guo,

Y. W.

; Chen,

Z. G.

; Wang,

W. P.

Correlation-preserving photo collage. IEEE Transactions on Visualization and Computer Graphics Vol. 24, No. 6, 1956-1968, 2018.

Crossref Google Scholar

[3]

Liang,

; Wang,

X. T.

; Zhang,

S. H.

; Hu,

S. M.

; Liu,

S. X.

PhotoRecomposer: Interactive photo recomposition by cropping. IEEE Transactions on Visualization and Computer Graphics Vol. 24, No. 10, 2728-2742, 2018.

Crossref Google Scholar

[4]

Pan,

X. J.

; Tang,

; Dong,

W. M.

; Ma,

C. Y.

; Meng,

Y. P.

; Huang,

F. Y.

; Lee,

T. Y.

; Xu,

C. S.

Content-based visual summarization for image collections. IEEE Transactions on Visualization and Computer Graphics Vol. 27, No. 4, 2298-2312, 2021.

Crossref Google Scholar

[5]

Wagemans,

; Elder,

J. H.

; Kubovy,

; Palmer,

S. E.

; Peterson,

M. A.

; Singh,

; von der Heydt,

A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychological Bulletin Vol. 138, No. 6, 1172-1217, 2012.

Crossref Google Scholar

[6]

Hübner,

; Fillinger,

M. G.

Comparison of objective measures for predicting perceptual balance and visual aesthetic preference. Frontiers in Psychology Vol. 7, 335, 2016.

Crossref Google Scholar

[7]

Hübner,

; Fillinger,

M. G.

Perceptual balance, stability, and aesthetic appreciation: Their relations depend on the picture type. i-Perception Vol. 10, No.3, 1-17, 2019.

Crossref Google Scholar

[8]

Wilson,

; Chatterjee,

The assessment of preference for balance: Introducing a new test. Empirical Studies of the Arts Vol. 23, No. 2, 165-180, 2005.

Crossref Google Scholar

[9]

Coello Coello,

; Lamont,

G. B.

; van Veldhuizen,

D. A.

Evolutionary Algorithms for Solving MultiObjective Problems. Springer US, 2007.

[10]

Tan,

; Song,

Y. Q.

; Liu,

S. X.

; Xie,

L. X.

ImageHive: Interactive content-aware image summarization. IEEE Computer Graphics and Applications Vol. 32, No. 1, 46-55, 2012.

Crossref Google Scholar

[11]

Cao,

; Chan,

A. B.

; Lau,

R. W. H.

Automatic stylistic manga layout. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 141, 2012.

Crossref Google Scholar

[12]

Cao,

; Lau,

R. W. H.

; Chan,

A. B.

Look over here: Attention-directing composition of manga elements. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 94, 2014.

Crossref Google Scholar

[13]

Chen,

; Lu,

A. D.

; Hu,

S. M.

Visual storylines: Semantic visualization of movie sequence. Computers & Graphics Vol. 36, No. 4, 241-249, 2012.

Crossref Google Scholar

[14]

Jing,

G. M.

; Hu,

Y. T.

; Guo,

Y. W.

; Yu,

Y. Z.

; Wang,

W. P.

Content-aware Video2Comics with manga-style layout. IEEE Transactions on Multimedia Vol. 17, No. 12, 2122-2133, 2015.

Crossref Google Scholar

[15]

Han,

X. T.

; Zhang,

C. Y.

; Lin,

W. Y.

; Xu,

M. L.

; Sheng,

; Mei,

Tree-based visualization and optimization for image collection. IEEE Transactions on Cybernetics Vol. 46, No. 6, 1286-1300, 2016.

Crossref Google Scholar

[16]

Wu,

Z. P.

; Aizawa,

Very fast generation of content-preserved photo collage under canvas size constraint. Multimedia Tools and Applications Vol. 75, No. 4, 1813-1841, 2016.

Crossref Google Scholar

[17]

Zheng,

X. R.

; Qiao,

X. T.

; Cao,

; Lau,

R. W. H.

Content-aware generative modeling of graphic design layouts. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 133, 2019.

Crossref Google Scholar

[18]

Gan,

; Zhang,

; Sun,

Z. X.

; Zhang,

Qualitative photo collage by quartet analysis and active learning. Computers & Graphics Vol. 88, 35-44, 2020.

Crossref Google Scholar

[19]

Kong,

; Shen,

X. H.

; Lin,

; Mech,

; Fowlkes,

Photo aesthetics ranking network with attributes and content adaptation. In: Computer Vision - ECCV 2016. Lecture Notes in Computer Science, Vol. 9905. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 662-679, 2016.

Crossref

[20]

Song,

; Tang,

; Dong,

W. M.

; Huang,

F. Y.

; Xu,

C. S.

Balance-based photo posting. In: Proceedings of the SIGGRAPH Asia 2019 Posters, Article No. 20, 2019.

Crossref

[21]

Deb,

Multi-Objective Optimization Using Evolutionary Algorithms. New York: Wiley, 2001.

[22]

Deb,

; Pratap,

; Agarwal,

; Meyarivan,

A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation Vol. 6, No. 2, 182-197, 2002.

Crossref Google Scholar

[23]

Li,

; Zhong,

Dimensionally aware multi-objective genetic programming for automatic crowd behavior modeling. ACM Transactions on Modeling and Computer Simulation Vol. 30, No. 3, Article No. 19, 2020.

Crossref Google Scholar

[24]

Lee,

; Shin,

; Issa,

R. R. A.

Multi-objective optimization of a free-form surface based on generative designs. In: Proceedings of the 18th International Conference on Computing in Civil and Building Engineering. Lecture Notes in Civil Engineering, Vol. 98. Toledo Santos, E.; Scheer, S. Eds. Springer Cham, 1252-1261, 2021.

Crossref

[25]

Rejeesh,

M. R.

; Thejaswini,

MOTF: Multi-objective Optimal Trilateral Filtering based partial moving frame algorithm for image denoising. Multimedia Tools and Applications Vol. 79, Nos. 37-38, 28411-28430, 2020.

Crossref Google Scholar

[26]

Su,

; Yin,

HypervolGAN: An efficient approach for GAN with multi-objective training function. arXiv preprint arXiv:2006.15228, 2020.

Google Scholar

[27]

Chen,

Z. B.

; Lin,

J. X.

; Liao,

; Chen,

C. W.

Full reference quality assessment for image retargeting based on natural scene statistics modeling and Bi-directional saliency similarity. IEEE Transactions on Image Processing Vol. 26, No. 11, 5138-5148, 2017.

Crossref Google Scholar

[28]

Zhang,

Y. B.

; Lin,

W. S.

; Li,

Q. H.

; Cheng,

W. T.

; Zhang,

X. F.

Multiple-level feature-based measure for retargeted image quality. IEEE Transactions on Image Processing Vol. 27, No. 1, 451-463, 2018.

Crossref Google Scholar

[29]

Ma,

; Xu,

; Zhang,

Y. C.

; Yan,

Y. H.

; Ngan,

K. N.

No-reference retargeted image quality assessment based on pairwise rank learning. IEEE Transactions on Multimedia Vol. 18, No. 11, 2228-2237, 2016.

Crossref Google Scholar

[30]

Liang,

; Liu,

Y. J.

; Gutierrez,

Objective quality prediction of image retargeting algorithms. IEEE Transactions on Visualization and Computer Graphics Vol. 23, No. 2, 1099-1110, 2017.

Crossref Google Scholar

[31]

Sheng,

K. K.

; Dong,

W. M.

; Ma,

C. Y.

; Mei,

; Huang,

F. Y.

; Hu,

B. G.

Attention-based multi-patch aggregation for image aesthetic assessment. In: Proceedings of the 26th ACM International Conference on Multimedia, 879-886, 2018.

Crossref

[32]

Sheng,

K. K.

; Dong,

W. M.

; Huang,

H. B.

; Chai,

M. L.

; Zhang,

; Ma,

C. Y.

; Hu,

B.-G.

Learning to assess visual aesthetics of food images. Computational Visual Media Vol. 7, No. 1, 139-152, 2021.

Crossref Google Scholar

[33]

Xu,

M. N.

; Zhong,

J. X.

; Ren,

Y. R.

; Liu,

; Li,

Context-aware attention network for predicting image aesthetic subjectivity. In: Proceedings of the 28th ACM International Conference on Multimedia, 798-806, 2020.

[34]

Simonyan,

; Zisserman,

Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations, 2015.

[35]

Thömmes,

; Hübner,

Instagram likes for architectural photos can be predicted by quantitative balance measures and curvature. Frontiers in Psychology Vol. 9, 1050, 2018.

Crossref Google Scholar

Computational Visual Media

Volume 8 Issue 2,
June 2022

Pages 199-212

DOI: 10.1007/s41095-021-0221-0

Cite this article:

Song Y, Tang F, Dong W, et al. Non-dominated sorting based multi-page photo collage. Computational Visual Media, 2022, 8(2): 199-212. https://doi.org/10.1007/s41095-021-0221-0

896

Views

Downloads

Crossref

Web of Science

Scopus

CSCD

Google Scholar
Citation

Altmetrics

Received: 21 January 2021

Accepted: 01 March 2021

Published: 06 December 2021

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.