Journal Home > Volume 2 , Issue 3
Computational Visual Media 2016, 2(3): 257-266 https://doi.org/10.1007/s41095-016-0054-4
Research Article | Open Access | Issue | Published: 16 June 2016

Discriminative subgraphs for discovering family photos

Show Author's Information Changmin Choi1,  YoonSeok Lee1,  Sung-Eui Yoon1( )
Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, Republic of Korea.
Keywords:
image classification, subgraph mining, social context, group photographs
Cite this article:
Choi C, Lee Y, Yoon S-E. Discriminative subgraphs for discovering family photos. Computational Visual Media, 2016, 2(3): 257-266. https://doi.org/10.1007/s41095-016-0054-4
Download citation
EndNote(RIS)
BibTeX

556

Views

28

Downloads

Citations

4

Crossref

N/A

WoS

4

Scopus

0

CSCD

Abstract Full text About this article

We propose to use discriminative subgraphs to discover family photos from group photos in an efficient and effective way. Group photos are represented as face graphs by identifying social contexts such as age, gender, and face position. The previous work utilized bag-of-word models and considered frequent subgraphs from all group photos as features for classification. This approach, however, produces numerous subgraphs, resulting in high dimensions. Furthermore, some of them are not discriminative. To solve these issues, we adopt a state-of-the-art, frequent subgraph mining method that removes non-discriminative subgraphs. We also use TF-IDF normalization, which is more suitable for the bag-of-word model. To validate our method, we experiment in two datasets. Our method shows consistently better performance, higher accuracy in lower feature dimensions, compared to the previous method. We also integrate our method with the recent Microsoft face recognition API and release it in a public website.


menu
Abstract
Full text
Outline
About this article

Discriminative subgraphs for discovering family photos

Show Author's information Changmin Choi1, YoonSeok Lee1, Sung-Eui Yoon1( )
Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, Republic of Korea.

Abstract

We propose to use discriminative subgraphs to discover family photos from group photos in an efficient and effective way. Group photos are represented as face graphs by identifying social contexts such as age, gender, and face position. The previous work utilized bag-of-word models and considered frequent subgraphs from all group photos as features for classification. This approach, however, produces numerous subgraphs, resulting in high dimensions. Furthermore, some of them are not discriminative. To solve these issues, we adopt a state-of-the-art, frequent subgraph mining method that removes non-discriminative subgraphs. We also use TF-IDF normalization, which is more suitable for the bag-of-word model. To validate our method, we experiment in two datasets. Our method shows consistently better performance, higher accuracy in lower feature dimensions, compared to the previous method. We also integrate our method with the recent Microsoft face recognition API and release it in a public website.

Keywords: image classification, subgraph mining, social context, group photographs

References(27)

[1]
Krizhevsky, A. Learning multiple layers of features from tiny images. Technical Report. University of Toronto, 2009.
[2]
Chen, Y.-Y.; Hsu, W. H.; Liao, H.-Y. M. Discovering informative social subgraphs and predicting pairwise relationships from group photos. In: Proceedings of the 20th ACM International Conference on Multimedia, 669-678, 2012.
DOI
[3]
Gallagher, A. C.; Chen, T. Understanding images of groups of people. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 256-263, 2009.
DOI
[4]
Murillo, A. C.; Kwak, I. S.; Bourdev, L.; Kriegman, D.; Belongie, S. Urban tribes: Analyzing group photos from a social perspective. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 28-35, 2012.
DOI
[5]
Wang, G.; Gallagher, A.; Luo, J.; Forsyth, D. Seeing people in social context: Recognizing people and social relationships. In: Lecture Notes in Computer Science, Vol. 6315. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer Berlin Heidelberg, 169-182, 2010.
[6]
Chiu, Y.-I.; Li, C.; Huang, C.-R.; Chung, P.-C.; Chen, T. Efficient graph based spatial face context representation and matching. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 2001-2005, 2013.
DOI
[7]
Gallagher, A. C.; Chen, T. Finding rows of people in group images. In: Proceedings of IEEE International Conference on Multimedia and Expo, 602-605, 2009.
DOI
[8]
Choi, W.; Chao, Y.-W.; Pantofaru, C.; Savarese, S. Discovering groups of people in images. In: Lecture Notes in Computer Science, Vol. 8692. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer International Publishing, 417-433, 2014.
[9]
Shu, H.; Gallagher, A.; Chen, H.; Chen, T. Face–graph matching for classifying groups of people. In: Proceedings of IEEE International Conference on Image Processing, 2425-2429, 2013.
DOI
[10]
Chiu, Y.-I.; Hsu, R.-Y.; Huang, C.-R. Spatial face context with gender information for group photo similarity assessment. In: Proceedings of the 22nd International Conference on Pattern Recognition, 2673-2678, 2014.
DOI
[11]
Shimizu, K.; Nitta, N.; Nakai, Y.; Babaguchi, N. Classification based group photo retrieval with bag of people features. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, Article No. 6, 2012.
DOI
[12]
Zhang, T.; Chao, H.; Willis, C.; Tretter, D. Consumer image retrieval by estimating relation tree from family photo collections. In: Proceedings of the ACM International Conference on Image and Video Retrieval, 143-150, 2010.
DOI
[13]
Singla, P.; Kautz, H.; Luo, J.; Gallagher, A. Discovery of social relationships in consumer photo collections using Markov logic. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 1-7, 2008.
DOI
[14]
Song, Z.; Wang, M.; Hua, X.-s.; Yan, S. Predicting occupation via human clothing and contexts. In: Proceedings of International Conference on Computer Vision, 1084-1091, 2011.
DOI
[15]
Lowe, D. G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision Vol. 60, No. 2, 91-110, 2004.
DOI Google Scholar
[16]
Oliva, A.; Torralba, A. Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision Vol. 42, No. 3, 145-175, 2001.
DOI Google Scholar
[17]
Jiang, C.; Coenen, F.; Zito, M. A survey of frequent subgraph mining algorithms. The Knowledge Engineering Review Vol. 28, No. 1, 75-105, 2013.
DOI Google Scholar
[18]
Cook, S. A. The complexity of theorem-proving procedures. In: Proceedings of the 3rd Annual ACM Symposium on Theory of Computing, 151-158, 1971.
DOI
[19]
Wang, M.; Lai, Y.-K.; Liang, Y.; Martin, R. R.; Hu, S.-M. BiggerPicture: Data-driven image extrapolation using graph matching. ACM Transactions on Graphics Vol. 33, No. 6, Article No. 173, 2014.
DOI Google Scholar
[20]
Chen, X.; Zhou, B.; Guo, Y.; Xu, F.; Zhao, Q. Structure guided texture inpainting through multi-scale patches and global optimization for image completion. Science China Information Sciences Vol. 57, No. 1, 1-16, 2014.
DOI Google Scholar
[21]
Li, H.; Wu, W.; Wu, E. Robust interactive image segmentation via graph-based manifold ranking. Computational Visual Media Vol. 1, No. 3, 183-195, 2015.
DOI Google Scholar
[22]
Hu, S.-M.; Zhang, F.-L.; Wang, M.; Martin, R. R.; Wang, J. PatchNet: A patch-based image representation for interactive library-driven image editing. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 196, 2013.
DOI Google Scholar
[23]
Taigman, Y.; Yang, M.; Ranzato, M.; Wolf, L. DeepFace: Closing the gap to human-level performance in face verification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1701-1708, 2014.
DOI
[24]
Zhao, W.; Chellappa, R.; Phillips, P. J.; Rosenfeld, A. Face recognition: A literature survey. ACM Computing Surveys Vol. 35, No. 4, 399-458, 2003.
DOI Google Scholar
[25]
Yan, X.; Han, J. gSpan: Graph-based substructure pattern mining. In: Proceedings of the 2002 IEEE International Conference on Data Mining, 721-724, 2002.
[26]
Thoma, M.; Cheng, H.; Gretton, A.; Han, J.; Kriegel, H.-P.; Smola, A.; Song, L.; Yu, P. S.; Yan, X.; Borgwardt, K. M. Discriminative frequent subgraph mining with optimality guarantees. Statistical Analysis and Data Mining: The ASA Data Science Journal Vol. 3, No. 5, 302-318, 2010.
DOI Google Scholar
[27]
Jones, K. S. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation Vol. 28, No. 1, 11-21, 1972.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Revised: 01 February 2016
Accepted: 13 April 2016
Published: 16 June 2016
Issue date: September 2016

Copyright

© The Author(s) 2016

Acknowledgements

We are thankful to our lab members for valuable feedbacks, and to Ph.D. Yan-Ying Chen for sharing her dataset. This work was supported in part by MSIP/IITP (Nos. R0126-16-1108, R0101-16-0176) and MSIP/NRF (No. 2013-067321).

Rights and permissions

This article is published with open access at Springerlink.com

The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http:// creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.

Return