Journal Home > Volume 1 , Issue 1
CAAI Artificial Intelligence Research 2022, 1(1): 79-92 https://doi.org/10.26599/AIR.2022.9150006
Open Access | Issue | Published: 28 August 2022

Multi-Label Image Classification with Weak Correlation Prior

Show Author's Information Xiao Ouyang1,  Ruidong Fan1,  Hong Tao1( ),  Chenping Hou1( )
Department of Systems Science, National University of Defense Technology, Changsha 410073, China
Keywords:
image recognition, label correlation, multi-label classification, weakly-supervised learning
Cite this article:
Ouyang X, Fan R, Tao H, et al. Multi-Label Image Classification with Weak Correlation Prior. CAAI Artificial Intelligence Research, 2022, 1(1): 79-92. https://doi.org/10.26599/AIR.2022.9150006
Download citation
EndNote(RIS)
BibTeX

1606

Views

221

Downloads

Citations

0

Crossref

N/A

WoS

N/A

Scopus

N/A

CSCD

Abstract Full text About this article

Image classification is vital and basic in many data analysis domains. Since real-world images generally contain multiple diverse semantic labels, it amounts to a typical multi-label classification problem. Traditional multi-label image classification relies on a large amount of training data with plenty of labels, which requires a lot of human and financial costs. By contrast, one can easily obtain a correlation matrix of concerned categories in current scene based on the historical image data in other application scenarios. How to perform image classification with only label correlation priors, without specific and costly annotated labels, is an important but rarely studied problem. In this paper, we propose a model to classify images with this kind of weak correlation prior. We use label correlation to recapitulate the sample similarity, employ the prior information to decompose the projection matrix when regressing the label indication matrix, and introduce the ℓ2,1 norm to select features for each image. Finally, experimental results on several image datasets demonstrate that the proposed model has distinct advantages over current state-of-the-art multi-label classification methods.


menu
Abstract
Full text
Outline
About this article

Multi-Label Image Classification with Weak Correlation Prior

Show Author's information Xiao Ouyang1, Ruidong Fan1, Hong Tao1( ), Chenping Hou1( )
Department of Systems Science, National University of Defense Technology, Changsha 410073, China

Abstract

Image classification is vital and basic in many data analysis domains. Since real-world images generally contain multiple diverse semantic labels, it amounts to a typical multi-label classification problem. Traditional multi-label image classification relies on a large amount of training data with plenty of labels, which requires a lot of human and financial costs. By contrast, one can easily obtain a correlation matrix of concerned categories in current scene based on the historical image data in other application scenarios. How to perform image classification with only label correlation priors, without specific and costly annotated labels, is an important but rarely studied problem. In this paper, we propose a model to classify images with this kind of weak correlation prior. We use label correlation to recapitulate the sample similarity, employ the prior information to decompose the projection matrix when regressing the label indication matrix, and introduce the ℓ2,1 norm to select features for each image. Finally, experimental results on several image datasets demonstrate that the proposed model has distinct advantages over current state-of-the-art multi-label classification methods.

Keywords: image recognition, label correlation, multi-label classification, weakly-supervised learning

References(34)

1
J. Wang, Y. Yang, J. Mao, Z. Huang, C. Huang, and W. Xu. CNN-RNN: A unified framework for multi-label image classification, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 2285–2294.
DOI
2

J. Zhang, Q. Wu, C. Shen, J. Zhang, and J. Lu, Multilabel image classification with regional latent semantic dependencies, IEEE Trans. Multim., vol. 20, no. 10, pp. 2801–2813, 2018.
DOI Google Scholar
3
J. Lanchantin, T. Wang, V. Ordonez, and Y. Qi, General multi-label image classification with transformers, in Proc. 2021 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Nashville, TN, USA, 2021, pp. 16478–16488.
DOI
4

S. Wen, W. Liu, Y. Yang, P. Zhou, Z. Guo, Z. Yan, Y. Chen, and T. Huang, Multilabel image classification via feature/label co-projection, IEEE Trans. Syst. Man Cybern. Syst., vol. 51, no. 11, pp. 7250–7259, 2021.
DOI Google Scholar
5
Y. Liu, L. Sheng, J. Shao, J. Yan, S. Xiang, and C. Pan, Multi-label image classification via knowledge distillation from weakly-supervised detection, in Proc. 26th ACM Multimedia Conf. Multimedia Conference, Seoul, Republic of Korea, 2018, pages 700–708.
DOI
6

Y. Xia, L. Nie, L. Zhang, Y. Yang, R. Hong, and X. Li, Weakly supervised multilabel clustering and its applications in computer vision, IEEE Trans. Cybern., vol. 46, no. 12, pp. 3220–3232, 2016.
DOI Google Scholar
7
H. M. Chu, C. K. Yeh, and Y. C. F. Wang, Deep generative models for weakly-supervised multi-label classification, in Proc. Computer Vision - ECCV 2018 - 15th European Conf., Munich, Germany, 2018, pp. 409–425.
DOI
8

R. Cabral, F. De la Torre, J. P. Costeira, and A. Bernardino, Matrix completion for weaklysupervised multi-label image classification, IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 1, pp. 121–135, 2015.
DOI Google Scholar
9
P. Nodet, V. Lemaire, A. Bondu, A. Cornuéjols, and A. Ouorou. From weakly supervised learning to biquality learning: An introduction, arXiv preprint arXiv: 2012.09632, 2020.
DOI
10
M. Hu, H. Han, S. Shan, and X. Chen, Weakly supervised image classification through noise regularization, in Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019, pp. 11517–11525.
DOI
11

L. Sun, G. Lyu, S. Feng, and X. Huang, Beyond missing: Weakly-supervised multi-label learning with incomplete and noisy labels, Appl. Intell., vol. 51, no. 3, pp. 1552–1564, 2021.
DOI Google Scholar
12

B. Frenay and M. Verleysen, Classification in the presence of label noise: A Survey, IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 5, pp. 845–869, 2014.
DOI Google Scholar
13

J. Wu, A. Guo, V. S. Sheng, P. Zhao, and Z. Cui, An active learning approach for multi-label image classification with sample noise, Int. J. Pattern Recognit. Artif. Intell., vol. 32, no. 3, p. 1850005, 2018.
DOI Google Scholar
14
Z. Younes, F. Abdallah, and T. Denœux, Evidential multi-label classification approach to learning from data with imprecise labels, in Proc. 13th Int. Conf. Information Processing and Management of Uncertainty, Dortmund, Germany, 2010, pp. 119–128.
DOI
15
J. Mojoo, K. Kurosawa, and T. Kurita, Deep CNN with graph Laplacian regularization for multi-label image annotation, in Proc. Image Analysis and Recognition - 14th Int. Conf., Montreal, Canada, 2017, pp. 19–26.
DOI
16

H. Guo, L. Han, S. Su, and Z. Sun, Deep multi-instance multi-label learning for image annotation, Int. J. Pattern Recognit. Artif. Intell., vol. 32, no. 3, p. 1859005, 2018.
DOI Google Scholar
17
S. Y. Li and Y. Jiang, Multi-label crowdsourcing learning with incomplete annotations, in Proc. 15th Pacific Rim Int. Conf. Artificial Intelligence, Nanjing, China, 2018, pp. 232–245.
DOI
18
S. S. Bucak, R. Jin, and A. K. Jain, Multi-label learning with incomplete class assignments, in Proc. 24th IEEE Conf. Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 2011, pp. 2801–2808.
DOI
19
X. Kong, Z. Wu, L. J. Li, R. Zhang, P. S. Yu, H. Wu, and W. Fan, Large-scale multi-label learning with incomplete label assignments, in Proc. 2014 SIAM Int. Conf. Data Mining, Philadelphia, PA, USA, 2014, pp. 920–928.
DOI
20

M. R. Boutell, J. Luo, X. Shen, and C. M. Brown, Learning multi-label scene classification, Pattern Recognit., vol. 37, no. 9, pp. 1757–1771, 2004.
DOI Google Scholar
21
Y. Li, Y. Song, and J. Luo, Improving pairwise ranking for multi-label image classification, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 1837–1845.
DOI
22
G. Tsoumakas and I. P. Vlahavas, Random k-labelsets: An ensemble method for multilabel classification, in Proc. 18th European Conf. Machine Learning, Warsaw, Poland, 2007, pp. 406–417.
DOI
23

J. Shan, C. Hou, H. Tao, W. Zhuge, and D. Yi, Co-learning binary classifiers for LP-based multi-label classification, Cogn. Syst. Res., vol. 55, pp. 146–152, 2019.
DOI Google Scholar
24

M. L. Zhang and Z. H. Zhou, ML-KNN: A lazy learning approach to multi-label learning, Pattern Recognit., vol. 40, no. 7, pp. 2038–2048, 2007.
DOI Google Scholar
25
A. Elisseeff and J. Weston, A kernel method for multi-labelled classification, in Proc. 14th Int. Conf. Neural Information Processing Systems: Natural and Synthetic, Vancouver, Canada, 2001, pp. 681−687.
26
A. Clare and R. D. King, Knowledge discovery in multi-label phenotype data, in Proc. Principles of Data Mining and Knowledge Discovery, 5th European Conf., Freiburg, Germany, 2001, pp. 42–53.
DOI
27

Y. Wei, W. Xia, M. Lin, J. Huang, B. Ni, J. Dong, Y. Zhao, and S. Yan, HCP: A flexible CNN framework for multi-label image classification, IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 9, pp. 1901–1907, 2016.
DOI Google Scholar
28

M. L. Zhang and J. P. Fang, Partial multi-label learning via credible label elicitation, IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 10, pp. 3587–3599, 2021.
DOI Google Scholar
29
A. Cotter, J. Keshet, and N. Srebro, Explicit approximations of the Gaussian kernel, arXiv preprint arXiv: 1109.4603, 2011.
30
M. L. Zhang and K. Zhang, Multi-label learning by exploiting label dependency, in Proc. 16th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Washington, DC, USA, 2010, pp. 999–1008.
DOI
31

H. Hu, R. Wang, F. Nie, X. Yang, and W. Yu, Fast unsupervised feature selection with anchor graph and ℓ2,1-norm regularization, Multim. Tools Appl., vol. 77, no. 17, pp. 22099–22113, 2018.
DOI Google Scholar
32

H. Tao, C. Hou, F. Nie, Y. Jiao, and D. Yi, Effective discriminative feature selection with nontrivial solution, IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 4, pp. 796–808, 2016.
DOI Google Scholar
33

M. L. Zhang, J. P. Fang, and Y. B. Wang, BiLabel-specific features for multi-label classification, ACM Trans. Knowl. Discov. Data, vol. 16, no. 1, p. 18, 2022.
DOI Google Scholar
34
Y. Liu, R. Jin, and L. Yang, Semi-supervised multi-label learning by constrained non-negative matrix factorization, in Proc. 21st National Conf. Artificial Intelligence, Boston, MA, USA, 2006, pp. 421–426.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 19 April 2022
Revised: 14 August 2022
Accepted: 20 August 2022
Published: 28 August 2022
Issue date: September 2022

Copyright

© The author(s) 2022

Acknowledgements

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Nos. 61922087, 61906201, 62006238, and 62136005), and the Natural Science Fund for Distinguished Young Scholars of Hunan Province (No. 2019JJ20020).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return