Journal Home >

Image classification is vital and basic in many data analysis domains. Since real-world images generally contain multiple diverse semantic labels, it amounts to a typical multi-label classification problem. Traditional multi-label image classification relies on a large amount of training data with plenty of labels, which requires a lot of human and financial costs. By contrast, one can easily obtain a correlation matrix of concerned categories in current scene based on the historical image data in other application scenarios. How to perform image classification with only label correlation priors, without specific and costly annotated labels, is an important but rarely studied problem. In this paper, we propose a model to classify images with this kind of weak correlation prior. We use label correlation to recapitulate the sample similarity, employ the prior information to decompose the projection matrix when regressing the label indication matrix, and introduce the $ℓ2,1$ norm to select features for each image. Finally, experimental results on several image datasets demonstrate that the proposed model has distinct advantages over current state-of-the-art multi-label classification methods.

Abstract
Full text
Outline

# Multi-Label Image Classification with Weak Correlation Prior

Show Author's information Xiao Ouyang1Ruidong Fan1Hong Tao1( )Chenping Hou1( )
Department of Systems Science, National University of Defense Technology, Changsha 410073, China

## Abstract

Image classification is vital and basic in many data analysis domains. Since real-world images generally contain multiple diverse semantic labels, it amounts to a typical multi-label classification problem. Traditional multi-label image classification relies on a large amount of training data with plenty of labels, which requires a lot of human and financial costs. By contrast, one can easily obtain a correlation matrix of concerned categories in current scene based on the historical image data in other application scenarios. How to perform image classification with only label correlation priors, without specific and costly annotated labels, is an important but rarely studied problem. In this paper, we propose a model to classify images with this kind of weak correlation prior. We use label correlation to recapitulate the sample similarity, employ the prior information to decompose the projection matrix when regressing the label indication matrix, and introduce the $ℓ2,1$ norm to select features for each image. Finally, experimental results on several image datasets demonstrate that the proposed model has distinct advantages over current state-of-the-art multi-label classification methods.

Keywords: image recognition, label correlation, multi-label classification, weakly-supervised learning

## References(34)

1
J. Wang, Y. Yang, J. Mao, Z. Huang, C. Huang, and W. Xu. CNN-RNN: A unified framework for multi-label image classification, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 2285–2294.
2

J. Zhang, Q. Wu, C. Shen, J. Zhang, and J. Lu, Multilabel image classification with regional latent semantic dependencies, IEEE Trans. Multim., vol. 20, no. 10, pp. 2801–2813, 2018.

3
J. Lanchantin, T. Wang, V. Ordonez, and Y. Qi, General multi-label image classification with transformers, in Proc. 2021 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Nashville, TN, USA, 2021, pp. 16478–16488.
4

S. Wen, W. Liu, Y. Yang, P. Zhou, Z. Guo, Z. Yan, Y. Chen, and T. Huang, Multilabel image classification via feature/label co-projection, IEEE Trans. Syst. Man Cybern. Syst., vol. 51, no. 11, pp. 7250–7259, 2021.

5
Y. Liu, L. Sheng, J. Shao, J. Yan, S. Xiang, and C. Pan, Multi-label image classification via knowledge distillation from weakly-supervised detection, in Proc. 26th ACM Multimedia Conf. Multimedia Conference, Seoul, Republic of Korea, 2018, pages 700–708.
6

Y. Xia, L. Nie, L. Zhang, Y. Yang, R. Hong, and X. Li, Weakly supervised multilabel clustering and its applications in computer vision, IEEE Trans. Cybern., vol. 46, no. 12, pp. 3220–3232, 2016.

7
H. M. Chu, C. K. Yeh, and Y. C. F. Wang, Deep generative models for weakly-supervised multi-label classification, in Proc. Computer Vision - ECCV 2018 - 15th European Conf., Munich, Germany, 2018, pp. 409–425.
8

R. Cabral, F. De la Torre, J. P. Costeira, and A. Bernardino, Matrix completion for weaklysupervised multi-label image classification, IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 1, pp. 121–135, 2015.

9
P. Nodet, V. Lemaire, A. Bondu, A. Cornuéjols, and A. Ouorou. From weakly supervised learning to biquality learning: An introduction, arXiv preprint arXiv: 2012.09632, 2020.
10
M. Hu, H. Han, S. Shan, and X. Chen, Weakly supervised image classification through noise regularization, in Proc. 2019 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019, pp. 11517–11525.
11

L. Sun, G. Lyu, S. Feng, and X. Huang, Beyond missing: Weakly-supervised multi-label learning with incomplete and noisy labels, Appl. Intell., vol. 51, no. 3, pp. 1552–1564, 2021.

12

B. Frenay and M. Verleysen, Classification in the presence of label noise: A Survey, IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 5, pp. 845–869, 2014.

13

J. Wu, A. Guo, V. S. Sheng, P. Zhao, and Z. Cui, An active learning approach for multi-label image classification with sample noise, Int. J. Pattern Recognit. Artif. Intell., vol. 32, no. 3, p. 1850005, 2018.

14
Z. Younes, F. Abdallah, and T. Denœux, Evidential multi-label classification approach to learning from data with imprecise labels, in Proc. 13th Int. Conf. Information Processing and Management of Uncertainty, Dortmund, Germany, 2010, pp. 119–128.
15
J. Mojoo, K. Kurosawa, and T. Kurita, Deep CNN with graph Laplacian regularization for multi-label image annotation, in Proc. Image Analysis and Recognition - 14th Int. Conf., Montreal, Canada, 2017, pp. 19–26.
16

H. Guo, L. Han, S. Su, and Z. Sun, Deep multi-instance multi-label learning for image annotation, Int. J. Pattern Recognit. Artif. Intell., vol. 32, no. 3, p. 1859005, 2018.

17
S. Y. Li and Y. Jiang, Multi-label crowdsourcing learning with incomplete annotations, in Proc. 15th Pacific Rim Int. Conf. Artificial Intelligence, Nanjing, China, 2018, pp. 232–245.
18
S. S. Bucak, R. Jin, and A. K. Jain, Multi-label learning with incomplete class assignments, in Proc. 24th IEEE Conf. Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 2011, pp. 2801–2808.
19
X. Kong, Z. Wu, L. J. Li, R. Zhang, P. S. Yu, H. Wu, and W. Fan, Large-scale multi-label learning with incomplete label assignments, in Proc. 2014 SIAM Int. Conf. Data Mining, Philadelphia, PA, USA, 2014, pp. 920–928.
20

M. R. Boutell, J. Luo, X. Shen, and C. M. Brown, Learning multi-label scene classification, Pattern Recognit., vol. 37, no. 9, pp. 1757–1771, 2004.

21
Y. Li, Y. Song, and J. Luo, Improving pairwise ranking for multi-label image classification, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 1837–1845.
22
G. Tsoumakas and I. P. Vlahavas, Random k-labelsets: An ensemble method for multilabel classification, in Proc. 18th European Conf. Machine Learning, Warsaw, Poland, 2007, pp. 406–417.
23

J. Shan, C. Hou, H. Tao, W. Zhuge, and D. Yi, Co-learning binary classifiers for LP-based multi-label classification, Cogn. Syst. Res., vol. 55, pp. 146–152, 2019.

24

M. L. Zhang and Z. H. Zhou, ML-KNN: A lazy learning approach to multi-label learning, Pattern Recognit., vol. 40, no. 7, pp. 2038–2048, 2007.

25
A. Elisseeff and J. Weston, A kernel method for multi-labelled classification, in Proc. 14th Int. Conf. Neural Information Processing Systems: Natural and Synthetic, Vancouver, Canada, 2001, pp. 681&#8722;687.
26
A. Clare and R. D. King, Knowledge discovery in multi-label phenotype data, in Proc. Principles of Data Mining and Knowledge Discovery, 5th European Conf., Freiburg, Germany, 2001, pp. 42–53.
27

Y. Wei, W. Xia, M. Lin, J. Huang, B. Ni, J. Dong, Y. Zhao, and S. Yan, HCP: A flexible CNN framework for multi-label image classification, IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 9, pp. 1901–1907, 2016.

28

M. L. Zhang and J. P. Fang, Partial multi-label learning via credible label elicitation, IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 10, pp. 3587–3599, 2021.

29
A. Cotter, J. Keshet, and N. Srebro, Explicit approximations of the Gaussian kernel, arXiv preprint arXiv: 1109.4603, 2011.
30
M. L. Zhang and K. Zhang, Multi-label learning by exploiting label dependency, in Proc. 16th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Washington, DC, USA, 2010, pp. 999–1008.
31

H. Hu, R. Wang, F. Nie, X. Yang, and W. Yu, Fast unsupervised feature selection with anchor graph and $ℓ2,1$-norm regularization, Multim. Tools Appl., vol. 77, no. 17, pp. 22099–22113, 2018.

32

H. Tao, C. Hou, F. Nie, Y. Jiao, and D. Yi, Effective discriminative feature selection with nontrivial solution, IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 4, pp. 796–808, 2016.

33

M. L. Zhang, J. P. Fang, and Y. B. Wang, BiLabel-specific features for multi-label classification, ACM Trans. Knowl. Discov. Data, vol. 16, no. 1, p. 18, 2022.

34
Y. Liu, R. Jin, and L. Yang, Semi-supervised multi-label learning by constrained non-negative matrix factorization, in Proc. 21st National Conf. Artificial Intelligence, Boston, MA, USA, 2006, pp. 421–426.
Publication history
Acknowledgements
Rights and permissions

## Publication history

Revised: 14 August 2022
Accepted: 20 August 2022
Published: 28 August 2022
Issue date: September 2022