Journal Home > Volume 6 , Issue 4
Computational Visual Media 2020, 6(4): 477-487 https://doi.org/10.1007/s41095-020-0184-6
Research Article | Open Access | Issue | Published: 01 October 2020

A new dataset of dog breed images and a benchmark for fine-grained classification

Show Author's Information Ding-Nan Zou1,2,  Song-Hai Zhang1( ),  Tai-Jiang Mu1,  Min Zhang3
Department of Computer Science and Technology, BNRist, Tsinghua University, Beijing 100084, China
NaJiu Company, Hunan 410022, China
Harvard Medical School, Brigham and Women’s Hospital, Boston, MA 02115, USA
Keywords:
fine-grained classification, dog, dataset, benchmark
Cite this article:
Zou D-N, Zhang S-H, Mu T-J, et al. A new dataset of dog breed images and a benchmark for fine-grained classification. Computational Visual Media, 2020, 6(4): 477-487. https://doi.org/10.1007/s41095-020-0184-6
Download citation
EndNote(RIS)
BibTeX

745

Views

49

Downloads

Citations

17

Crossref

N/A

WoS

16

Scopus

1

CSCD

Abstract Full text About this article

In this paper, we introduce an image dataset for fine-grained classification of dog breeds: the Tsinghua Dogs Dataset. It is currently the largest dataset for fine-grained classification of dogs, including 130 dog breeds and 70,428 real-world images. It has only one dog in each image and provides annotated bounding boxes for the whole body and head. In comparison to previous similar datasets, it contains more breeds and more carefully chosen images for each breed. The diversity within each breed is greater, with between 200 and 7000+ images for each breed. Annotation of the whole body and head makes the dataset not only suitable for the improvement of fine-grained image classification models based on overall features, but also for those locating local informative parts. We show that dataset provides a tough challenge by benchmarking several state-of-the-art deep neural models. The dataset is available for academic purposes at https://cg.cs.tsinghua.edu.cn/ThuDogs/.


menu
Abstract
Full text
Outline
About this article

A new dataset of dog breed images and a benchmark for fine-grained classification

Show Author's information Ding-Nan Zou1,2, Song-Hai Zhang1( ), Tai-Jiang Mu1, Min Zhang3
Department of Computer Science and Technology, BNRist, Tsinghua University, Beijing 100084, China
NaJiu Company, Hunan 410022, China
Harvard Medical School, Brigham and Women’s Hospital, Boston, MA 02115, USA

Abstract

In this paper, we introduce an image dataset for fine-grained classification of dog breeds: the Tsinghua Dogs Dataset. It is currently the largest dataset for fine-grained classification of dogs, including 130 dog breeds and 70,428 real-world images. It has only one dog in each image and provides annotated bounding boxes for the whole body and head. In comparison to previous similar datasets, it contains more breeds and more carefully chosen images for each breed. The diversity within each breed is greater, with between 200 and 7000+ images for each breed. Annotation of the whole body and head makes the dataset not only suitable for the improvement of fine-grained image classification models based on overall features, but also for those locating local informative parts. We show that dataset provides a tough challenge by benchmarking several state-of-the-art deep neural models. The dataset is available for academic purposes at https://cg.cs.tsinghua.edu.cn/ThuDogs/.

Keywords: fine-grained classification, dog, dataset, benchmark

References(47)

[1]
S. Cai,; W. Zuo,; L. Zhang, Higher-order integration of hierarchical convolutional activations for fine-grained visual categorization. In: Proceedings of the IEEE International Conference on Computer Vision, 511-520, 2017.
[2]
Y. Cui,; Y. Song,; C. Sun,; A. Howard,; S. J. Belongie, Large scale fine-grained categorization and domain-specific transfer learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4109-4118, 2018.
[3]
Y. Wang,; V. I. Morariu,; L. S. Davis, Learning a discriminative filter bank within a CNN for fine-grained recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4148-4157, 2018.
[4]
Z. Yang,; T. G. Luo,; D. Wang,; Z. Q. Hu,; J. Gao,; L. W. Wang, Learning to navigate for fine-grained classification. In: Computer Vision - ECCV 2018. Lecture Notes in Computer Science Vol. 11218. V. Ferrari,; M. Hebert,; C. Sminchisescu,; Y. Weiss, Eds. Springer Cham, 438-454, 2018.
[5]
A. Khosla,; N. Jayadevaprakash,; B. Yao,; F.-F. Li, Novel dataset for fine-grained image categorization. In: Proceedings of the 1st Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition, 2011.
[6]
A. Krizhevsky,; I. Sutskever,; G. E. Hinton, ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, Vol. 1, 1097-1105, 2012.
[7]
L. Chen,; M. Yang, Semi-supervised dictionary learning with label propagation for image classification. Computational Visual Media Vol. 3, No. 1, 83-94, 2017.
Google Scholar
[8]
K. X. Chen,; X. J. Wu, Component SPD matrices: A low-dimensional discriminative data descriptor for image set classification. Computational Visual Media Vol. 4, No. 3, 245-252, 2018.
Google Scholar
[9]
J. Y. Ren,; X. J. Wu, Vectorial approximations of infinite-dimensional covariance descriptors for image classification. Computational Visual Media Vol. 3, No. 4, 379-385, 2017.
Google Scholar
[10]
C. Wah,; S. Branson,; P. Welinder,; P. Perona,; S. Belongie, The Caltech-UCSD Birds-200-2011 Dataset. Computation & Neural Systems Technical Report, CNS-TR-2011-001. California Institute of Technology, 2011.
[11]
J. Liu,; A. Kanazawa,; D. Jacobs,; P. Belhumeur, Dog breed classification using part localization. In: Proceedings of the 12th European Conference on Computer Vision, Vol. Part I, 172-185, 2012.
[12]
T. Berg,; P. N. Belhumeur, POOF: Part-based one-vs.-one features for fine-grained categorization, face verification, and attribute estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 955-962, 2013.
[13]
S. Branson,; G. V. Horn,; S. Belongie,; P. Perona, Bird species categorization using pose normalized deep convolutional nets. arXiv preprint arXiv:1406.2952, 2014.
[14]
N. Zhang,; J. Donahue,; R. Girshick,; T. Darrell, Part-based R-CNNs for fine-grained category detection. In: Computer Vision-ECCV 2014. Lecture Notes in Computer Science Vol. 8689. D. Fleet,; T. Pajdla,; B. Schiele,; T. Tuytelaars, Eds. Springer Cham, 834-849, 2014.
[15]
D. Lin,; X. Shen,; C. Lu,; J. Jia, Deep LAC: Deep localization, alignment and classification for fine-grained recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1666-1674, 2015.
[16]
M. Lam,; B. Mahasseni,; S. Todorovic, Fine-grained recognition as HSnet search for informative image parts. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6497-6506, 2017.
[17]
Y. Chen,; Y. Bai,; W. Zhang,; T. Mei, Destruction and construction learning for finegrained image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5157-5166, 2019.
[18]
W. F. Ge,; X. R. Lin,; Y. Z. Yu, Weakly supervised complementary parts models for fine-grained image classification from the bottom up. arXiv preprint arXiv:1903.02827, 2019.
[19]
R. Y. Du,; D. L. Chang,; A. K. Bhunia,; J. Y. Xie,; Z. Y. Ma,; Y. Z. Song,; J. Guo, Fine-grained visual classification via progressive multi-granularity training of jigsaw patches. arXiv preprint arXiv:2003.03836, 2020.
[20]
H. Zheng,; J. Fu,; T. Mei,; J. Luo, Learning multi-attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE International Conference on Computer Vision, 5219-5227, 2017.
[21]
J. Fu,; H. Zheng,; T. Mei, Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4476-4484, 2017.
[22]
H. Zheng,; J. Fu,; Z. Zha,; J. Luo,; Looking for the devil in the details: Learning trilinear attention sampling network for fine-grained image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5012-5021, 2019.
[23]
F. Zhang,; M. Li,; G. Zhai,; Y. Liu, Three-branch and multi-scale learning for fine-grained image recognition (TBMSL-Net). arXiv preprint arXiv:2003.09150, 2020.
[24]
G. L. Sun,; H. Cholakkal,; S. Khan,; F. S. Khan,; L. Shao, Fine-grained recognition: Accounting for subtle differences between similar classes. arXiv preprint arXiv:1912.06842, 2019.
[25]
T.-Y. Lin,; A. RoyChowdhury,; S. Maji, Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE international conference on computer vision, 1449-1457, 2015.
[26]
Y. Gao,; O. Beijbom,; N. Zhang,; T. Darrell, Compact bilinear pooling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 317-326, 2016.
[27]
C. Yu,; X. Zhao,; Q. Zheng,; P. Zhang,; X. You, Hierarchical bilinear pooling for fine-grained visual recognition. In: Computer Vision-ECCV 2018. Lecture Notes in Computer Science Vol. 11220. V. Ferrari,; M. Hebert,; C. Sminchisescu,; Y. Weiss, Eds. Springer Cham, 595-610, 2018.
[28]
Y. Wang,; J. Choi,; V. I. Morariu,; L. S. Davis, Mining discriminative triplets of patches for fine-grained classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1163-1172, 2016.
[29]
X. Zhang,; F. Zhou,; Y. Lin,; S. Zhang, Embedding label structures for finegrained feature representation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1114-1123, 2016.
[30]
A. Dubey,; O. Gupta,; R. Raskar,; N. Naik, Maximum-entropy fine grained classification. arXiv preprint arXiv:1809.05934, 2018.
[31]
Q. Qian,; R. Jin,; S. Zhu,; Y. Lin, Fine-grained visual categorization via multi-stage metric learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3716-3724, 2015.
[32]
M. Sun,; Y. Yuan,; F. Zhou,; E. Ding, Multi-attention multi-class constraint for fine-grained image recognition. In: Computer Vision-ECCV 2018. Lecture Notes in Computer Science Vol. 11220. V. Ferrari,; M. Hebert,; C. Sminchisescu,; Y. Weiss, Eds. Springer Cham, 834-850, 2018.
[33]
A. Dubey,; O. Gupta,; P. Guo,; R. Raskar,; R. Farrell,; N. Naik, Pairwise confusion for fine-grained visual classification. In: Computer Vision-ECCV 2018. Lecture Notes in Computer Science Vol. 11216. V. Ferrari,; M. Hebert,; C. Sminchisescu,; Y. Weiss, Eds. Springer Cham, 71-88, 2018.
[34]
P. Zhuang,; Y. Wang,; Y. Qiao, Learning attentive pairwise interaction for fine-grained classification. arXiv preprint arXiv:2002.10191, 2020.
[35]
Z. Xu,; S. Huang,; Y. Zhang,; D. Tao, Augmenting strong supervision using web data for finegrainedcategorization. In: Proceedings of the IEEE International Conference on Computer Vision, 2524-2532, 2015.
[36]
L. Niu,; A. Veeraraghavan,; A. Sabharwal, Fine-grained classification using heterogeneous web data and auxiliary categories. arXiv preprint arXiv:1811.07567, 2018.
[37]
A. Torralba,; A. A. Efros, Unbiased look at dataset bias. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1521-1528, 2011.
[38]
T. Hu,; H. G. Qi,; Q. M. Huang,; Y. Lu, See better before looking closer: Weakly supervised data augmentation network for fine-grained visual classification. arXiv preprint arXiv:1901.09891, 2019.
[39]
J. Krause,; M. Stark,; J. Deng,; Fei-Fei. L. 3D object representations for fine-grained categorization. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 554-561, 2013.
[40]
S. Maji,; E. Rahtu,; J. Kannala,; M. Blaschko,; A. Vedaldi, Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013.
[41]
M. Nilsback,; A. Zisserman, Automated flower classification over a large number of classes. In: Proceedings of the 6th Indian Conference on Computer Vision, Graphics & Image Processing, 722-729, 2008.
[42]
J. Deng,; W. Dong,; R. Socher,; L. Li,; K. Li,; L. Fei-Fei, ImageNet: A large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 248-255, 2009.
[43]
M. Everingham,; L. van Gool,; C. K. I. Williams,; J. Winn,; A. Zisserman, The pascal visual object classes (VOC) challenge. International Journal of Computer Vision Vol. 88, No. 2, 303-338, 2010.
Google Scholar
[44]
T. Lin,; M. Maire,; S. Belongie,; L. Bourdev,; R. Girshick,; J. Hays,; P. Perona,; D. Ramanan,; C. L. Zitnick,; P. Dollár, Microsoft COCO: Common objects in context. arXiv preprint arXiv:1405.0312, 2014.
[45]
Z. Wang,; A. C. Bovik,; H. R. Sheikh,; E. P. Simoncelli, Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing Vol. 13, No. 4, 600-612, 2004.
Google Scholar
[46]
B. C. Russell,; A. Torralba,; K. P. Murphy,; W. T. Freeman, LabelMe: A database and web-based tool for image annotation. International Journal of Computer Vision Vol. 77, Nos. 1-3, 157-173, 2008.
Google Scholar
[47]
G. Huang,; Z. Liu,; L. van der Maaten,; K. Q. Weinberger, Densely connected convolutional networks. arXiv preprint arXiv:1608.06993, 2016.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 18 May 2020
Accepted: 14 June 2020
Published: 01 October 2020
Issue date: December 2020

Copyright

© The Author(s) 2020

Acknowledgements

The authors would like to thank Wei-Yu Xie for his assistance on paper writing, and also thank Qiu Xin and Zhi-Ping Zhang for much help on image processing and labeling. This work was supported by the National Natural Science Foundation of China (Project Nos. 61521002 and 61772298), a Research Grant of Beijing Higher Institution Engineering Research Center, and Tsinghua-Tencent Joint Laboratory for Internet Innovation Technology.

Rights and permissions

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Return