Journal Home > Volume 8 , Issue 4
Computational Visual Media 2022, 8(4): 553-570 https://doi.org/10.1007/s41095-021-0254-4
Research Article | Open Access | Issue | Published: 16 May 2022

Automatic location and semantic labeling of landmarks on 3D human body models

Show Author's Information Shan Luo1,  Qitong Zhang1,  Jieqing Feng1( )
State Key Lab of CAD&CG, Zhejiang University, Hangzhou 310058, China
Keywords:
3D human body model, descriptor, land-marking, shape alignment
Cite this article:
Luo S, Zhang Q, Feng J. Automatic location and semantic labeling of landmarks on 3D human body models. Computational Visual Media, 2022, 8(4): 553-570. https://doi.org/10.1007/s41095-021-0254-4
Download citation
EndNote(RIS)
BibTeX

857

Views

73

Downloads

Citations

1

Crossref

0

WoS

1

Scopus

0

CSCD

Abstract Full text About this article

Landmarks on human body models are of great significance for applications such as digital anthropometry and clothing design. The diversity of pose and shape of human body models and the semantic gap make landmarking a challenging problem. Inthis paper, a learning-based method is proposed to locate landmarks on human body models by analyzing the relationship between geometric descriptors and semantic labels of landmarks. A shape alignmentalgorithm is proposed to align human body models to break symmetric ambiguity. A symmetry-awaredescriptor is proposed based on the structure of the human body models, which is robust to both pose and shape variations in human body models. AnAdaBoost regression algorithm is adopted to establish the correspondence between several descriptors and semantic labels of the landmarks. Quantitative and qualitative analyses and comparisons show that the proposed method can obtain more accurate landmarks and distinguish symmetrical landmarks semantically. Additionally, a dataset of landmarked human body models is also provided, containing 271 human body models collected from current human body datasets; each model has 17 landmarks labeled manually.


menu
Abstract
Full text
Outline
About this article

Automatic location and semantic labeling of landmarks on 3D human body models

Show Author's information Shan Luo1, Qitong Zhang1, Jieqing Feng1( )
State Key Lab of CAD&CG, Zhejiang University, Hangzhou 310058, China

Abstract

Landmarks on human body models are of great significance for applications such as digital anthropometry and clothing design. The diversity of pose and shape of human body models and the semantic gap make landmarking a challenging problem. Inthis paper, a learning-based method is proposed to locate landmarks on human body models by analyzing the relationship between geometric descriptors and semantic labels of landmarks. A shape alignmentalgorithm is proposed to align human body models to break symmetric ambiguity. A symmetry-awaredescriptor is proposed based on the structure of the human body models, which is robust to both pose and shape variations in human body models. AnAdaBoost regression algorithm is adopted to establish the correspondence between several descriptors and semantic labels of the landmarks. Quantitative and qualitative analyses and comparisons show that the proposed method can obtain more accurate landmarks and distinguish symmetrical landmarks semantically. Additionally, a dataset of landmarked human body models is also provided, containing 271 human body models collected from current human body datasets; each model has 17 landmarks labeled manually.

Keywords: 3D human body model, descriptor, land-marking, shape alignment

References(45)

[1]
Creusot, C.; Pears, N.; Austin, J. A machine-learning approach to keypoint detection and landmarking on 3D meshes. International Journal of Computer Vision Vol. 102, Nos. 1-3, 146-179, 2013.
DOI Google Scholar
[2]
Wang, H.; Guo, J.; Yan, D. M.; Quan, W.; Zhang, X. Learning 3D keypoint descriptors for non-rigid shape matching. In: Computer Vision - ECCV 2018. Lecture Notes in Computer Science, Vol. 11212. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 3-20, 2018.
[3]
Guo, Y. L.; Bennamoun, M.; Sohel, F.; Lu, M.; Wan, J. W. 3D object recognition in cluttered scenes with local surface features: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 36, No. 11, 2270-2287, 2014.
DOI Google Scholar
[4]
Yang, Y.; Fu, X. M.; Chai, S. M.; Xiao, S. W.; Liu, L. G. Volume-enhanced compatible remeshing of 3D models. IEEE Transactions on Visualization and Computer Graphics Vol. 25, No. 10, 2999-3010, 2019.
DOI Google Scholar
[5]
Jones, P. R. M.; Rioux, M. Three-dimensional surfaceanthropometry: Applications to the human body. Optics and Lasers in Engineering Vol. 28, No. 2, 89-117, 1997.
DOI Google Scholar
[6]
Treleaven, P.; Wells, J. 3D body scanning and healthcare applications. Computer Vol. 40, No. 7, 28-34, 2007.
DOI Google Scholar
[7]
You, Y.; Lou, Y. J.; Li, C. K.; Cheng, Z. J.; Li, L. W.; Ma, L. Z.; Lu, C.; Wang, W. KeypointNet: A large-scale 3D keypoint dataset aggregated from numerous human annotations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13644-13653, 2020.
DOI
[8]
Allen, B.; Curless, B.; Popović, Z. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Transactions on Graphics Vol. 22, No. 3, 587-594, 2003.
DOI Google Scholar
[9]
Giachetti, A.; Mazzi, E.; Piscitelli, F.; Aono, M.; Hamza, A. B.; Bonis, T.; Claes, P.; Godil, A.; Li, C.; Ovsjanikov, M.; et al. SHREC’14 track: Automatic location of landmarks used in manual anthropometry. In: Eurographics Workshop on 3D Object Retrieval (2014). Bustos, B.; Tabia, H.; Vandeborre, J. P.; Veltkamp, R. Eds. The Eurographics Association, 2014.
[10]
Sung, M.; Su, H.; Yu, R.; Guibas, L. Deep functional dictionaries: Learning consistent semantic structures on 3D models from functions. In: Proceedings of the 32nd Conference on Neural Information Processing Systems, 2018.
[11]
Chaouch, M.; Verroust-Blondet, A. Alignment of 3D models. Graphical Models Vol. 71, No. 2, 63-76, 2009.
DOI Google Scholar
[12]
Haim, N.; Segol, N.; Ben-Hamu, H.; Maron, H.; Lipman, Y. Surface networks via general covers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 632-641, 2019.
DOI
[13]
Hanocka, R.; Hertz, A.; Fish, N.; Giryes, R.; Fleishman, S.; Cohen-Or, D. MeshCNN: A network with an edge. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 90, 2019.
DOI Google Scholar
[14]
Wiersma, R.; Eisemann, E.; Hildebrandt, K. CNNs on surfaces using rotation-equivariant features. ACM Transactions on Graphics Vol. 39, No. 4, Article No. 92, 2020.
DOI Google Scholar
[15]
Johnson, A. E.; Hebert, M. Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 21, No. 5, 433-449, 1999.
DOI Google Scholar
[16]
Frome, A.; Huber, D.; Kolluri, R.; Bülow, T.; Malik, J. Recognizing objects in range data using regional point descriptors. In: Computer Vision - ECCV 2004. Lecture Notes in Computer Science, Vol. 3023. Pajdla, T.; Matas, J. Eds. Springer Berlin Heidelberg, 224-237, 2004.
DOI
[17]
Shapira, L.; Shamir, A.; Cohen-Or, D. Consistent mesh partitioning and skeletonisation using the shape diameter function. The Visual Computer Vol. 24, No. 4, 249-259, 2008.
DOI Google Scholar
[18]
Rustamov, R. M. Laplace-Beltrami eigenfunctions for deformation invariant shape representation. In: Proceedings of the 5th Eurographics Symposium on Geometry Processing, 225-233, 2007.
[19]
Sun, J.; Ovsjanikov, M.; Guibas, L. A concise and provably informative multi-scale signature based on heat diffusion. Computer Graphics Forum Vol. 28, No. 5, 1383-1392, 2009.
DOI Google Scholar
[20]
Bronstein, M. M.; Kokkinos, I. Scale-invariant heat kernel signatures for non-rigid shape recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1704-1711, 2010.
DOI
[21]
Aubry, M.; Schlickewei, U.; Cremers, D. The wave kernel signature: A quantum mechanical approach to shape analysis. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 1626-1633, 2011.
DOI
[22]
Meng, W.; Yi, F. Local diffusion map signature for symmetry-aware non-rigid shape correspondence. In: Proceedings of the 24th ACM International Conference on Multimedia, 526-530, 2016.
[23]
Ren, J.; Poulenard, A.; Wonka, P.; Ovsjanikov, M. Continuous and orientation-preserving correspondences via functional maps. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 248, 2018.
DOI Google Scholar
[24]
Wang, Y. Q.; Guo, J. W.; Yan, D. M.; Wang, K.; Zhang, X. P. A robust local spectral descriptor for matching non-rigid shapes with incompatible shape structures. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6224-6233, 2019.
DOI
[25]
Li, Y.; Zhong, Y. Q. Automatic detecting anthropometric landmarks based on spin image. Textile Research Journal Vol. 82, No. 6, 622-632, 2012.
DOI Google Scholar
[26]
Wuhrer, S.; Azouz, Z. B.; Shu, C. Semi-automatic prediction of landmarks on human models in varying poses. In: Proceedings of the Canadian Conference on Computer and Robot Vision, 136-142, 2010.
DOI
[27]
Azouz, Z. B.; Shu, C.; Mantel, A. Automatic locating of anthropometric landmarks on 3D human models.In: Proceedings of the International Symposium on 3D Data Processing, Visualization, and Transmission, 750-757, 2006.
DOI
[28]
Lovato, C.; Castellani, U.; Zancanaro, C.; Giachetti, A. Automatic labelling of anatomical landmarks on 3D body scans. Graphical Models Vol. 76, No. 6, 648-657, 2014.
DOI Google Scholar
[29]
Shu, Z. Y.; Xin, S. Q.; Xu, X.; Liu, L. G.; Kavan, L. Detecting 3D points of interest using multiple features and stacked auto-encoder. IEEE Transactions on Visualization and Computer Graphics Vol. 25, No. 8, 2583-2596, 2019.
DOI Google Scholar
[30]
Xi, P. C.; Shu, C.; Goubran, R. Localizing 3-D anatomical landmarks using deep convolutional neural networks. In: Proceedings of the 14th Conference on Computer and Robot Vision, 197-204, 2017.
[31]
Yi, L.; Su, H.; Guo, X. W.; Guibas, L. SyncSpecCNN: Synchronized spectral CNN for 3D shape segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6584-6592, 2017.
DOI
[32]
Zhou, Z. K.; Hao, S. J. Anatomical landmark detection on 3D human shapes by hierarchically utilizing multipleshape features. Neurocomputing Vol. 253, 162-168, 2017.
DOI Google Scholar
[33]
Marin, R.; Melzi, S.; Rodolà, E.; Castellani, U. FARM: Functional automatic registration method for 3D human bodies. Computer Graphics Forum Vol. 39, No. 1, 160-173, 2020.
DOI Google Scholar
[34]
Guo, J. W.; Wang, H. Y.; Cheng, Z. L.; Zhang, X. P.; Yan, D. M. Learning local shape descriptors for computing non-rigid dense correspondence. Computational Visual Media Vol. 6, No. 1, 95-112, 2020.
DOI Google Scholar
[35]
Luo, S.; Feng, J. Q. Symmetry-aware kinematic skeleton generation of a 3D human body model. Multimedia Tools and Applications Vol. 79, Nos. 29-30, 20579-20602, 2020.
DOI Google Scholar
[36]
Baran, I.; Popović, J. Automatic rigging and animation of 3D characters. ACM Transactions on Graphics Vol. 26, No. 3, 72-es, 2007.
DOI Google Scholar
[37]
Anguelov, D.; Srinivasan, P.; Koller, D.; Thrun, S.; Davis, J. SCAPE: Shape completion and animation of people. ACM Transactions on Graphics Vol. 24, No. 3, 408-416, 2005.
DOI Google Scholar
[38]
Bogo, F.; Romero, J.; Loper, M.; Black, M. J. FAUST: Dataset and evaluation for 3D mesh registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3794-3801, 2014.
DOI
[39]
Yang, Y. P.; Yu, Y.; Zhou, Y.; Du, S. D.; Davis, J.; Yang, R. G. Semantic parametric reshaping of human body models. In: Proceedings of the 2nd International Conference on 3D Vision, 41-48, 2014.
DOI
[40]
Maron, H.; Galun, M.; Aigerman, N.; Trope, M.; Dym,N.; Yumer, E.; Kim, V. G.; Lipman, Y. Convolutional neural networks on surfaces via seamless toric covers. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 71, 2017.
DOI Google Scholar
[41]
Loper, M.; Mahmood, N.; Romero, J.; Pons-Moll, G.; Black, M. J. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics Vol. 34, No. 6, Article No. 248, 2015.
DOI Google Scholar
[42]
Chen, X. B.; Golovinskiy, A.; Funkhouser, T. Abenchmark for 3D mesh segmentation. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 73, 2009.
DOI Google Scholar
[43]
Xu, Y. F.; Fan, T. Q.; Xu, M. Y.; Zeng, L.; Qiao, Y.SpiderCNN: Deep learning on point sets withparameterized convolutional filters. In: Computer Vision -ECCV 2018. Lecture Notes in Computer Science, Vol. 11212. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 90-105, 2018.
[44]
Wang, Y.; Sun, Y. B.; Liu, Z. W.; Sarma, S. E.; Bronstein, M. M.; Solomon, J. M. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics Vol. 38, No. 5, Article No. 146, 2019.
DOI Google Scholar
[45]
Wu, W. X.; Qi, Z. A.; Li, F. X. PointConv: Deepconvolutional networks on 3D point clouds. In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9613-9622, 2019.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 02 July 2021
Accepted: 08 September 2021
Published: 16 May 2022
Issue date: December 2022

Copyright

© The Author(s) 2021.

Acknowledgements

This work was jointly supported by the National Natural Science Foundation of China under Grant Nos. 61732015, 61932018, and 61472349.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.

Return