Review Article
Open Access
Issue
Published:
10 June 2020
Open Access
Issue
Published:
10 June 2020
A survey on deep geometry learning: From a representation perspective
Lin Gao1(
)
)
Cite this article:
Xiao Y-P, Lai Y-K, Zhang F-L, et al.
A survey on deep geometry learning: From a representation perspective.
Computational Visual Media,
2020, 6(2): 113-133.
https://doi.org/10.1007/s41095-020-0174-8
Download citation
1110
Views
85
Downloads
Citations
66
Crossref
N/A
WoS
65
Scopus
9
CSCD
Researchers have achieved great success in dealing with 2D images using deep learning. In recent years, 3D computer vision and geometry deep learning have gained ever more attention. Many advanced techniques for 3D shapes have been proposed for different applications. Unlike 2D images, which can be uniformly represented by a regular grid of pixels, 3D shapes have various representations, such as depth images, multi-view images, voxels, point clouds, meshes, implicit surfaces, etc. The performance achieved in different applications largely depends on the representa-tion used, and there is no unique representation that works well for all applications. Therefore, in this survey, we review recent developments in deep learning for 3D geometry from a representation perspective, summarizing the advantages and disadvantages of different representations for different applications. We also present existing datasets in these representations and further discuss future research directions.
menu
Abstract
Full text
Outline
About this article
A survey on deep geometry learning: From a representation perspective
Lin Gao1(
)
)
Abstract
Researchers have achieved great success in dealing with 2D images using deep learning. In recent years, 3D computer vision and geometry deep learning have gained ever more attention. Many advanced techniques for 3D shapes have been proposed for different applications. Unlike 2D images, which can be uniformly represented by a regular grid of pixels, 3D shapes have various representations, such as depth images, multi-view images, voxels, point clouds, meshes, implicit surfaces, etc. The performance achieved in different applications largely depends on the representa-tion used, and there is no unique representation that works well for all applications. Therefore, in this survey, we review recent developments in deep learning for 3D geometry from a representation perspective, summarizing the advantages and disadvantages of different representations for different applications. We also present existing datasets in these representations and further discuss future research directions.
Keywords:
3D shape representation, geometry learning;neural networks, computer graphics
References(120)
[1]
M. M. Bronstein,; J. Bruna,; Y. LeCun,; A. Szlam,; P. Vandergheynst, Geometric deep learning: Going beyond Euclidean data. IEEE Signal Processing Magazine Vol. 34, No. 4, 18-42, 2017.
[2]
E. Ahmed,; A. Saint,; A. E. R. Shabayek,; K. Cherenkova,; R. Das,; G. Gusev,; D. Aouada,; B. Ottersten, Deep learning advances on different 3D data representations: A survey. arXiv preprint arXiv:1808.01462, 1, 2018.
[3]
Y. Guo,; H. Wang,; Q. Hu,; H. Liu,; L. Liu,; M. Bennamoun, Deep learning for 3D point clouds: A survey. arXiv preprint arXiv:1912.12033, 2019.
[4]
A. Krizhevsky,; I. Sutskever,; G. E. Hinton, ImageNet classification with deep convolutional neural networks. In: Proceedings of the Advances in Neural Information Processing Systems, 1097-1105, 2012.
[5]
Y. LeCun,; K. Kavukcuoglu,; C. Farabet, Convolutional networks and applications in vision. In: Proceedings of the IEEE International Symposium on Circuits and Systems, 253-256, 2010.
[6]
R. Q. Charles,; S. Hao,; K. C. Mo,; L. J. Guibas, PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 652-660, 2017.
[7]
C. R. Qi,; L. Yi,; H. Su,; L. J. Guibas, PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the Advances in Neural Information Processing Systems, 5099-5108, 2017.
[8]
L. Mescheder,; M. Oechsle,; M. Niemeyer,; S. Nowozin,; A. Geiger, Occupancy networks: Learning 3D reconstruction in function space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4460-4470, 2019.
[9]
Q. Xu,; W. Wang,; D. Ceylan,; R. Mech,; U. Neumann, DISN: Deep implicit surface network for high-quality single-view 3D reconstruction. In: Proceedings of the Advances in Neural Information Processing Systems, 490-500, 2019.
[10]
W. E. Lorensen,; H. E. Cline, Marching cubes: A high resolution 3D surface construction algorithm. ACM SIGGRAPH Computer Graphics Vol. 21, No. 4, 163-169, 1987.
[11]
C. H. Zou,; E. Yumer,; J. M. Yang,; D. Ceylan,; D. Hoiem, 3D-PRNN: Generating shape primitives with recurrent neural networks. In: Proceedings of the IEE International Conference on Computer Vision, 900-909, 2017.
[12]
J. Li,; K. Xu,; S. Chaudhuri,; E. Yumer,; H. Zhang,; L. Guibas, GRASS: Generative recursive autoencoders for shape structures. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 52, 2017.
[13]
Z. R. Wu,; S. R. Song,; A. Khosla,; F. Yu,; L. G. Zhang,; X. O. Tang,; J. Xiao, 3D ShapeNets: A deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1912-1920, 2015.
[14]
H. Su,; S. Maji,; E. Kalogerakis,; E. Learned-Miller, Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of the IEEE International Conference on Computer Vision, 945-953, 2015.
[15]
J. Masci,; D. Boscaini,; M. M. Bronstein,; P. Vandergheynst, Geodesic convolutional neural networks on Riemannian manifolds. In: Proceedings of the IEEE International Conference on Computer Vision Workshop, 37-45, 2015.
[16]
D. Eigen,; C. Puhrsch,; R. Fergus, Depth map prediction from a single image using a multi-scale deep network. In: Proceedings of the Advances in Neural Information Processing Systems, 2366-2374, 2014.
[17]
L. Gao,; Y.-K. Lai,; D. Liang,; S.-Y. Chen,; S. Xia, Efficient and flexible deformation representation for data-driven surface modeling. ACM Transactions on Graphics Vol. 35, No. 5, Article No. 158, 2016.
[18]
C. B. Choy,; D. F. Xu,; J. Gwak,; K. Chen,; S. Savarese, 3D-R2N2: A unified approach for single and multi-view 3D object reconstruction. In: Computer Vision - ECCV 2016. Lecture Notes in Computer Science, Vol. 9912. B. Leibe,; J. Matas,; N. Sebe,; M. Welling, Eds. Springer Cham, 628-644, 2016.
[19]
J. Wu,; C. Zhang,; T. Xue,; B. Freeman,; J. Tenenbaum, Learning a probabilistic latent space of object shapes via 3D generativeadversarial modeling. In: Proceedings of the Advances in Neural Information Processing Systems, 82-90, 2016.
[20]
H. Q. Fan,; H. Su,; L. Guibas, A point set generation network for 3D object reconstruction from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 605-613, 2017.
[21]
G. Riegler,; A. O. Ulusoy,; A. Geiger, OctNet: Learning deep 3D representations at high resolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3577-3586, 2017.
[22]
P.-S. Wang,; Y. Liu,; Y.-X. Guo,; C.-Y. Sun,; X. Tong, O-CNN: Octree-based convolutional neural networks for 3D shape analysis. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 72, 2017.
[23]
N. Y. Wang,; Y. D. Zhang,; Z. W. Li,; Y. W. Fu,; W. Liu,; Y. G. Jiang, Pixel2Mesh: Generating 3D mesh models from single RGB images. In: Computer Vision - ECCV 2018. Lecture Notes in Computer Science, Vol. 11215. V. Ferrari,; M. Hebert,; C. Sminchisescu,; Y. Weiss, Eds. Springer Cham, 55-71, 2018.
[24]
Y. Li,; R. Bu,; M. Sun,; W. Wu,; X. Di,; B. Chen, PointCNN: Convolution on xtransformed points. In: Proceedings of the Advances in Neural Information Processing Systems, 820-830, 2018.
[25]
J. J. Park,; P. Florence,; J. Straub,; R. Newcombe,; S. Lovegrove, DeepSDF: Learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
[26]
Z. Q. Chen,; H. Zhang, Learning implicit fields for generative shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5939-5948, 2019.
[27]
L. Gao,; J. Yang,; T. Wu,; Y.-J. Yuan,; H. Fu,; Y.-K. Lai,; H. Zhang, SDM-NET: Deep generative network for structured deformable mesh. ACM Transactions on Graphics Vol. 38, No. 6, Article No. 243, 2019.
[28]
K. Mo,; P. Guerrero,; L. Yi,; H. Su,; P. Wonka,; N. J. Mitra,; L. J. Guibas, StructureNet: Hierarchical graph networks for 3D shape generation. ACM Transactions on Graphics Vol. 38, No. 6, Article No. 242, 2019.
[29]
R. Hanocka,; A. Hertz,; N. Fish,; R. Giryes,; S. Fleishman,; D. Cohen-Or, MeshCNN: A network with an edge. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 90, 2019.
[30]
S. Liu,; S. Saito,; W. Chen,; H. Li, Learning to infer implicit surfaces without 3D supervision. In: Proceedings of the Advances in Neural Information Processing Systems, 8293-8304, 2019.
[31]
Z. Chen,; A. Tagliasacchi,; H. Zhang, BSP-Net: Generating compact meshes via binary space partitioning. arXiv preprint arXiv:1911.06971, 2019.
[32]
T. Jeruzalski,; B. Deng,; M. Norouzi,; J. Lewis,; G. Hinton,; A. Tagliasacchi, NASA: Neural articulated shape approximation. arXiv preprint arXiv:1912.03207, 2019.
[33]
R. Socher,; B. Huval,; B. Bath,; C. D. Manning,; A. Y. Ng, Convolutional-recursive deep learning for 3D object classification. In: Proceedings of the Advances in Neural Information Processing Systems, 656-664, 2012.
[34]
S. Gupta,; R. Girshick,; P. Arbeláez,; J. Malik, Learning rich features from RGB-D images for object detection and segmentation. In: Computer Vision - ECCV 2014. Lecture Notes in Computer Science, Vol. 8695. D. Fleet,; T. Pajdla,; B. Schiele,; T. Tuytelaars, Eds. Springer, Cham, 345-360, 2014.
[35]
S. Gupta,; P. Arbelaez,; R. Girshick,; J. Malik, Aligning 3D models to RGB-D images of cluttered scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4731-4740, 2015.
[36]
S. R. Song,; J. X. Xiao, Deep sliding shapes for amodal 3D object detection in RGB-D images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 808-816, 2016.
[37]
C. R. Qi,; H. Su,; M. NieBner,; A. Dai,; M. Y. Yan,; L. J. Guibas, Volumetric and multi-view CNNs for object classification on 3D data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5648-5656, 2016.
[38]
G. E. Hinton,; S. Osindero,; Y. W. Teh, A fast learning algorithm for deep belief nets. Neural Computation Vol. 18, No. 7, 1527-1554, 2006.
[39]
D. Maturana,; S. Scherer, 3D convolutional neural networks for landing zone detection from LiDAR. In: Proceedings of the IEEE International Conference on Robotics and Automation, 3471-3478, 2015.
[40]
D. Maturana,; S. Scherer, VoxNet: A 3D convolutional neural network for real-time object recognition. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 922-928, 2015.
[41]
I. Goodfellow,; J. Pouget-Abadie,; M. Mirza,; B. Xu,; D. Warde-Farley,; S. Ozair,; A. Courville,; Y. Bengio, Generative adversarial nets. In: Proceedings of the Advances in Neural Information Processing Systems, 2672-2680, 2014.
[42]
P. Vincent,; H. Larochelle,; Y. Bengio,; P. A. Manzagol, Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on Machine learning, 1096-1103, 2008.
[43]
P. Vincent,; H. Larochelle,; I. Lajoie,; Y. Bengio,; P.-A. Manzagol, Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research Vol. 11, 3371-3408, 2010.
[44]
A. Sharma,; O. Grau,; M. Fritz, VConv-DAE: Deep volumetric shape learning without object labels. In: Computer Vision - ECCV 2016 Workshops. Lecture Notes in Computer Science, Vol. 9915. G. Hua,; H. Jégou, Eds. Springer Cham, 236-250, 2016.
[45]
R. Girdhar,; D. F. Fouhey,; M. Rodriguez,; A. Gupta, Learning a predictable and generative vector representation for objects. In: Computer Vision - ECCV 2016. Lecture Notes in Computer Science, Vol. 9910. B. Leibe,; J. Matas,; N. Sebe,; M. Welling, Eds. Springer Cham, 484-499, 2016.
[46]
S. Hochreiter,; J. Schmidhuber, Long short-term memory. Neural Computation Vol. 9, No. 8, 1735-1780, 1997.
[47]
K. Cho,; B. Van Merriënboer,; C. Gulcehre,; D. Bahdanau,; F. Bougares,; H. Schwenk,; Y. Bengio, Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014.
[48]
A. B. L. Larsen,; S. K. Sønderby,; H. Larochelle,; O. Winther, Autoencoding beyond pixels using a learned similarity metric. arXiv preprint arXiv:1512.09300, 2015.
[49]
M. Lin,; Q. Chen,; S. Yan, Network in network. arXiv preprint arXiv:1312.4400, 2013.
[50]
N. Sedaghat,; M. Zolfaghari,; E. Amiri,; T. Brox, Orientation-boosted voxel nets for 3D object recognition. In: Proceedings of the British Machine Vision Conference, 2017.
[51]
Y. Li,; S. Pirk,; H. Su,; C. R. Qi,; L. J. Guibas, FPNN: Field probing neural networks for 3D data. In: Proceedings of the Advances in Neural Information Processing Systems, 307-315, 2016.
[52]
D. Meagher, Geometric modeling using octree encoding. Computer Graphics and Image Processing Vol. 19, No. 2, 129-147, 1982.
[53]
C. Hane,; S. Tulsiani,; J. Malik, Hierarchical surface prediction for 3D object reconstruction. In: Proceedings of the International Conference on 3D Vision, 412-420, 2017.
[54]
M. Tatarchenko,; A. Dosovitskiy,; T. Brox, Octree generating networks: Efficient convolutional architectures for high-resolution 3D outputs. In: Proceedings of the IEEE International Conference on Computer Vision, 2088-2096, 2017.
[55]
P.-S. Wang,; C.-Y. Sun,; Y. Liu,; X. Tong, Adaptive O-CNN: A patch-based deep representation of 3D shapes. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 217, 2018.
[56]
Y. Rubner,; C. Tomasi,; L. J. Guibas The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision Vol. 40, No. 2, 99-121, 2000.
[57]
Y. Wang,; Y. B. Sun,; Z. W. Liu,; S. E. Sarma,; M. M. Bronstein,; J. M. Solomon, Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics Vol. 38, No. 5, Article No. 146, 2019.
[58]
R. Klokov,; V. Lempitsky, Escape from cells: Deep kd-networks for the recognition of 3D point cloud models. In: Proceedings of the IEEE International Conference on Computer Vision, 863-872, 2017.
[59]
Y. Q. Yang,; C. Feng,; Y. R. Shen,; D. Tian, FoldingNet: Point cloud auto-encoder via deep grid deformation. In: Proceedings of the IEEE/ CVF Conference on Computer Vision and Pattern Recognition, 206-215, 2018.
[60]
E. Mehr,; A. Jourdan,; N. Thome,; M. Cord,; V. Guitteny, DiscoNet: Shapes learning on disconnected manifolds for 3D editing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 3474-3483, 2019.
[61]
H. Y. Meng,; L. Gao,; Y. K. Lai,; D. Manocha, VV-net: Voxel VAE net with group convolutions for point cloud segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 8500-8508, 2019.
[62]
L. Q. Yu,; X. Z. Li,; C. W. Fu,; D. Cohen-Or,; P. A. Heng, PU-Net: Point cloud upsampling network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2790-2799, 2018.
[63]
Y. F. Wang,; S. H. Wu,; H. Huang,; D. Cohen-Or,; O. Sorkine-Hornung, Patch-based progressive 3D point set upsampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5958-5967, 2019.
[64]
R. H. Li,; X. Z. Li,; C.W. Fu,; D. Cohen-Or,; P.A. Heng, PU-GAN: A point cloud upsampling adversarial network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 7203-7212, 2019.
[65]
Y. Wang,; J. Solomon, Deep closest point: Learning representations for point cloud registration. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 3523-3532, 2019.
[66]
P. J. Besl,; N. D. McKay, Method for registration of 3-D shapes. In: Proceedings of the SPIE 1611, Sensor Fusion IV: Control Paradigms and Data Structures, 586-606, 1992.
[67]
A. Sinha,; J. Bai,; K. Ramani, Deep learning 3D shape surfaces using geometry images. In: Computer Vision - ECCV 2016. Lecture Notes in Computer Science, Vol. 9910. B. Leibe,; J. Matas,; N. Sebe,; M. Welling, Eds. Springer Cham, 223-240, 2016.
[68]
H. Maron,; M. Galun,; N. Aigerman,; M. Trope,; N. Dym,; E. Yumer,; V. G. Kim,; Y. Lipman, Convolutional neural networks on surfaces via seamless toric covers. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 71, 2017.
[69]
A. Sinha,; A. Unmesh,; Q. X. Huang,; K. Ramani, SurfNet: Generating 3D shape surfaces using deep residual networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6040-6049, 2017.
[70]
B. G. Shi,; S. Bai,; Z. C. Zhou,; X. Bai, DeepPano: Deep panoramic representation for 3-D shape recognition. IEEE Signal Processing Letters Vol. 22, No. 12, 2339-2343, 2015.
[71]
J. W. Huang,; H. T. Zhang,; L. Yi,; T. Funkhouser,; M. NieBner,; L. J. Guibas, TextureNet: Consistent local parametrizations for learning from high-resolution signals on meshes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4440-4449, 2019.
[72]
J. Bruna,; W. Zaremba,; A. Szlam,; Y. LeCun, Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203, 2013.
[73]
M. Henaff,; J. Bruna,; Y. LeCun, Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163, 2015.
[74]
M. Defferrard,; X. Bresson,; P. Vandergheynst, Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the Advances in Neural Information Processing Systems, 3844-3852, 2016.
[75]
T. N. Kipf,; M. Welling, Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
[76]
J. Atwood,; D. Towsley, Diffusionconvolutional neural networks. In: Proceedings of the Advances in Neural Information Processing Systems, 1993-2001, 2016.
[77]
N. Verma,; E. Boyer,; J. Verbeek, FeaStNet: Feature-steered graph convolutions for 3D shape analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2598-2606, 2018.
[78]
D. Boscaini,; J. Masci,; S. Melzi,; M. M. Bronstein,; U. Castellani,; P. Vandergheynst, Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks. Computer Graphics Forum Vol. 34, No. 5, 13-23, 2015.
[79]
D. Boscaini,; J. Masci,; E. Rodolà,; M. Bronstein, Learning shape correspondence with anisotropic convolutional neural networks. In: Proceedings of the Advances in Neural Information Processing Systems, 3189-3197, 2016.
[80]
H. T. Xu,; M. Dong,; Z. C. Zhong, Directionally convolutional networks for 3D shape segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, 2698-2707, 2017.
[81]
F. Monti,; D. Boscaini,; J. Masci,; E. Rodola,; J. Svoboda,; M. M. Bronstein, Geometric deep learning on graphs and manifolds using mixture model CNNs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5115-5124, 2017.
[82]
M. Fey,; J. E. Lenssen,; F. Weichert,; H. Müller, SplineCNN: Fast geometric deep learning with continuous B-spline kernels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 869-877, 2018.
[83]
H. Pan,; S. Liu,; Y. Liu,; X. Tong, Convolutional neural networks on 3D surfaces using parallel frames. arXiv preprint arXiv:1808.04952, 2018.
[84]
Y.-L. Qiao,; L. Gao,; J. Yang,; P. L. Rosin,; Y.-K. Lai,; X. Chen, LaplacianNet: Learning on 3D meshes with Laplacian encoding and pooling. arXiv preprint arXiv:1910.14063, 2019.
[85]
C. Wen,; Y. D. Zhang,; Z. W. Li,; Y. W. Fu, Pixel2Mesh++: Multi-view 3D mesh generation via deformation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 1042-1051, 2019.
[86]
T. Groueix,; M. Fisher,; V. G. Kim,; B. C. Russell,; M. Aubry, A papier-Mache approach to learning 3D surface generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 216-224, 2018.
[87]
H. Ben-Hamu,; H. Maron,; I. Kezurer,; G. Avineri,; Y. Lipman, Multi-chart generative surface modeling. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 215, 2019.
[88]
J. Y. Pan,; X. G. Han,; W. K. Chen,; J. P. Tang,; K. Jia, Deep mesh reconstruction from single RGB images via topology modification networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 9964-9973, 2019.
[89]
J. P. Tang,; X. G. Han,; J. Y. Pan,; K. Jia,; X. Tong, A skeleton-bridged deep learning approach for generating meshes of complex topologies from single RGB images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4541-4550, 2019.
[90]
C. Nash,; Y. Ganin,; S. Eslami,; P. W. Battaglia PolyGen: An autoregressive generative model of 3D meshes. arXiv preprint arXiv:2002.10880, 2020.
[91]
A. Vaswani,; N. Shazeer,; N. Parmar,; J. Uszkoreit,; L. Jones,; A. N. Gomez,; L. Kaiser,; I. Polosukhin, Attention is all you need. In: Proceedings of the Advances in Neural Information Processing Systems, 5998-6008, 2017.
[92]
K. Genova,; F. Cole,; D. Vlasic,; A. Sarna,; W. Freeman,; T. Funkhouser, Learning shape templates with structured implicit functions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 7154-7164, 2019.
[93]
K. Genova,; F. Cole,; A. Sud,; A. Sarna,; T. Funkhouser, Deep structured implicit functions. arXiv preprint arXiv:1912.06126, 2019.
[94]
R. Wu,; Y. Zhuang,; K. Xu,; H. Zhang,; B. Chen, PQ-NET: A generative part seq2seq network for 3D shapes. arXiv preprint arXiv:1911.10949, 2019.
[95]
R. Socher,; C. C. Lin,; C. Manning,; A. Y. Ng, Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th International Conference on Machine Learning, 129-136, 2011.
[96]
Z. Wu,; X. Wang,; D. Lin,; D. Lischinski,; D. Cohen-Or,; H. Huang, SAGNet: Structure-aware generative network for 3D shape modeling. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 91, 2019.
[97]
H. Wang,; N. Schor,; R. Hu,; H. Huang,; D. Cohen-Or,; H. Huang, Global-tolocal generative model for 3D shapes. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 214, 2018.
[98]
Q. Y. Tan,; L. Gao,; Y. K. Lai,; S. H. Xia, Variational autoencoders for deforming 3D mesh models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5841-5850, 2018.
[99]
L. Gao,; Y.K. Lai,; J. Yang,; L.-X. Zhang,; S. H. Xia,; L. Kobbelt, Sparse data driven mesh deformation. IEEE Transactions on Visualization and Computer Graphics , 2019.
[100]
Q. Tan,; L. Gao,; Y.-K. Lai,; J. Yang,; S. Xia, Mesh-based autoencoders for localized deformation component analysis. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
[101]
D. K. Duvenaud,; D. Maclaurin,; J. Iparraguirre,; R. Bombarell,; T. Hirzel,; A. Aspuru-Guzik,; R. P. Adams, Convolutional networks on graphs for learning molecular fingerprints. In: Proceedings of the Advances in Neural Information Processing Systems, 2224-2232, 2015.
[102]
L. Gao,; J. Yang,; Y.-L. Qiao,; Y.-K. Lai,; P. L. Rosin,; W. Xu,; S. Xia, Automatic unpaired shape deformation transfer. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 237, 2018.
[103]
S. S. Huang,; H. B. Fu,; L. Y. Wei,; S. M. Hu, Support substructures: Support-induced part-level structural representation. IEEE Transactions on Visualization and Computer Graphics Vol. 22, No. 8, 2024-2036, 2016.
[104]
Y.-J. Yuan,; Y.-K. Lai,; J. Yang,; H. Fu,; L. Gao, Mesh variational autoencoders with edge contraction pooling. arXiv preprint arXiv:1908.02507, 2019.
[105]
Q. Y. Tan,; Z. R. Pan,; L. Gao,; D. Manocha, Realtime simulation of thin-shell deformable materials using CNN-based mesh embedding. IEEE Robotics and Automation Letters Vol. 5, No. 2, 2325-2332, 2020.
[106]
N. Silberman,; R. Fergus, Indoor scene segmentation using a structured light sensor. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2011.
[107]
N. Silberman,; D. Hoiem,; P. Kohli,; R. Fergus, Indoor segmentation and support inference from RGBD images. In: Computer Vision - ECCV 2012. Lecture Notes in Computer Science, Vol. 7576. A. Fitzgibbon,; S. Lazebnik,; P. Perona,; Y. Sato,; C. Schmid, Eds. Springer Berlin Heidelberg, 746-760, 2012.
[108]
A. Geiger,; P. Lenz,; C. Stiller,; R. Urtasun, Vision meets robotics: The KITTI dataset. The International Journal of Robotics Research Vol. 32, No. 11, 1231-1237, 2013.
[109]
A. Dai,; A. X. Chang,; M. Savva,; M. Halber,; T. Funkhouser,; M. Niessner, ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5828-5839, 2017.
[110]
Y. P. Cao,; Z. N. Liu,; Z. F. Kuang,; L. Kobbelt,; S. M. Hu, Learning to reconstruct high-quality 3D shapes with cascaded fully convolutional networks In: Computer Vision - ECCV 2018. Lecture Notes in Computer Science, Vol. 11213. V. Ferrari,; M. Hebert,; C. Sminchisescu,; Y. Weiss, Eds. Springer Cham, 626-643, 2018.
[111]
A. X. Chang,; T. Funkhouser,; L. Guibas,; P. Hanrahan,; Q. Huang,; Z. Li,; S. Savarese,; M. Savva,; S. Song,; H. Su, et al. ShapeNet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012, 2015.
[112]
Y. Xiang,; W. Kim,; W. Chen,; J. W. Ji,; C. Choy,; H. Su,; R. Mottaghi,; L. Guibas,; S. Savarese, ObjectNet3D: A large scale database for 3D object recognition. In: Computer Vision - ECCV 2016. Lecture Notes in Computer Science, Vol. 9912. B. Leibe,; J. Matas,; N. Sebe,; M. Welling, Eds. Springer Cham, 160-176, 2016.
[113]
S. R. Song,; F. Yu,; A. Zeng,; A. X. Chang,; M. Savva,; T. Funkhouser, Semantic scene completion from a single depth image. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[114]
K. C. Mo,; S. L. Zhu,; A. X. Chang,; L. Yi,; S. Tripathi,; L. J. Guibas,; H. Su, PartNet: A large-scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 909-918, 2019.
[115]
H. Fu,; R. Jia,; L. Gao,; M. Gong,; B. Zhao,; S. Maybank,; D. Tao, 3D-FUTURE: 3D FUrniture shape with TextURE. 2020. Available at https://tianchi.aliyun.com/specials/promotion/alibaba-3d-future.
[116]
A. M. Bronstein,; M. M. Bronstein,; R. Kimmel, Numerical Geometry of Non-Rigid Shapes. Springer Science & Business Media, 2008.
[117]
F. Bogo,; J. Romero,; M. Loper,; M. J. Black, FAUST: Dataset and evaluation for 3D mesh registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3794-3801, 2014.
[118]
N. Mahmood,; N. Ghorbani,; N. F. Troje,; G. Pons-Moll,; M. Black, AMASS: Archive of motion capture as surface shapes. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 5442-5451, 2019.
[119]
X. H. Liu,; Z. Z. Han,; Y.S. Liu,; M. Zwicker, Point2Sequence: Learning the shape representation of 3D point clouds with an attention-based sequence to sequence network. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 33, 8778-8785, 2019.
[120]
L. Gao,; L. X. Zhang,; H. Y. Meng,; Y. H. Ren,; Y. K. Lai,; L. Kobbelt, PRS-Net: Planar reflective symmetry detection net for 3D models. arXiv preprint arXiv:1910.06511, 2019.
Publication history
Copyright
Acknowledgements
Rights and permissions
Publication history
Received: 16 February 2020
Revised: 16 February 2020
Accepted: 17 April 2020
Published:
10 June 2020
Issue date: June 2020
Copyright
© The Author(s) 2020
Acknowledgements
This work was supported by the National Natural Science Foundation of China (61828204, 61872440), Beijing Municipal Natural Science Foundation (L182016), Youth Innovation Promotion Association CAS, CCF-Tencent Open Fund, Royal Society-Newton Advanced Fellowship (NAF\R2\192151), and the Royal Society (IES\R1\180126).
Rights and permissions
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.
FEEDBACK 
TOP