Review Article
Open Access
Issue
Published:
18 May 2023
Open Access
Issue
Published:
18 May 2023
A survey of deep learning-based 3D shape generation
Tai-Jiang Mu1(
),
),
Cite this article:
Xu Q-C, Mu T-J, Yang Y-L.
A survey of deep learning-based 3D shape generation.
Computational Visual Media,
2023, 9(3): 407-442.
https://doi.org/10.1007/s41095-022-0321-5
Download citation
1410
Views
259
Downloads
Citations
4
Crossref
1
WoS
2
Scopus
0
CSCD
Deep learning has been successfully used for tasks in the 2D image domain. Research on 3D computer vision and deep geometry learning has also attracted attention. Considerable achievements have been made regarding feature extraction and discrimination of 3D shapes. Following recent advances in deep generative models such as generative adversarial networks, effective generation of 3D shapes has become an active research topic. Unlike 2D images with a regular grid structure, 3D shapes have various representations, such as voxels, point clouds, meshes, and implicit functions. For deep learning of 3D shapes, shape representation has to be taken into account as there is no unified representation that can cover all tasks well. Factors such as the representativeness of geometry and topology often largely affect the quality of the generated 3D shapes. In this survey, we comprehensively review works on deep-learning-based 3D shape generation by classifying and discussing them in terms of the underlying shape representation and the architecture of the shape generator. The advantages and disadvantages of each class are further analyzed. We also consider the 3D shape datasets commonly used for shape generation. Finally, we present several potential research directions that hopefully can inspire future works on this topic.
menu
Abstract
Full text
Outline
About this article
A survey of deep learning-based 3D shape generation
Tai-Jiang Mu1(
),
),
Abstract
Deep learning has been successfully used for tasks in the 2D image domain. Research on 3D computer vision and deep geometry learning has also attracted attention. Considerable achievements have been made regarding feature extraction and discrimination of 3D shapes. Following recent advances in deep generative models such as generative adversarial networks, effective generation of 3D shapes has become an active research topic. Unlike 2D images with a regular grid structure, 3D shapes have various representations, such as voxels, point clouds, meshes, and implicit functions. For deep learning of 3D shapes, shape representation has to be taken into account as there is no unified representation that can cover all tasks well. Factors such as the representativeness of geometry and topology often largely affect the quality of the generated 3D shapes. In this survey, we comprehensively review works on deep-learning-based 3D shape generation by classifying and discussing them in terms of the underlying shape representation and the architecture of the shape generator. The advantages and disadvantages of each class are further analyzed. We also consider the 3D shape datasets commonly used for shape generation. Finally, we present several potential research directions that hopefully can inspire future works on this topic.
Keywords:
deep learning, generative models, 3D representations, geometry learning
References(226)
[1]
Zhang, Z. Y. Microsoft kinect sensor and its effect. IEEE MultiMedia Vol. 19, No. 2, 4–10, 2012.
[2]
Chang, A. X.; Funkhouser, T.; Guibas, L.; Hanrahan, P.; Huang, Q. X.; Li, Z. M.; Savarese, S.; Savva, M.; Song, S. R.; Su, H.; et al. ShapeNet: An information-rich 3D model repository. arXiv preprint arXiv:1512.03012, 2015.
[3]
Deng, J.; Dong, W.; Socher, R.; Li, L. J.; Kai, L.; Li, F. F. ImageNet: A large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 248–255, 2009.
[4]
Kirk, D. NVIDIA cuda software and gpu parallel computing architecture. In: Proceedings of the 6th International Symposium on Memory Management, 103–104, 2007.
[5]
Guo, M. H.; Xu, T. X.; Liu, J. J.; Liu, Z. N.; Jiang, P. T.; Mu, T. J.; Zhang, S. H.; Martin, R. R.; Cheng, M. M.; Hu, S. M. Attention mechanisms in computer vision: A survey. Computational Visual Media Vol. 8, No. 3, 331–368, 2022.
[6]
Cao, W. M.; Yan, Z. Y.; He, Z. Q.; He, Z. H. A comprehensive survey on geometric deep learning. IEEE Access Vol. 8, 35929–35949, 2020.
[7]
Karras, T.; Laine, S.; Aila, T. M. A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4396–4405, 2019.
[8]
Ramesh, A.; Pavlov, M.; Goh, G.; Gray, S.; Voss, C.; Radford, A.; Chen, M.; Sutskever, I. In: Proceedings of the 38th International Conference on Machine Learning, 8821–8831, 2021.
[9]
Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X. H.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An image is worth 16x16 words: Transformers for image recognition at scale. In: Proceedings of the International Conference on Learning Representations, 2021.
[10]
Huang, J. H.; Huang, S. S.; Song, H. X.; Hu, S. M. DI-fusion: Online implicit 3D reconstruction with deep priors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8928–8937, 2021.
[11]
Mildenhall, B.; Srinivasan, P. P.; Tancik, M.; Barron, J. T.; Ramamoorthi, R.; Ng, R. NeRF: Representing scenes as neural radiance fields for view synthesis. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12346. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 405–421, 2020.
[12]
Saito, S.; Huang, Z.; Natsume, R.; Morishima, S.; Li, H.; Kanazawa, A. PIFu: Pixel-aligned implicit function for high-resolution clothed human digitization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2304–2314, 2019.
[13]
Wu, Z. R.; Song, S. R.; Khosla, A.; Yu, F.; Zhang, L. G.; Tang, X. O.; Xiao, J. X. 3D ShapeNets: A deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1912–1920, 2015.
[14]
Yan, X.; Yang, J.; Yumer, E.; Guo, Y.; Lee, H. Perspective transformer nets: Learning single-view 3D object reconstruction without 3D supervision. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, 1704–1712, 2016.
[15]
Girdhar, R.; Fouhey, D. F.; Rodriguez, M.; Gupta, A. Learning a predictable and generative vector representation for objects. In: Computer Vision – ECCV 2016. Lecture Notes in Computer Science, Vol. 9910. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 484–499, 2016.
[16]
Choy, C. B.; Xu, D. F.; Gwak, J.; Chen, K.; Savarese, S. 3D-R2N2: A unified approach for single and multi-view 3D object reconstruction. In: Computer Vision – ECCV 2016. Lecture Notes in Computer Science, Vol. 9912. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 628–644, 2016.
[17]
Wu, J.; Wang, Y.; Xue, T.; Sun, X.; Freeman, B.; Tenenbaum, J. MarrNet: 3D shape reconstruction via 2.5D sketches. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, 540–550, 2017.
[18]
Wu, J. J.; Zhang, C. K.; Zhang, X. M.; Zhang, Z. T.; Freeman, W. T.; Tenenbaum, J. B. Learning shape priors for single-view 3D completion and reconstruction. In: Computer Vision – ECCV 2018. Lecture Notes in Computer Science, Vol. 11215. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 673–691, 2018.
[19]
Zhang, X.; Zhang, Z.; Zhang, C.; Tenenbaum, J.; Freeman, B.; Wu, J. Learning to reconstruct shapes from unseen classes. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2263–2274, 2018.
[20]
Kar, A.; Häne, C.; Malik, J. Learning a multi-view stereo machine. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, 364–375, 2017.
[21]
Liu, S. K.; Giles, L.; Ororbia, A. Learning a hierarchical latent-variable model of 3D shapes. In: Proceedings of the International Conference on 3D Vision, 542–551, 2018.
[22]
Tatarchenko, M.; Dosovitskiy, A.; Brox, T. Octree generating networks: Efficient convolutional architectures for high-resolution 3D outputs. In: Proceedings of the IEEE International Conference on Computer Vision, 2107–2115, 2017.
[23]
Häne, C.; Tulsiani, S.; Malik, J. Hierarchical surface prediction for 3D object reconstruction. In: Proceedings of the International Conference on 3D Vision, 412–420, 2017.
[24]
Cao, Y. P.; Liu, Z. N.; Kuang, Z. F.; Kobbelt, L.; Hu, S. M. Learning to reconstruct high-quality 3D shapes with cascaded fully convolutional networks. In: Computer Vision – ECCV 2018. Lecture Notes in Computer Science, Vol. 11213. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 626–643, 2018.
[25]
Liu, Z. N.; Cao, Y. P.; Kuang, Z. F.; Kobbelt, L.; Hu, S. M. High-quality textured 3D shape reconstruction with cascaded fully convolutional networks. IEEE Transactions on Visualization and Computer Graphics Vol. 27, No. 1, 83–97, 2021.
[26]
Xie, H. Z.; Yao, H. X.; Sun, X. S.; Zhou, S. C.; Zhang, S. P. Pix2Vox: Context-aware 3D reconstruction from single and multi-view images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2690–2698, 2019.
[27]
Yang, S.; Xu, M.; Xie, H. Z.; Perry, S.; Xia, J. H. Single-view 3D object reconstruction from shape priors in memory. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3151–3160, 2021.
[28]
Wu, J.; Zhang, C.; Xue, T.; Freeman, B.; Tenenbaum, J. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, 82–90, 2016.
[29]
Smith, E. J.; Meger, D. Improved adversarial systems for 3D object generation and reconstruction. In: Proceedings of the 1st Annual Conference on Robot Learning, 87–96, 2017.
[30]
Zhu, J. Y.; Zhang, Z.; Zhang, C.; Wu, J.; Torralba, A.; Tenenbaum, J.; Freeman, B. Visual object networks: Image generation with disentangled 3D representation. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 118–129, 2018.
[31]
Chen, K.; Choy, C. B.; Savva, M.; Chang, A. X.; Funkhouser, T.; Savarese, S. Text2Shape: Generating shapes from natural language by learning joint embeddings. In: Computer Vision – ACCV 2018. Lecture Notes in Computer Science, Vol. 11363. Jawahar, C.; Li, H.; Mori, G.; Schindler, K. Eds. Springer Cham, 100–116, 2019.
[32]
Knyaz, V. A.; Kniaz, V. V.; Remondino, F. Image-to-voxel model translation with conditional adversarial networks. In: Computer Vision – ECCV 2018 Workshops. Lecture Notes in Computer Science, Vol. 11129. Leal-Taixé, L.; Roth, S. Eds. Springer Cham, 601–618, 2019.
[33]
Gadelha, M.; Maji, S.; Wang, R. 3D shape induction from 2D views of multiple objects. In: Proceedings of the International Conference on 3D Vision, 402–411, 2017.
[34]
Li, X.; Dong, Y.; Peers, P.; Tong, X. Synthesizing 3D shapes from silhouette image collections using multi-projection generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5530–5539, 2019.
[35]
Khan, S. H.; Guo, Y.; Hayat, M.; Barnes, N. Unsupervised primitive discovery for improved 3D generative modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9731–9740, 2019.
[36]
Henzler, P.; Mitra, N.; Ritschel, T. Escaping plato’s cave: 3D shape from adversarial rendering. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 9983–9992, 2019.
[37]
Chen, Z. Q.; Kim, V. G.; Fisher, M.; Aigerman, N.; Zhang, H.; Chaudhuri, S. DECOR-GAN: 3D shape detailization by conditional refinement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15735–15744, 2021.
[38]
Brock, A.; Lim, T.; Ritchie, J. M.; Weston, N. Generative and discriminative voxel modeling with convolutional neural networks. arXiv preprint arXiv:1608.04236, 2016.
[39]
Balashova, E.; Singh, V.; Wang, J. P.; Teixeira, B.; Chen, T.; Funkhouser, T. Structure-aware shape synthesis. In: Proceedings of the International Conference on 3D Vision, 140–149, 2018.
[40]
Mittal, P.; Cheng, Y. C.; Singh, M.; Tulsiani, S. AutoSDF: Shape priors for 3D completion, reconstruction and generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 306–315, 2022.
[41]
Huang, W.; Lai, B.; Xu, W.; Tu, Z. 3D volumetric modeling with introspective neural networks. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 33, No. 01, 8481–8488, 2019.
[42]
Ibing, M.; Kobsik, G.; Kobbelt, L. Octree transformer: Autoregressive 3D shape generation on hierarchically structured sequences. arXiv preprint arXiv:2111.12480, 2021.
[43]
Xie, J.; Zheng, Z.; Gao, R.; Wang, W.; Zhu, S. C.; Wu, Y. N. Learning descriptor networks for 3D shape synthesis and analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8629–8638, 2018.
[44]
Xie, J.; Zheng, Z.; Gao, R.; Wang, W.; Zhu, S. C.; Wu, Y. N. Generative VoxelNet: Learning energy-based models for 3D shape synsynthesis and analysis. IEEE Transactions on Pattern Analysisand Machine Intelligence Vol. 44, No. 5, 2468–2484, 2022.
[45]
Gadelha, M.; Wang, R.; Maji, S. Multiresolution tree networks for 3D point cloud processing. In: Computer Vision – ECCV 2018. Lecture Notes in Computer Science, Vol. 11211. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 105–122, 2018.
[46]
Yang, Y. Q.; Feng, C.; Shen, Y. R.; Tian, D. FoldingNet: Point cloud auto-encoder via deep grid deformation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 206–215, 2018.
[47]
Zamorski, M.; Zięba, M.; Klukowski, P.; Nowak, R.; Kurach, K.; Stokowiec, W.; Trzciński, T. Adversarial autoencoders for compact representations of 3D point clouds. Computer Vision and Image Understanding Vol. 193, 102921, 2020.
[48]
Kurenkov, A.; Ji, J. W.; Garg, A.; Mehta, V.; Gwak, J.; Choy, C.; Savarese, S. DeformNet: Free-form deformation network for 3D shape reconstruction from a single image. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 858–866, 2018.
[49]
Fan, H. Q.; Su, H.; Guibas, L. A point set generation network for 3D object reconstruction from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2463–2471, 2017.
[50]
Wei, Y.; Liu, S. H.; Zhao, W.; Lu, J. W. Conditional single-view shape generation for multi-view stereo reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9643–9652, 2019.
[51]
Hu, T.; Lin, G.; Han, Z. Z.; Zwicker, M. Learning to generate dense point clouds with textures on multiple categories. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 2169–2178, 2021.
[52]
Lin, C. H.; Kong, C.; Lucey, S. Learning efficient point cloud generation for dense 3D object reconstruction. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 32, No. 1, 7114–7121, 2018.
[53]
Insafutdinov, E.; Dosovitskiy, A. Unsupervised learning of shape and pose with differentiable point clouds. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2807–2817, 2018.
[54]
Chen, C.; Han, Z. Z.; Liu, Y. S.; Zwicker, M. Unsupervised learning of fine structure generation for 3D point clouds by 2D projection matching. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 12446–12457, 2021.
[55]
Komarichev, A.; Hua, J.; Zhong, Z. C. Learning geometry-aware joint latent space for simultaneous multimodal shape generation. Computer Aided Geometric Design Vol. 93, 102076, 2022.
[56]
Gal, R.; Bermano, A.; Zhang, H.; Cohen-Or, D. MRGAN: Multi-rooted 3D shape generation with unsupervised part disentanglement. arXiv preprint arXiv:2007.12944, 2020.
[57]
Achlioptas, P.; Diamanti, O.; Mitliagkas, I.; Guibas, L. Learning representations and generative models for 3D point clouds. In: Proceedings of the 35th International Conference on Machine Learning, 40–49, 2018.
[58]
Valsesia, D.; Fracastoro, G.; Magli, E. Learning localized generative models for 3D point clouds via graph convolution. In: Proceedings of the 7th International Conference on Learning Representations, 2019.
[59]
Shu, D.; Park, S. W.; Kwon, J. 3D point cloud generative adversarial network based on tree structured graph convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 3858–3867, 2019.
[60]
Li, R. H.; Li, X. Z.; Fu, C. W.; Cohen-Or, D.; Heng, P. A. PU-GAN: A point cloud upsampling adversarial network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 7202–7211, 2019.
[61]
Ramasinghe, S.; Khan, S.; Barnes, N.; Gould, S. Spectral-GANs for high-resolution 3D point-cloudgeneration. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 8169–8176, 2020.
[62]
Li, Y. S.; Baciu, G. SAPCGAN: Self-attention based generative adversarial network for point clouds. In: Proceedings of the IEEE 19th International Conference on Cognitive Informatics & Cognitive Computing, 52–59, 2020.
[63]
Li, Y. S.; Baciu, G. HSGAN: Hierarchical graph learning for point cloud generation. IEEE Transactions on Image Processing Vol. 30, 4540–4554, 2021.
[64]
Li, R.; Li, X.; Hui, K. H.; Fu, C. W. SP-GAN: Sphere-guided 3D shape generation and manipulation. ACM Transactions on Graphics Vol. 40, No. 4, Article No. 151, 2021.
[65]
Tang, Y. Z.; Qian, Y.; Zhang, Q. J.; Zeng, Y. M.; Hou, J. H.; Zhe, X. F. WarpingGAN: Warping multiple uniform priors for adversarial 3D point cloud generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6387–6395, 2022.
[66]
Hui, L.; Xu, R.; Xie, J.; Qian, J. J.; Yang, J. Progressive point cloud deconvolution generation network. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12360. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 397–413, 2020.
[67]
Arshad, M. S.; Beksi, W. J. A progressive conditional generative adversarial network for generating dense and colored 3D point clouds. In: Proceedings of the International Conference on 3D Vision, 712–722, 2020.
[68]
Wen, C.; Yu, B. S.; Tao, D. C. Learning progressive point embeddings for 3D point cloud generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10261–10270, 2021.
[69]
Mo, K. C.; Wang, H.; Yan, X. C.; Guibas, L. PT2PC: Learning to generate 3D point cloud shapes from part tree conditions. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12351. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 683–701, 2020.
[70]
Yang, X. M.; Wu, Y. A.; Zhang, K. Y.; Jin, C. CPCGAN: A controllable 3D point cloud generative adversarial network with semantic label generating. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 35, No. 4, 3154–3162, 2021.
[71]
Kim, J.; Yoo, J.; Lee, J.; Hong, S. SetVAE: Learning hierarchical composition for generative modeling of set-structured data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15054–15063, 2021.
[72]
Li, S. D.; Liu, M. M.; Walder, C. EditVAE: Unsupervised parts-aware controllable 3D point cloud shape generation. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 36, No. 2, 1386–1394, 2022.
[73]
Yang, G. D.; Huang, X.; Hao, Z. K.; Liu, M. Y.; Belongie, S.; Hariharan, B. PointFlow: 3D point cloud generation with continuous normalizing flows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 4540–4549, 2019.
[74]
Klokov, R.; Boyer, E.; Verbeek, J. Discrete point flow networks for efficient point cloud generation. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12368. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 694–710, 2020.
[75]
Kim, H.; Lee, H.; Kang, W. H.; Lee, J. Y.; Kim, N. S. SoftFlow: Probabilistic framework for normalizing flow on manifolds. In: Proceedings of the 34th International Conference on Neural Information Processing Systems, 16388–16397, 2020.
[76]
Pumarola, A.; Popov, S.; Moreno-Noguer, F.; Ferrari, V. C-flow: Conditional generative flow models for images and 3D point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7946–7955, 2020.
[77]
Postels, J.; Liu, M. Y.; Spezialetti, R.; Van Gool, L.; Tombari, F. Go with the flows: Mixtures of normalizing flows for point cloud generation and reconstruction. In: Proceedings of the International Conference on 3D Vision, 1249–1258, 2021.
[78]
Sun, Y. B.; Wang, Y.; Liu, Z. W.; Siegel, J. E.; Sarma, S. E. PointGrow: Autoregressively learned point cloud generation with self-attention. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 61–70, 2020.
[79]
Luo, S. T.; Hu, W. Diffusion probabilistic models for 3D point cloud generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2836–2844, 2021.
[80]
Zhou, L. Q.; Du, Y. L.; Wu, J. J. 3D shape generation and completion through point-voxel diffusion. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 5806–5815, 2021.
[81]
Xie, J.; Xu, Y.; Zheng, Z.; Zhu, S. C.; Wu, Y. N. Generative PointNet: Deep energy-based learning on unordered point sets for 3D generation, reconstruction and classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14971–14980, 2021.
[82]
Cai, R. J.; Yang, G. D.; Averbuch-Elor, H.; Hao, Z. K.; Belongie, S.; Snavely, N.; Hariharan, B. Learning gradient fields for shape generation. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12348. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 364–381, 2020.
[83]
Groueix, T.; Fisher, M.; Kim, V. G.; Russell, B. C.; Aubry, M. A Papier-Mache approach to learning 3D surface generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 216–224, 2018.
[84]
Pontes, J. K.; Kong, C.; Sridharan, S.; Lucey, S.; Eriksson, A.; Fookes, C. Image2Mesh: A learning framework for single image 3D reconstruction. In: Computer Vision – ACCV 2018. Lecture Notes in Computer Science, Vol. 11361. Jawahar, C.; Li, H.; Mori, G.; Schindler, K. Eds. Springer Cham, 365–381, 2019.
[85]
Wang, N. Y.; Zhang, Y. D.; Li, Z. W.; Fu, Y. W.; Liu, W.; Jiang, Y. G. Pixel2Mesh: Generating 3D mesh models from single RGB images. In: Computer Vision – ECCV 2018. Lecture Notes in Computer Science, Vol. 11215. Ferrari, V.; Hebert, M.; Sminchisescu, C.; Weiss, Y. Eds. Springer Cham, 55–71, 2018.
[86]
Wen, C.; Zhang, Y. D.; Li, Z. W.; Fu, Y. W. Pixel2Mesh: Multi-view 3D mesh generation via deformation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 1042–1051, 2019.
[87]
Pan, J. Y.; Han, X. G.; Chen, W. K.; Tang, J. P.; Jia, K. Deep mesh reconstruction from single RGB images via topology modification networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 9963–9972, 2019.
[88]
Shi, Y.; Ni, B. B.; Liu, J. X.; Rong, D. Y.; Qian, Y.; Zhang, W. J. Geometric granularity aware pixel-to-mesh. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 13077–13086, 2021.
[89]
Tang, J. P.; Han, X. G.; Pan, J. Y.; Jia, K.; Tong, X. A skeleton-bridged deep learning approach for generating meshes of complex topologies from single RGB images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4536–4545, 2019.
[90]
Gkioxari, G.; Johnson, J.; Malik, J. Mesh R-CNN. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 9784–9794, 2019.
[91]
Hui, K. H.; Li, R. H.; Hu, J. Y.; Fu, C. W. Neural template: Topology-aware reconstruction and disentangled generation of 3D meshes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18551–18561, 2022.
[92]
Zhang, S. H.; Guo, Y. C.; Gu, Q. W. Sketch2Model: View-aware 3D modeling from single free-hand sketches. In: Proceedings of the IEEE/CVF Con-ference on Computer Vision and Pattern Recognition, 6000–6017, 2021.
[93]
Chen, W.; Ling, H.; Gao, J.; Smith, E.; Lehtinen, J.; Jacobson, A.; Fidler, S. Learning to predict 3D objects with an interpolation-based differentiable renderer. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, 9609–9619, 2019.
[94]
Grigorev, A.; Iskakov, K.; Ianina, A.; Bashirov, R.; Zakharkin, I.; Vakhitov, A.; Lempitsky, V. StylePeople: A generative model of fullbody human avatars. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5147–5156, 2021.
[95]
Pavllo, D.; Spinks, G.; Hofmann, T.; Moens, M. F.; Lucchi, A. Convolutional generation of textured 3D meshes. In: Proceedings of the 34th International Conference on Neural Information Processing Systems, 870–882, 2020.
[96]
Pavllo, D.; Kohler, J.; Hofmann, T.; Lucchi, A. Learning generative models of textured 3D meshes from real-world images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 13859–13869, 2021.
[97]
Tan, Q.; Gao, L.; Lai, Y. K.; Xia, S. Variational autoencoders for deforming 3D mesh models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5841–5850, 2018.
[98]
Gao, L.; Wu, T.; Yuan, Y. J.; Lin, M. X.; Lai, Y. K.; Zhang, H. TM-NET: Deep generative networks for textured meshes. ACM Transactions on Graphics Vol. 40, No. 6, Article No. 263, 2021.
[99]
Rezende, D. J.; Eslami, S.; Mohamed, S.; Battaglia, P.; Jaderberg, M.; Heess, N. Unsupervised learning of 3D structure from images. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, 5003–5011, 2016.
[100]
Henderson, P.; Tsiminaki, V.; Lampert, C. H. Leveraging 2D data to learn textured 3D mesh generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7495–7504, 2020.
[101]
Nash, C.; Ganin, Y.; Eslami, S. A.; Battaglia, P. PolyGen: An autoregressive generative model of 3D meshes. In: Proceedings of the 37th International Conference on Machine Learning, 7220–7229, 2020.
[102]
Park, J. J.; Florence, P.; Straub, J.; Newcombe, R.; Lovegrove, S. DeepSDF: Learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 165–174, 2019.
[103]
Mescheder, L.; Oechsle, M.; Niemeyer, M.; Nowozin, S.; Geiger, A. Occupancy networks: Learning 3D reconstruction in function space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4455–4465, 2019.
[104]
Xu, Q.; Wang, W.; Ceylan, D.; Mech, R.; Neumann, U. DISN: Deep implicit surface network for high-quality single-view 3D reconstruction. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, 492–502, 2019.
[105]
Liu, S.; Saito, S.; Chen, W.; Li, H. Learning to infer implicit surfaces without 3D supervision. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, 8295–8306, 2019.
[106]
Peng, S. Y.; Niemeyer, M.; Mescheder, L.; Pollefeys, M.; Geiger, A. Convolutional occupancy networks. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12348. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 523–540, 2020.
[107]
Liu, S. L.; Guo, H. X.; Pan, H.; Wang, P. S.; Tong, X.; Liu, Y. Deep implicit moving least-squares functions for 3D reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1788–1797, 2021.
[108]
Chibane, J.; Alldieck, T.; Pons-Moll, G. Implicit functions in feature space for 3D shape reconstruction and completion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6968–6979, 2020.
[109]
Li, M. Y.; Zhang, H. D2IM-net: Learning detail disentangled implicit fields from single images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10241–10250, 2021.
[110]
Poursaeed, O.; Fisher, M.; Aigerman, N.; Kim, V. G. Coupling explicit and implicit surface representations for generative 3D modeling. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12355. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 667–683, 2020.
[111]
Jiang, C. Y.; Sud, A.; Makadia, A.; Huang, J. W.; Nießner, M.; Funkhouser, T. Local implicit grid representations for 3D scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6000–6009, 2020.
[112]
Jiang, C.; Marcus, P. Hierarchical detail enhancing mesh-based shape generation with 3D generative adversarial network. arXiv preprint arXiv:1709.07581, 2017.
[113]
Chen, Z. Q.; Zhang, H. Learning implicit fields for generative shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5932–5941, 2019.
[114]
Ibing, M.; Lim, I.; Kobbelt, L. 3D shape generation with grid-based implicit functions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13554–13563, 2021.
[115]
Mezghanni, M.; Boulkenafed, M.; Lieutier, A.; Ovsjanikov, M. Physically-aware generative network for 3D shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9326–9337, 2021.
[116]
Tang, J. H.; Chen, W. K.; Yang, J.; Wang, B.; Liu, S. R.; Yang, B.; Gao, L. OctField: Hierarchical implicit functions for 3D modeling. In: Proceedings of the 35th Conference on Neural Information Processing Systems, 12648–12660, 2021.
[117]
Sanghi, A.; Chu, H.; Lambourne, J. G.; Wang, Y.; Cheng, C. Y.; Fumero, M.; Malekshan, K. R. CLIP-forge: Towards zero-shot text-to-shape generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18582–18592, 2022.
[118]
Yan, X. G.; Lin, L. Q.; Mitra, N. J.; Lischinski, D.; Cohen-Or, D.; Huang, H. ShapeFormer: Transformer-based shape completion via sparse representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6229–6239, 2022.
[119]
Liu, Z. Z.; Wang, Y.; Qi, X. J.; Fu, C. W. Towards implicit text-guided 3D shape generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 17875–17885, 2022.
[120]
Zou, C. H.; Yumer, E.; Yang, J. M.; Ceylan, D.; Hoiem, D. 3D-PRNN: Generating shape primitives with recurrent neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, 900–909, 2017.
[121]
Schor, N.; Katzir, O.; Zhang, H.; Cohen-Or, D. CompoNet: Learning to generate the unseen by part synthesis and composition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 8758–8767, 2019.
[122]
Wu, R. D.; Zhuang, Y. X.; Xu, K.; Zhang, H.; Chen, B. Q. PQ-NET: A generative part Seq2Seq network for 3D shapes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 826–835, 2020.
[123]
Yin, K. X.; Chen, Z. Q.; Chaudhuri, S.; Fisher, M.; Kim, V. G.; Zhang, H. COALESCE: Component assembly by learning to synthesize connections. In: Proceedings of the International Conference on 3D Vision, 61–70, 2020.
[124]
Kawana, Y.; Mukuta, Y.; Harada, T. Neural star domain as primitive representation. In: Proceedings of the 34th International Conference on Neural Information Processing Systems, 7875–7886, 2020.
[125]
Li, J.; Xu, K.; Chaudhuri, S.; Yumer, E.; Zhang, H.; Guibas, L. GRASS: Generative recursive autoencoders for shape structures. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 52, 2017.
[126]
Wang, H.; Schor, N.; Hu, R. Z.; Huang, H. B.; Cohen-Or, D.; Huang, H. Global-to-local generative model for 3D shapes. ACM Transactions on Graphics Vol. 37, No. 6, Article No. 214, 2018.
[127]
Nash, C.; Williams, C. K. The shape variational autoencoder: A deep generative model of part-segmented 3D objects. Computer Graphics Forum Vol. 36, No. 5, 1–12, 2017.
[128]
Wu, Z. J.; Wang, X.; Lin, D.; Lischinski, D.; Cohen-Or, D.; Huang, H. SAGNet: Structure-aware generative network for 3D-shape modeling. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 91, 2019.
[129]
Mo, K. C.; Guerrero, P.; Yi, L.; Su, H.; Wonka, P.; Mitra, N. J.; Guibas, L. J. StructureNet: Hierarchical graph networks for 3D shape generation. ACM Transactions on Graphics Vol. 38, No. 6, Article No. 242, 2019.
[130]
Gao, L.; Yang, J.; Wu, T.; Yuan, Y. J.; Fu, H. B.; Lai, Y. K.; Zhang, H. SDM-NET: Deep generative network for structured deformable mesh. ACM Transactions on Graphics Vol. 38, No. 6, Article No. 243, 2019.
[131]
Yang, J.; Mo, K. C.; Lai, Y. K.; Guibas, L. J.; Gao, L. DSG-net: Learning disentangled structure and geometry for 3D shape generation. ACM Transactions on Graphics Vol. 42, No. 1, Article No. 1, 2023.
[132]
Jones, R. K.; Barton, T.; Xu, X.; Wang, K.; Jiang, E.; Guerrero, P.; Mitra, N. J.; Ritchie, D. ShapeAssembly: Learning to generate programs for 3D shape structure synthesis. ACM Transactions on Graphics Vol. 39, No. 6, Article No. 234, 2020.
[133]
Kalogerakis, E.; Chaudhuri, S.; Koller, D.; Koltun, V. A probabilistic model for component-based shape synthesis. ACM Transactions on Graphics Vol. 31, No. 4, Article No. 55, 2012.
[134]
Kim, V. G.; Li, W.; Mitra, N.; Chaudhuri, S.; DiVerdi, S.; Funkhouser, T. Learning part-based templates from large collections of 3D shapes. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 70, 2013.
[135]
Huang, H.; Kalogerakis, E.; Marlin, B. Analysis and synthesis of 3D shape families via deep-learned generative models of surfaces. Computer Graphics Forum Vol. 34, No. 5, 25–38, 2015.
[136]
Sung, M.; Su, H.; Kim, V. G.; Chaudhuri, S.; Guibas, L. ComplementMe: Weakly-supervised component suggestions for 3D modeling. ACM Transactions on Graphics Vol. 36, No. 6, Article No. 226, 2017.
[137]
Chen, Z. Q.; Tagliasacchi, A.; Zhang, H. BSP-net: Generating compact meshes via binary space partitioning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 42–51, 2020.
[138]
Paschalidou, D.; Katharopoulos, A.; Geiger, A.; Fidler, S. Neural parts: Learning expressive 3D shape abstractions with invertible neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3203–3214, 2021.
[139]
Xiao, Y. P.; Lai, Y. K.; Zhang, F. L.; Li, C. P.; Gao, L. A survey on deep geometry learning: From a representation perspective. Computational Visual Media Vol. 6, No. 2, 113–133, 2020.
[140]
Li, R. H.; Li, X. Z.; Heng, P. A.; Fu, C. W. PointAugment: An auto-augmentation framework for point cloud classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6377–6386, 2020.
[141]
Guo, M. H.; Cai, J. X.; Liu, Z. N.; Mu, T. J.; Martin, R. R.; Hu, S. M. PCT: Point cloud transformer. Computational Visual Media Vol. 7, No. 2, 187–199, 2021.
[142]
Huang, S. S.; Ma, Z. Y.; Mu, T. J.; Fu, H.; Hu, S. M. Supervoxel convolution for online 3D semantic segmentation. ACM Transactions on Graphics Vol. 40, No. 3, Article No. 34, 2021.
[143]
Huang, J. H.; Wang, H.; Birdal, T.; Sung, M.; Arrigoni, F.; Hu, S. M.; Guibas, L. MultiBodySync: Multi-body segmentation and motion estimation via 3D scan synchronization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7104–7114, 2021.
[144]
Bronstein, M. M.; Bruna, J.; LeCun, Y.; Szlam, A.; Vandergheynst, P. Geometric deep learning: Going beyond euclidean data. IEEE Signal Processing Magazine Vol. 34, No. 4, 18–42, 2017.
[145]
Maturana, D.; Scherer, S. 3D Convolutional Neural Networks for landing zone detection from LiDAR. In: Proceedings of the IEEE International Conference on Robotics and Automation, 3471–3478, 2015.
[146]
Maturana, D.; Scherer, S. VoxNet: A 3D Con-volutional Neural Network for real-time object recognition. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 922–928, 2015.
[147]
Charles, R. Q.; Hao, S.; Mo, K. C.; Guibas, L. J. PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 77–85, 2017.
[148]
Li, Y.; Bu, R.; Sun, M.; Wu, W.; Di, X.; Chen, B. PointCNN: Convolution on X -transformed points. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 828–838, 2018.
[149]
Hanocka, R.; Hertz, A.; Fish, N.; Giryes, R.; Fleishman, S.; Cohen-Or, D. MeshCNN: A network with an edge. ACM Transactions on Graphics Vol. 38, No. 4, Article No. 90, 2019.
[150]
Yuan, Y. J.; Lai, Y. K.; Yang, J.; Duan, Q.; Fu, H.; Gao, L. Mesh variational autoencoders with edge contraction pooling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 1105–1112, 2020.
[151]
Hu, S. M.; Liu, Z. N.; Guo, M. H.; Cai, J. X.; Huang, J. H.; Mu, T. J.; Martin, R. R. Subdivision-based mesh convolution networks. ACM Transactions on Graphics Vol. 41, No. 3, Article No. 25, 2022.
[152]
Lorensen, W. E.; Cline, H. E. Marching cubes: A high resolution 3D surface construction algorithm. In: Proceedings of the 14th Annual Conferenceon Computer Graphics and Interactive Techniques, 163–169, 1987.
[153]
Hinton, G. E.; Zemel, R. Autoencoders, minimum description length and Helmholtz free energy. In: Proceedings of the 6th International Conference on Neural Information Processing Systems, 3–10, 1993.
[154]
Goodfellow, I. J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, 2672–2680, 2014.
[155]
Kingma, D. P.; Welling, M. Auto-encoding variational Bayes. In: Proceedings of the International Conference on Learning Representations, 2014.
[156]
Dinh, L.; Krueger, D.; Bengio, Y. NICE: Non-linear independent components estimation. In: Proceedings of the International Conference on Learning Representations Workshops, 2015.
[157]
Dinh, L.; Sohl-Dickstein, J.; Bengio, S. Density estimation using Real NVP. In: Proceedings of the International Conference on Learning Representations, 2017.
[158]
Kingma, D. P.; Dhariwal, P. Glow: Generative flow with invertible 1×
[159]
Rezende, D. J.; Mohamed, S. Variational inference with normalizing flows. In: Proceedings of the 32nd International Conference on Machine Learning, 1530–1538, 2015.
[160]
Chen, R. T. Q.; Rubanova, Y.; Bettencourt, J.; Duvenaud, D. Neural ordinary differential equations. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 6572–6583, 2018.
[161]
Grathwohl, W.; Chen, R. T. Q.; Bettencourt, J.; Sutskever, I.; Duvenaud, D. FFJORD: Free-form continuous dynamics for scalable reversible generative models. In: Proceedings of the International Con-ference on Learning Representations, 2019.
[162]
Edwards, H.; Storkey, A. J. Towards a neural statistician. In: Proceedings of the International Conference on Learning Representations, 2017.
[163]
Riegler, G.; Ulusoy, A. O.; Geiger, A. OctNet: Learning deep 3D representations at high resolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6620–6629, 2017.
[164]
Wang, P. S.; Liu, Y.; Guo, Y. X.; Sun, C. Y.; Tong, X. O-CNN: Octree-based convolutional neural networks for 3D shape analysis. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 72, 2017.
[165]
Radford, A.; Metz, L.; Chintala, S. Unsupervised representation learning with deep convolutional generative adversarial networks. In: Proceedings of the International Conference on Learning Representations, 2016.
[166]
He, K. M.; Zhang, X. Y.; Ren, S. Q.; Sun, J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778, 2016.
[167]
Arjovsky, M.; Chintala, S.; Bottou, L. Wasserstein generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning, 214–223, 2017.
[168]
Gulrajani, I.; Ahmed, F.; Arjovsky, M.; Dumoulin, V.; Courville, A. Improved training of Wasserstein GANs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, 5769–5779, 2017.
[169]
Isola, P.; Zhu, J. Y.; Zhou, T. H.; Efros, A. A. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5967–5976, 2017.
[170]
Lazarow, J.; Jin, L.; Tu, Z. W. Introspective neural networks for generative modeling. In: Proceedings of the IEEE International Conference on Computer Vision, 2793–2802, 2017.
[171]
Razavi, A.; Van den Oord, A.; Vinyals, O. Generating diverse high-fidelity images with VQ-VAE-2. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems, 14866–14876, 2019.
[172]
Esser, P.; Rombach, R.; Ommer, B. Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12868–12878, 2021.
[173]
Devlin, J.; Chang, M.; Lee, K.; Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 4171–4186, 2019.
[174]
LeCun, Y.; Chopra, S.; Hadsell, R.; Ranzato, M.; Huang, F. A tutorial on energy-based learning. In: Predicting Structured Data. Bakir, G.; Hofman, T.; Scholkopf, B.; Smola, A.; Taskar, B. Eds. MIT Press, 2006.
[175]
Xie, J. W.; Lu, Y.; Zhu, S. C.; Wu, Y. N. A theory of generative ConvNet. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning, 2635–2644, 2016.
[176]
Qi, C. R.; Yi, L.; Su, H.; Guibas, L. J. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, 5105–5114, 2017.
[177]
Wang, Y.; Sun, Y. B.; Liu, Z. W.; Sarma, S. E.; Bronstein, M. M.; Solomon, J. M. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics Vol. 38, No. 5, Article No. 146, 2019.
[178]
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, 6000–6010, 2017.
[179]
Zhao, H.; Jiang, L.; Jia, J.; Torr, P. H.; Koltun, V. Point transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 16239–16248, 2021.
[180]
Pan, X.; Xia, Z.; Song, S.; Li, L. E.; Huang, G. 3D object detection with pointformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7459–7468, 2021.
[181]
Sederberg, T. W.; Parry, S. R. Free-form deformation of solid geometric models. In: Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques, 151–160, 1986.
[182]
Navaneet, K. L.; Mandikal, P.; Agarwal, M.; Babu, R. V. CAPNet: Continuous approximation projection for 3D point cloud reconstruction using 2D supervision. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 33, No. 01, 8819–8826, 2019.
[183]
Han, Z. Z.; Chen, C.; Liu, Y. S.; Zwicker, M. DRWR: A differentiable renderer without rendering for unsupervised 3D structure learning from silhouette images. In: Proceedings of the 37th International Conference on Machine Learning, 3994–4005, 2020.
[184]
Karras, T.; Laine, S.; Aittala, M.; Hellsten, J.; Lehtinen, J.; Aila, T. Analyzing and improving the image quality of StyleGAN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8107–8116, 2020.
[185]
Sohl-Dickstein, J.; Weiss, E. A.; Maheswaranathan, N.; Ganguli, S. Deep unsupervised learning using nonequilibrium thermodynamics. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning, 2256–2265, 2015.
[186]
Maron, H.; Galun, M.; Aigerman, N.; Trope, M.; Dym, N.; Yumer, E.; Kim, V. G.; Lipman, Y. Convolutional neural networks on surfaces via seamless toric covers. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 71, 2017.
[187]
Saquil, Y.; Xu, Q. C.; Yang, Y. L.; Hall, P. Rank3DGAN: Semantic mesh generation using relative attributes. In: Proceedings of the 34th AAAI Conference on Artificial Intelligenc, 5586–5594, 2020.
[188]
Aigerman, N.; Lipman, Y. Hyperbolic orbifold tutte embeddings. ACM Transactions on Graphics Vol. 34, No. 6, Article No. 190, 2015.
[189]
Bruna, J.; Zaremba, W.; Szlam, A.; LeCun, Y. Spectral networks and locally connected networks on graphs. In: Proceedings of the International Conference on Learning Representations, 2014.
[190]
Atwood, J.; Towsley, D. Diffusion-convolutional neural networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, 2001–2009, 2016.
[191]
Defferrard, M.; Bresson, X.; Vandergheynst, P. Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, 3844–3852, 2016.
[192]
Qiao, Y. L.; Gao, L.; Yang, J.; Rosin, P. L.; Lai, Y. K.; Chen, X. L. Learning on 3D meshes with Laplacian encoding and pooling. IEEE Transactions on Visualization and Computer Graphics Vol. 28, No. 2, 1317–1327, 2022.
[193]
Feng, Y.; Feng, Y.; You, H.; Zhao, X.; Gao, Y. MeshNet: Mesh neural network for 3D shape representation. Proceedings of the AAAI Conference on Artificial Intelligence Vol. 33, No. 01, 8279–8286, 2019.
[194]
Liu, H. T D.; Kim, V. G.; Chaudhuri, S.; Aigerman, N.; Jacobson, A. Neural subdivision. ACM Transactions on Graphics Vol. 39, No. 4, Article No. 124, 2020.
[195]
Hu, S. M.; Liang, D.; Yang, G. Y.; Yang, G. W.; Zhou, W. Y. Jittor: A novel deep learning framework with meta-operators and unified graph execution. Science China Information Sciences Vol. 63, No. 12, 222103, 2020.
[196]
He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, 2980–2988, 2017.
[197]
Gregor, K.; Danihelka, I.; Graves, A.; Rezende, D. J.; Wierstra, D. DRAW: A recurrent neural network for image generation. In: Proceedings of the International Conference on Machine Learning, 1462–1471, 2015.
[198]
Kato, H.; Ushiku, Y.; Harada, T. Neural 3D mesh renderer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3907–3916, 2018.
[199]
Liu, S.; Li, T.; Chen, W.; Li, H. Soft rasterizer: A differentiable renderer for image-based 3D reasoning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 7707–7716, 2019.
[200]
Pavlakos, G.; Choutas, V.; Ghorbani, N.; Bolkart, T.; Osman, A. A. A.; Tzionas, D.; Black, M. J. Expressive body capture: 3D hands, face, and body from a single image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10967–10977, 2019.
[201]
Michalkiewicz, M.; Pontes, J. K.; Jack, D.; Baktashmotlagh, M.; Eriksson, A. Deep level sets: Implicit surface representations for 3D shape inference. arXiv preprint arXiv:1901.06802, 2019.
[202]
Chibane, J.; Mir, A.; Pons-Moll, G. Neural unsigned distance fields for implicit function learning. In: Proceedings of the 34th International Conference on Neural Information Processing Systems, 21638–21652, 2020.
[203]
Venkatesh, R.; Karmali, T.; Sharma, S.; Ghosh, A.; Babu, R. V.; Jeni, L. A.; Singh, M. Deep implicit surface point prediction networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 12633–12642, 2021.
[204]
Aumentado-Armstrong, T.; Tsogkas, S.; Dickinson, S.; Jepson, A. Representing 3D shapes with probabilistic directed distance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19321–19332, 2022.
[205]
Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. Learning transferable visual models from natural language supervision. In: Proceedings of the International Conference on Machine Learning, 8748–8763, 2021.
[206]
Schuster, M.; Paliwal, K. K. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing Vol. 45, No. 11, 2673–2681, 1997.
[207]
Cho, K.; van Merrienboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 1724–1734, 2014.
[208]
Gao, L.; Lai, Y. K.; Yang, J.; Zhang, L. X.; Xia, S. H.; Kobbelt, L. Sparse data driven mesh deformation. IEEE Transactions on Visualization and Computer Graphics Vol. 27, No. 3, 2085–2100, 2021.
[209]
Sedaghat, N.; Zolfaghari, M.; Amiri, E.; Brox, T. Orientation-boosted voxel nets for 3D object recognition. In: Proceedings of the British Machine Vision Conference, 97.1–97.13, 2017.
[210]
Xiang, Y.; Kim, W.; Chen, W.; Ji, J.; Choy, C.; Su, H.; Mottaghi, R.; Guibas, L.; Savarese, S. ObjectNet3D: A large scale database for 3D object recognition. In: Computer Vision – ECCV 2016. Lecture Notes in Computer Science, Vol. 9912. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 160–176, 2016.
[211]
Song, S. R.; Yu, F.; Zeng, A.; Chang, A. X.; Savva, M.; Funkhouser, T. Semantic scene completion from a single depth image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 190–198, 2017.
[212]
Sun, X. Y.; Wu, J. J.; Zhang, X. M.; Zhang, Z. T.; Zhang, C. K.; Xue, T. F.; Tenenbaum, J. B.; Freeman, W. T. Pix3D: Dataset and methods for single-image 3D shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2974–2983, 2018.
[213]
Mo, K.; Zhu, S.; Chang, A. X.; Yi, L.; Tripathi, S.; Guibas, L. J.; Su, H. PartNet: A large-scale benchmark for fine-grained and hierarchical part-level 3D object understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 909–918, 2019.
[214]
Fu, H.; Jia, R. F.; Gao, L.; Gong, M. M.; Zhao, B. Q.; Maybank, S.; Tao, D. C. 3D-FUTURE: 3D furniture shape with TextURE. International Journal of Computer Vision Vol. 129, No. 12, 3313–3337, 2021.
[215]
Xiang, Y.; Mottaghi, R.; Savarese, S. Beyond PASCAL: A benchmark for 3D object detection in the wild. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 75–82, 2014.
[216]
Bronstein, A. M.; Bronstein, M. M.; Kimmel, R. Numerical Geometry of Non-Rigid Shapes. Springer New York, 2009.
[217]
Bogo, F.; Romero, J.; Loper, M.; Black, M. J. FAUST: Dataset and evaluation for 3D mesh registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3794–3801, 2014.
[218]
Song, S. R.; Lichtenberg, S. P.; Xiao, J. X. SUN RGB-D: A RGB-D scene understanding benchmark suite. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 567–576, 2015.
[219]
Choi, S.; Zhou, Q. Y.; Miller, S.; Koltun, V. A large dataset of object scans. arXiv preprint arXiv:1602.02481, 2016.
[220]
Welinder, P.; Branson, S.; Mita, T.; Wah, C.; Schroff, F.; Belongie, S. J.; Perona, P. Caltech-UCSD birds 200. Computation & Neural Systems Technical Report, 2010-001. California Institute of Technology, 2010. Available at https://resolver.caltech.edu/CaltechAUTHORS:20111026-155425465.
[221]
Shen, T. C.; Gao, J.; Yin, K. X.; Liu, M. Y.; Fidler, S. Deep marching tetrahedra: A hybrid representation for high-resolution 3D shape synthesis. In: Proceedings of the 35th Conference on Neural Information Processing Systems, 6087–6101, 2021.
[222]
Yuan, Y. J.; Sun, Y. T.; Lai, Y. K.; Ma, Y. W.; Jia, R. F.; Gao, L. NeRF-editing: Geometry editing of neural radiance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18332–18343, 2022.
[223]
Liao, Y. Y.; Donné, S.; Geiger, A. Deep marching cubes: Learning explicit surface representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2916–2925, 2018.
[224]
Zhang, S. H.; Zhang, S. K.; Xie, W. Y.; Luo, C. Y.; Yang, Y. L.; Fu, H. B. Fast 3D indoor scene synthesis by learning spatial relation priors of objects. IEEE Transactions on Visualization and Computer Graphics Vol. 28, No. 9, 3082–3092, 2022.
[225]
Qian, Y.; Hou, J.; Kwong, S.; He, Y. PUGeo-Net: A geometry-centric network for 3D point cloud upsampling. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12364. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J. M. Eds. Springer Cham, 752–769, 2020.
[226]
Liang, Y. Q.; Zhao, S. S.; Yu, B. S.; Zhang, J.; He, F. Z. MeshMAE: Masked autoencoders for 3D mesh data analysis. arXiv preprint arXiv:2207.10228, 2022.
Publication history
Copyright
Acknowledgements
Rights and permissions
Publication history
Received: 24 August 2022
Accepted: 26 October 2022
Published:
18 May 2023
Issue date: September 2023
Copyright
© The Author(s) 2023.
Acknowledgements
This work was supported by the National NaturalScience Foundation of China (Grant No. 61902210) and RCUK grant CAMERA (Grant Nos. EP/M023281/1, EP/T022523/1).
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
FEEDBACK 
TOP