A Survey of Recent Advances in Generative 3D Reconstruction

Inspired by the rapid progress of generative AI techniques, there have been huge advances made for the 3D (three-dimensional) reconstruction community, which promoted the traditional 3D reconstruction framework from deep implicit 3D reconstruction to generative 3D reconstruction, achieving more robust and expansive 3D reconstruction results with the help of generative AI models. Meanwhile, there is still a lack of corresponding review articles to provide a comprehensive analysis of recent advances from the perspective of 3D reconstruction. In response, this paper gives a comprehensive review for the generative 3D reconstruction approaches, especially on the recent advances made from the computer graphics and vision communities. Firstly, this paper mainly divides the recent generative 3D reconstruction approaches into four categories, including generative structure-from-motion/multiview-sterero (SfM/MVS), generative adversarial networks (GAN) based 3D reconstruction, diffusion-based 3D reconstruction, and cross-modal 3D reconstruction, which cover most generative-model aided 3D reconstruction work with a comprehensive review and analysis. Thereafter, some representative applications inspired by the generative 3D reconstruction including dynamic human avatars, 3D interactive editing, and autonomous driving are also reviewed. Besides, some major datasets widely used for the generative 3D reconstruction approaches are included. Finally, this paper makes a discussion of the potential future work in further improving the quality of generative 3D reconstruction, towards better and more intelligent 3D reconstruction and generation.