FRNeRF: Fusion and regularization fields for dynamic view synthesis

Novel space–time view synthesis for monocular video is a highly challenging task: both static and dynamic objects usually appear in the video, but only a single view of the current scene is available, resulting in inaccurate synthesis results. To address this challenge, we propose FRNeRF, a novel space–time view synthesis method with a fusion regularization field. Specifically, we design a 2D–3D fusion regularization field for the original dynamic neural field, which helps reduce blurring of dynamic objects in the scene. In addition, we add image prior features to the hierarchical sampling to solve the problem that the traditional hierarchical sampling strategy cannot obtain sufficient sampling points during training. We evaluate our method extensively on multiple datasets and show the results of dynamic space–time view synthesis. Our method achieves state-of-the-art performance both qualitatively and quantitatively. Code is available for research purposes at https://cic.tju.edu.cn/faculty/likun/projects/FRNerf.