Neural video field editing

Diffusion models have revolutionized text-driven video editing. However, applying these methods to real-world editing encounters two significant challenges: (ⅰ) the rapid increase in GPU memory demand as the number of frames grows, and (ⅱ) inconsistency between frames in edited videos. To address these issues, we propose NVEdit, a novel text-driven video editing framework designed to mitigate memory overhead and improve editing consistency for long real-world videos. Specifically, we construct a neural video field, based on tri-plane and sparse grid representations, to enable the encoding of videos with hundreds of frames in a memory-efficient manner. Next, we update the video field through off-the-shelf text-to-image (T2I) models to impart text-driven editing effects. A progressive optimization strategy is used to preserve original temporal priors. Importantly, both the neural video field and T2I model are adaptable and replaceable, thus providing a basis for future research. Experiments demonstrate the ability of our approach to edit hundreds of frames with strong inter-frame consistency. Our project is available at https://nvedit.github.io/.