VFI-E: Flow-Based Extrapolation for High-Fidelity Unsupervised Video Frame Interpolation

Video Frame Interpolation (VFI) seeks to enhance visual quality by synthesizing intermediate frames, effectively increasing the frame rate. While existing supervised VFI methods have shown remarkable success, they are heavily reliant on large labeled datasets of high frame-rate videos, posing significant challenges in terms of acquisition cost and scalability. To address this limitation, we present Video Frame Interpolation by Extrapolation (VFI-E), a novel unsupervised approach that leverages frame extrapolation as a self-supervisory signal for high-quality VFI. Our central hypothesis is that the fidelity of an interpolated frame can be effectively gauged by its ability to facilitate accurate frame extrapolation. VFI-E operates by first synthesizing an intermediate frame using a flow-based interpolation network. This interpolated frame, along with one of the input frames, is then fed into an extrapolation network tasked with reconstructing the other input frame. The reconstruction loss between the original and extrapolated input frames serves as the driving force for optimizing the interpolation network, eliminating the need for ground truth intermediate frames. Extensive experiments on benchmark datasets, including UCF101, Vimeo-90k, and Adobe240fps, demonstrate that our unsupervised VFI-E achieves performance comparable to state-of-the-art supervised methods. Specifically, VFI-E attains competitive Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) metrics on single-frame interpolation tasks and exhibits strong performance on multi-frame interpolation, showcasing its effectiveness and generalization capabilities.