Full-frame video stabilization via spatiotemporal transformers

Traditional video stabilization methods use a warping operation to smooth the camera path but result in missing regions in the video frames. To solve this issue, full-frame video stabilization techniques attempt to fill in the unidentified boundary regions, but their effectiveness is limited. In this work, we propose a full-frame video stabilization method using spatiotemporal transformers to fill the missing boundary regions after the warping operation. For training, we adopt a self-supervised strategy and improve it by incorporating temporal information. The proposed approach allows the utilization of redundant video information spatially and temporally while filling in missing regions. Experimental results show that our approach achieves superior results on popular video stabilization datasets. The code, pre-trained model, and video results are available at https://github.com/leventkaracan/VidStabFormer.