Video saliency prediction via gated fusion of dual-stream hybrid architecture

To balance long-range spatiotemporal dependency modeling and fine local feature preservation in video saliency prediction (VSP), we propose an end-to-end dynamic complementary fusion network (DcF-Net) featuring a dual-backbone fusion architecture. The spatiotemporal stream leverages improved multi-scale vision Transformers (MViTv2) to construct a spatiotemporal feature pyramid, thereby capturing multi-scale motion cues, while the ConvNeXt branch compensates for potential local detail loss. These complementary features are adaptively integrated via a spatiotemporal gated fusion (STGF) module that dynamically regulates inter-stream interaction weights, followed by a temporal fusion module (TFM) to enhance inter-frame coherence and stability. Extensive experiments on DHF1K, Hollywood-2, and UCF Sports datasets demonstrate significant performance gains over state-of-the-art models. Specifically, our model achieves AUC-J, SIM, s-AUC, CC, and NSS scores of 0.922, 0.414, 0.745, 0.570, and 3.284 on DHF1K, relative improvements of 0.5%, 2.5%, 1.5%, and 2.0% in AUC-J, CC, s-AUC, and NSS respectively over the previous best-performing TM2SP-Net. These results validate the capability of DcF-Net to accurately predict human visual saliency in diverse and dynamic scenes.