As the use of deepfake facial videos proliferate, the associated threats to social security and integrity cannot be overstated. Effective methods for detecting forged facial videos are thus urgently needed. While many deep learning-based facial forgery detection approaches show promise, they often fail to delve deeply into the complex relationships between image features and forgery indicators, limiting their effectiveness to specific forgery techniques. To address this challenge, we propose a dual-branch collaborative deepfake detection network. The network processes video frame images as input, where a specialized noise extraction module initially extracts the noise feature maps. Subsequently, the original facial images and corresponding noise maps are directed into two parallel feature extraction branches to concurrently learn texture and noise forgery clues. An attention mechanism is employed between the two branches to facilitate mutual guidance and enhancement of texture and noise features across four different scales. This dual-modal feature integration enhances sensitivity to forgery artifacts and boosts generalization ability across various forgery techniques. Features from both branches are then effectively combined and processed through a multi-layer perception layer to distinguish between real and forged video. Experimental results on benchmark deepfake detection datasets demonstrate that our approach outperforms existing state-of-the-art methods in terms of detection performance, accuracy, and generalization ability.

Voice portrait technology has explored and established the relationship between speakers’ voices and their facial features, aiming to generate corresponding facial characteristics by providing the voice of an unknown speaker. Due to its powerful advantages in image generation, Generative Adversarial Networks (GANs) have now been widely applied across various fields. The existing Voice2Face methods for voice portraits are primarily based on GANs trained on voice-face paired datasets. However, voice portrait models solely constructed on GANs face limitations in image generation quality and struggle to maintain facial similarity. Additionally, the training process is relatively unstable, thereby affecting the overall generative performance of the model. To overcome the above challenges, we propose a novel deep Generative Adversarial Network model for audio-visual synthesis, named AVP-GAN (Attention-enhanced Voice Portrait Model using Generative Adversarial Network). This model is based on a convolutional attention mechanism and is capable of generating corresponding facial images from the voice of an unknown speaker. Firstly, to address the issue of training instability, we integrate convolutional neural networks with deep GANs. In the network architecture, we apply spectral normalization to constrain the variation of the discriminator, preventing issues such as mode collapse. Secondly, to enhance the model’s ability to extract relevant features between the two modalities, we propose a voice portrait model based on convolutional attention. This model learns the mapping relationship between voice and facial features in a common space from both channel and spatial dimensions independently. Thirdly, to enhance the quality of generated faces, we have incorporated a degradation removal module and utilized pretrained facial GANs as facial priors to repair and enhance the clarity of the generated facial images. Experimental results demonstrate that our AVP-GAN achieved a cosine similarity of 0.511, outperforming the performance of our comparison model, and effectively achieved the generation of high-quality facial images corresponding to a speaker’s voice.