Qiao Tong, Xie Shichuang, Chen Yanli, Retraint Florent, Luo Xiangyang
IEEE Trans Pattern Anal Mach Intell. 2024 Jul;46(7):4654-4668. doi: 10.1109/TPAMI.2024.3356814. Epub 2024 Jun 5.
Nowadays, Deepfake videos are widely spread over the Internet, which severely impairs the public trustworthiness and social security. Although more and more reliable detectors have recently sprung up for resisting against that new-emerging tampering technique, some challengeable issues still need to be addressed, such that most of Deepfake video detectors under the framework of the supervised mechanism require a large scale of samples with accurate labels for training. When the amount of the training samples with the true labels are not enough or the training data are maliciously poisoned by adversaries, the supervised classifier is probably not reliable for detection. To tackle that tough issue, it is proposed to design a fully unsupervised Deepfake detector. In particular, in the whole procedure of training or testing, we have no idea of any information about the true labels of samples. First, we novelly design a pseudo-label generator for labeling the training samples, where the traditional hand-crafted features are used to characterize both types of samples. Second, the training samples with the pseudo-labels are fed into the proposed enhanced contrastive learner, in which the discriminative features are further extracted and continually refined by iteration on the guidance of the contrastive loss. Last, relying on the inter-frame correlation, we complete the final binary classification between real and fake videos. A large scale of experimental results empirically verify the effectiveness of our proposed unsupervised Deepfake detector on the benchmark datasets including FF++, Celeb-DF, DFD, DFDC, and UADFV. Furthermore, our proposed well-performed detector is superior to the current unsupervised method, and comparable to the baseline supervised methods. More importantly, when facing the problem of the labeled data poisoned by malicious adversaries or insufficient data for training, our proposed unsupervised Deepfake detector performs its powerful superiority.
如今，深度伪造视频在互联网上广泛传播，这严重损害了公众信任和社会安全。尽管最近涌现出越来越多可靠的检测器来抵御这种新兴的篡改技术，但仍有一些具有挑战性的问题需要解决，例如在监督机制框架下的大多数深度伪造视频检测器需要大量带有准确标签的样本进行训练。当带有真实标签的训练样本数量不足或训练数据被对手恶意污染时，监督分类器在检测时可能并不可靠。为了解决这个棘手的问题，本文提出设计一种完全无监督的深度伪造检测器。具体而言，在整个训练或测试过程中，我们对样本的真实标签一无所知。首先，我们创新性地设计了一个伪标签生成器来标记训练样本，其中传统的手工制作特征用于表征两种类型的样本。其次，将带有伪标签的训练样本输入到提出的增强对比学习器中，在对比损失的引导下，通过迭代进一步提取并不断细化判别特征。最后，依靠帧间相关性，我们完成真实视频和伪造视频之间的最终二分类。大量实验结果通过实证验证了我们提出的无监督深度伪造检测器在包括FF++、Celeb-DF、DFD、DFDC和UADFV在内的基准数据集上的有效性。此外，我们提出的性能良好的检测器优于当前的无监督方法，并且与基线监督方法相当。更重要的是，当面临被恶意对手污染的标记数据或训练数据不足的问题时，我们提出的无监督深度伪造检测器表现出强大的优势。
