Cognitive and Affective Neuroscience Unit, Department of Psychology, University of Zurich, Zurich, Switzerland.
Phonetics and Speech Sciences Group, Department of Computational Linguistics, University of Zurich, Zurich, Switzerland.
Commun Biol. 2024 Jun 11;7(1):711. doi: 10.1038/s42003-024-06372-6.
Deepfakes are viral ingredients of digital environments, and they can trick human cognition into misperceiving the fake as real. Here, we test the neurocognitive sensitivity of 25 participants to accept or reject person identities as recreated in audio deepfakes. We generate high-quality voice identity clones from natural speakers by using advanced deepfake technologies. During an identity matching task, participants show intermediate performance with deepfake voices, indicating levels of deception and resistance to deepfake identity spoofing. On the brain level, univariate and multivariate analyses consistently reveal a central cortico-striatal network that decoded the vocal acoustic pattern and deepfake-level (auditory cortex), as well as natural speaker identities (nucleus accumbens), which are valued for their social relevance. This network is embedded in a broader neural identity and object recognition network. Humans can thus be partly tricked by deepfakes, but the neurocognitive mechanisms identified during deepfake processing open windows for strengthening human resilience to fake information.
深度伪造是数字环境中极具传染性的因素,它们可以欺骗人类认知,将假的误认为真的。在这里,我们测试了 25 名参与者对接受或拒绝音频深度伪造中重新创建的身份的神经认知敏感性。我们使用先进的深度伪造技术,从自然说话者那里生成高质量的语音身份克隆。在身份匹配任务中,参与者对深度伪造声音的表现处于中等水平,这表明存在欺骗水平和对深度伪造身份欺骗的抵制。在大脑水平上,单变量和多变量分析一致地揭示了一个中央皮质纹状体网络,该网络可以解码声音的声学模式和深度伪造水平(听觉皮层),以及具有社会相关性的自然说话者身份(伏隔核)。该网络嵌入在更广泛的神经身份和对象识别网络中。因此,人类可能会被深度伪造所欺骗,但在深度伪造处理过程中确定的神经认知机制为增强人类对虚假信息的适应能力打开了窗口。