Pan Yi-Lun, Chen Jun-Cheng, Wu Ja-Ling
Department of Computer Science and Information Engineering, National Taiwan University, Taipei 116, Taiwan.
National Center for High-Performance Computing, Hsinchu 300, Taiwan.
Entropy (Basel). 2023 Feb 1;25(2):272. doi: 10.3390/e25020272.
Privacy protection data processing has been critical in recent years when pervasively equipped mobile devices could easily capture high-resolution personal images and videos that may disclose personal information. We propose a new controllable and reversible privacy protection system to address the concern in this work. The proposed scheme can automatically and stably anonymize and de-anonymize face images with one neural network and provide strong security protection with multi-factor identification solutions. Furthermore, users can include other attributes as identification factors, such as passwords and specific facial attributes. Our solution lies in a modified conditional-GAN-based training framework, the Multi-factor Modifier (MfM), to simultaneously accomplish the function of multi-factor facial anonymization and de-anonymization. It can successfully anonymize face images while generating realistic faces satisfying the conditions specified by the multi-factor features, such as gender, hair colors, and facial appearance. Furthermore, MfM can also de-anonymize de-identified faces to their corresponding original ones. One crucial part of our work is design of physically meaningful information-theory-based loss functions, which include mutual information between authentic and de-identification images and mutual information between original and re-identification images. Moreover, extensive experiments and analyses show that, with the correct multi-factor feature information, the MfM can effectively achieve nearly perfect reconstruction and generate high-fidelity and diverse anonymized faces to defend attacks from hackers better than other methods with compatible functionalities. Finally, we justify the advantages of this work through perceptual quality comparison experiments. Our experiments show that the resulting LPIPS (with a value of 0.35), FID (with a value of 28), and SSIM (with a value of 0.95) of MfM demonstrate significantly better de-identification effects than state-of-the-art works. Additionally, the MfM we designed can achieve re-identification, which improves real-world practicability.
近年来，隐私保护数据处理至关重要，因为无处不在的移动设备能够轻易捕捉高分辨率的个人图像和视频，而这些图像和视频可能会泄露个人信息。在这项工作中，我们提出了一种新的可控且可逆的隐私保护系统来解决这一问题。所提出的方案可以通过一个神经网络自动且稳定地对人脸图像进行匿名化和去匿名化处理，并通过多因素识别解决方案提供强大的安全保护。此外，用户可以将其他属性作为识别因素，如密码和特定的面部特征。我们的解决方案基于一个经过修改的基于条件生成对抗网络的训练框架——多因素修改器（MfM），以同时实现多因素人脸匿名化和去匿名化的功能。它能够在生成满足多因素特征（如性别、发色和面部外观）指定条件的逼真人脸的同时，成功地对人脸图像进行匿名化处理。此外，MfM还可以将已去识别的人脸恢复为其相应的原始人脸。我们工作的一个关键部分是设计基于物理意义的信息论损失函数，其中包括真实图像与去识别图像之间的互信息以及原始图像与重新识别图像之间的互信息。此外，大量的实验和分析表明，在拥有正确的多因素特征信息的情况下，MfM能够有效地实现近乎完美的重建，并生成高保真且多样的匿名人脸，比其他具有兼容功能的方法更能抵御黑客攻击。最后，我们通过感知质量比较实验证明了这项工作的优势。我们的实验表明，MfM所得到的LPIPS（值为0.35）、FID（值为28）和SSIM（值为0.95）显示出比现有技术显著更好的去识别效果。此外，我们设计的MfM能够实现重新识别，这提高了在现实世界中的实用性。
