Du Kangning, Zhang Jiyu, Cao Lin, Guo Yanan, Sun Wenwen
Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing, 100101, China.
Key Laboratory of Information and Communication Systems, Ministry of Information Industry, Beijing Information Science and Technology University, Beijing, 100101, China.
Sci Rep. 2025 May 8;15(1):16056. doi: 10.1038/s41598-025-00574-6.
Sketch face synthesis aims to generate sketch images from photos. Recently, contrastive learning, which maps and aligns information across diverse modalities, has found extensive application in image translation. However, when applying traditional contrastive learning to sketch face synthesis, the random sampling strategy and the imbalance between positive and negative samples result in poor performance of synthesized sketch images regarding local details. To address the above challenges, we propose A Facial Structure Sampling Contrastive Learning Method for Sketch Facial Synthesis. Firstly, we propose a region-constrained sampling module that utilizes the distribution map of facial structure obtained by a dual-branch attention mechanism to segment the input photos into diverse regions, thereby providing regional constraints for sample selection. Subsequently, we propose a dynamic sampling strategy that dynamically adjusts the sampling frequency based on the feature density in the distribution map, thereby alleviating sample imbalance. Additionally, to diminish the background influence and enhance the delineation of character contours, we introduce the mask derived from the input photo as an additional input. Finally, to further enhance the quality of the synthesized sketch images, we introduce pixel-wise loss and perceptual loss. The CUFS dataset experiment demonstrates that our method generates high-quality sketch images, outperforming existing state-of-the-art methods in subjective and objective evaluations.
草图人脸合成旨在从照片生成草图图像。最近，跨不同模态映射和对齐信息的对比学习在图像翻译中得到了广泛应用。然而，将传统对比学习应用于草图人脸合成时，随机采样策略以及正负样本之间的不平衡导致合成草图图像在局部细节方面性能不佳。为应对上述挑战，我们提出了一种用于草图人脸合成的面部结构采样对比学习方法。首先，我们提出一个区域约束采样模块，该模块利用通过双分支注意力机制获得的面部结构分布图将输入照片分割成不同区域，从而为样本选择提供区域约束。随后，我们提出一种动态采样策略，该策略基于分布图中的特征密度动态调整采样频率，从而减轻样本不平衡。此外，为减少背景影响并增强人物轮廓的描绘，我们引入从输入照片派生的掩码作为额外输入。最后，为进一步提高合成草图图像的质量，我们引入逐像素损失和感知损失。CUFS数据集实验表明，我们的方法生成高质量的草图图像，在主观和客观评估中均优于现有的最先进方法。
