Khowaja Sunder Ali, Almakdi Sultan, Memon Muhammad Ali, Khuwaja Parus, Sulaiman Adel, Alqahtani Ali, Shaikh Asadullah, Alghamdi Abdullah
Department of Telecommunication, Faculty of Eng. And Tech, University of Sindh, Jamshoro, Sindh, 76090, Pakistan.
Department of Computer Science, College of Computer Science and Information Systems, Najran University, Najran, 61441, Najran, Saudi Arabia.
Heliyon. 2024 Feb 28;10(5):e27012. doi: 10.1016/j.heliyon.2024.e27012. eCollection 2024 Mar 15.
The field of neural style transfer refers to the re-rendering of content image while fusing the features of a style image. The recent studies either focus on multiple style transfer or arbitrary style transfer while using perceptual and fixpoint content losses in their respective network architectures. The aforementioned losses provide notable stylization results but lack the liberty of style control to the user. Consequently, the stylization results also compromise the preservation of details with respect to the content image. This work proposes the hierarchical style transfer network (HSTN) for the image stylization task that could provide the user with the liberty to control the degree of incurred style via denoising parameter. The HSTN incorporates the proposed fixpoint control loss that preserves details from the content image and the addition of denoising CNN network (DnCNN) and denoising loss for allowing the user to control the level of stylization. The encoder-decoder block, the DnCNN block, and the loss network block make the basic building blocks of HSTN. Extensive experiments have been carried out, and the results are compared with existing works to demonstrate the effectiveness of HSTN. The subjective user evaluation shows that the HSTN's stylization represents the best fusion of style and generates unique stylization results while preserving the content image details, which is evident by acquiring 12% better results than the second-best performing method. It has also been observed that the proposed work is amongst the studies that achieve the best trade-off regarding content and style classification scores, i.e. 37.64% and 60.27%, respectively.
神经风格迁移领域是指在融合风格图像特征的同时对内容图像进行重新渲染。最近的研究要么专注于多风格迁移，要么专注于任意风格迁移，同时在各自的网络架构中使用感知和定点内容损失。上述损失提供了显著的风格化效果，但缺乏用户对风格控制的自由度。因此，风格化结果在内容图像细节的保留方面也有所妥协。这项工作提出了用于图像风格化任务的分层风格迁移网络（HSTN），它可以通过去噪参数为用户提供控制风格化程度的自由度。HSTN包含了所提出的定点控制损失，该损失可保留内容图像的细节，还添加了去噪卷积神经网络（DnCNN）和去噪损失，以允许用户控制风格化水平。编码器 - 解码器模块、DnCNN模块和损失网络模块构成了HSTN的基本构建块。已经进行了广泛的实验，并将结果与现有工作进行比较，以证明HSTN的有效性。主观用户评估表明，HSTN的风格化代表了风格的最佳融合，并在保留内容图像细节的同时生成独特的风格化结果，这一点很明显，其获得的结果比表现第二好的方法高出12%。还观察到，所提出的工作是在内容和风格分类分数方面实现最佳权衡的研究之一，即分别为37.64%和60.27%。
