Zeng Lu, Chen Xuan, Shi Xiaoshuang, Tao Shen Heng
IEEE Trans Neural Netw Learn Syst. 2025 Apr;36(4):7711-7724. doi: 10.1109/TNNLS.2024.3394511. Epub 2025 Apr 4.
The presence of label noise in the training data has a profound impact on the generalization of deep neural networks (DNNs). In this study, we introduce and theoretically demonstrate a simple feature noise (FN) method, which directly adds noise to the features of training data and can enhance the generalization of DNNs under label noise. Specifically, we conduct theoretical analyses to reveal that label noise leads to weakened DNN generalization by loosening the generalization bound, and FN results in better DNN generalization by imposing an upper bound on the mutual information between the model weights and the features, which constrains the generalization bound. Furthermore, we conduct a qualitative analysis to discuss the ideal type of FN that obtains good label noise generalization. Finally, extensive experimental results on several popular datasets demonstrate that the FN method can significantly enhance the label noise generalization of state-of-the-art methods. The source codes of the FN method are available on https://github.com/zlzenglu/FN.
训练数据中标签噪声的存在对深度神经网络（DNN）的泛化有深远影响。在本研究中，我们引入并从理论上论证了一种简单的特征噪声（FN）方法，该方法直接向训练数据的特征添加噪声，并能在标签噪声下增强DNN的泛化能力。具体而言，我们进行理论分析以揭示标签噪声通过放宽泛化边界导致DNN泛化能力减弱，而FN通过对模型权重与特征之间的互信息施加上限来实现更好的DNN泛化，这限制了泛化边界。此外，我们进行定性分析以讨论获得良好标签噪声泛化的理想FN类型。最后，在几个流行数据集上的大量实验结果表明，FN方法可以显著增强现有方法的标签噪声泛化能力。FN方法的源代码可在https://github.com/zlzenglu/FN上获取。
