IEEE Trans Image Process. 2021;30:5984-5996. doi: 10.1109/TIP.2021.3089942. Epub 2021 Jun 30.
Label smoothing is an effective regularization tool for deep neural networks (DNNs), which generates soft labels by applying a weighted average between the uniform distribution and the hard label. It is often used to reduce the overfitting problem of training DNNs and further improve classification performance. In this paper, we aim to investigate how to generate more reliable soft labels. We present an Online Label Smoothing (OLS) strategy, which generates soft labels based on the statistics of the model prediction for the target category. The proposed OLS constructs a more reasonable probability distribution between the target categories and non-target categories to supervise DNNs. Experiments demonstrate that based on the same classification models, the proposed approach can effectively improve the classification performance on CIFAR-100, ImageNet, and fine-grained datasets. Additionally, the proposed method can significantly improve the robustness of DNN models to noisy labels compared to current label smoothing approaches. The source code is available at our project page: https://mmcheng.net/ols/.
标签平滑是一种有效的深度学习网络（DNN）正则化工具，它通过在均匀分布和硬标签之间应用加权平均来生成软标签。它通常用于减少训练 DNN 的过拟合问题，并进一步提高分类性能。在本文中，我们旨在研究如何生成更可靠的软标签。我们提出了一种在线标签平滑（OLS）策略，它基于模型对目标类别的预测统计信息生成软标签。所提出的 OLS 在目标类和非目标类之间构建了更合理的概率分布，以监督 DNN。实验表明，基于相同的分类模型，所提出的方法可以有效地提高 CIFAR-100、ImageNet 和细粒度数据集上的分类性能。此外，与当前的标签平滑方法相比，所提出的方法可以显著提高 DNN 模型对噪声标签的鲁棒性。源代码可在我们的项目页面上获得：https://mmcheng.net/ols/。
