Yang Shuo, Wu Songhua, Yang Erkun, Han Bo, Liu Yang, Xu Min, Niu Gang, Liu Tongliang
IEEE Trans Pattern Anal Mach Intell. 2023 Dec;45(12):14055-14068. doi: 10.1109/TPAMI.2023.3301876. Epub 2023 Nov 3.
In label-noise learning, estimating the transition matrix is a hot topic as the matrix plays an important role in building statistically consistent classifiers. Traditionally, the transition from clean labels to noisy labels (i.e., clean-label transition matrix (CLTM)) has been widely exploited on class-dependent label-noise (wherein all samples in a clean class share the same label transition matrix). However, the CLTM cannot handle the more common instance-dependent label-noise well (wherein the clean-to-noisy label transition matrix needs to be estimated at the instance level by considering the input quality). Motivated by the fact that classifiers mostly output Bayes optimal labels for prediction, in this paper, we study to directly model the transition from Bayes optimal labels to noisy labels (i.e., Bayes-Label Transition Matrix (BLTM)) and learn a classifier to predict Bayes optimal labels. Note that given only noisy data, it is ill-posed to estimate either the CLTM or the BLTM. But favorably, Bayes optimal labels have no uncertainty compared with the clean labels, i.e., the class posteriors of Bayes optimal labels are one-hot vectors while those of clean labels are not. This enables two advantages to estimate the BLTM, i.e., (a) a set of examples with theoretically guaranteed Bayes optimal labels can be collected out of noisy data; (b) the feasible solution space is much smaller. By exploiting the advantages, this work proposes a parametrical model for estimating the instance-dependent label-noise transition matrix by employing a deep neural network, leading to better generalization and superior classification performance.
在标签噪声学习中，估计转移矩阵是一个热门话题，因为该矩阵在构建统计上一致的分类器中起着重要作用。传统上，从干净标签到噪声标签的转移（即干净标签转移矩阵（CLTM））已被广泛用于类依赖的标签噪声（其中干净类中的所有样本共享相同的标签转移矩阵）。然而，CLTM不能很好地处理更常见的实例依赖的标签噪声（其中需要通过考虑输入质量在实例级别估计干净到噪声的标签转移矩阵）。鉴于分类器大多输出贝叶斯最优标签进行预测这一事实，在本文中，我们研究直接对从贝叶斯最优标签到噪声标签的转移进行建模（即贝叶斯标签转移矩阵（BLTM）），并学习一个分类器来预测贝叶斯最优标签。请注意，仅给定噪声数据时，估计CLTM或BLTM都是不适定的。但有利的是，与干净标签相比，贝叶斯最优标签没有不确定性，即贝叶斯最优标签的类后验是独热向量，而干净标签的类后验不是。这使得估计BLTM有两个优势，即：（a）可以从噪声数据中收集一组具有理论保证的贝叶斯最优标签的示例；（b）可行解空间要小得多。通过利用这些优势，这项工作提出了一种参数模型，通过使用深度神经网络来估计实例依赖的标签噪声转移矩阵，从而实现更好的泛化和卓越的分类性能。
