Department of Biomedical Engineering, Georgia Institute of Technology, Emory University, Atlanta, GA 30332, USA.
INESC TEC-Institute for Systems and Computer Engineering, Technology and Science, 4200-465 Porto, Portugal.
Sensors (Basel). 2022 Sep 21;22(19):7166. doi: 10.3390/s22197166.
Label noise is omnipresent in the annotations process and has an impact on supervised learning algorithms. This work focuses on the impact of label noise on the performance of learning models by examining the effect of random and class-dependent label noise on a binary classification task: quality assessment for photoplethysmography (PPG). PPG signal is used to detect physiological changes and its quality can have a significant impact on the subsequent tasks, which makes PPG quality assessment a particularly good target for examining the impact of label noise in the field of biomedicine. Random and class-dependent label noise was introduced separately into the training set to emulate the errors associated with fatigue and bias in labeling data samples. We also tested different representations of the PPG, including features defined by domain experts, 1D raw signal and 2D image. Three different classifiers are tested on the noisy training data, including support vector machine (SVM), XGBoost, 1D Resnet and 2D Resnet, which handle three representations, respectively. The results showed that the two deep learning models were more robust than the two traditional machine learning models for both the random and class-dependent label noise. From the representation perspective, the 2D image shows better robustness compared to the 1D raw signal. The logits from three classifiers are also analyzed, the predicted probabilities intend to be more dispersed when more label noise is introduced. From this work, we investigated various factors related to label noise, including representations, label noise type, and data imbalance, which can be a good guidebook for designing more robust methods for label noise in future work.
标签噪声在注释过程中普遍存在，并对监督学习算法产生影响。本工作通过研究随机标签噪声和类别相关标签噪声对二分类任务（光电容积脉搏波（PPG）质量评估）的影响，重点研究了标签噪声对学习模型性能的影响。PPG 信号用于检测生理变化，其质量对后续任务有重大影响，这使得 PPG 质量评估成为检验生物医学领域标签噪声影响的特别理想的目标。分别在训练集中引入随机标签噪声和类别相关标签噪声，以模拟与数据样本标记疲劳和偏差相关的误差。我们还测试了 PPG 的不同表示形式，包括由领域专家定义的特征、1D 原始信号和 2D 图像。在有噪声的训练数据上测试了三种不同的分类器，包括支持向量机（SVM）、XGBoost、1D Resnet 和 2D Resnet，分别处理三种表示形式。结果表明，两种深度学习模型对随机标签噪声和类别相关标签噪声都比两种传统机器学习模型更具鲁棒性。从表示形式的角度来看，与 1D 原始信号相比，2D 图像显示出更好的鲁棒性。还分析了三个分类器的对数几率，当引入更多的标签噪声时，预测概率的分布意图更加分散。通过这项工作，我们研究了与标签噪声相关的各种因素，包括表示形式、标签噪声类型和数据不平衡，这可以为未来设计更鲁棒的标签噪声方法提供良好的指导。
