Tschida Jordan, Chandrashekar Mayanka, Peluso Alina, Fox Zachary, Krawczuk Patrycja, Murdock Dakota, Wu Xiao-Cheng, Gounley John, Hanson Heidi A
Advanced Computing for Health Sciences, Oak Ridge National Laboratory, Oak Ridge, TN 37830, United States.
Department of Epidemiology, Louisiana State University New Orleans School of Public Health, New Orleans, LA 70112, United States.
JAMIA Open. 2025 May 9;8(3):ooaf033. doi: 10.1093/jamiaopen/ooaf033. eCollection 2025 Jun.
This work evaluated algorithmic bias in biomarkers classification using electronic pathology reports from female breast cancer cases. Bias was assessed across 5 subgroups: cancer registry, race, Hispanic ethnicity, age at diagnosis, and socioeconomic status.
We utilized 594 875 electronic pathology reports from 178 121 tumors diagnosed in Kentucky, Louisiana, New Jersey, New Mexico, Seattle, and Utah to train 2 deep-learning algorithms to classify breast cancer patients using their biomarkers test results. We used balanced error rate (BER), demographic parity (DP), equalized odds (EOD), and equal opportunity (EOP) to assess bias.
We found differences in predictive accuracy between registries, with the highest accuracy in the registry that contributed the most data (Seattle Registry, BER ratios for all registries >1.25). BER showed no significant algorithmic bias in extracting biomarkers (estrogen receptor, progesterone receptor, human epidermal growth factor receptor 2) for race, Hispanic ethnicity, age at diagnosis, or socioeconomic subgroups (BER ratio <1.25). DP, EOD, and EOP all showed insignificant results.
We observed significant differences in BER by registry, but no significant bias using the DP, EOD, and EOP metrics for socio-demographic or racial categories. This highlights the importance of employing a diverse set of metrics for a comprehensive evaluation of model fairness.
A thorough evaluation of algorithmic biases that may affect equality in clinical care is a critical step before deploying algorithms in the real world. We found little evidence of algorithmic bias in our biomarker classification tool. Artificial intelligence tools to expedite information extraction from clinical records could accelerate clinical trial matching and improve care.
本研究利用女性乳腺癌病例的电子病理报告评估生物标志物分类中的算法偏差。在五个亚组中评估偏差：癌症登记处、种族、西班牙裔、诊断年龄和社会经济地位。
我们利用肯塔基州、路易斯安那州、新泽西州、新墨西哥州、西雅图和犹他州诊断的178121例肿瘤的594875份电子病理报告，训练两种深度学习算法，根据生物标志物检测结果对乳腺癌患者进行分类。我们使用平衡错误率（BER）、人口统计学均等（DP）、均等赔率（EOD）和均等机会（EOP）来评估偏差。
我们发现不同登记处之间的预测准确性存在差异，提供数据最多的登记处（西雅图登记处）准确性最高（所有登记处的BER比率>1.25）。在按种族、西班牙裔、诊断年龄或社会经济亚组提取生物标志物（雌激素受体、孕激素受体、人表皮生长因子受体2）时，BER未显示出显著的算法偏差（BER比率<1.25）。DP、EOD和EOP均显示无显著结果。
我们观察到不同登记处的BER存在显著差异，但使用DP、EOD和EOP指标评估社会人口统计学或种族类别时未发现显著偏差。这突出了采用多种指标对模型公平性进行全面评估的重要性。
在将算法应用于现实世界之前，对可能影响临床护理平等性的算法偏差进行全面评估是关键一步。我们在生物标志物分类工具中几乎没有发现算法偏差的证据。加快从临床记录中提取信息的人工智能工具可以加速临床试验匹配并改善护理。
