Jacobs Michael A, Hausmann Leslie R M, Handzel Robert M, Schmidt Susanne, Jacobs Carly A, Hall Daniel E
From the Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA.
Division of General Internal Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA.
Ann Surg Open. 2024 Sep 25;5(4):e490. doi: 10.1097/AS9.0000000000000490. eCollection 2024 Dec.
Our objective was to assess potential racial bias within the Risk Analysis Index (RAI).
Patient risk measures are rarely tested for racial bias. Measures of frailty, like the RAI, need to be evaluated for poor predictive performance among Black patients.
Retrospective cohort study using April 2010-March 2019 Veterans Affairs Surgical Quality Improvement Program and 2010-2019 National Surgical Quality Improvement Program data. The performance of the RAI and several potential variants were compared between Black and White cases using various metrics to predict mortality (180-day for Veterans Affairs Surgical Quality Improvement Program, 30-day for National Surgical Quality Improvement Program).
Using the current, clinical threshold, the RAI performed as good or better among Black cases across various performance metrics White. When a higher threshold was used, Black cases had higher true positive rates but lower true negative rates, yielding 2.0% higher balanced accuracy. No RAI variant noticeably eliminated bias, improved parity across both true positives and true negatives, or improved overall model performance.
The RAI tends to predict mortality among Black patients better than it predicts mortality among White patients. As existing bias-reducing techniques were not effective, further research into bias-reducing techniques is needed, especially for clinical risk predictions. We recommend using the RAI for both statistical analysis of surgical cohorts and quality improvement programs, such as the Surgical Pause.
我们的目的是评估风险分析指数（RAI）中潜在的种族偏见。
患者风险测量很少进行种族偏见测试。像RAI这样的虚弱测量方法，需要针对黑人患者预测性能不佳的情况进行评估。
采用回顾性队列研究，使用2010年4月至2019年3月退伍军人事务部外科质量改进计划和2010 - 2019年国家外科质量改进计划的数据。使用各种指标预测死亡率（退伍军人事务部外科质量改进计划为180天，国家外科质量改进计划为30天），比较黑人和白人病例中RAI及几种潜在变体的性能。
使用当前的临床阈值，在各种性能指标上，RAI在黑人病例中的表现与白人一样好或更好。当使用更高的阈值时，黑人病例的真阳性率更高，但真阴性率更低，平衡准确率高出2.0%。没有RAI变体能够显著消除偏见、改善真阳性和真阴性的均等性或提高整体模型性能。
RAI对黑人患者死亡率的预测往往比对白人患者死亡率的预测更好。由于现有的减少偏见技术无效，需要进一步研究减少偏见的技术，特别是对于临床风险预测。我们建议将RAI用于外科队列的统计分析和质量改进计划，如手术暂停。
