Liu Xiaoli, Shen Max, Lie Margaret, Zhang Zhongheng, Liu Chao, Li Deyu, Mark Roger G, Zhang Zhengbo, Celi Leo Anthony
Center for Artificial Intelligence in Medicine, The General Hospital of PLA, Beijing, China.
School of Biological Science and Medical Engineering, Beihang University, Beijing, China.
Crit Care Explor. 2024 Jan 17;6(1):e1033. doi: 10.1097/CCE.0000000000001033. eCollection 2024 Jan.
Although illness severity scoring systems are widely used to support clinical decision-making and assess ICU performance, their potential bias across different age, sex, and primary language groups has not been well-studied.
We aimed to identify potential bias of Sequential Organ Failure Assessment (SOFA) and Acute Physiology and Chronic Health Evaluation (APACHE) IVa scores via large ICU databases.
SETTING/PATIENTS: This multicenter, retrospective study was conducted using data from the Medical Information Mart for Intensive Care (MIMIC) and eICU Collaborative Research Database. SOFA and APACHE IVa scores were obtained from ICU admission. Hospital mortality was the primary outcome. Discrimination (area under receiver operating characteristic [AUROC] curve) and calibration (standardized mortality ratio [SMR]) were assessed for all subgroups.
Not applicable.
A total of 196,310 patient encounters were studied. Discrimination for both scores was worse in older patients compared with younger patients and female patients rather than male patients. In MIMIC, discrimination of SOFA in non-English primary language speakers patients was worse than that of English speakers (AUROC 0.726 vs. 0.783, < 0.0001). Evaluating calibration via SMR showed statistically significant underestimations of mortality when compared with overall cohort in the oldest patients for both SOFA and APACHE IVa, female patients (1.09) for SOFA, and non-English primary language patients (1.38) for SOFA in MIMIC.
Differences in discrimination and calibration of two scores across varying age, sex, and primary language groups suggest illness severity scores are prone to bias in mortality predictions. Caution must be taken when using them for quality benchmarking and decision-making among diverse real-world populations.
尽管疾病严重程度评分系统被广泛用于支持临床决策和评估重症监护病房（ICU）的表现，但其在不同年龄、性别和主要语言群体中的潜在偏差尚未得到充分研究。
设计、背景与患者：我们旨在通过大型ICU数据库确定序贯器官衰竭评估（SOFA）和急性生理与慢性健康状况评价系统（APACHE）IVa评分的潜在偏差。
背景/患者：本多中心回顾性研究使用了重症监护医学信息集市（MIMIC）和电子ICU协作研究数据库的数据。SOFA和APACHE IVa评分在ICU入院时获取。医院死亡率是主要结局。对所有亚组评估了区分度（受试者操作特征曲线下面积[AUROC]）和校准度（标准化死亡率比[SMR]）。
不适用。
共研究了196310例患者。与年轻患者相比，两个评分在老年患者中的区分度更差，在女性患者而非男性患者中也是如此。在MIMIC数据库中，以非英语为主要语言的患者中SOFA评分的区分度低于以英语为主要语言的患者（AUROC分别为0.726和0.783，P<0.0001）。通过SMR评估校准度显示，与总体队列相比，在最年长患者中，SOFA和APACHE IVa评分均存在统计学上显著的死亡率低估；在女性患者中，SOFA评分存在死亡率低估（SMR为1.09）；在MIMIC数据库中，以非英语为主要语言的患者中，SOFA评分存在死亡率低估（SMR为1.38）。
两个评分在不同年龄、性别和主要语言群体中的区分度和校准度存在差异，这表明疾病严重程度评分在死亡率预测中容易出现偏差。在将其用于不同实际人群的质量基准和决策时必须谨慎。
