Tenet Healthcare, Nashville, TN.
The Intensivist Group/Sound Physicians, Tacoma, WA.
Crit Care Med. 2018 Jun;46(6):e481-e488. doi: 10.1097/CCM.0000000000003011.
Risk adjustment algorithms for ICU mortality are necessary for measuring and improving ICU performance. Existing risk adjustment algorithms are not widely adopted. Key barriers to adoption include licensing and implementation costs as well as labor costs associated with human-intensive data collection. Widespread adoption of electronic health records makes automated risk adjustment feasible. Using modern machine learning methods and open source tools, we developed and evaluated a retrospective risk adjustment algorithm for in-hospital mortality among ICU patients. The Risk of Inpatient Death score can be fully automated and is reliant upon data elements that are generated in the course of usual hospital processes.
One hundred thirty-one ICUs in 53 hospitals operated by Tenet Healthcare.
A cohort of 237,173 ICU patients discharged between January 2014 and December 2016.
The data were randomly split into training (36 hospitals), and validation (17 hospitals) data sets. Feature selection and model training were carried out using the training set while the discrimination, calibration, and accuracy of the model were assessed in the validation data set.
Model discrimination was evaluated based on the area under receiver operating characteristic curve; accuracy and calibration were assessed via adjusted Brier scores and visual analysis of calibration curves. Seventeen features, including a mix of clinical and administrative data elements, were retained in the final model. The Risk of Inpatient Death score demonstrated excellent discrimination (area under receiver operating characteristic curve = 0.94) and calibration (adjusted Brier score = 52.8%) in the validation dataset; these results compare favorably to the published performance statistics for the most commonly used mortality risk adjustment algorithms.
Low adoption of ICU mortality risk adjustment algorithms impedes progress toward increasing the value of the healthcare delivered in ICUs. The Risk of Inpatient Death score has many attractive attributes that address the key barriers to adoption of ICU risk adjustment algorithms and performs comparably to existing human-intensive algorithms. Automated risk adjustment algorithms have the potential to obviate known barriers to adoption such as cost-prohibitive licensing fees and significant direct labor costs. Further evaluation is needed to ensure that the level of performance observed in this study could be achieved at independent sites.
为了衡量和改善 ICU 的绩效，需要为 ICU 死亡率制定风险调整算法。现有的风险调整算法并未被广泛采用。采用的主要障碍包括许可和实施成本，以及与密集型数据收集相关的人工成本。电子健康记录的广泛应用使得自动风险调整成为可能。我们使用现代机器学习方法和开源工具，开发并评估了一种 ICU 住院患者院内死亡率的回顾性风险调整算法。住院死亡风险评分可以完全自动化，并且依赖于在常规医院流程中产生的数据元素。
由 Tenet Healthcare 运营的 53 家医院的 131 个 ICU。
2014 年 1 月至 2016 年 12 月期间出院的 237173 名 ICU 患者队列。
数据随机分为训练（36 家医院）和验证（17 家医院）数据集。使用训练集进行特征选择和模型训练，而模型的判别、校准和准确性则在验证数据集中进行评估。
基于接收者操作特征曲线下面积评估模型判别能力；通过调整后的 Brier 评分和校准曲线的直观分析评估准确性和校准。最终模型保留了 17 个特征，包括临床和管理数据元素的混合。住院死亡风险评分在验证数据集中表现出出色的判别能力（接收者操作特征曲线下面积=0.94）和校准能力（调整后的 Brier 评分=52.8%）；与最常用的死亡率风险调整算法的发布性能统计数据相比，这些结果具有可比性。
ICU 死亡率风险调整算法的低采用率阻碍了提高 ICU 所提供医疗服务价值的进展。住院死亡风险评分具有许多吸引人的属性，可解决 ICU 风险调整算法采用的主要障碍，并且与现有的密集型人工算法性能相当。自动化风险调整算法有可能消除采用成本过高的许可费和大量直接劳动力成本等已知障碍。需要进一步评估，以确保在本研究中观察到的性能水平能够在独立站点实现。
