Frechman Erica, Jaeger Byron C, Kowalkowski Marc, Williamson Jeff D, Lenoir Kristin M, Palakshappa Jessica A, Wells Brian J, Callahan Kathryn E, Pajewski Nicholas M, Gabbard Jennifer L
Section on Gerontology and Geriatric Medicine, Department of Internal Medicine, Wake Forest University School of Medicine, Winston-Salem, NC 27157, United States.
Department of Biostatistics and Data Science, Wake Forest University School of Medicine, Winston-Salem, NC 27157, United States.
J Am Med Inform Assoc. 2025 Jul 1;32(7):1110-1119. doi: 10.1093/jamia/ocaf062.
To examine the discrimination, calibration, and algorithmic fairness of the Epic End of Life Care Index (EOL-CI).
We assessed the EOL-CI's performance by estimating area under the receiver operating characteristic curve (AUC), sensitivity, and positive and negative predictive values in community-dwelling adults ≥65 years of age in a single health system in the Southeastern United States. Algorithmic fairness was examined by comparing the model's performance across sex, race, and ethnicity subgroups. Using a machine learning approach, we also explored local re-calibration of the EOL-CI considering additional information on past hospitalizations and frailty.
Among 215 731 patients (median age = 74 years, 57% female, 12% of Black race), 10% were classified as medium risk (15-44) and 3% as high risk (≥45) by the EOL-CI. The observed 1-year mortality rate was 3%. The EOL-CI had an AUC 0.82 for 1-year mortality, with a positive predictive value of 22%. Predictive performance was generally similar across sex and race subgroups, though the EOL-CI displayed better performance with increasing age and in older adults with 2 or more outpatient encounters in the past 24 months. Local re-calibration of the EOL-CI was required to provide absolute estimates of mortality risk, and calibration was further improved when the EOL-CI was augmented with data on inpatient hospitalizations and frailty.
The EOL-CI demonstrates reasonable discrimination, albeit with better performance in older adults and in those with greater health system contact.
Local refinement and calibration of the EOL-CI score is required to provide direct estimates of prognosis, with the goal of making the EOL-CI a more a valuable tool at the point of care for identifying patients who would benefit from targeted palliative care interventions and proactive care planning.
评估Epic临终关怀指数（EOL-CI）的辨别力、校准度和算法公平性。
我们通过估计美国东南部单一医疗系统中65岁及以上社区居住成年人的受试者操作特征曲线下面积（AUC）、敏感性以及阳性和阴性预测值，来评估EOL-CI的性能。通过比较模型在性别、种族和族裔亚组中的性能来检验算法公平性。使用机器学习方法，我们还考虑了过去住院和虚弱状况的额外信息，探索了EOL-CI的局部重新校准。
在215731名患者（中位年龄 = 74岁，57%为女性，12%为黑人种族）中，EOL-CI将10%的患者分类为中度风险（15 - 44），3%为高风险（≥45）。观察到的1年死亡率为3%。EOL-CI对1年死亡率的AUC为0.82，阳性预测值为22%。预测性能在性别和种族亚组中总体相似，不过EOL-CI在年龄增长以及过去24个月内有2次或更多门诊就诊的老年人中表现更好。需要对EOL-CI进行局部重新校准以提供死亡风险的绝对估计值，并且当EOL-CI增加住院和虚弱数据时，校准得到进一步改善。
EOL-CI表现出合理的辨别力，尽管在老年人和与医疗系统接触较多的人群中表现更好。
需要对EOL-CI评分进行局部细化和校准以提供预后的直接估计值，目标是使EOL-CI成为在护理点更有价值的工具，用于识别将从针对性姑息治疗干预和积极护理计划中受益的患者。
