Sharma Vishal, Joon Tanya, Kulkarni Vinaykumar, Samanani Salim, Simpson Scot H, Voaklander Don, Eurich Dean
2-040 Li Ka Shing Center for Health Research Innovation, School of Public Health, University of Alberta, Edmonton, Alberta T6G 2E1, Canada.
OKAKI Health Intelligence, Edmonton, Alberta, Canada.
Int J Med Inform. 2023 Oct;178:105177. doi: 10.1016/j.ijmedinf.2023.105177. Epub 2023 Aug 11.
To develop a machine-learning (ML) model using administrative data to estimate risk of adverse outcomes within 30-days of a benzodiazepine (BZRA) dispensation in older adults for use by health departments/regulators.
DESIGN, SETTING AND PARTICIPANTS: This study was conducted in Alberta, Canada during 2018-2019 in Albertans 65 years of age and older. Those with any history of malignancy or palliative care were excluded.
Each BZRA dispensation from a community pharmacy served as the unit of analysis.
ML algorithms were developed on 2018 administrative data to predict risk of any-cause hospitalization, emergency department visit or death within 30-days of a BZRA dispensation. Validation on 2019 administrative data was done using XGBoost to evaluate discrimination, calibration and other relevant metrics on ranked predictions. Daily and quarterly predictions were simulated on 2019 data.
65,063 study participants were included which represented 633,333 BZRA dispensation during 2018-2019. The validation set had 314,615 dispensations linked to 55,928 all-cause outcomes representing a pre-test probability of 17.8%. C-statistic for the XGBoost model was 0.75. Measuring risk at the end of 2019, the top 0.1 percentile of predicted risk had a LR + of 40.31 translating to a post-test probability of 90%. Daily and quarterly classification simulations resulted in uninformative predictions with positive likelihood ratios less than 10 in all risk prediction categories. Previous history of admissions was ranked highest in variable importance.
Developing ML models using only administrative health data may not provide health regulators with sufficient informative predictions to use as decision aids for potential interventions, especially if considering daily or quarterly classifications of BZRA risks in older adults. ML models may be informative for this context if yearly classifications are preferred. Health regulators should have access to other types of data to improve ML prediction.
利用管理数据开发一种机器学习（ML）模型，以估计老年人在苯二氮䓬类药物（BZRA）配药后30天内出现不良后果的风险，供卫生部门/监管机构使用。
设计、设置和参与者：本研究于2018 - 2019年在加拿大艾伯塔省对65岁及以上的艾伯塔人进行。排除有任何恶性肿瘤或姑息治疗病史的人。
社区药房每次BZRA配药作为分析单位。
基于2018年管理数据开发ML算法，以预测BZRA配药后30天内任何原因导致的住院、急诊就诊或死亡风险。使用XGBoost对2019年管理数据进行验证，以评估排序预测的辨别力、校准度和其他相关指标。对2019年数据进行每日和每季度预测模拟。
纳入65,063名研究参与者，代表2018 - 2019年期间633,333次BZRA配药。验证集有314,615次配药与55,928例全因结局相关，预测试概率为17.8%。XGBoost模型的C统计量为0.75。在2019年底测量风险时，预测风险最高的0.1百分位数的LR +为40.31，转化为测试后概率为90%。每日和每季度分类模拟在所有风险预测类别中产生的预测信息不足，阳性似然比小于10。既往住院史在变量重要性方面排名最高。
仅使用管理健康数据开发ML模型可能无法为卫生监管机构提供足够的信息性预测，以用作潜在干预措施的决策辅助工具，特别是在考虑老年人BZRA风险按日或按季度分类时。如果倾向于按年分类，ML模型在此背景下可能具有信息价值。卫生监管机构应获取其他类型的数据以改善ML预测。
