Government Health and Human Services, IBM Watson Health, Bethesda, Maryland, USA.
Food and Drug Administration, Silver Spring, Maryland, USA.
J Am Med Inform Assoc. 2021 Jul 14;28(7):1507-1517. doi: 10.1093/jamia/ocab036.
Claims-based algorithms are used in the Food and Drug Administration Sentinel Active Risk Identification and Analysis System to identify occurrences of health outcomes of interest (HOIs) for medical product safety assessment. This project aimed to apply machine learning classification techniques to demonstrate the feasibility of developing a claims-based algorithm to predict an HOI in structured electronic health record (EHR) data.
We used the 2015-2019 IBM MarketScan Explorys Claims-EMR Data Set, linking administrative claims and EHR data at the patient level. We focused on a single HOI, rhabdomyolysis, defined by EHR laboratory test results. Using claims-based predictors, we applied machine learning techniques to predict the HOI: logistic regression, LASSO (least absolute shrinkage and selection operator), random forests, support vector machines, artificial neural nets, and an ensemble method (Super Learner).
The study cohort included 32 956 patients and 39 499 encounters. Model performance (positive predictive value [PPV], sensitivity, specificity, area under the receiver-operating characteristic curve) varied considerably across techniques. The area under the receiver-operating characteristic curve exceeded 0.80 in most model variations.
For the main Food and Drug Administration use case of assessing risk of rhabdomyolysis after drug use, a model with a high PPV is typically preferred. The Super Learner ensemble model without adjustment for class imbalance achieved a PPV of 75.6%, substantially better than a previously used human expert-developed model (PPV = 44.0%).
It is feasible to use machine learning methods to predict an EHR-derived HOI with claims-based predictors. Modeling strategies can be adapted for intended uses, including surveillance, identification of cases for chart review, and outcomes research.
基于索赔的算法被用于食品和药物管理局监测主动风险识别和分析系统，以识别医疗产品安全评估中感兴趣的健康结果（HOI）的发生。本项目旨在应用机器学习分类技术，展示开发基于索赔算法预测结构化电子健康记录（EHR）数据中 HOI 的可行性。
我们使用了 2015-2019 年 IBM MarketScan Explorys 索赔-EMR 数据集，在患者水平上链接行政索赔和 EHR 数据。我们专注于单一的 HOI，即肌溶解症，由 EHR 实验室检测结果定义。使用基于索赔的预测因子，我们应用机器学习技术预测 HOI：逻辑回归、LASSO（最小绝对收缩和选择算子）、随机森林、支持向量机、人工神经网络和集成方法（Super Learner）。
研究队列包括 32956 名患者和 39499 次就诊。模型性能（阳性预测值[PPV]、敏感性、特异性、接收者操作特征曲线下面积）在不同技术之间差异很大。大多数模型变体的接收者操作特征曲线下面积均超过 0.80。
对于食品和药物管理局主要使用案例，即在药物使用后评估肌溶解症风险，通常首选具有高 PPV 的模型。未经平衡分类调整的 Super Learner 集成模型的 PPV 为 75.6%，明显优于先前使用的人类专家开发的模型（PPV=44.0%）。
使用机器学习方法基于索赔预测因子预测 EHR 衍生的 HOI 是可行的。建模策略可以根据预期用途进行调整，包括监测、确定病例进行图表审查和结局研究。
