Janssen Research & Development, 920 Route 202, Raritan, NJ 08869, USA; OHDSI Collaborators, Observational Health Data Sciences and Informatics (OHDSI), 622 West 168th Street, PH-20, New York, NY 10032, USA.
OHDSI Collaborators, Observational Health Data Sciences and Informatics (OHDSI), 622 West 168th Street, PH-20, New York, NY 10032, USA; Columbia University, 622 West 168th Street, PH20, New York, NY 10032, USA.
J Biomed Inform. 2019 Sep;97:103258. doi: 10.1016/j.jbi.2019.103258. Epub 2019 Jul 29.
The primary approach for defining disease in observational healthcare databases is to construct phenotype algorithms (PAs), rule-based heuristics predicated on the presence, absence, and temporal logic of clinical observations. However, a complete evaluation of PAs, i.e., determining sensitivity, specificity, and positive predictive value (PPV), is rarely performed. In this study, we propose a tool (PheValuator) to efficiently estimate a complete PA evaluation.
We used 4 administrative claims datasets: OptumInsight's de-identified Clinformatics™ Datamart (Eden Prairie,MN); IBM MarketScan Multi-State Medicaid); IBM MarketScan Medicare Supplemental Beneficiaries; and IBM MarketScan Commercial Claims and Encounters from 2000 to 2017. Using PheValuator involves (1) creating a diagnostic predictive model for the phenotype, (2) applying the model to a large set of randomly selected subjects, and (3) comparing each subject's predicted probability for the phenotype to inclusion/exclusion in PAs. We used the predictions as a 'probabilistic gold standard' measure to classify positive/negative cases. We examined 4 phenotypes: myocardial infarction, cerebral infarction, chronic kidney disease, and atrial fibrillation. We examined several PAs for each phenotype including 1-time (1X) occurrence of the diagnosis code in the subject's record and 1-time occurrence of the diagnosis in an inpatient setting with the diagnosis code as the primary reason for admission (1X-IP-1stPos).
Across phenotypes, the 1X PA showed the highest sensitivity/lowest PPV among all PAs. 1X-IP-1stPos yielded the highest PPV/lowest sensitivity. Specificity was very high across algorithms. We found similar results between algorithms across datasets.
PheValuator appears to show promise as a tool to estimate PA performance characteristics.
在观察性医疗保健数据库中定义疾病的主要方法是构建表型算法（PA），这是一种基于临床观察的存在、缺失和时间逻辑的基于规则的启发式方法。然而，很少对 PA 进行全面评估，即确定敏感性、特异性和阳性预测值（PPV）。在这项研究中，我们提出了一种工具（PheValuator）来有效地估计完整的 PA 评估。
我们使用了 4 个管理索赔数据集：OptumInsight 的去标识 Clinformatics™Datamart（明尼苏达州伊登草原）；IBM MarketScan 多州医疗补助；IBM MarketScan 医疗保险补充受益人和 IBM MarketScan 商业索赔和就诊记录，时间范围为 2000 年至 2017 年。使用 PheValuator 涉及（1）为表型创建诊断预测模型，（2）将模型应用于大量随机选择的受试者，以及（3）将每个受试者的表型预测概率与 PA 的纳入/排除进行比较。我们使用预测作为“概率金标准”测量来对阳性/阴性病例进行分类。我们研究了 4 种表型：心肌梗死、脑梗死、慢性肾脏病和心房颤动。我们为每种表型研究了几种 PA，包括在受试者记录中出现诊断代码 1 次（1X）和在住院环境中出现诊断代码 1 次，诊断代码是入院的主要原因（1X-IP-1stPos）。
在所有 PA 中，1X PA 显示出最高的敏感性/最低的 PPV。1X-IP-1stPos 产生了最高的 PPV/最低的敏感性。特异性在所有算法中都非常高。我们在数据集之间的算法中发现了类似的结果。
PheValuator 似乎是一种估计 PA 性能特征的有前途的工具。
