Janssen Research and Development LLC, 1125 Trenton-Harbourton Road, Room K30205, PO Box 200, Titusville, NJ, 08560, USA,
Drug Saf. 2013 Oct;36 Suppl 1:S33-47. doi: 10.1007/s40264-013-0097-8.
Methodological research to evaluate the performance of methods requires a benchmark to serve as a referent comparison. In drug safety, the performance of analyses of spontaneous adverse event reporting databases and observational healthcare data, such as administrative claims and electronic health records, has been limited by the lack of such standards.
To establish a reference set of test cases that contain both positive and negative controls, which can serve the basis for methodological research in evaluating methods performance in identifying drug safety issues.
Systematic literature review and natural language processing of structured product labeling was performed to identify evidence to support the classification of drugs as either positive controls or negative controls for four outcomes: acute liver injury, acute kidney injury, acute myocardial infarction, and upper gastrointestinal bleeding.
Three-hundred and ninety-nine test cases comprised of 165 positive controls and 234 negative controls were identified across the four outcomes. The majority of positive controls for acute kidney injury and upper gastrointestinal bleeding were supported by randomized clinical trial evidence, while the majority of positive controls for acute liver injury and acute myocardial infarction were only supported based on published case reports. Literature estimates for the positive controls shows substantial variability that limits the ability to establish a reference set with known effect sizes.
A reference set of test cases can be established to facilitate methodological research in drug safety. Creating a sufficient sample of drug-outcome pairs with binary classification of having no effect (negative controls) or having an increased effect (positive controls) is possible and can enable estimation of predictive accuracy through discrimination. Since the magnitude of the positive effects cannot be reliably obtained and the quality of evidence may vary across outcomes, assumptions are required to use the test cases in real data for purposes of measuring bias, mean squared error, or coverage probability.
为评估方法的性能而进行的方法学研究需要一个基准作为参考比较。在药物安全性方面，由于缺乏此类标准，自发性不良事件报告数据库和观察性医疗保健数据（如行政索赔和电子健康记录）的分析方法的性能受到限制。
建立一个包含阳性和阴性对照的测试案例参考集，该参考集可作为评估方法在识别药物安全性问题方面的性能的方法学研究的基础。
对结构化产品标签进行系统文献回顾和自然语言处理，以确定证据，以支持将药物分类为阳性对照或阴性对照的证据，用于四个结局：急性肝损伤、急性肾损伤、急性心肌梗死和上消化道出血。
在四个结局中，共确定了 399 个测试案例，其中包括 165 个阳性对照和 234 个阴性对照。急性肾损伤和上消化道出血的大多数阳性对照均得到随机临床试验证据的支持，而急性肝损伤和急性心肌梗死的大多数阳性对照仅基于已发表的病例报告得到支持。阳性对照的文献估计显示出很大的变异性，这限制了建立具有已知效应大小的参考集的能力。
可以建立一个测试案例参考集，以促进药物安全性方面的方法学研究。创建具有无效应（阴性对照）或增加效应（阳性对照）的二元分类的药物-结局对的足够样本是可能的，并且可以通过区分来估计预测准确性。由于阳性效果的幅度不能可靠地获得，并且证据的质量可能因结局而异，因此需要假设才能在真实数据中使用测试案例来衡量偏差、均方误差或覆盖概率。
