Almenoff June S, LaCroix Karol K, Yuen Nancy A, Fram David, DuMouchel William
Global Clinical Safety and Pharmacovigilance, GlaxoSmithKline, Research Triangle Park, North Carolina 27709-3398, USA.
Drug Saf. 2006;29(10):875-87. doi: 10.2165/00002018-200629100-00005.
There is increasing interest in using disproportionality-based signal detection methods to support postmarketing safety surveillance activities. Two commonly used methods, empirical Bayes multi-item gamma Poisson shrinker (MGPS) and proportional reporting ratio (PRR), perform differently with respect to the number and types of signals detected. The goal of this study was to compare and analyse the performance characteristics of these two methods, to understand why they differ and to consider the practical implications of these differences for a large, industry-based pharmacovigilance department.
We compared the numbers and types of signals of disproportionate reporting (SDRs) obtained with MGPS and PRR using two postmarketing safety databases and a simulated database. We recorded signal counts and performed a qualitative comparison of the drug-event combinations signalled by the two methods as well as a sensitivity analysis to better understand how the thresholds commonly used for these methods impact their performance.
PRR detected more SDRs than MGPS. We observed that MGPS is less subject to confounding by demographic factors because it employs stratification and is more stable than PRR when report counts are low. Simulation experiments performed using published empirical thresholds demonstrated that PRR detected false-positive signals at a rate of 1.1%, while MGPS did not detect any statistical false positives. In an attempt to separate the effect of choice of signal threshold from more fundamental methodological differences, we performed a series of experiments in which we modified the conventional threshold values for each method so that each method detected the same number of SDRs for the example drugs studied. This analysis, which provided quantitative examples of the relationship between the published thresholds for the two methods, demonstrates that the signalling criterion published for PRR has a higher signalling frequency than that published for MGPS.
The performance differences between the PRR and MGPS methods are related to (i) greater confounding by demographic factors with PRR; (ii) a higher tendency of PRR to detect false-positive signals when the number of reports is small; and (iii) the conventional thresholds that have been adapted for each method. PRR tends to be more 'sensitive' and less 'specific' than MGPS. A high-specificity disproportionality method, when used in conjunction with medical triage and investigation of critical medical events, may provide an efficient and robust approach to applying quantitative methods in routine postmarketing pharmacovigilance.
基于不成比例性的信号检测方法在支持上市后安全性监测活动方面越来越受到关注。两种常用方法，经验贝叶斯多项目伽马泊松收缩器（MGPS）和比例报告比（PRR），在检测到的信号数量和类型方面表现不同。本研究的目的是比较和分析这两种方法的性能特征，理解它们为何不同，并考虑这些差异对大型、基于行业的药物警戒部门的实际影响。
我们使用两个上市后安全性数据库和一个模拟数据库，比较了用MGPS和PRR获得的不成比例报告信号（SDR）的数量和类型。我们记录信号计数，并对两种方法发出信号的药物-事件组合进行定性比较，以及进行敏感性分析，以更好地理解这些方法常用的阈值如何影响其性能。
PRR检测到的SDR比MGPS多。我们观察到MGPS受人口统计学因素的混杂影响较小，因为它采用分层方法，并且在报告计数较低时比PRR更稳定。使用已发表的经验阈值进行的模拟实验表明，PRR检测到假阳性信号的比率为1.1%，而MGPS未检测到任何统计学上的假阳性。为了将信号阈值选择的影响与更基本的方法学差异分开，我们进行了一系列实验，在这些实验中我们修改了每种方法的传统阈值，以便每种方法对所研究的示例药物检测到相同数量的SDR。该分析提供了两种方法已发表阈值之间关系的定量示例，表明为PRR公布的信号标准比为MGPS公布的信号标准具有更高的信号频率。
PRR和MGPS方法之间的性能差异与以下因素有关：（i）PRR受人口统计学因素的混杂影响更大；（ii）当报告数量较少时，PRR检测到假阳性信号的倾向更高；（iii）为每种方法调整的传统阈值。PRR往往比MGPS更“敏感”，而“特异性”更低。一种高特异性的不成比例性方法，与医疗分诊和对关键医疗事件的调查结合使用时，可能为在常规上市后药物警戒中应用定量方法提供一种有效且稳健的方法。
