Ioannidis John P A
Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.
JAMA. 2005 Jul 13;294(2):218-28. doi: 10.1001/jama.294.2.218.
Controversy and uncertainty ensue when the results of clinical research on the effectiveness of interventions are subsequently contradicted. Controversies are most prominent when high-impact research is involved.
To understand how frequently highly cited studies are contradicted or find effects that are stronger than in other similar studies and to discern whether specific characteristics are associated with such refutation over time.
All original clinical research studies published in 3 major general clinical journals or high-impact-factor specialty journals in 1990-2003 and cited more than 1000 times in the literature were examined.
The results of highly cited articles were compared against subsequent studies of comparable or larger sample size and similar or better controlled designs. The same analysis was also performed comparatively for matched studies that were not so highly cited.
Of 49 highly cited original clinical research studies, 45 claimed that the intervention was effective. Of these, 7 (16%) were contradicted by subsequent studies, 7 others (16%) had found effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged. Five of 6 highly-cited nonrandomized studies had been contradicted or had found stronger effects vs 9 of 39 randomized controlled trials (P = .008). Among randomized trials, studies with contradicted or stronger effects were smaller (P = .009) than replicated or unchallenged studies although there was no statistically significant difference in their early or overall citation impact. Matched control studies did not have a significantly different share of refuted results than highly cited studies, but they included more studies with "negative" results.
Contradiction and initially stronger effects are not unusual in highly cited research of clinical interventions and their outcomes. The extent to which high citations may provoke contradictions and vice versa needs more study. Controversies are most common with highly cited nonrandomized studies, but even the most highly cited randomized trials may be challenged and refuted over time, especially small ones.
当关于干预措施有效性的临床研究结果随后被反驳时，争议和不确定性就会随之而来。当涉及到高影响力的研究时，争议最为突出。
了解被高度引用的研究被反驳或发现比其他类似研究更强效应的频率，并辨别随着时间推移，特定特征是否与这种反驳相关。
对1990年至2003年发表在3种主要综合临床期刊或高影响因子专业期刊上且在文献中被引用超过1000次的所有原创临床研究进行审查。
将被高度引用文章的结果与随后样本量相当或更大、设计相似或控制更好的研究结果进行比较。对未被如此高度引用的匹配研究也进行了类似分析。
在49项被高度引用的原创临床研究中，45项声称干预措施有效。其中，7项（16%）被随后的研究反驳，另外7项（16%）发现的效应比随后的研究更强，20项（44%）得到重复验证，11项（24%）在很大程度上未受到挑战。6项被高度引用的非随机研究中有5项被反驳或发现效应更强，而39项随机对照试验中有9项（P = 0.008）。在随机试验中，被反驳或效应更强的研究比得到重复验证或未受挑战的研究规模更小（P = 0.009），尽管它们在早期或总体引用影响力上没有统计学显著差异。匹配对照研究被反驳结果的比例与被高度引用的研究没有显著差异，但它们包含更多有“阴性”结果的研究。
在关于临床干预措施及其结果的高引用研究中，反驳和最初更强的效应并不罕见。高引用可能引发反驳的程度以及反之亦然的情况需要更多研究。争议在被高度引用的非随机研究中最为常见，但即使是被引用最多的随机试验随着时间推移也可能受到挑战和反驳，尤其是规模较小的试验。
