Interdisciplinary Centre for Ethics, Jagiellonian University, Kraków, Poland; Institute of Philosophy, Jagiellonian University, Grodzka 52, Kraków, Poland.
Stud Hist Philos Sci. 2022 Feb;91:159-167. doi: 10.1016/j.shpsa.2021.11.007. Epub 2021 Dec 15.
Literature-based meta-analysis is a standard technique applied to pool results of individual studies used in medicine and social sciences. It has been criticized for being too malleable to constrain results, averaging incomparable values, lacking a measure of evidence's strength, and problems with a systematic bias of individual studies. We argue against using literature-based meta-analysis of RCTs to assess treatment efficacy and show that therapeutic decisions based on meta-analytic average are not optimal given the full scope of existing evidence. The argument proceeds with discussing examples and analyzing the properties of some standard meta-analytic techniques. First, we demonstrate that meta-analysis can lead to reporting statistically significant results despite the treatment's limited efficacy. Second, we show that meta-analytic confidence intervals are too narrow compared to the variability of treatment outcomes reported by individual studies. Third, we argue that literature-based meta-analysis is not a reliable measurement instrument. Finally, we show that meta-analysis averages out the differences among studies and leads to a loss of information. Despite these problems, literature-based meta-analysis is useful for the assessment of harms. We support two alternative approaches to evidence amalgamation: meta-analysis of individual patient data (IPD) and qualitative review employing mechanistic evidence.
基于文献的荟萃分析是一种应用于医学和社会科学中个体研究结果的标准技术。它被批评为过于灵活，无法限制结果，平均不可比的值，缺乏证据强度的衡量标准，以及个体研究的系统偏差问题。我们反对使用基于文献的 RCT 荟萃分析来评估治疗效果，并表明基于荟萃分析平均值的治疗决策在考虑到现有证据的全部范围时并不是最优的。该论点通过讨论示例和分析一些标准荟萃分析技术的特性来进行。首先，我们证明尽管治疗效果有限，荟萃分析仍可能导致报告具有统计学意义的结果。其次，我们表明荟萃分析置信区间与个体研究报告的治疗结果变异性相比过于狭窄。第三，我们认为基于文献的荟萃分析不是可靠的测量工具。最后，我们表明荟萃分析掩盖了研究之间的差异，并导致信息丢失。尽管存在这些问题，基于文献的荟萃分析对于评估危害仍然是有用的。我们支持两种证据综合的替代方法：个体患者数据（IPD）荟萃分析和基于机制证据的定性综述。
