Greenland S, O'Rourke K
Department of Epidemiology, UCLA School of Public Health and Department of Statistics, UCLA College of Letters and Science, 22333 Swenson Drive, Topanga, CA 90290, USA.
Biostatistics. 2001 Dec;2(4):463-71. doi: 10.1093/biostatistics/2.4.463.
Results from better quality studies should in some sense be more valid or more accurate than results from other studies, and as a consequence should tend to be distributed differently from results of other studies. To date, however, quality scores have been poor predictors of study results. We discuss possible reasons and remedies for this problem. It appears that 'quality' (whatever leads to more valid results) is of fairly high dimension and possibly non-additive and nonlinear, and that quality dimensions are highly application-specific and hard to measure from published information. Unfortunately, quality scores are often used to contrast, model, or modify meta-analysis results without regard to the aforementioned problems, as when used to directly modify weights or contributions of individual studies in an ad hoc manner. Even if quality would be captured in one dimension, use of quality scores in summarization weights would produce biased estimates of effect. Only if this bias were more than offset by variance reduction would such use be justified. From this perspective, quality weighting should be evaluated against formal bias-variance trade-off methods such as hierarchical (random-coefficient) meta-regression. Because it is unlikely that a low-dimensional appraisal will ever be adequate (especially over different applications), we argue that response-surface estimation based on quality items is preferable to quality weighting. Quality scores may be useful in the second stage of a hierarchical response-surface model, but only if the scores are reconstructed to maximize their correlation with bias.
在某种意义上，高质量研究的结果应该比其他研究的结果更有效或更准确，因此其分布应该与其他研究的结果有所不同。然而，迄今为止，质量评分一直是研究结果的不良预测指标。我们讨论了这个问题的可能原因和解决方法。看来，“质量”（无论什么导致更有效的结果）具有相当高的维度，可能是非加性和非线性的，而且质量维度高度特定于应用，很难从已发表的信息中衡量。不幸的是，质量评分经常被用来对比、建模或修改荟萃分析结果，而不考虑上述问题，例如在临时直接修改个别研究的权重或贡献时。即使质量可以在一个维度上被捕捉到，在汇总权重中使用质量评分也会产生效应的偏差估计。只有当这种偏差被方差减少所抵消时，这种使用才是合理的。从这个角度来看，质量加权应该根据正式的偏差-方差权衡方法进行评估，例如分层（随机系数）元回归。由于低维度评估不太可能足够（特别是在不同的应用中），我们认为基于质量项目的响应面估计比质量加权更可取。质量评分在分层响应面模型的第二阶段可能有用，但前提是评分要重新构建以最大化它们与偏差的相关性。
