Lakens Daniël, Etz Alexander J
Human Technology Interaction Group, Eindhoven University of Technology, Eindhoven, the Netherlands.
Department of Cognitive Sciences, University of California, Irvine, CA, USA.
Soc Psychol Personal Sci. 2017 Nov;8(8):875-881. doi: 10.1177/1948550617693058. Epub 2017 May 5.
Psychology journals rarely publish nonsignificant results. At the same time, it is often very unlikely (or "too good to be true") that a set of studies yields exclusively significant results. Here, we use likelihood ratios to explain when sets of studies that contain a mix of significant and nonsignificant results are likely to be true or "too true to be bad." As we show, mixed results are not only likely to be observed in lines of research but also, when observed, often provide evidence for the alternative hypothesis, given reasonable levels of statistical power and an adequately controlled low Type 1 error rate. Researchers should feel comfortable submitting such lines of research with an internal meta-analysis for publication. A better understanding of probabilities, accompanied by more realistic expectations of what real sets of studies look like, might be an important step in mitigating publication bias in the scientific literature.
心理学杂志很少发表无显著结果的研究。与此同时，一组研究仅产生显著结果的情况通常极不可能（或“好得令人难以置信”）。在此，我们使用似然比来解释何时包含显著结果和无显著结果混合的研究集可能是真实的，或者“好到不像是假的”。正如我们所表明的，混合结果不仅很可能在一系列研究中被观察到，而且当被观察到时，在合理的统计效力水平和充分控制的低一类错误率的情况下，通常会为备择假设提供证据。研究人员应该放心地提交带有内部元分析的此类研究系列以供发表。更好地理解概率，并对实际研究集的样子有更现实的期望，可能是减轻科学文献中发表偏倚的重要一步。
