Rouder Jeffrey N, Morey Richard D, Verhagen Josine, Province Jordan M, Wagenmakers Eric-Jan
University of Missouri.
University of Cardiff.
Top Cogn Sci. 2016 Jul;8(3):520-47. doi: 10.1111/tops.12214.
The field of psychology, including cognitive science, is vexed by a crisis of confidence. Although the causes and solutions are varied, we focus here on a common logical problem in inference. The default mode of inference is significance testing, which has a free lunch property where researchers need not make detailed assumptions about the alternative to test the null hypothesis. We present the argument that there is no free lunch; that is, valid testing requires that researchers test the null against a well-specified alternative. We show how this requirement follows from the basic tenets of conventional and Bayesian probability. Moreover, we show in both the conventional and Bayesian framework that not specifying the alternative may lead to rejections of the null hypothesis with scant evidence. We review both frequentist and Bayesian approaches to specifying alternatives, and we show how such specifications improve inference. The field of cognitive science will benefit because consideration of reasonable alternatives will undoubtedly sharpen the intellectual underpinnings of research.
包括认知科学在内的心理学领域正饱受信心危机的困扰。尽管原因和解决方案多种多样，但我们在此关注推理中一个常见的逻辑问题。默认的推理模式是显著性检验，它具有“免费午餐”属性，即研究人员无需对备择假设做出详细假设就能检验原假设。我们提出的观点是不存在“免费午餐”；也就是说，有效的检验要求研究人员针对一个明确指定的备择假设来检验原假设。我们展示了这一要求是如何从传统概率和贝叶斯概率的基本原理推导出来的。此外，我们在传统框架和贝叶斯框架中都表明，不明确指定备择假设可能会导致在证据不足的情况下拒绝原假设。我们回顾了指定备择假设的频率主义方法和贝叶斯方法，并展示了这些指定如何改进推理。认知科学领域将从中受益，因为考虑合理的备择假设无疑会加强研究的知识基础。
