Rao V N Vimal, Bye Jeffrey K, Varma Sashank
Department of Educational Psychology, University of Minnesota, 56 E River Road Room 250, Minneapolis, MN, 55455, USA.
School of Interactive Computing and School of Psychology, Technology Square Research Building, Georgia Institute of Technology, 85 5Th St NW, Atlanta, GA, 30308, USA.
Cogn Res Princ Implic. 2024 May 3;9(1):27. doi: 10.1186/s41235-024-00553-x.
The .05 boundary within Null Hypothesis Statistical Testing (NHST) "has made a lot of people very angry and been widely regarded as a bad move" (to quote Douglas Adams). Here, we move past meta-scientific arguments and ask an empirical question: What is the psychological standing of the .05 boundary for statistical significance? We find that graduate students in the psychological sciences show a boundary effect when relating p-values across .05. We propose this psychological boundary is learned through statistical training in NHST and reading a scientific literature replete with "statistical significance". Consistent with this proposal, undergraduates do not show the same sensitivity to the .05 boundary. Additionally, the size of a graduate student's boundary effect is not associated with their explicit endorsement of questionable research practices. These findings suggest that training creates distortions in initial processing of p-values, but these might be dampened through scientific processes operating over longer timescales.
零假设统计检验（NHST）中的0.05界限“让很多人非常愤怒，并且被广泛认为是一个糟糕的举措”（引用道格拉斯·亚当斯的话）。在此，我们抛开元科学争论，提出一个实证问题：统计显著性的0.05界限在心理学上处于什么地位？我们发现，心理学专业的研究生在比较跨越0.05的p值时会表现出界限效应。我们认为这种心理界限是通过在NHST中的统计训练以及阅读充斥着“统计显著性”的科学文献习得的。与这一观点一致的是，本科生对0.05界限没有表现出同样的敏感性。此外，研究生的界限效应大小与他们对可疑研究行为的明确认可无关。这些发现表明，训练会在p值的初始处理过程中造成扭曲，但这些扭曲可能会通过在更长时间尺度上运行的科学过程得到缓解。
