Stunt Jonah, van Grootel Leonie, Bouter Lex, Trafimow David, Hoekstra Trynke, de Boer Michiel
Department of Health Sciences, Section of Methodology and Applied Statistics, Vrije Universiteit, Amsterdam, The Netherlands.
Department of Radiation Oncology, Erasmus Medical Center, Rotterdam, The Netherlands.
PLoS One. 2021 Oct 15;16(10):e0258330. doi: 10.1371/journal.pone.0258330. eCollection 2021.
Null Hypothesis Significance Testing (NHST) is the most familiar statistical procedure for making inferences about population effects. Important problems associated with this method have been addressed and various alternatives that overcome these problems have been developed. Despite its many well-documented drawbacks, NHST remains the prevailing method for drawing conclusions from data. Reasons for this have been insufficiently investigated. Therefore, the aim of our study was to explore the perceived barriers and facilitators related to the use of NHST and alternative statistical procedures among relevant stakeholders in the scientific system.
Individual semi-structured interviews and focus groups were conducted with junior and senior researchers, lecturers in statistics, editors of scientific journals and program leaders of funding agencies. During the focus groups, important themes that emerged from the interviews were discussed. Data analysis was performed using the constant comparison method, allowing emerging (sub)themes to be fully explored. A theory substantiating the prevailing use of NHST was developed based on the main themes and subthemes we identified.
Twenty-nine interviews and six focus groups were conducted. Several interrelated facilitators and barriers associated with the use of NHST and alternative statistical procedures were identified. These factors were subsumed under three main themes: the scientific climate, scientific duty, and reactivity. As a result of the factors, most participants feel dependent in their actions upon others, have become reactive, and await action and initiatives from others. This may explain why NHST is still the standard and ubiquitously used by almost everyone involved.
Our findings demonstrate how perceived barriers to shift away from NHST set a high threshold for actual behavioral change and create a circle of interdependency between stakeholders. By taking small steps it should be possible to decrease the scientific community's strong dependence on NHST and p-values.
零假设显著性检验（NHST）是用于推断总体效应的最常见统计程序。与该方法相关的重要问题已得到解决，并且已经开发出各种克服这些问题的替代方法。尽管NHST有许多已被充分记录的缺点，但它仍然是从数据中得出结论的主要方法。对此的原因尚未得到充分研究。因此，我们研究的目的是探讨科学系统中相关利益者在使用NHST和替代统计程序方面所感知到的障碍和促进因素。
对初级和高级研究人员、统计学讲师、科学期刊编辑以及资助机构的项目负责人进行了个人半结构化访谈和焦点小组讨论。在焦点小组讨论中，对访谈中出现的重要主题进行了讨论。使用持续比较法进行数据分析，以便充分探索新出现的（子）主题。基于我们确定的主要主题和子主题，构建了一个支持NHST普遍使用的理论。
进行了29次访谈和6次焦点小组讨论。确定了与使用NHST和替代统计程序相关的几个相互关联的促进因素和障碍。这些因素归纳为三个主要主题：科学氛围、科学职责和反应性。由于这些因素，大多数参与者在行动上感到依赖他人，变得具有反应性，并等待他人的行动和倡议。这或许可以解释为什么NHST仍然是几乎所有相关人员使用的标准且普遍的方法。
我们的研究结果表明，人们所感知到的背离NHST的障碍如何为实际行为改变设定了高门槛，并在利益相关者之间形成了相互依赖的循环。通过采取小步骤，应该有可能降低科学界对NHST和p值的强烈依赖。
