Suppr超能文献

科学不是一个信号检测问题。

Science is not a signal detection problem.

机构信息

Department of Psychology, University of California San Diego, La Jolla, CA 92093

Department of Psychology, University of California San Diego, La Jolla, CA 92093.

出版信息

Proc Natl Acad Sci U S A. 2020 Mar 17;117(11):5559-5567. doi: 10.1073/pnas.1914237117. Epub 2020 Mar 3.

Abstract

The perceived replication crisis and the reforms designed to address it are grounded in the notion that science is a binary signal detection problem. However, contrary to null hypothesis significance testing (NHST) logic, the magnitude of the underlying effect size for a given experiment is best conceptualized as a random draw from a continuous distribution, not as a random draw from a dichotomous distribution (null vs. alternative). Moreover, because continuously distributed effects selected using a < 0.05 filter must be inflated, the fact that they are smaller when replicated (reflecting regression to the mean) is no reason to sound the alarm. Considered from this perspective, recent replication efforts suggest that most published < 0.05 scientific findings are "true" (i.e., in the correct direction), with observed effect sizes that are inflated to varying degrees. We propose that original science is a screening process, one that adopts NHST logic as a useful fiction for selecting true effects that are potentially large enough to be of interest to other scientists. Unlike original science, replication science seeks to precisely measure the underlying effect size associated with an experimental protocol via large- direct replication, without regard for statistical significance. Registered reports are well suited to (often resource-intensive) direct replications, which should focus on influential findings and be published regardless of outcome. Conceptual replications play an important but separate role in validating theories. However, because they are part of NHST-based original science, conceptual replications cannot serve as the field's self-correction mechanism. Only direct replications can do that.

摘要

被感知的复制危机以及为解决这一危机而进行的改革,其基础是这样一种观念,即科学是一个二元信号检测问题。然而,与零假设显著性检验(NHST)逻辑相反,给定实验的潜在效应大小最好被概念化为来自连续分布的随机抽取,而不是来自二分分布(零假设与备择假设)的随机抽取。此外,由于使用 < 0.05 滤波器选择的连续分布效应必然会被夸大,因此当它们被复制时(反映出向均值回归)较小,这并不是发出警报的理由。从这个角度来看,最近的复制努力表明,大多数已发表的 < 0.05 的科学发现都是“真实的”(即,朝着正确的方向),观察到的效应大小在不同程度上被夸大了。我们提出,原始科学是一个筛选过程,它采用 NHST 逻辑作为一种有用的虚构,以选择潜在足够大、可能引起其他科学家兴趣的真实效应。与原始科学不同,复制科学旨在通过大型直接复制,精确测量与实验方案相关的潜在效应大小,而不考虑统计显著性。注册报告非常适合(通常资源密集型)直接复制,直接复制应侧重于有影响力的发现,无论结果如何都应发表。概念复制在验证理论方面发挥着重要但独立的作用。然而,由于它们是基于 NHST 的原始科学的一部分,概念复制不能作为该领域的自我修正机制。只有直接复制才能做到这一点。

相似文献

1
Science is not a signal detection problem.科学不是一个信号检测问题。
Proc Natl Acad Sci U S A. 2020 Mar 17;117(11):5559-5567. doi: 10.1073/pnas.1914237117. Epub 2020 Mar 3.
4
Bayesian inference of population prevalence.贝叶斯推断种群流行率。
Elife. 2021 Oct 6;10:e62461. doi: 10.7554/eLife.62461.

引用本文的文献

本文引用的文献

1
A manifesto for reproducible science.可重复科学宣言。
Nat Hum Behav. 2017 Jan 10;1(1):0021. doi: 10.1038/s41562-016-0021.
3
The impact of sleep on eyewitness identifications.睡眠对目击证人辨认的影响。
R Soc Open Sci. 2019 Dec 4;6(12):170501. doi: 10.1098/rsos.170501. eCollection 2019 Dec.
6
Replicator degrees of freedom allow publication of misleading failures to replicate.复制者自由度允许发表具有误导性的未能复制的结果。
Proc Natl Acad Sci U S A. 2019 Dec 17;116(51):25535-25545. doi: 10.1073/pnas.1910951116. Epub 2019 Nov 25.
7
What's next for Registered Reports?注册报告的下一步是什么?
Nature. 2019 Sep;573(7773):187-189. doi: 10.1038/d41586-019-02674-6.
8
The forgotten history of signal detection theory.信号检测理论的遗忘历史。
J Exp Psychol Learn Mem Cogn. 2020 Feb;46(2):201-233. doi: 10.1037/xlm0000732. Epub 2019 Jun 27.
10
Rein in the four horsemen of irreproducibility.控制住不可重复性的四大因素。
Nature. 2019 Apr;568(7753):435. doi: 10.1038/d41586-019-01307-2.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验