Berkeley Initiative for Transparency in the Social Sciences (BITSS), University of California, Berkeley.
Department of Psychology, University of Toronto.
J Pers Soc Psychol. 2017 Aug;113(2):254-261. doi: 10.1037/pspi0000106.
Finkel, Eastwick, and Reis (2016; FER2016) argued the post-2011 methodological reform movement has focused narrowly on replicability, neglecting other essential goals of research. We agree multiple scientific goals are essential, but argue, however, a more fine-grained language, conceptualization, and approach to replication is needed to accomplish these goals. Replication is the general empirical mechanism for testing and falsifying theory. Sufficiently methodologically similar replications, also known as direct replications, test the basic existence of phenomena and ensure cumulative progress is possible a priori. In contrast, increasingly methodologically dissimilar replications, also known as conceptual replications, test the relevance of auxiliary hypotheses (e.g., manipulation and measurement issues, contextual factors) required to productively investigate validity and generalizability. Without prioritizing replicability, a field is not empirically falsifiable. We also disagree with FER2016's position that "bigger samples are generally better, but . . . that very large samples could have the downside of commandeering resources that would have been better invested in other studies" (abstract). We identify problematic assumptions involved in FER2016's modifications of our original research-economic model, and present an improved model that quantifies when (and whether) it is reasonable to worry that increasing statistical power will engender potential trade-offs. Sufficiently powering studies (i.e., >80%) maximizes both research efficiency and confidence in the literature (research quality). Given that we are in agreement with FER2016 on all key open science points, we are eager to start seeing the accelerated rate of cumulative knowledge development of social psychological phenomena such a sufficiently transparent, powered, and falsifiable approach will generate. (PsycINFO Database Record
芬克尔、伊斯威克和里斯（2016 年；FER2016）认为，2011 年后的方法论改革运动仅仅狭隘地关注于可重复性，忽视了研究的其他重要目标。我们同意多个科学目标是必要的，但我们认为，需要更精细的语言、概念化和复制方法来实现这些目标。复制是检验和证伪理论的一般经验机制。足够方法学上相似的复制，也称为直接复制，检验现象的基本存在，并确保累积进展在事先是可能的。相比之下，越来越多方法学上不同的复制，也称为概念复制，检验辅助假设（例如，操纵和测量问题、背景因素）的相关性，这些假设是对有效性和可推广性进行富有成效的研究所必需的。如果不优先考虑可重复性，那么一个领域就无法进行经验验证。我们也不同意 FER2016 的立场，即“更大的样本通常更好，但……过大的样本可能会有征用资源的缺点，这些资源本来可以更好地投资于其他研究”（摘要）。我们确定了 FER2016 对我们原始研究经济模型的修改所涉及的有问题的假设，并提出了一个改进的模型，该模型量化了何时（以及是否）有合理的理由担心增加统计能力会产生潜在的权衡。充分赋能研究（即>80%）可以最大化研究效率和对文献（研究质量）的信心。鉴于我们在所有关键的开放科学观点上都与 FER2016 达成一致，我们渴望看到这种充分透明、赋能和可证伪的方法将加速社会心理现象的累积知识发展。
