Olsen Jerome, Mosen Johanna, Voracek Martin, Kirchler Erich
Faculty of Psychology, Department of Applied Psychology: Work, Education and Economy, University of Vienna, Universitaetsstrasse 7, 1010 Vienna, Austria.
Faculty of Psychology, Department of Basic Psychological Research and Research Methods, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria.
R Soc Open Sci. 2019 Dec 18;6(12):190738. doi: 10.1098/rsos.190738. eCollection 2019 Dec.
The replicability of research findings has recently been disputed across multiple scientific disciplines. In constructive reaction, the research culture in psychology is facing fundamental changes, but investigations of research practices that led to these improvements have almost exclusively focused on academic researchers. By contrast, we investigated the statistical reporting quality and selected indicators of questionable research practices (QRPs) in psychology students' master's theses. In a total of 250 theses, we investigated utilization and magnitude of standardized effect sizes, along with statistical power, the consistency and completeness of reported results, and possible indications of -hacking and further testing. Effect sizes were reported for 36% of focal tests (median = 0.19), and only a single formal power analysis was reported for sample size determination (median observed power 1 - = 0.67). revealed inconsistent -values in 18% of cases, while 2% led to decision errors. There were no clear indications of -hacking or further testing. We discuss our findings in the light of promoting open science standards in teaching and student supervision.
研究结果的可重复性最近在多个科学学科中受到质疑。作为建设性的回应,心理学的研究文化正面临着根本性的变革,但对导致这些改进的研究实践的调查几乎完全集中在学术研究人员身上。相比之下,我们调查了心理学专业学生硕士论文中的统计报告质量以及可疑研究行为(QRPs)的选定指标。在总共250篇论文中,我们研究了标准化效应量的使用情况和大小,以及统计功效、报告结果的一致性和完整性,以及可能存在的“p值操纵”和进一步检验的迹象。在36%的重点检验中报告了效应量(中位数 = 0.19),并且仅报告了一项用于确定样本量的正式功效分析(观察到的中位数功效1 - = 0.67)。在18%的案例中发现了不一致的p值,而2%导致了决策错误。没有明显的“p值操纵”或进一步检验的迹象。我们根据在教学和学生指导中推广开放科学标准来讨论我们的研究结果。