Department of Psychology.
Department of Management, Technology, and Economics.
J Exp Psychol Learn Mem Cogn. 2023 Apr;49(4):575-589. doi: 10.1037/xlm0001146. Epub 2022 Sep 8.
Relying on shared tasks and stimuli to conduct research can enhance the replicability of findings and allow a community of researchers to collect large data sets across multiple experiments. This approach is particularly relevant for experiments in spatial navigation, which often require the development of unfamiliar large-scale virtual environments to test participants. One challenge with shared platforms is that undetected technical errors, rather than being restricted to individual studies, become pervasive across many studies. Here, we discuss the discovery of a software bug in a virtual environment platform used to investigate individual differences in spatial navigation: Virtual Silcton. The bug, which was difficult to detect for several reasons, resulted in storing the absolute value of a direction in a pointing task rather than the signed direction and rendered the original sign of the direction unrecoverable. To assess the impact of the bug on published findings, we collected a new data set for comparison. Results revealed that although the bug caused suppression in pointing errors and had different effects across people (less accurate navigators had more suppression), the effect of the bug on published data is small, partially explaining the difficulty in detecting the bug. We also used the new data set to develop a tool that allows researchers who have previously used Virtual Silcton to evaluate the impact of the bug on their findings. We summarize the ways that shared open materials, shared data, and collaboration can pave the way for better science to prevent errors in the future. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
依靠共享任务和刺激物进行研究可以提高研究结果的可重复性，并允许研究人员在多个实验中收集大量数据集。这种方法对于空间导航实验特别有用，因为空间导航实验通常需要开发不熟悉的大规模虚拟环境来测试参与者。共享平台的一个挑战是，未被检测到的技术错误不仅限于个别研究，而是会在许多研究中普遍存在。在这里，我们讨论了在一个用于研究空间导航个体差异的虚拟环境平台 Virtual Silcton 中发现的一个软件错误。这个错误由于多种原因很难被发现，导致在指向任务中存储的是方向的绝对值而不是有符号的方向，并且使方向的原始符号无法恢复。为了评估该错误对已发表研究结果的影响，我们收集了一个新的数据集进行比较。结果表明，尽管该错误导致指向错误减少，并且对不同的人有不同的影响（导航能力较差的人减少更多），但该错误对已发表数据的影响很小，部分解释了难以检测到该错误的原因。我们还使用新数据集开发了一个工具，允许以前使用过 Virtual Silcton 的研究人员评估该错误对他们发现的影响。我们总结了共享开放材料、共享数据和合作可以为未来更好的科学铺平道路，以防止错误的方式。（PsycInfo 数据库记录（c）2023 APA，保留所有权利）。
