Faculty of Psychology, Technische Universität Dresden, Dresden, Germany.
Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden.
Behav Res Methods. 2018 Aug;50(4):1345-1358. doi: 10.3758/s13428-018-1036-5.
The internet-based assessment of response time (RT) and error rate (ERR) has recently become a well-validated alternative to traditional laboratory-based assessment, because methodological research has provided evidence for negligible setting- and setup-related differences in RT and ERR measures of central tendency. However, corresponding data on potential differences in the variability of such performance measures are still lacking, to date. Hence, the aim of this study was to conduct internet-based mental chronometry in both poorly standardized domestic and highly standardized laboratory environments and to compare the variabilities of the corresponding performance measures. Using the Millisecond Inquisit4Web software, 127 men and women completed three different RT-based cognitive paradigms (i.e., go/no-go, two-back, and number-letter). Each participant completed all paradigms in two environments (i.e., at home and in the laboratory), with a time lag of seven days and in a counterbalanced order. Mixed-effects modeling was employed to estimate the between-setting variability across a comprehensive set of performance measures, including conventional measures of central tendency (i.e., mean RT and ERR) and further measures characterizing the joint distribution of RT/ERR. The latter measures were estimated using the diffusion model. The results suggested negligible differences between the domestic and laboratory settings. Thus, this study provides novel evidence suggesting that the statistical power of internet-based mental chronometry is commonly not compromised by increased environmental variance. The within- and between-session reliabilities were in a satisfactory range-that is, comparable to performance measures collected offline in laboratory settings. In consequence, our results support the broad applicability, robustness, and cost efficiency of mental chronometry assessment using the internet.
基于互联网的反应时间 (RT) 和错误率 (ERR) 评估最近已成为传统基于实验室的评估的有效替代方法，因为方法学研究为 RT 和 ERR 集中趋势测量的设置和设置相关差异可忽略不计提供了证据。然而，迄今为止，关于此类性能测量的可变性存在潜在差异的相应数据仍然缺乏。因此，本研究的目的是在标准化程度较差的国内和高度标准化的实验室环境中进行基于互联网的心理计时，并比较相应性能测量的可变性。使用毫秒 Inquisit4Web 软件，127 名男性和女性完成了三种不同的基于 RT 的认知范式（即 Go/No-Go、2-Back 和数字字母）。每个参与者在两个环境（即家庭和实验室）中完成所有范式，时间间隔为七天，并且以平衡的顺序进行。混合效应模型用于估计在一系列全面的性能测量中跨设置的可变性，包括集中趋势的常规测量（即平均 RT 和 ERR）以及进一步描述 RT/ERR 联合分布的测量。后者的测量使用扩散模型进行估计。结果表明家庭环境和实验室环境之间几乎没有差异。因此，本研究提供了新的证据，表明基于互联网的心理计时的统计能力通常不会因环境方差的增加而受到影响。在会话内和会话间的可靠性均处于令人满意的范围内，即与实验室环境中离线收集的性能测量相当。因此，我们的结果支持使用互联网进行心理计时评估的广泛适用性、稳健性和成本效益。
