Bipp Tanja, Wee Serena, Walczok Marvin, Hansal Laura
Department of Psychology, Heidelberg University, 69117 Heidelberg, Germany.
School of Psychological Science, The University of Western Australia, Crawley, WA 6009, Australia.
J Intell. 2024 Dec 16;12(12):129. doi: 10.3390/jintelligence12120129.
Technological advances have introduced new methods for assessing psychological constructs, moving beyond traditional paper-pencil tests. Game-related assessments (GRAs) offer several advantages for research and practice, though questions about their construct validity persist. This meta-analysis investigated the relationship between indicators derived from computer-based games and traditional cognitive ability measures, examining whether measurement scope (single vs. multiple indicators) or measurement medium of cognitive ability (computer-based vs. paper-pencil) influences this relationship. We identified 52 eligible samples stemming from 44 papers, including data from over 6100 adult participants. The results from three-stage mixed-effects meta-analyses showed an overall observed correlation of = 0.30 ( < 0.001; corrected = 0.45) between GRA indicators and traditional cognitive ability measures with substantial heterogeneity in effect sizes. Stronger relationships were found when cognitive ability was measured by multiple indicators, but no differences emerged based on the measurement medium of cognitive ability. Furthermore, GRAs intended to assess cognitive ability did not show stronger relationships with traditional measures of cognitive ability than GRAs not specifically used to measure cognitive ability. Overall, our findings suggest that GRAs are related to traditional cognitive ability measures. However, the overall effect size raises questions about whether GRAs and traditional measures capture the same aspects of cognitive ability or if GRAs also measure other constructs beyond cognitive ability.
技术进步引入了评估心理结构的新方法，超越了传统的纸笔测试。与游戏相关的评估（GRA）在研究和实践中具有诸多优势，尽管其结构效度问题依然存在。这项荟萃分析调查了基于计算机游戏得出的指标与传统认知能力测量方法之间的关系，考察了测量范围（单一指标与多个指标）或认知能力的测量媒介（基于计算机的与纸笔的）是否会影响这种关系。我们从44篇论文中确定了52个符合条件的样本，包括来自6100多名成年参与者的数据。三阶段混合效应荟萃分析的结果显示，GRA指标与传统认知能力测量方法之间的总体观察相关性为r = 0.30（p < 0.001；校正r = 0.45），效应大小存在显著异质性。当通过多个指标测量认知能力时，发现了更强的相关性，但基于认知能力的测量媒介未出现差异。此外，旨在评估认知能力的GRA与传统认知能力测量方法之间的关系，并不比未专门用于测量认知能力的GRA更强。总体而言，我们的研究结果表明GRA与传统认知能力测量方法相关。然而，总体效应大小引发了关于GRA和传统测量方法是否捕捉到认知能力的相同方面，或者GRA是否还测量了认知能力之外的其他结构的问题。
