Liu Zheng, Hu Mengzhen, Zheng Yuanrui, Sui Jie, Chuan-Peng Hu
School of Psychology, Nanjing Normal University, Nanjing, China.
School of Humanities and Social Science, The Chinese University of Hong Kong-Shenzhen, Shenzhen, China.
Behav Res Methods. 2025 Jan 2;57(1):37. doi: 10.3758/s13428-024-02538-6.
The self-matching task (SMT) is widely used to investigate the cognitive mechanisms underlying the self-prioritization effect (SPE), wherein performance is enhanced for self-associated stimuli compared to other-associated ones. Although the SMT robustly elicits the SPE, there is a lack of data quantifying the reliability of this paradigm. This is problematic, given the prevalence of the reliability paradox in cognitive tasks: many well-established cognitive tasks demonstrate relatively low reliability when used to evaluate individual differences, despite exhibiting replicable effects at the group level. To fill this gap, this preregistered study investigated the reliability of SPE derived from the SMT using a multiverse approach, combining all possible indicators and baselines reported in the literature. We first examined the robustness of 24 SPE measures across 42 datasets (N = 2250) using a meta-analytical approach. We then calculated the split-half reliability (r) and intraclass correlation coefficient (ICC2) for each SPE measure. Our findings revealed a robust group-level SPE across datasets. However, when evaluating individual differences, SPE indices derived from reaction time (RT) and efficiency exhibited relatively higher, compared to other SPE indices, but still unsatisfied split-half reliability (approximately 0.5). The reliability across multiple time points, as assessed by ICC2, RT, and efficiency, demonstrated moderate levels of test-retest reliability (close to 0.5). These findings revealed the presence of a reliability paradox in the context of SMT-based SPE assessment. We discussed the implications of how to enhance individual-level reliability using this paradigm for future study design.
自我匹配任务（SMT）被广泛用于研究自我优先效应（SPE）背后的认知机制，在该效应中，与其他相关刺激相比，自我相关刺激的表现会得到增强。尽管SMT能有力地引发SPE，但缺乏量化该范式可靠性的数据。鉴于认知任务中可靠性悖论的普遍存在，这是个问题：许多成熟的认知任务在用于评估个体差异时显示出相对较低的可靠性，尽管在组水平上表现出可重复的效应。为填补这一空白，这项预先注册的研究采用多宇宙方法，结合文献中报道的所有可能指标和基线，研究了源自SMT的SPE的可靠性。我们首先使用元分析方法检查了42个数据集（N = 2250）中24种SPE测量方法的稳健性。然后，我们计算了每种SPE测量方法的分半信度（r）和组内相关系数（ICC2）。我们的研究结果显示，各数据集之间存在稳健的组水平SPE。然而，在评估个体差异时，与其他SPE指标相比，源自反应时间（RT）和效率的SPE指标相对较高，但分半信度仍不令人满意（约为0.5）。通过ICC2、RT和效率评估的多个时间点的可靠性显示出中等水平的重测信度（接近0.5）。这些发现揭示了在基于SMT的SPE评估背景下存在可靠性悖论。我们讨论了如何使用该范式提高个体水平可靠性对未来研究设计的影响。
