Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Margarete-Höppel-Platz 1, 97080, Würzburg, Germany.
Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstraße 1, 04103, Leipzig, Germany.
Behav Res Methods. 2022 Dec;54(6):2993-3014. doi: 10.3758/s13428-021-01739-7. Epub 2022 Feb 15.
Task-based measures that capture neurocognitive processes can help bridge the gap between brain and behavior. To transfer tasks to clinical application, reliability is a crucial benchmark because it imposes an upper bound to potential correlations with other variables (e.g., symptom or brain data). However, the reliability of many task readouts is low. In this study, we scrutinized the retest reliability of a probabilistic reversal learning task (PRLT) that is frequently used to characterize cognitive flexibility in psychiatric populations. We analyzed data from N = 40 healthy subjects, who completed the PRLT twice. We focused on how individual metrics are derived, i.e., whether data were partially pooled across participants and whether priors were used to inform estimates. We compared the reliability of the resulting indices across sessions, as well as the internal consistency of a selection of indices. We found good to excellent reliability for behavioral indices as derived from mixed-effects models that included data from both sessions. The internal consistency was good to excellent. For indices derived from computational modeling, we found excellent reliability when using hierarchical estimation with empirical priors and including data from both sessions. Our results indicate that the PRLT is well equipped to measure individual differences in cognitive flexibility in reinforcement learning. However, this depends heavily on hierarchical modeling of the longitudinal data (whether sessions are modeled separately or jointly), on estimation methods, and on the combination of parameters included in computational models. We discuss implications for the applicability of PRLT indices in psychiatric research and as diagnostic tools.
基于任务的测量方法可以捕捉神经认知过程，有助于弥合大脑和行为之间的差距。为了将任务转化为临床应用，可靠性是一个至关重要的基准，因为它对与其他变量（例如症状或大脑数据）的潜在相关性施加了上限。然而，许多任务读数的可靠性较低。在这项研究中，我们仔细研究了概率反转学习任务（PRLT）的重测可靠性，该任务常用于描述精神科人群的认知灵活性。我们分析了 N = 40 名健康受试者的两次 PRLT 数据。我们专注于个体指标是如何得出的，即数据是否在参与者之间部分汇总以及是否使用先验信息来告知估计值。我们比较了两次测试中结果指标的可靠性，以及一系列指标的内部一致性。我们发现，从包含两个测试的混合效应模型中得出的行为指标具有良好到极好的可靠性。内部一致性也很好。对于从计算模型中得出的指标，当使用包含经验先验的分层估计并包含两个测试的数据时，我们发现其可靠性很高。我们的研究结果表明，PRLT 非常适合测量强化学习中认知灵活性的个体差异。然而，这在很大程度上取决于纵向数据的分层建模（是否分别或共同对测试进行建模）、估计方法以及包含在计算模型中的参数组合。我们讨论了 PRLT 指标在精神科研究和作为诊断工具中的适用性的影响。
