Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health (CAMH), Toronto, ON, Canada.
Department of Psychiatry, University of Toronto, Toronto, ON, Canada.
PLoS One. 2024 Nov 18;19(11):e0312255. doi: 10.1371/journal.pone.0312255. eCollection 2024.
The development of computational models for studying mental disorders is on the rise. However, their psychometric properties remain understudied, posing a risk of undermining their use in empirical research and clinical translation. Here we investigated test-retest reliability (with a 2-week interval) of a computational assay probing advice-taking under volatility with a Hierarchical Gaussian Filter (HGF) model. In a sample of 39 healthy participants, we found the computational measures to have largely poor reliability (intra-class correlation coefficient or ICC < 0.5), on par with the behavioral measures of task performance. Further analysis revealed that reliability was substantially impacted by intrinsic measurement noise (indicated by parameter recovery analysis) and to a smaller extent by practice effects. However, a large portion of within-subject variance remained unexplained and may be attributable to state-like fluctuations. Despite the poor test-retest reliability, we found the assay to have face validity at the group level. Overall, our work highlights that the different sources of variance affecting test-retest reliability need to be studied in greater detail. A better understanding of these sources would facilitate the design of more psychometrically sound assays, which would improve the quality of future research and increase the probability of clinical translation.
研究精神障碍的计算模型的发展正在兴起。然而，它们的心理测量特性仍未得到充分研究，这有可能破坏它们在实证研究和临床转化中的应用。在这里，我们研究了使用分层高斯滤波器（HGF）模型探测在波动性下采取建议的计算测定的重测信度（间隔 2 周）。在 39 名健康参与者的样本中，我们发现计算测量值的可靠性大多很差（组内相关系数或 ICC <0.5），与任务表现的行为测量值相当。进一步的分析表明，可靠性受到内在测量噪声（由参数恢复分析表示）的显著影响，受练习效应的影响较小。然而，大部分个体内差异仍然无法解释，可能归因于状态样波动。尽管重测信度较差，但我们发现该测定在组水平上具有表面效度。总体而言，我们的工作强调需要更详细地研究影响重测信度的不同方差源。更好地理解这些来源将有助于设计更具心理测量学意义的测定，从而提高未来研究的质量并增加临床转化的可能性。
