O'Laughlin Kristine D, Cheng Britte Haugan, Volponi Joshua J, Lorentz John David A, Obregon Sophia A, Younger Jessica Wise, Gazzaley Adam, Uncapher Melina R, Anguera Joaquin A
Neuroscape, Department of Neurology, University of California San Francisco, San Francisco, CA, United States.
J Med Internet Res. 2025 Apr 21;27:e60041. doi: 10.2196/60041.
Executive functions (EFs) predict positive life outcomes and educational attainment. Consequently, it is imperative that our measures of EF constructs are both reliable and valid, with advantages for research tools that offer efficiency and remote capabilities.
The objective of this study was to evaluate reliability and validity evidence for a mobile, adaptive measure of EFs called Adaptive Cognitive Evaluation-Explorer (ACE-X).
We collected data from 2 cohorts of participants: a test-retest sample (N=246, age: mean 35.75, SD 11.74 y) to assess consistency of ACE-X task performance over repeated administrations and a validation sample involving child or adolescent (5436/6052, 89.82%; age: mean 12.78, SD 1.60 years) and adult participants (484/6052, 8%; age: mean 38.11, SD 14.96 years) to examine consistency of metrics, internal structures, and invariance of ACE-X task performance. A subset of participants (132/6052, 2.18%; age: mean 37.04, SD 13.23 years) also completed a similar set of cognitive tasks using the Inquisit platform to assess the concurrent validity of ACE-X.
Intraclass correlation coefficients revealed most ACE-X tasks were moderately to very reliable across repeated assessments (intraclass correlation coefficient=0.45-0.79; P<.001). Moreover, in comparisons of internal structures of ACE-X task performance, model fit indices suggested that a network model based on partial correlations was the best fit to the data (χ=40.13; P=.06; comparative fit index=0.99; root mean square error of approximation=0.03, 90% CI 0.00-0.05; Bayesian information criterion=5075.87; Akaike information criterion=4917.71) and that network edge weights are invariant across both younger and older adult participants. A Spinglass community detection algorithm suggested ACE-X task performance can be described by 3 communities (selected in 85% of replications): set reconfiguration, attentional control, and interference resolution. On the other hand, Pearson correlation coefficients indicated mixed results for the concurrent validity comparisons between ACE-X and Inquisit (r=-.05-.62, P<.001-.76).
These findings suggest that ACE-X is a reliable and valid research tool for understanding EFs and their relations to outcome measures.
执行功能（EFs）可预测积极的生活结果和教育成就。因此，至关重要的是，我们对EF结构的测量既可靠又有效，这对于具有高效性和远程功能的研究工具具有优势。
本研究的目的是评估一种名为自适应认知评估探索者（ACE-X）的移动自适应EF测量方法的可靠性和有效性证据。
我们从两组参与者中收集数据：一个重测样本（N = 246，年龄：平均35.75岁，标准差11.74岁），以评估ACE-X任务表现在重复施测中的一致性；一个验证样本，包括儿童或青少年（5436/6052，89.82%；年龄：平均12.78岁，标准差1.60岁）和成人参与者（484/6052，8%；年龄：平均38.11岁，标准差14.96岁），以检查ACE-X任务表现的指标一致性、内部结构和不变性。一部分参与者（132/6052，2.18%；年龄：平均37.04岁，标准差13.23岁）还使用Inquisit平台完成了一组类似的认知任务，以评估ACE-X的同时效度。
组内相关系数显示，大多数ACE-X任务在重复评估中具有中等至非常高的可靠性（组内相关系数 = 0.45 - 0.79；P <.001）。此外，在ACE-X任务表现的内部结构比较中，模型拟合指数表明基于偏相关的网络模型最适合数据（χ = 40.13；P =.06；比较拟合指数 = 0.99；近似均方根误差 = 0.03，90%置信区间0.00 - 0.05；贝叶斯信息准则 = 5075.87；赤池信息准则 = 4917.71），并且网络边权重在年轻和年长的成年参与者中是不变的。一种Spinglass社区检测算法表明，ACE-X任务表现可以由3个社区来描述（在85%的重复中被选中）：集合重新配置、注意力控制和干扰解决。另一方面，Pearson相关系数表明ACE-X和Inquisit之间的同时效度比较结果不一（r = -0.05 - 0.62，P <.001 -.76）。
这些发现表明，ACE-X是一种可靠且有效的研究工具，可用于理解EFs及其与结果测量之间的关系。
