Department of Psychology, University of Windsor.
Department of Physical Medicine and Rehabilitation, Harvard Medical School.
Psychol Assess. 2021 Jan;33(1):90-96. doi: 10.1037/pas0000958. Epub 2020 Oct 29.
To assess noncredible performance on the NIH Toolbox Cognitive Battery (NIHTB-CB), we developed embedded validity indicators (EVIs). Data were collected from 98 adults (54.1% female) as part of a prospective multicenter cross-sectional study at 4 mild traumatic brain injury (mTBI) specialty clinics. Traditional EVIs and novel item-based EVIs were developed for the NIHTB-CB using the Medical Symptom Validity Test (MSVT) as criterion. The signal detection profile of individual EVIs varied greatly. Multivariate models had superior classification accuracy. Failing ≥4 traditional EVIs at the liberal cutoff or ≥3 at the conservative cutoff produced a good combination of sensitivity (.57 to .61) and specificity (.92 to .94) to MSVT. Combining the traditional and item-based EVIs improved sensitivity (.65 to .70) at comparable specificity (.91 to .95). In conclusion, newly developed EVIs within the NIHTB-CB effectively discriminated between patients who passed versus failed the MSVT. Aggregating EVIs within the same category into validity composites improved signal detection over univariate cutoffs. Item-based EVIs improved classification accuracy over that of traditional EVIs. However, the marginal gains hardly justify the burden of extra calculations. The newly introduced EVIs require cross-validation before wide-spread research or clinical application. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
为了评估 NIH 工具包认知电池（NIHTB-CB）上的不可信表现，我们开发了嵌入式有效性指标（EVIs）。数据来自 98 名成年人（54.1%为女性），作为 4 个轻度创伤性脑损伤（mTBI）专业诊所的前瞻性多中心横断面研究的一部分。使用医学症状效度测试（MSVT）作为标准，为 NIHTB-CB 开发了传统的 EVIs 和基于新项目的 EVIs。个体 EVIs 的信号检测特征差异很大。多变量模型具有更高的分类准确性。在宽松截止值下失败≥4 个传统 EVIs 或在保守截止值下失败≥3 个传统 EVIs 时，对 MSVT 的敏感性（.57 至.61）和特异性（.92 至.94）具有良好的组合。将传统和基于项目的 EVIs 相结合可提高敏感性（.65 至.70），同时保持特异性（.91 至.95）不变。总之，NIHTB-CB 中开发的新 EVIs 可有效区分通过和未通过 MSVT 的患者。将同一类别内的 EVIs 聚合到有效性综合指标中可提高比单变量截止值更好的信号检测。基于项目的 EVIs 提高了传统 EVIs 的分类准确性。然而，边际收益几乎不能证明额外计算的负担是合理的。新引入的 EVIs 需要在广泛研究或临床应用之前进行交叉验证。
