Department of Psychiatry, University of Illinois College of Medicine , Chicago, IL, USA.
Department of Psychology, Roosevelt University , Chicago, IL, USA.
J Clin Exp Neuropsychol. 2020 Jul;42(5):459-472. doi: 10.1080/13803395.2020.1758634. Epub 2020 May 13.
Embedded performance validity tests (PVTs) allow for continuous and economical validity assessment during neuropsychological evaluations; however, similar to their freestanding counterparts, a limitation of well-validated embedded PVTs is that the majority are memory-based. This study cross-validated several previously identified non-memory-based PVTs derived from language, processing speed, and executive functioning tests within a single mixed clinical neuropsychiatric sample with and without cognitive impairment.
This cross-sectional study included data from 124 clinical patients who underwent outpatient neuropsychological evaluation. Validity groups were determined by four independent criterion PVTs (failing ≤1 or ≥2), resulting in 98 valid (68% cognitively impaired) and 26 invalid performances. In total, 23 previously identified embedded PVTs derived from Verbal Fluency (VF), Trail Making Test (TMT), Stroop (SCWT), and Wisconsin Card Sorting Test (WCST) were examined.
All VF, SCWT, and TMT PVTs, along with WCST Categories, significantly differed between validity groups (=.05-.22) with areas under the curve (AUCs) of.65-.81 and 19-54% sensitivity (≥89% specificity) at optimal cut-scores. When subdivided by impairment status, all PVTs except for WCST Failures to Maintain Set were significant (AUCs =.75-94) with 33-85% sensitivity (≥90% specificity) in the cognitively unimpaired group. Among the cognitively impaired group, most VF, TMT, and SCWT PVTs remained significant, albeit with decreased accuracy (AUCs =.65-.76) and sensitivities (19-54%) at optimal cut-scores, whereas all WCST PVTs were nonsignificant. Across groups, SCWT embedded PVTs evidenced the strongest psychometric properties.
VF, TMT, and SCWT embedded PVTs generally demonstrated moderate accuracy for identifying invalid neuropsychological performance. However, performance on these non-memory-based PVTs from processing speed and executive functioning tests are not immune to the effects of cognitive impairment, such that alternate cut-scores (with reduced sensitivity if adequate specificity is maintained) are indicated in cases where the clinical history is consistent with cognitive impairment. In contrast, WCST indices generally had poor accuracy.
嵌入式绩效有效性测试（PVT）允许在神经心理评估过程中进行连续且经济有效的有效性评估；然而，与独立的 PVT 类似，经过充分验证的嵌入式 PVT 的一个局限性是，它们大多基于记忆。本研究在一个混合的临床神经精神科样本中交叉验证了几种先前确定的非记忆性 PVT，这些 PVT 源自语言、处理速度和执行功能测试。
本横断面研究纳入了 124 名接受门诊神经心理评估的临床患者的数据。通过四个独立的标准 PVT（≤1 或≥2 失败）确定有效性组，结果得出 98 个有效（68%认知受损）和 26 个无效表现。总共检查了 23 种先前确定的嵌入式 PVT，这些 PVT 源自言语流畅性（VF）、连线测试（TMT）、Stroop（SCWT）和威斯康星卡片分类测试（WCST）。
所有 VF、SCWT 和 TMT PVT，以及 WCST 类别，在有效性组之间均有显著差异（=.05-.22），曲线下面积（AUCs）为.65-.81，最佳截断分数的敏感性为 19-54%（特异性≥89%）。按损伤状态细分时，除 WCST 未能维持设置外，所有 PVT 均具有统计学意义（AUCs=.75-94），在认知正常组中的敏感性为 33-85%（特异性≥90%）。在认知受损组中，大多数 VF、TMT 和 SCWT PVT 仍然具有统计学意义，尽管在最佳截断分数处准确性降低（AUCs=.65-.76），敏感性降低（19-54%），而所有 WCST PVT 均无统计学意义。在所有组中，SCWT 嵌入式 PVT 表现出最强的心理测量特性。
VF、TMT 和 SCWT 嵌入式 PVT 通常具有中等的准确性，可用于识别无效的神经心理表现。然而，来自处理速度和执行功能测试的这些非记忆性 PVT 的表现并不免受认知损伤的影响，因此，如果临床病史与认知损伤一致，则需要替代截断分数（在保持足够特异性的情况下降低敏感性）。相比之下，WCST 指数的准确性通常较差。
