Pacific Metrics Corporation, 1 Lower Ragsdale Drive, Monterey, CA, 93940, USA.
Qual Life Res. 2018 Jul;27(7):1683-1693. doi: 10.1007/s11136-017-1650-1. Epub 2017 Jul 14.
Most computerized adaptive testing (CAT) applications in patient-reported outcomes (PRO) measurement to date are reliability-centric, with a primary objective of maximizing measurement efficiency. A key concern and a potential threat to validity is that, when left unconstrained, individual CAT administrations could have items with systematically different attributes, e.g., sub-domain coverage. This paper aims to provide a solution to the problem from an optimal test design framework using the shadow-test approach to CAT.
Following the approach, a case study was conducted using the PROMIS (Patient-Reported Outcomes Measurement Information System) fatigue item bank both with empirical and simulated response data. Comparisons between CAT administrations without and with the enforcement of content and item pool usage constraints were examined.
The unconstrained CAT exhibited a high degree of variation in items selected from different substrata of the item bank. Contrastingly, the shadow-test approach delivered CAT administrations conforming to all specifications with a minimal loss in measurement efficiency.
The optimal test design and shadow-test approach to CAT provide a flexible framework for solving complex test-assembly problems with better control of their domain coverage than for the conventional use of CAT in PRO measurement. Applications in a wide array of PRO domains are expected to lead to more controlled and balanced use of CAT in the field.
迄今为止，大多数用于患者报告结局（PRO）测量的计算机化自适应测试（CAT）应用程序都是以可靠性为中心的，主要目标是最大限度地提高测量效率。一个关键的关注点和潜在的有效性威胁是，当不受约束时，个体 CAT 管理可能会有具有系统差异的项目，例如，子域覆盖。本文旨在从使用 CAT 的影子测试方法的最优测试设计框架提供一个解决方案。
按照该方法，使用 PROMIS（患者报告结果测量信息系统）疲劳项目库进行了案例研究，同时使用了实证和模拟响应数据。检查了没有和执行内容和项目池使用约束的 CAT 管理之间的比较。
无约束 CAT 从项目库的不同子层中选择的项目具有高度的变化性。相比之下，影子测试方法提供了符合所有规范的 CAT 管理，并且在测量效率方面损失最小。
CAT 的最优测试设计和影子测试方法为解决复杂的测试组装问题提供了一个灵活的框架，与 PRO 测量中传统的 CAT 使用相比，它对其领域覆盖范围有更好的控制。在广泛的 PRO 领域的应用有望在该领域更有效地控制和平衡 CAT 的使用。
