Department of Quantitative Health Sciences, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, USA.
Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, 625 Michigan Ave, 27th Floor, Chicago, IL, 60611, USA.
Qual Life Res. 2023 May;32(5):1355-1367. doi: 10.1007/s11136-022-03255-3. Epub 2022 Sep 24.
This study compares classical test theory and item response theory frameworks to determine reliable change. Reliable change followed by anchoring to the change in categorically distinct responses on a criterion measure is a useful method to detect meaningful change on a target measure.
Adult cancer patients were recruited from five cancer centers. Baseline and follow-up assessments at 6 weeks were administered. We investigated short forms derived from PROMIS® item banks on anxiety, depression, fatigue, pain intensity, pain interference, and sleep disturbance. We detected reliable change using reliable change index (RCI). We derived the T-scores corresponding to the RCI calculated under IRT and CTT frameworks using PROMIS® short forms. For changes that were reliable, meaningful change was identified using patient-reported change in PRO-CTCAE by at least one level. For both CTT and IRT approaches, we applied one-sided tests to detect reliable improvement or worsening using RCI. We compared the percentages of patients with reliable change and reliable/meaningful change.
The amount of change in T score corresponding to RCI of 1.65 ranged from 5.1 to 9.2 depending on domains. The amount of change corresponding to RCI of 1.65 varied across the score range, and the minimum change ranged from 3.0 to 8.2 depending on domains. Across domains, the RCI and RCI classified 80% to 98% of the patients consistently. When there was disagreement, the RCI tended to identify more patients as having reliably changed compared to RCI if scores at both timepoints were in the range of 43 to 78 in anxiety, 45 to 70 in depression, 38 to 80 in fatigue, 35 to 78 in sleep disturbance, and 48 to 74 in pain interference, due to smaller standard errors in these ranges using the IRT method. The CTT method found more changes compared to IRT for the pain intensity domain that was shorter in length. Using RCI, 22% to 66% had reliable change in either direction depending on domains, and among these patients, 62% to 83% had meaningful change. Using RCI, 37% to 68% had reliable change in either direction, and among these patients, 62% to 81% had meaningful change.
Applying the two-step criteria demonstrated in this study, we determined how much change is needed to declare reliable change at different levels of baseline scores. We offer reference values for percentage of patients who meaningfully change for investigators using the PROMIS instruments in oncology.
本研究比较了经典测试理论和项目反应理论框架，以确定可靠的变化。可靠的变化后，锚定到分类上截然不同的反应在标准测量上的变化，是一种检测目标测量上有意义变化的有用方法。
从五个癌症中心招募成年癌症患者。在 6 周时进行基线和随访评估。我们研究了来自 PROMIS®项目库的焦虑、抑郁、疲劳、疼痛强度、疼痛干扰和睡眠障碍的简短形式。我们使用可靠变化指数（RCI）检测可靠变化。我们使用 PROMIS 简短形式计算 IRT 和 CTT 框架下与 RCI 相对应的 T 分数。对于可靠的变化，使用 PRO-CTCAE 患者报告的变化至少一级来确定有意义的变化。对于 CTT 和 IRT 方法，我们使用 RCI 通过单侧检验检测可靠的改善或恶化。我们比较了具有可靠变化和可靠/有意义变化的患者的百分比。
与 RCI 为 1.65 对应的 T 分数变化量因域而异，范围为 5.1 至 9.2。与 RCI 为 1.65 对应的变化量在分数范围内变化，并且最小变化因域而异，范围为 3.0 至 8.2。在所有领域，RCI 和 RCI 一致地将 80%至 98%的患者分类。当存在分歧时，如果两个时间点的分数都在焦虑的 43 到 78 之间，抑郁的 45 到 70 之间，疲劳的 38 到 80 之间，睡眠干扰的 35 到 78 之间，疼痛干扰的 48 到 74 之间，那么 RCI 比 RCI 更倾向于识别更多的患者具有可靠的变化，这是因为在这些范围内使用 IRT 方法的标准误差较小。对于疼痛强度域，由于长度较短，CTT 方法比 IRT 方法发现更多的变化。使用 RCI，根据域的不同，有 22%至 66%的患者有可靠的变化，在这些患者中，有 62%至 83%的患者有有意义的变化。使用 RCI，有 37%至 68%的患者有可靠的变化，在这些患者中，有 62%至 81%的患者有有意义的变化。
应用本研究中提出的两步标准，我们确定了在不同的基线分数水平下需要多大的变化才能宣布可靠的变化。我们为使用 PROMIS 工具在肿瘤学中进行研究的研究人员提供了有意义变化的患者百分比的参考值。
