Stover Angela M, McLeod Lori D, Langer Michelle M, Chen Wen-Hung, Reeve Bryce B
Department of Health Policy and Management, University of North Carolina at Chapel Hill, 1101-G McGavran-Greenberg Hall (CB# 7411), Chapel Hill, NC, 27599, USA.
Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill School of Medicine, 101 Manning Drive, Chapel Hill, NC, 27599, USA.
J Patient Rep Outcomes. 2019 Jul 30;3(1):50. doi: 10.1186/s41687-019-0130-5.
This paper is part of a series comparing different psychometric approaches to evaluate patient-reported outcome (PRO) measures using the same items and dataset. We provide an overview and example application to demonstrate 1) using item response theory (IRT) to identify poor and well performing items; 2) testing if items perform differently based on demographic characteristics (differential item functioning, DIF); and 3) balancing IRT and content validity considerations to select items for short forms.
Model fit, local dependence, and DIF were examined for 51 items initially considered for the Patient-Reported Outcomes Measurement Information System® (PROMIS®) Depression item bank. Samejima's graded response model was used to examine how well each item measured severity levels of depression and how well it distinguished between individuals with high and low levels of depression. Two short forms were constructed based on psychometric properties and consensus discussions with instrument developers, including psychometricians and content experts. Calibrations presented here are for didactic purposes and are not intended to replace official PROMIS parameters or to be used for research.
Of the 51 depression items, 14 exhibited local dependence, 3 exhibited DIF for gender, and 9 exhibited misfit, and these items were removed from consideration for short forms. Short form 1 prioritized content, and thus items were chosen to meet DSM-V criteria rather than being discarded for lower discrimination parameters. Short form 2 prioritized well performing items, and thus fewer DSM-V criteria were satisfied. Short forms 1-2 performed similarly for model fit statistics, but short form 2 provided greater item precision.
IRT is a family of flexible models providing item- and scale-level information, making it a powerful tool for scale construction and refinement. Strengths of IRT models include placing respondents and items on the same metric, testing DIF across demographic or clinical subgroups, and facilitating creation of targeted short forms. Limitations include large sample sizes to obtain stable item parameters, and necessary familiarity with measurement methods to interpret results. Combining psychometric data with stakeholder input (including people with lived experiences of the health condition and clinicians) is highly recommended for scale development and evaluation.
本文是一个系列文章的一部分，该系列文章比较了使用相同条目和数据集来评估患者报告结局（PRO）指标的不同心理测量方法。我们提供了一个概述和示例应用，以展示：1）使用项目反应理论（IRT）来识别表现不佳和良好的条目；2）测试条目是否根据人口统计学特征表现不同（差异项目功能，DIF）；3）平衡IRT和内容效度考量以选择简短形式的条目。
对最初考虑纳入患者报告结局测量信息系统（PROMIS®）抑郁条目库的51个条目进行了模型拟合、局部依赖性和DIF检验。使用Samejima的等级反应模型来检验每个条目对抑郁严重程度的测量效果以及区分抑郁水平高和低的个体的效果。基于心理测量特性以及与包括心理测量学家和内容专家在内的工具开发者的共识讨论，构建了两个简短形式。此处呈现的校准仅用于教学目的，并非旨在取代官方的PROMIS参数或用于研究。
在51个抑郁条目中，14个表现出局部依赖性，3个在性别方面表现出DIF，9个表现出不拟合，这些条目被排除在简短形式的考虑范围之外。简短形式1优先考虑内容，因此选择条目以满足《精神疾病诊断与统计手册》第五版（DSM-V）标准，而不是因为较低的区分参数而被舍弃。简短形式2优先考虑表现良好的条目，因此满足的DSM-V标准较少。简短形式1 - 2在模型拟合统计方面表现相似，但简短形式2提供了更高的条目精度。
IRT是一族灵活的模型，可提供条目和量表层面的信息，使其成为量表构建和完善的强大工具。IRT模型的优势包括将受访者和条目置于同一度量标准上、跨人口统计学或临床亚组测试DIF以及便于创建有针对性的简短形式。局限性包括需要大样本量以获得稳定的条目参数，以及需要熟悉测量方法以解释结果。强烈建议在量表开发和评估中结合心理测量数据与利益相关者的意见（包括有该健康状况实际经历的人和临床医生）。
