Garrard Lili, Price Larry R, Bott Marjorie J, Gajewski Byron J
Department of Biostatistics, University of Kansas Medical Center, Mail Stop 1026, 3901 Rainbow Blvd., Kansas City, KS, 66160, USA.
College of Education, Texas State University, San Marcos, TX, 78666, USA.
BMC Med Res Methodol. 2015 Sep 29;15:77. doi: 10.1186/s12874-015-0071-5.
Developing valid and reliable patient-reported outcome measures (PROMs) is a critical step in promoting patient-centered health care, a national priority in the U.S. Small populations or rare diseases often pose difficulties in developing PROMs using traditional methods due to small samples.
To overcome the small sample size challenge while maintaining psychometric soundness, we propose an innovative Ordinal Bayesian Instrument Development (OBID) method that seamlessly integrates expert and participant data in a Bayesian item response theory (IRT) with a probit link model framework. Prior distributions obtained from expert data are imposed on the IRT model parameters and are updated with participants' data. The efficiency of OBID is evaluated by comparing its performance to classical instrument development performance using actual and simulation data. RESULTS AND DISCUSSION : The overall performance of OBID (i.e., more reliable parameter estimates, smaller mean squared errors (MSEs) and higher predictive validity) is superior to that of classical approaches when the sample size is small (e.g. less than 100 subjects). Although OBID may exhibit larger bias, it reduces the MSEs by decreasing variances. Results also closely align with recommendations in the current literature that six subject experts will be sufficient for establishing content validity evidence. However, in the presence of highly biased experts, three experts will be adequate.
This study successfully demonstrated that the OBID approach is more efficient than the classical approach when the sample size is small. OBID promises an efficient and reliable method for researchers and clinicians in future PROMs development for small populations or rare diseases.
开发有效且可靠的患者报告结局测量工具（PROMs）是促进以患者为中心的医疗保健的关键一步，这是美国的一项国家优先事项。由于样本量小，小群体或罕见疾病在使用传统方法开发PROMs时常常面临困难。
为了在保持心理测量稳健性的同时克服小样本量挑战，我们提出了一种创新的有序贝叶斯工具开发（OBID）方法，该方法在贝叶斯项目反应理论（IRT）与概率单位链接模型框架中无缝整合专家和参与者数据。从专家数据获得的先验分布被施加到IRT模型参数上，并用参与者的数据进行更新。通过使用实际数据和模拟数据将OBID的性能与经典工具开发性能进行比较，来评估OBID的效率。
当样本量较小时（例如少于100名受试者），OBID的整体性能（即更可靠的参数估计、更小的均方误差（MSE）和更高的预测效度）优于经典方法。尽管OBID可能表现出较大的偏差，但它通过降低方差来减少MSE。结果也与当前文献中的建议密切一致，即六位主题专家足以建立内容效度证据。然而，在存在高度偏差的专家的情况下，三位专家就足够了。
本研究成功证明，当样本量较小时，OBID方法比经典方法更有效。OBID为研究人员和临床医生在未来针对小群体或罕见疾病的PROMs开发中提供了一种高效且可靠的方法。
