临床试验中顺序确定的观察者间一致性（Kappa）测量可能与观察者表现的变化无关。
Sequentially Determined Measures of Interobserver Agreement (Kappa) in Clinical Trials May Vary Independent of Changes in Observer Performance.
机构信息
Decision Sciences, IQVIA, Durham, NC, USA.
Immunology, Eli Lilly, Indianapolis, IN, 46285, USA.
出版信息
Ther Innov Regul Sci. 2020 May;54(3):681-686. doi: 10.1007/s43441-019-00102-5. Epub 2020 Jan 6.
BACKGROUND
Cohen's kappa is a statistic that estimates interobserver agreement. It was originally introduced to help develop diagnostic tests. Interpretative readings of 2 observers, for example, of a mammogram or other imaging, were compared at a single point in time. It is known that kappa depends on the prevalence of disease and that, therefore, kappas across different settings are hard to compare.
METHODS
Using simulation, we examine an analogous situation, not previously described, that occurs in clinical trials where sequential measurements are obtained to evaluate disease progression or clinical improvement over time.
RESULTS
We show that weighted kappa, used for multilevel outcomes, changes during the trial even if we keep the performance of the observer constant.
CONCLUSIONS
Kappa and closely related measures can therefore only be used with great difficulty, if at all, in quality assurance in clinical trials.
背景
科恩的 kappa 是一种用于估计观察者间一致性的统计量。它最初是为了帮助开发诊断测试而引入的。例如，对乳腺 X 线照片或其他影像学检查的两位观察者的解释性读数，在单个时间点进行比较。已知 kappa 取决于疾病的流行程度，因此，不同环境下的 kappas 很难进行比较。
方法
我们使用模拟方法研究了一种以前未描述的类似情况，这种情况发生在临床试验中，为了评估疾病的进展或随着时间的推移临床改善，需要进行连续测量。
结果
我们表明，即使观察者的表现保持不变，用于多水平结果的加权 kappa 在试验期间也会发生变化。
结论
因此，如果要在临床试验的质量保证中使用 kappa 和密切相关的度量标准，则只能非常困难地使用，如果可以使用的话。
Zotero 插件