Massachusetts General Hospital and Harvard Medical School, Biostatistics Center, 50 Staniford Street, Suite 560, Boston, MA 02114, USA.
Stat Med. 2010 Mar 15;29(6):617-26. doi: 10.1002/sim.3819.
Many large-scale studies have recently been carried out to assess the reliability of diagnostic procedures, such as mammography for the detection of breast cancer. The large numbers of raters and subjects involved raise new challenges in how to measure agreement in these types of studies. An important motivator of these studies is the identification of factors that contribute to the often wide discrepancies observed between raters' classifications, such as a rater's experience, in order to improve the reliability of the diagnostic process of interest. Incorporating covariate information into the agreement model is a key component in addressing these questions. Few agreement models are currently available that jointly model larger numbers of raters and subjects and incorporate covariate information. In this paper, we extend a recently developed population-based model and measure of agreement for binary ratings to incorporate covariate information using the class of generalized linear mixed models with a probit link function. Important information on factors related to the subjects and raters can be included as fixed and/or random effects in the model. We demonstrate how agreement can be assessed between subgroups of the raters and/or subjects, for example, comparing agreement between experienced and less experienced raters. Simulation studies are carried out to test the performance of the proposed models and measures of agreement. Application to a large-scale breast cancer study is presented.
最近进行了许多大规模的研究，以评估诊断程序的可靠性，例如用于检测乳腺癌的乳房 X 光检查。涉及的大量评估者和对象提出了如何在这些类型的研究中衡量一致性的新挑战。这些研究的一个重要动机是确定导致评估者分类之间经常出现广泛差异的因素，例如评估者的经验，以提高感兴趣的诊断过程的可靠性。将协变量信息纳入一致性模型是解决这些问题的关键组成部分。目前可用的少数一致性模型可联合对更多的评估者和对象进行建模，并纳入协变量信息。在本文中，我们扩展了最近开发的基于人群的二进制评分一致性模型和度量标准，以使用具有概率链接函数的广义线性混合模型类来纳入协变量信息。与受试者和评估者相关的重要信息可以作为固定和/或随机效应包含在模型中。我们展示了如何在评估者和/或受试者的子组之间评估一致性，例如，比较经验丰富的评估者和经验较少的评估者之间的一致性。进行了模拟研究以测试所提出的模型和一致性度量的性能。介绍了对大规模乳腺癌研究的应用。
