Departments of Radiology and Biostatistics, The University of Iowa, 3710 Medical Laboratories, 200 Hawkins Drive, Iowa City, 52242-1077, IA, U.S.A.
Stat Med. 2018 Jun 15;37(13):2067-2093. doi: 10.1002/sim.7616. Epub 2018 Apr 2.
For the typical diagnostic radiology study design, each case (ie, patient) undergoes several diagnostic tests (or modalities) and the resulting images are interpreted by several readers. Often, each reader is asked to assign a confidence-of-disease rating to each case for each test, and the diagnostic tests are compared with respect to reader-performance outcomes that are functions of the reader receiver operating characteristic (ROC) curves, such as the area under the ROC curve. These reader-performance outcomes are frequently analyzed using the Obuchowski and Rockette method, which allows conclusions to generalize to both the reader and case populations. The simulation model proposed by Roe and Metz (RM) in 1997 emulates confidence-of-disease data collected from such studies and has been an important tool for empirically evaluating various reader-performance analysis methods. However, because the RM model parameters are expressed in terms of a continuous decision variable rather than in terms of reader-performance outcomes, it has not been possible to evaluate the realism of the RM model. I derive the relationships between the RM and Obuchowski-Rockette model parameters for the empirical area under the ROC curve reader-performance outcome. These relationships make it possible to evaluate the realism of the RM parameter models and to assess the performance of Obuchowski-Rockette parameter estimates. An example illustrates the application of the relationships for assessing the performance of a proposed upper one-sided confidence bound for the Obuchowski-Rockette test-by-reader variance component, which is useful for sample size estimation.
对于典型的诊断放射学研究设计，每个病例（即患者）接受多项诊断测试（或方式），并由多名读者对结果图像进行解读。通常，每位读者会被要求对每个病例的每个测试分配疾病置信度评分，然后比较诊断测试，以评估读者表现结果，这些结果是读者接收机操作特征（ROC）曲线的函数，例如 ROC 曲线下的面积。这些读者表现结果通常使用 Obuchowski 和 Rockette 方法进行分析，该方法允许将结论推广到读者和病例群体。Roe 和 Metz 于 1997 年提出的 Roe-Metz (RM) 模拟模型模拟了此类研究中收集的疾病置信度数据，是对各种读者表现分析方法进行实证评估的重要工具。然而，由于 RM 模型参数是用连续决策变量表示的，而不是用读者表现结果表示的，因此无法评估 RM 模型的现实性。我推导出了 RM 模型和 Obuchowski-Rockette 模型参数之间的关系，用于实证 ROC 曲线读者表现结果的面积。这些关系使得评估 RM 参数模型的现实性和评估 Obuchowski-Rockette 参数估计的性能成为可能。一个示例说明了如何应用这些关系来评估拟议的 Obuchowski-Rockette 测试间读者方差分量单侧置信上限的性能，这对于样本量估计很有用。
