From the Department of Radiology and Biomedical Imaging, University of California San Francisco, 513 Parnassus Ave, S255, Box 0628, San Francisco, CA 94143 (C.W.H., M.A.O.); Liver Imaging Group, Department of Radiology, University of California, San Diego, San Diego, Calif (C.W.H., C.P., T.D., D.T.M., K.J.F., C.B.S.); Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY (V.C., N.H.); Department of Radiology, Yonsei University, Seoul, South Korea (J.Y.C.); Department of Radiology, University of California Irvine, Orange, Calif (S.L., R.K.); Computational and Applied Statistics Laboratory, University of California San Diego, San Diego, Calif (T.W., A.G.); Department of Radiology, New York University, New York, NY (J.B.); Department of Radiology, University of Florida, Jacksonville, Fla (C.L.); Department of Radiology, University of Kentucky, Lexington, Ky (J.T.L., J.W.O.); Department of Radiology, Fundación Santa Fe de Bogotá, Bogotá, Colombia (D.A.A.); Department of Radiology, University of Michigan, Ann Arbor, Mich (M.M.L., M.S.D., W.M.); Department of Radiology, Allegheny Health Network, Pittsburgh, Pa (A.R.); Department of Radiology, Icahn School of Medicine at Mount Sinai, New York, NY (S.C.L.); Department of Radiology, New York-Presbyterian/Weill Cornell Medical Center, New York, NY (A.S.K., E.M.H.); Departments of Radiology and Medicine, Duke University Medical Center, New York, NY (M.R.B.); Section of Radiology, Department of Biomedicine, Neuroscience and Advanced Diagnostics (BiND), University Hospital Paolo Giaccone, Palermo, Italy (G.B.); Department of Radiology, University of California Los Angeles, Los Angeles, Calif (M.L.D.); Department of Radiology, Radiation Oncology and Nuclear Medicine, Université de Montréal, Montréal, Canada (A.T., M.C.); Department of Radiology, Oregon Health & Science University, Portland, Ore (A.F.); CEDRUL-Centro de Diagnóstico por Imagem, João Pessoa, Brazil (E.A.C.); Department of Radiology, University of California Davis, Sacramento, Calif (M.T.C., J.P.M.); Radiology Limited, Tucson, Ariz (B.K.); Department of Abdominal Imaging, University of Texas MD Anderson Cancer Center, Houston, Tex (K.M.E., V.R.S., K.B.); Department of Radiology, Naval Medical Center San Diego, San Diego, Calif (R.M.M.); University of São Paulo/Hospital Sírio-Libanês, São Paulo, Brazil (N.H.); Department of Radiology, University of Kansas, Kansas City, Kan (S.B., R.A.); Sir H. N. Reliance Foundation Hospital and Research Centre, Mumbai, India (K.G.); Department of Radiology, California Pacific Medical Center, San Francisco, Calif (C.R.K.); Department of Radiology, Massachusetts General Hospital, Boston, Mass (A.K.); The 3rd Affiliated Hospital, Sun Yat-sen University, Guangzhou, China (J.W.); Inland Imaging, Spokane, Wash (I.C.); Sutter Medical Group, Sacramento, Calif (B.B.); Austin Health, Melbourne, Australia (M.G.); Department of Radiology, University of Washington, Seattle, Wash (G.M.C.).
Radiology. 2023 Jun;307(5):e222855. doi: 10.1148/radiol.222855.
Background Various limitations have impacted research evaluating reader agreement for Liver Imaging Reporting and Data System (LI-RADS). Purpose To assess reader agreement of LI-RADS in an international multicenter multireader setting using scrollable images. Materials and Methods This retrospective study used deidentified clinical multiphase CT and MRI and reports with at least one untreated observation from six institutions and three countries; only qualifying examinations were submitted. Examination dates were October 2017 to August 2018 at the coordinating center. One untreated observation per examination was randomly selected using observation identifiers, and its clinically assigned features were extracted from the report. The corresponding LI-RADS version 2018 category was computed as a rescored clinical read. Each examination was randomly assigned to two of 43 research readers who independently scored the observation. Agreement for an ordinal modified four-category LI-RADS scale (LR-1, definitely benign; LR-2, probably benign; LR-3, intermediate probability of malignancy; LR-4, probably hepatocellular carcinoma [HCC]; LR-5, definitely HCC; LR-M, probably malignant but not HCC specific; and LR-TIV, tumor in vein) was computed using intraclass correlation coefficients (ICCs). Agreement was also computed for dichotomized malignancy (LR-4, LR-5, LR-M, and LR-TIV), LR-5, and LR-M. Agreement was compared between research-versus-research reads and research-versus-clinical reads. Results The study population consisted of 484 patients (mean age, 62 years ± 10 [SD]; 156 women; 93 CT examinations, 391 MRI examinations). ICCs for ordinal LI-RADS, dichotomized malignancy, LR-5, and LR-M were 0.68 (95% CI: 0.61, 0.73), 0.63 (95% CI: 0.55, 0.70), 0.58 (95% CI: 0.50, 0.66), and 0.46 (95% CI: 0.31, 0.61) respectively. Research-versus-research reader agreement was higher than research-versus-clinical agreement for modified four-category LI-RADS (ICC, 0.68 vs 0.62, respectively; = .03) and for dichotomized malignancy (ICC, 0.63 vs 0.53, respectively; = .005), but not for LR-5 ( = .14) or LR-M ( = .94). Conclusion There was moderate agreement for LI-RADS version 2018 overall. For some comparisons, research-versus-research reader agreement was higher than research-versus-clinical reader agreement, indicating differences between the clinical and research environments that warrant further study. © RSNA, 2023 See also the editorials by Johnson and Galgano and Smith in this issue.
背景 各种局限性影响了评估 Liver Imaging Reporting and Data System（LI-RADS）读者一致性的研究。目的 本研究旨在使用可滚动图像在国际多中心多读者环境中评估 LI-RADS 的读者一致性。材料与方法 本回顾性研究使用了来自六个机构和三个国家的经去识别的临床多期 CT 和 MRI 及至少有一个未经治疗观察的报告；仅提交合格的检查。检查日期为 2017 年 10 月至 2018 年 8 月在协调中心。使用观察标识符随机选择每个检查的一个未经治疗的观察，并从报告中提取其临床分配的特征。相应的 2018 年版 LI-RADS 类别作为重新评分的临床阅读计算。每个检查随机分配给 43 名研究读者中的两名，他们独立对观察结果进行评分。使用组内相关系数（intraclass correlation coefficients，ICC）计算了用于有序修改后的四分类 LI-RADS 量表（LR-1，肯定良性；LR-2，可能良性；LR-3，恶性可能性中等；LR-4，可能肝细胞癌[hepatocellular carcinoma，HCC]；LR-5，肯定 HCC；LR-M，可能恶性但非 HCC 特异性；LR-TIV，静脉内肿瘤）的一致性。还计算了恶性（LR-4、LR-5、LR-M 和 LR-TIV）、LR-5 和 LR-M 的二分法的一致性。比较了研究内-研究内和研究内-临床阅读的一致性。结果 研究人群包括 484 例患者（平均年龄，62 岁±10[标准差]；156 例女性；93 例 CT 检查，391 例 MRI 检查）。有序 LI-RADS、恶性二分法、LR-5 和 LR-M 的 ICC 分别为 0.68（95%置信区间：0.61，0.73）、0.63（95%置信区间：0.55，0.70）、0.58（95%置信区间：0.50，0.66）和 0.46（95%置信区间：0.31，0.61）。对于修改后的四分类 LI-RADS（ICC，0.68 与 0.62，分别； =.03）和恶性二分法（ICC，0.63 与 0.53，分别； =.005），研究内-研究内读者的一致性高于研究内-临床读者的一致性，但对于 LR-5（ =.14）或 LR-M（ =.94）则不然。结论 总体而言，LI-RADS 版本 2018 的一致性中等。对于某些比较，研究内-研究内读者的一致性高于研究内-临床读者的一致性，表明临床和研究环境之间存在差异，需要进一步研究。 ©2023 RSNA。另见本期 Johnson 和 Galgano 及 Smith 的社论。
