人类与人工智能在局灶性皮质发育不良检测中的定量比较

A Quantitative Comparison Between Human and Artificial Intelligence in the Detection of Focal Cortical Dysplasia.

作者信息

Walger Lennart, Bauer Tobias, Kügler David, Schmitz Matthias H, Schuch Fabiane, Arendt Christophe, Baumgartner Tobias, Birkenheier Johannes, Borger Valeri, Endler Christoph, Grau Franziska, Immanuel Christian, Kölle Markus, Kupczyk Patrick, Lakghomi Asadeh, Mackert Sarah, Neuhaus Elisabeth, Nordsiek Julia, Odenthal Anna-Maria, Dague Karmele Olaciregui, Ostermann Laura, Pukropski Jan, Racz Attila, von der Ropp Klaus, Schmeel Frederic Carsten, Schrader Felix, Sitter Aileen, Unruh-Pinheiro Alexander, Voigt Marilia, Vychopen Martin, von Wedel Philip, von Wrede Randi, Attenberger Ulrike, Vatter Hartmut, Philipsen Alexandra, Becker Albert, Reuter Martin, Hattingen Elke, Sander Josemir W, Radbruch Alexander, Surges Rainer, Rüber Theodor

机构信息

From the Department of Neuroradiology, University Hospital Bonn, Bonn, Germany (L.W., T. Bauer, M.H.S., F.G., A.L., F.C.S., A. Radbruch, T.R.); Department of Epileptology, University Hospital Bonn, Bonn, Germany (L.W., T. Bauer, M.H.S., F. Schuch, T. Baumgartner, K.O.D., L.O., J.P., A. Racz, K.v.d.R., A.U.-P., P.v.W., R.v.W., R.S., T.R.); German Center for Neurodegenerative Diseases, Bonn, Germany (D.K., M.R., A. Radbruch); Department of Neuroradiology, Goethe University Frankfurt, Frankfurt, Germany (C.A., E.N., E.H.); Department of Neurology, University Hospital Bonn, Bonn, Germany (J.B., J.N.); Department of Neurosurgery, University Hospital Bonn, Bonn, Germany (V.B., M. Vychopen, H.V.); Department of Diagnostic and Interventional Radiology, University Hospital Bonn, Bonn, Germany (C.E., C.I., P.K., A.L., A.-M.O., M. Voigt, U.A.); Department of Psychiatry and Psychotherapy, University Hospital Bonn, Bonn, Germany (M.K., S.M., F. Schrader, A.S., A.P.); Chair of Economic & Social Policy, WHU-Otto Beisheim School of Management, Vallendar, Germany (P.v.W.); Department of Neuropathology, University Hospital Bonn, Bonn, Germany (A.B.); A.A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA (M.R.); Department of Radiology, Harvard Medical School, Boston, MA (M.R.); Department of Clinical and Experimental Epilepsy, UCL Queen Square Institute of Neurology, London, United Kingdom (J.W.S.); Chalfont Centre for Epilepsy, Chalfont St Peter, United Kingdom (J.W.S.); Stichting Epilepsie Instellingen Nederland, Heemstede, the Netherland (J.W.S.); Department of Neurology, West China Hospital, Sichuan University, Chengdu, China (J.W.S.); and Center for Medical Data Usability and Translation, University of Bonn, Bonn, Germany (A. Radbruch, T.R.).

出版信息

Invest Radiol. 2025 Apr 1;60(4):253-259. doi: 10.1097/RLI.0000000000001125. Epub 2024 Oct 23.

DOI:10.1097/RLI.0000000000001125

PMID:39437019

Abstract

OBJECTIVES

Artificial intelligence (AI) is thought to improve lesion detection. However, a lack of knowledge about human performance prevents a comparative evaluation of AI and an accurate assessment of its impact on clinical decision-making. The objective of this work is to quantitatively evaluate the ability of humans to detect focal cortical dysplasia (FCD), compare it to state-of-the-art AI, and determine how it may aid diagnostics.

MATERIALS AND METHODS

We prospectively recorded the performance of readers in detecting FCDs using single points and 3-dimensional bounding boxes. We acquired predictions of 3 AI models for the same dataset and compared these to readers. Finally, we analyzed pairwise combinations of readers and models.

RESULTS

Twenty-eight readers, including 20 nonexpert and 5 expert physicians, reviewed 180 cases: 146 subjects with FCD (median age: 25, interquartile range: 18) and 34 healthy control subjects (median age: 43, interquartile range: 19). Nonexpert readers detected 47% (95% confidence interval [CI]: 46, 49) of FCDs, whereas experts detected 68% (95% CI: 65, 71). The 3 AI models detected 32%, 51%, and 72% of FCDs, respectively. The latter, however, also predicted more than 13 false-positive clusters per subject on average. Human performance was improved in the presence of a transmantle sign ( P < 0.001) and cortical thickening ( P < 0.001). In contrast, AI models were sensitive to abnormal gyration ( P < 0.01) or gray-white matter blurring ( P < 0.01). Compared with single experts, expert-expert pairs detected 13% (95% CI: 9, 18) more FCDs ( P < 0.001). All AI models increased expert detection rates by up to 19% (95% CI: 15, 24) ( P < 0.001). Nonexpert+AI pairs could still outperform single experts by up to 13% (95% CI: 10, 17).

CONCLUSIONS

This study pioneers the comparative evaluation of humans and AI for FCD lesion detection. It shows that AI and human predictions differ, especially for certain MRI features of FCD, and, thus, how AI may complement the diagnostic workup.

摘要

目的

人们认为人工智能（AI）可改善病变检测。然而，由于缺乏对人类表现的了解，无法对AI进行比较评估，也无法准确评估其对临床决策的影响。本研究的目的是定量评估人类检测局灶性皮质发育异常（FCD）的能力，将其与最先进的AI进行比较，并确定其如何辅助诊断。

材料与方法

我们前瞻性地记录了读者使用单点和三维边界框检测FCD的表现。我们获取了3个AI模型对同一数据集的预测结果，并将其与读者的结果进行比较。最后，我们分析了读者和模型的两两组合。

结果

28名读者，包括20名非专家医生和5名专家医生，对180例病例进行了评估：146例FCD患者（中位年龄：25岁，四分位间距：18岁）和34名健康对照者（中位年龄：43岁，四分位间距：19岁）。非专家读者检测出47%（95%置信区间[CI]：46，49）的FCD，而专家检测出68%（95%CI：65，71）。3个AI模型分别检测出32%、51%和72%的FCD。然而，后者平均每个受试者还预测出超过13个假阳性簇。存在跨皮质征（P<0.001）和皮质增厚（P<0.001）时，人类的表现有所改善。相比之下，AI模型对异常脑回（P<0.01）或灰白质模糊（P<0.01）敏感。与单个专家相比，专家-专家对检测出的FCD多13%（95%CI：9，18）（P<0.001）。所有AI模型使专家的检测率提高了19%（95%CI：15，24）（P<0.001）。非专家+AI组合仍可使检测率比单个专家高出13%（95%CI：10，17）。