State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, China.
Department of Radiology, Sun Yat-Sen University Cancer Center, Guangzhou, China.
Neuro Oncol. 2023 Mar 14;25(3):544-556. doi: 10.1093/neuonc/noac192.
Errors have seldom been evaluated in computer-aided detection on brain metastases. This study aimed to analyze false negatives (FNs) and false positives (FPs) generated by a brain metastasis detection system (BMDS) and by readers.
A deep learning-based BMDS was developed and prospectively validated in a multicenter, multireader study. Ad hoc secondary analysis was restricted to the prospective participants (148 with 1,066 brain metastases and 152 normal controls). Three trainees and 3 experienced radiologists read the MRI images without and with the BMDS. The number of FNs and FPs per patient, jackknife alternative free-response receiver operating characteristic figure of merit (FOM), and lesion features associated with FNs were analyzed for the BMDS and readers using binary logistic regression.
The FNs, FPs, and the FOM of the stand-alone BMDS were 0.49, 0.38, and 0.97, respectively. Compared with independent reading, BMDS-assisted reading generated 79% fewer FNs (1.98 vs 0.42, P < .001); 41% more FPs (0.17 vs 0.24, P < .001) but 125% more FPs for trainees (P < .001); and higher FOM (0.87 vs 0.98, P < .001). Lesions with small size, greater number, irregular shape, lower signal intensity, and located on nonbrain surface were associated with FNs for readers. Small, irregular, and necrotic lesions were more frequently found in FNs for BMDS. The FPs mainly resulted from small blood vessels for the BMDS and the readers.
Despite the improvement in detection performance, attention should be paid to FPs and small lesions with lower enhancement for radiologists, especially for less-experienced radiologists.
在脑转移瘤的计算机辅助检测中，错误很少被评估。本研究旨在分析脑转移瘤检测系统（BMDS）和阅片者产生的假阴性（FN）和假阳性（FP）。
开发了一种基于深度学习的 BMDS，并在多中心、多阅片者研究中进行了前瞻性验证。专门的二次分析仅限于前瞻性参与者（148 例，共 1066 个脑转移瘤和 152 个正常对照）。3 名受训者和 3 名有经验的放射科医生在没有和有 BMDS 的情况下阅读 MRI 图像。使用二元逻辑回归分析 BMDS 和阅片者的每位患者的 FN 和 FP 数量、刀切替代自由响应接收器工作特性（FOM）以及与 FN 相关的病变特征。
独立 BMDS 的 FN、FP 和 FOM 分别为 0.49、0.38 和 0.97。与独立阅片相比，BMDS 辅助阅片可减少 79%的 FN（1.98 比 0.42，P <.001）；增加 41%的 FP（0.17 比 0.24，P <.001），但增加了 125%的受训者 FP（P <.001）；FOM 更高（0.87 比 0.98，P <.001）。阅片者 FN 与病灶小、数量多、形状不规则、信号强度低、位于脑表面外有关。BMDS 的 FN 中更常出现小、不规则和坏死性病变。FP 主要由 BMDS 和阅片者的小血管引起。
尽管检测性能有所提高，但放射科医生仍应注意 FP 和较小的增强程度较低的病变，特别是对经验较少的放射科医生。
