Ziegler Joceline, Dobsch Philipp, Rozema Marten, Zuber-Jerger Ina, Weigand Kilian, Reuther Stefan, Müller Martina, Kandulski Arne
Unetiq GmbH, München, Germany.
Department of Internal Medicine I, Gastroenterology, Hepatology, Endocrinology, Rheumatology and Infectious Diseases, University Hospital Regensburg, Regensburg, Germany.
Gastrointest Endosc. 2025 Apr;101(4):830-842.e2. doi: 10.1016/j.gie.2024.09.001. Epub 2024 Sep 13.
Deep learning algorithms gained attention for detection (computer-aided detection [CADe]) of biliary tract cancer in digital single-operator cholangioscopy (dSOC). We developed a multimodal convolutional neural network (CNN) for detection (CADe), characterization and discriminating (computer-aided diagnosis [CADx]) between malignant, inflammatory, and normal biliary tissue in raw dSOC videos. In addition, clinical metadata were included in the CNN algorithm to overcome limitations of image-only models.
Based on dSOC videos and images of 111 patients (total of 15,158 still frames), a real-time CNN-based algorithm for CADe and CADx was developed and validated. We established an image-only model and metadata injection approach. In addition, frame-wise and case-based predictions on complete dSOC video sequences were validated. Model embeddings were visualized, and class activation maps highlighted relevant image regions.
The concatenation-based CADx approach achieved a per-frame area under the receiver-operating characteristic curve of .871, sensitivity of .809 (95% CI, .784-.832), specificity of .773 (95% CI, .761-.785), positive predictive value of .450 (95% CI, .423-.467), and negative predictive value of .946 (95% CI, .940-.954) with respect to malignancy on 5715 test frames from complete videos of 20 patients. For case-based diagnosis using average prediction scores, 6 of 8 malignant cases and all 12 benign cases were identified correctly.
Our algorithm distinguishes malignant and inflammatory bile duct lesions in dSOC videos, indicating the potential of CNN-based diagnostic support systems for both CADe and CADx. The integration of non-image data can improve CNN-based support systems, targeting current challenges in the assessment of biliary strictures.
深度学习算法在数字单操作者胆管镜检查（dSOC）中用于胆管癌检测（计算机辅助检测[CADe]）受到关注。我们开发了一种多模态卷积神经网络（CNN），用于在原始dSOC视频中检测（CADe）、表征和区分（计算机辅助诊断[CADx]）恶性、炎症性和正常胆管组织。此外，临床元数据被纳入CNN算法以克服仅图像模型的局限性。
基于111例患者的dSOC视频和图像（共15158帧静止图像），开发并验证了一种基于实时CNN的CADe和CADx算法。我们建立了仅图像模型和元数据注入方法。此外，对完整dSOC视频序列进行逐帧和基于病例的预测验证。对模型嵌入进行可视化，并通过类激活映射突出显示相关图像区域。
基于串联的CADx方法在来自20例患者完整视频的5715个测试帧上，针对恶性肿瘤的受试者操作特征曲线下每帧面积为0.871，灵敏度为0.809（95%CI，0.784 - 0.832），特异性为0.773（95%CI，0.761 - 0.785），阳性预测值为0.450（95%CI，0.423 - 0.467），阴性预测值为0.946（95%CI，0.940 - 0.954）。对于使用平均预测分数的基于病例的诊断，8例恶性病例中的6例和所有12例良性病例被正确识别。
我们的算法可区分dSOC视频中的恶性和炎症性胆管病变，表明基于CNN的诊断支持系统在CADe和CADx方面的潜力。非图像数据的整合可改善基于CNN的支持系统，应对当前胆管狭窄评估中的挑战。
