Department of Urology, CHU Pellegrin, Place Amélie Raba Léon F-33000 Bordeaux, France.
Department of Multidisciplinary Functional Explorations, AP-HP, Tenon Hospital, INSERM UMRS F-1155, Sorbonne University, Paris, France.
Phys Med Biol. 2022 Aug 16;67(16). doi: 10.1088/1361-6560/ac8592.
To assess the performance and added value of processing complete digital endoscopic video sequences for the automatic recognition of stone morphological features during a standard-of-care intra-operative session.A computer-aided video classifier was developed to predictthe morphology of stone using an intra-operative digital endoscopic video acquired in a clinical setting. Using dedicated artificial intelligence (AI) networks, the proposed pipeline selects adequate frames in steady sequences of the video, ensures the presence of (potentially fragmented) stones and predicts the stone morphologies on a frame-by-frame basis. The automatic endoscopic stone recognition (A-ESR) is subsequently carried out by mixing all collected morphological observations.The proposed technique was evaluated on pure (i.e. include one morphology) and mixed (i.e. include at least two morphologies) stones involving 'Ia/Calcium Oxalate Monohydrate' (COM), 'IIb/Calcium Oxalate Dihydrate' (COD) and 'IIIb/Uric Acid' (UA) morphologies. The gold standard ESR was provided by a trained endo-urologist and confirmed by microscopy and infra-red spectroscopy. For the AI-training, 585 static images were collected (349 and 236 observations of stone surface and section, respectively) and used. Using the proposed video classifier, 71 digital endoscopic videos were analyzed: 50 exhibited only one morphological type and 21 displayed two. Taken together, both pure and mixed stone types yielded a mean diagnostic performances as follows: balanced accuracy = [88 ± 6] (min = 81)%, sensitivity = [80 ± 13] (min = 69)%, specificity = [95 ± 2] (min = 92)%, precision = [78 ± 12] (min = 62)% and F1-score = [78 ± 7] (min = 69)%.These results demonstrate that AI applied on digital endoscopic video sequences is a promising tool for collecting morphological information during the time-course of the stone fragmentation process without resorting to any human intervention for stone delineation or the selection of adequate steady frames.
评估处理完整数字内窥镜视频序列以自动识别标准护理术中腔内会话中结石形态特征的性能和附加值。开发了一种计算机辅助视频分类器，使用在临床环境中获得的术中数字内窥镜视频来预测结石的形态。使用专用的人工智能 (AI) 网络，该管道选择视频中稳定序列中的适当帧，确保（潜在的）结石的存在，并逐帧预测结石形态。随后通过混合所有收集的形态观察结果来进行自动内窥镜结石识别 (A-ESR)。该技术在涉及 'Ia/草酸单水合物' (COM)、'IIb/草酸二水合物' (COD) 和 'IIIb/尿酸' (UA) 形态的纯（即包含一种形态）和混合（即包含至少两种形态）结石上进行了评估。由经过培训的内镜泌尿科医生提供的金标准 ESR，并通过显微镜和红外光谱进行确认。对于 AI 培训，共收集了 585 张静态图像（结石表面和结石剖面分别为 349 次和 236 次观察）并加以使用。使用提出的视频分类器分析了 71 个数字内窥镜视频：50 个仅显示一种形态类型，21 个显示两种形态类型。总的来说，纯结石和混合结石类型的平均诊断性能如下：平衡准确率 = [88 ± 6]（最小值 = 81）%，敏感度 = [80 ± 13]（最小值 = 69）%，特异性 = [95 ± 2]（最小值 = 92）%，精确率 = [78 ± 12]（最小值 = 62）%和 F1 分数 = [78 ± 7]（最小值 = 69）%。这些结果表明，应用于数字内窥镜视频序列的人工智能是一种很有前途的工具，可在不依赖于结石描绘或选择适当稳定帧的情况下，在结石碎裂过程的时间过程中收集形态信息。
