Surgical Device Innovation Office, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan.
Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan.
JAMA Surg. 2023 Aug 1;158(8):e231131. doi: 10.1001/jamasurg.2023.1131. Epub 2023 Aug 9.
Automatic surgical skill assessment with artificial intelligence (AI) is more objective than manual video review-based skill assessment and can reduce human burden. Standardization of surgical field development is an important aspect of this skill assessment.
To develop a deep learning model that can recognize the standardized surgical fields in laparoscopic sigmoid colon resection and to evaluate the feasibility of automatic surgical skill assessment based on the concordance of the standardized surgical field development using the proposed deep learning model.
DESIGN, SETTING, AND PARTICIPANTS: This retrospective diagnostic study used intraoperative videos of laparoscopic colorectal surgery submitted to the Japan Society for Endoscopic Surgery between August 2016 and November 2017. Data were analyzed from April 2020 to September 2022.
Videos of surgery performed by expert surgeons with Endoscopic Surgical Skill Qualification System (ESSQS) scores higher than 75 were used to construct a deep learning model able to recognize a standardized surgical field and output its similarity to standardized surgical field development as an AI confidence score (AICS). Other videos were extracted as the validation set.
Videos with scores less than or greater than 2 SDs from the mean were defined as the low- and high-score groups, respectively. The correlation between AICS and ESSQS score and the screening performance using AICS for low- and high-score groups were analyzed.
The sample included 650 intraoperative videos, 60 of which were used for model construction and 60 for validation. The Spearman rank correlation coefficient between the AICS and ESSQS score was 0.81. The receiver operating characteristic (ROC) curves for the screening of the low- and high-score groups were plotted, and the areas under the ROC curve for the low- and high-score group screening were 0.93 and 0.94, respectively.
The AICS from the developed model strongly correlated with the ESSQS score, demonstrating the model's feasibility for use as a method of automatic surgical skill assessment. The findings also suggest the feasibility of the proposed model for creating an automated screening system for surgical skills and its potential application to other types of endoscopic procedures.
基于人工智能（AI）的自动手术技能评估比基于手动视频审查的技能评估更客观，并且可以减轻人力负担。手术领域的标准化是这种技能评估的一个重要方面。
开发一种能够识别腹腔镜乙状结肠切除术标准化手术领域的深度学习模型，并评估基于所提出的深度学习模型的标准化手术领域发展一致性的自动手术技能评估的可行性。
设计、设置和参与者：这项回顾性诊断研究使用了 2016 年 8 月至 2017 年 11 月期间向日本内镜外科学会提交的腹腔镜结直肠手术的术中视频。数据的分析于 2020 年 4 月至 2022 年 9 月进行。
使用专家外科医生的手术视频，这些外科医生的内镜手术技能资格系统（ESSQS）评分高于 75 分，来构建一种能够识别标准化手术领域并将其与标准化手术领域发展的相似性作为人工智能置信度评分（AICS）输出的深度学习模型。其他视频被提取为验证集。
定义得分低于或高于平均值 2 个标准差的视频为低得分组和高得分组。分析了 AICS 与 ESSQS 评分之间的相关性，以及使用 AICS 对低得分组和高得分组的筛选性能。
该样本包括 650 个术中视频，其中 60 个用于模型构建，60 个用于验证。AICS 与 ESSQS 评分之间的斯皮尔曼等级相关系数为 0.81。绘制了用于筛选低得分组和高得分组的受试者工作特征（ROC）曲线，低得分组和高得分组的 ROC 曲线下面积分别为 0.93 和 0.94。
所开发模型的 AICS 与 ESSQS 评分强烈相关，表明该模型在自动手术技能评估方法中的可行性。研究结果还表明，所提出的模型用于创建手术技能自动筛选系统的可行性及其在其他类型的内镜手术中的潜在应用。
