Surgical Device Innovation Office, National Cancer Center Hospital East, 6-5-1, Kashiwanoha, Kashiwa, Chiba, 277-8577, Japan.
Department of Colorectal Surgery, National Cancer Center Hospital East, 6-5-1, Kashiwanoha, Kashiwa, Chiba, 277-8577, Japan.
Sci Rep. 2022 Jul 22;12(1):12575. doi: 10.1038/s41598-022-16923-8.
Clarifying the generalizability of deep-learning-based surgical-instrument segmentation networks in diverse surgical environments is important in recognizing the challenges of overfitting in surgical-device development. This study comprehensively evaluated deep neural network generalizability for surgical instrument segmentation using 5238 images randomly extracted from 128 intraoperative videos. The video dataset contained 112 laparoscopic colorectal resection, 5 laparoscopic distal gastrectomy, 5 laparoscopic cholecystectomy, and 6 laparoscopic partial hepatectomy cases. Deep-learning-based surgical-instrument segmentation was performed for test sets with (1) the same conditions as the training set; (2) the same recognition target surgical instrument and surgery type but different laparoscopic recording systems; (3) the same laparoscopic recording system and surgery type but slightly different recognition target laparoscopic surgical forceps; (4) the same laparoscopic recording system and recognition target surgical instrument but different surgery types. The mean average precision and mean intersection over union for test sets 1, 2, 3, and 4 were 0.941 and 0.887, 0.866 and 0.671, 0.772 and 0.676, and 0.588 and 0.395, respectively. Therefore, the recognition accuracy decreased even under slightly different conditions. The results of this study reveal the limited generalizability of deep neural networks in the field of surgical artificial intelligence and caution against deep-learning-based biased datasets and models.Trial Registration Number: 2020-315, date of registration: October 5, 2020.
明确深度学习为基础的手术器械分割网络在不同手术环境中的泛化能力，对于认识手术器械开发中过拟合的挑战非常重要。本研究使用从 128 个术中视频中随机提取的 5238 张图像，全面评估了深度学习网络在手术器械分割中的泛化能力。视频数据集包含 112 例腹腔镜结直肠切除术、5 例腹腔镜远端胃切除术、5 例腹腔镜胆囊切除术和 6 例腹腔镜部分肝切除术病例。对于测试集，采用了以下 4 种情况进行基于深度学习的手术器械分割：(1)与训练集相同的条件；(2)相同的识别目标手术器械和手术类型，但不同的腹腔镜记录系统；(3)相同的腹腔镜记录系统和手术类型，但识别目标腹腔镜手术器械略有不同；(4)相同的腹腔镜记录系统和识别目标手术器械，但不同的手术类型。测试集 1、2、3 和 4 的平均精度和平均交并比分别为 0.941 和 0.887、0.866 和 0.671、0.772 和 0.676 以及 0.588 和 0.395。因此，即使在条件略有不同的情况下，识别准确率也会下降。本研究结果揭示了深度学习网络在手术人工智能领域的泛化能力有限，并提醒人们注意基于深度学习的有偏数据集和模型。
2020-315，注册日期：2020 年 10 月 5 日。
