Department for General, Visceral and Transplant Surgery, Heidelberg University Hospital, Heidelberg, Germany.
National Center for Tumor Diseases (NCT), Heidelberg, Germany.
Surg Endosc. 2024 Aug;38(8):4316-4328. doi: 10.1007/s00464-024-10958-w. Epub 2024 Jun 13.
BACKGROUND: Laparoscopic cholecystectomy is a very frequent surgical procedure. However, in an ageing society, less surgical staff will need to perform surgery on patients. Collaborative surgical robots (cobots) could address surgical staff shortages and workload. To achieve context-awareness for surgeon-robot collaboration, the intraoperative action workflow recognition is a key challenge. METHODS: A surgical process model was developed for intraoperative surgical activities including actor, instrument, action and target in laparoscopic cholecystectomy (excluding camera guidance). These activities, as well as instrument presence and surgical phases were annotated in videos of laparoscopic cholecystectomy performed on human patients (n = 10) and on explanted porcine livers (n = 10). The machine learning algorithm Distilled-Swin was trained on our own annotated dataset and the CholecT45 dataset. The validation of the model was conducted using a fivefold cross-validation approach. RESULTS: In total, 22,351 activities were annotated with a cumulative duration of 24.9 h of video segments. The machine learning algorithm trained and validated on our own dataset scored a mean average precision (mAP) of 25.7% and a top K = 5 accuracy of 85.3%. With training and validation on our dataset and CholecT45, the algorithm scored a mAP of 37.9%. CONCLUSIONS: An activity model was developed and applied for the fine-granular annotation of laparoscopic cholecystectomies in two surgical settings. A machine recognition algorithm trained on our own annotated dataset and CholecT45 achieved a higher performance than training only on CholecT45 and can recognize frequently occurring activities well, but not infrequent activities. The analysis of an annotated dataset allowed for the quantification of the potential of collaborative surgical robots to address the workload of surgical staff. If collaborative surgical robots could grasp and hold tissue, up to 83.5% of the assistant's tissue interacting tasks (i.e. excluding camera guidance) could be performed by robots.
背景：腹腔镜胆囊切除术是一种非常常见的手术。然而，在老龄化社会中，需要更少的外科医生为患者做手术。协作式手术机器人（cobot）可以解决外科医生短缺和工作负荷的问题。为了实现外科医生-机器人协作的上下文感知，术中动作工作流程识别是一个关键挑战。
方法：为腹腔镜胆囊切除术（不包括摄像指导）中的手术活动（包括演员、器械、动作和目标）开发了手术过程模型。在对人类患者（n=10）和离体猪肝脏（n=10）进行的腹腔镜胆囊切除术中的视频中，对这些活动以及器械的存在和手术阶段进行了注释。在我们自己的注释数据集和 CholecT45 数据集上，使用机器学习算法 Distilled-Swin 进行了训练。通过五重交叉验证方法对模型进行了验证。
结果：共注释了 22351 个活动，视频片段的总时长为 24.9 小时。在我们自己的数据集和 CholecT45 上进行训练和验证的机器学习算法的平均精度（mAP）得分为 25.7%，K=5 的准确率为 85.3%。在我们的数据集和 CholecT45 上进行训练和验证的算法的 mAP 得分为 37.9%。
结论：开发并应用了一种活动模型，用于在两种手术环境下对腹腔镜胆囊切除术进行精细粒度的注释。在我们自己的注释数据集和 CholecT45 上训练的机器识别算法的性能优于仅在 CholecT45 上训练的算法，并且可以很好地识别常见的活动，但不能识别不常见的活动。对注释数据集的分析允许量化协作式手术机器人解决外科医生工作负荷的潜力。如果协作式手术机器人能够抓取和握持组织，那么机器人可以完成助手 83.5%的组织交互任务（即不包括摄像指导）。
Cochrane Database Syst Rev. 2014-3-12
JBJS Essent Surg Tech. 2025-8-15
Cochrane Database Syst Rev. 2024-7-4
Int J Comput Assist Radiol Surg. 2024-7
Patterns (N Y). 2023-8-4
Med Image Anal. 2023-10
Med Image Anal. 2023-5
NPJ Digit Med. 2022-12-22
Nat Rev Gastroenterol Hepatol. 2023-3
NPJ Digit Med. 2022-10-28
Zotero 插件
