Fukuta Atsuhisa, Yamashita Shogo, Maniwa Junnosuke, Tamaki Akihiko, Kondo Takuya, Kawakubo Naonori, Nagata Kouji, Matsuura Toshiharu, Tajiri Tatsuro
Department of Pediatric Surgery, Reproductive and Developmental Medicine, Faculty of Medical Sciences, Kyushu University, 3-1-1 Maidashi, Higashi-ku, Fukuoka, 812-8582, Japan.
Exawizards Inc., Tokyo, Japan.
Int J Comput Assist Radiol Surg. 2025 Mar;20(3):597-603. doi: 10.1007/s11548-024-03253-5. Epub 2024 Aug 19.
The development of innovative solutions, such as simulator training and artificial intelligence (AI)-powered tutoring systems, has significantly changed surgical trainees' environments to receive the intraoperative instruction necessary for skill acquisition. In this study, we developed a new objective assessment system using AI for forceps manipulation in a surgical training simulator.
Laparoscopic exercises were recorded using an iPad®, which provided top and side views. Top-view movies were used for AI learning of forceps trajectory. Side-view movies were used as supplementary information to assess the situation. We used an AI-based posture estimation method, DeepLabCut (DLC), to recognize and positionally measure the forceps in the operating field. Tracking accuracy was quantitatively evaluated by calculating the pixel differences between the annotation points and the points predicted by the AI model. Tracking stability at specified key points was verified to assess the AI model.
We selected a random sample to evaluate tracking accuracy quantitatively. This sample comprised 5% of the frames not used for AI training from the complete set of video frames. We compared the AI detection positions and correct positions and found an average pixel discrepancy of 9.2. The qualitative evaluation of the tracking stability was good at the forceps hinge; however, forceps tip tracking was unstable during rotation.
The AI-based forceps tracking system can visualize and evaluate laparoscopic surgical skills. Improvements in the proposed system and AI self-learning are expected to enable it to distinguish the techniques of expert and novice surgeons accurately. This system is a useful tool for surgeon training and assessment.
创新解决方案的发展，如模拟器训练和人工智能（AI）驱动的辅导系统，已显著改变了外科受训人员获取技能所需术中指导的环境。在本研究中，我们开发了一种使用人工智能的新型客观评估系统，用于手术训练模拟器中的镊子操作。
使用iPad®记录腹腔镜练习，该设备可提供顶视图和侧视图。顶视图视频用于人工智能对镊子轨迹的学习。侧视图视频用作评估情况的补充信息。我们使用基于人工智能的姿态估计方法DeepLabCut（DLC）来识别手术视野中的镊子并进行位置测量。通过计算注释点与人工智能模型预测点之间的像素差异来定量评估跟踪准确性。在指定关键点验证跟踪稳定性以评估人工智能模型。
我们选择了一个随机样本进行跟踪准确性的定量评估。该样本包括从完整视频帧集中未用于人工智能训练的5%的帧。我们比较了人工智能检测位置和正确位置，发现平均像素差异为9.2。在镊子铰链处，跟踪稳定性的定性评估良好；然而，在旋转过程中镊子尖端跟踪不稳定。
基于人工智能的镊子跟踪系统可以可视化和评估腹腔镜手术技能。预计所提出系统和人工智能自学的改进将使其能够准确区分专家和新手外科医生的技术。该系统是外科医生培训和评估的有用工具。
