Hegde Shruti R, Namazi Babak, Iyengar Niyenth, Cao Sarah, Desir Alexis, Marques Carolina, Mahnken Heidi, Dumas Ryan P, Sankaranarayanan Ganesh
Department of Surgery, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd., Dallas, TX, 75390-9159, USA.
Surg Endosc. 2024 Jan;38(1):158-170. doi: 10.1007/s00464-023-10482-3. Epub 2023 Nov 9.
Video-based review is paramount for operative performance assessment but can be laborious when performed manually. Hierarchical Task Analysis (HTA) is a well-known method that divides any procedure into phases, steps, and tasks. HTA requires large datasets of videos with consistent definitions at each level. Our aim was to develop an AI model for automated segmentation of phases, steps, and tasks for laparoscopic cholecystectomy videos using a standardized HTA.
A total of 160 laparoscopic cholecystectomy videos were collected from a publicly available dataset known as cholec80 and from our own institution. All videos were annotated for the beginning and ending of a predefined set of phases, steps, and tasks. Deep learning models were then separately developed and trained for the three levels using a 3D Convolutional Neural Network architecture.
Four phases, eight steps, and nineteen tasks were defined through expert consensus. The training set for our deep learning models contained 100 videos with an additional 20 videos for hyperparameter optimization and tuning. The remaining 40 videos were used for testing the performance. The overall accuracy for phases, steps, and tasks were 0.90, 0.81, and 0.65 with the average F1 score of 0.86, 0.76 and 0.48 respectively. Control of bleeding and bile spillage tasks were most variable in definition, operative management, and clinical relevance.
The use of hierarchical task analysis for surgical video analysis has numerous applications in AI-based automated systems. Our results show that our tiered method of task analysis can successfully be used to train a DL model.
基于视频的评估对于手术操作性能评估至关重要，但手动进行时可能很费力。层次任务分析（HTA）是一种众所周知的方法，它将任何手术过程分为阶段、步骤和任务。HTA需要每个级别都有一致定义的大量视频数据集。我们的目标是开发一种人工智能模型，用于使用标准化的HTA对腹腔镜胆囊切除术视频中的阶段、步骤和任务进行自动分割。
从一个名为cholec80的公开可用数据集以及我们自己的机构收集了总共160个腹腔镜胆囊切除术视频。所有视频都针对一组预定义的阶段、步骤和任务的开始和结束进行了标注。然后使用三维卷积神经网络架构分别为三个级别开发和训练深度学习模型。
通过专家共识定义了四个阶段、八个步骤和十九个任务。我们深度学习模型的训练集包含100个视频，另有20个视频用于超参数优化和调整。其余40个视频用于测试性能。阶段、步骤和任务的总体准确率分别为0.90、0.81和0.65，平均F1分数分别为0.86、0.76和0.48。出血和胆汁渗漏任务的控制在定义、手术管理和临床相关性方面变化最大。
将层次任务分析用于手术视频分析在基于人工智能的自动化系统中有许多应用。我们的结果表明，我们的分层任务分析方法可以成功用于训练深度学习模型。
