Guédon Annetje C P, Meij Senna E P, Osman Karim N M M H, Kloosterman Helena A, van Stralen Karlijn J, Grimbergen Matthijs C M, Eijsbouts Quirijn A J, van den Dobbelsteen John J, Twinanda Andru P
Department of Clinical Physics, Spaarne Gasthuis, Spaarnepoort 1, 2134TM, Hoofddorp, the Netherlands.
Department of Biomechanical Engineering, Delft University of Technology, Delft, the Netherlands.
Surg Endosc. 2021 Nov;35(11):6150-6157. doi: 10.1007/s00464-020-08110-5. Epub 2020 Nov 25.
Operating room planning is a complex task as pre-operative estimations of procedure duration have a limited accuracy. This is due to large variations in the course of procedures. Therefore, information about the progress of procedures is essential to adapt the daily operating room schedule accordingly. This information should ideally be objective, automatically retrievable and in real-time. Recordings made during endoscopic surgeries are a potential source of progress information. A trained observer is able to recognize the ongoing surgical phase from watching these videos. The introduction of deep learning techniques brought up opportunities to automatically retrieve information from surgical videos. The aim of this study was to apply state-of-the art deep learning techniques on a new set of endoscopic videos to automatically recognize the progress of a procedure, and to assess the feasibility of the approach in terms of performance, scalability and practical considerations.
A dataset of 33 laparoscopic cholecystectomies (LC) and 35 total laparoscopic hysterectomies (TLH) was used. The surgical tools that were used and the ongoing surgical phases were annotated in the recordings. Neural networks were trained on a subset of annotated videos. The automatic recognition of surgical tools and phases was then assessed on another subset. The scalability of the networks was tested and practical considerations were kept up.
The performance of the surgical tools and phase recognition reached an average precision and recall between 0.77 and 0.89. The scalability tests showed diverging results. Legal considerations had to be taken into account and a considerable amount of time was needed to annotate the datasets.
This study shows the potential of deep learning to automatically recognize information contained in surgical videos. This study also provides insights in the applicability of such a technique to support operating room planning.
手术室规划是一项复杂的任务，因为术前对手术时长的估计准确性有限。这是由于手术过程存在很大差异。因此，有关手术进展的信息对于相应调整每日手术室日程安排至关重要。理想情况下，该信息应客观、可自动获取且为实时信息。内镜手术过程中的记录是手术进展信息的一个潜在来源。经过训练的观察者能够通过观看这些视频识别正在进行的手术阶段。深度学习技术的引入带来了从手术视频中自动获取信息的机会。本研究的目的是将最先进的深度学习技术应用于一组新的内镜视频，以自动识别手术进展，并从性能、可扩展性和实际考虑等方面评估该方法的可行性。
使用了一个包含33例腹腔镜胆囊切除术（LC）和35例全腹腔镜子宫切除术（TLH）的数据集。记录中标注了所使用的手术工具和正在进行的手术阶段。神经网络在一部分标注视频上进行训练。然后在另一部分视频上评估手术工具和阶段的自动识别情况。测试了网络的可扩展性，并考虑了实际因素。
手术工具和阶段识别的性能达到了平均精度和召回率在0.77至0.89之间。可扩展性测试结果不一。必须考虑法律因素，并且标注数据集需要大量时间。
本研究展示了深度学习自动识别手术视频中所含信息的潜力。本研究还为这种支持手术室规划的技术的适用性提供了见解。
