Department for Translational Surgical Oncology, National Center for Tumor Diseases (NCT), Partner Site Dresden, Dresden, Germany.
Department of General, Visceral and Transplant Surgery, University of Heidelberg, Heidelberg, Germany.
Int J Comput Assist Radiol Surg. 2019 Jun;14(6):1089-1095. doi: 10.1007/s11548-019-01966-6. Epub 2019 Apr 9.
The course of surgical procedures is often unpredictable, making it difficult to estimate the duration of procedures beforehand. This uncertainty makes scheduling surgical procedures a difficult task. A context-aware method that analyses the workflow of an intervention online and automatically predicts the remaining duration would alleviate these problems. As basis for such an estimate, information regarding the current state of the intervention is a requirement.
Today, the operating room contains a diverse range of sensors. During laparoscopic interventions, the endoscopic video stream is an ideal source of such information. Extracting quantitative information from the video is challenging though, due to its high dimensionality. Other surgical devices (e.g., insufflator, lights, etc.) provide data streams which are, in contrast to the video stream, more compact and easier to quantify. Though whether such streams offer sufficient information for estimating the duration of surgery is uncertain. In this paper, we propose and compare methods, based on convolutional neural networks, for continuously predicting the duration of laparoscopic interventions based on unlabeled data, such as from endoscopic image and surgical device streams.
The methods are evaluated on 80 recorded laparoscopic interventions of various types, for which surgical device data and the endoscopic video streams are available. Here the combined method performs best with an overall average error of 37% and an average halftime error of approximately 28%.
In this paper, we present, to our knowledge, the first approach for online procedure duration prediction using unlabeled endoscopic video data and surgical device data in a laparoscopic setting. Furthermore, we show that a method incorporating both vision and device data performs better than methods based only on vision, while methods only based on tool usage and surgical device data perform poorly, showing the importance of the visual channel.
手术过程通常难以预测，因此很难事先估计手术时间。这种不确定性使得安排手术程序成为一项困难的任务。一种上下文感知的方法，可以在线分析干预的工作流程，并自动预测剩余时间，将缓解这些问题。作为这种估计的基础，需要有关干预当前状态的信息。
如今，手术室中包含了多种传感器。在腹腔镜干预期间，内窥镜视频流是此类信息的理想来源。但是，由于其高维度，从视频中提取定量信息具有挑战性。其他手术设备（例如，注气机、灯等）提供的数据流与视频流相比，更紧凑且更容易量化。尽管这些流是否提供足够的信息来估计手术持续时间还不确定。在本文中，我们提出并比较了基于卷积神经网络的方法，这些方法可基于未标记的数据（例如内窥镜图像和手术设备流）连续预测腹腔镜干预的持续时间。
该方法在 80 个记录的各种类型的腹腔镜干预中进行了评估，这些干预中可获得手术设备数据和内窥镜视频流。在这里，组合方法的总体平均误差为 37%，平均半衰期误差约为 28%，性能最佳。
在本文中，我们提出了一种在腹腔镜环境中使用未标记的内窥镜视频数据和手术设备数据进行在线手术持续时间预测的方法，据我们所知，这是首次提出这种方法。此外，我们表明，结合使用视觉和设备数据的方法比仅基于视觉的方法表现更好，而仅基于工具使用和手术设备数据的方法表现不佳，这表明视觉通道的重要性。
