Research Group MITI, Klinikum rechts der Isar, TUM School of Medicine and Health, Technical University of Munich, Munich, Germany.
Department of Surgery, Klinikum rechts der Isar, TUM School of Medicine and Health, Technical University of Munich, Munich, Germany.
Int J Comput Assist Radiol Surg. 2024 Oct;19(10):1939-1945. doi: 10.1007/s11548-024-03220-0. Epub 2024 Jul 15.
Video-based intra-abdominal instrument tracking for laparoscopic surgeries is a common research area. However, the tracking can only be done with instruments that are actually visible in the laparoscopic image. By using extra-abdominal cameras to detect trocars and classify their occupancy state, additional information about the instrument location, whether an instrument is still in the abdomen or not, can be obtained. This can enhance laparoscopic workflow understanding and enrich already existing intra-abdominal solutions.
A data set of four laparoscopic surgeries recorded with two time-synchronized extra-abdominal 2D cameras was generated. The preprocessed and annotated data were used to train a deep learning-based network architecture consisting of a trocar detection, a centroid tracker and a temporal model to provide the occupancy state of all trocars during the surgery.
The trocar detection model achieves an F1 score of . The prediction of the occupancy state yields an F1 score of , providing a first step towards enhanced surgical workflow understanding.
The current method shows promising results for the extra-abdominal tracking of trocars and their occupancy state. Future advancements include the enlargement of the data set and incorporation of intra-abdominal imaging to facilitate accurate assignment of instruments to trocars.
基于视频的腹腔镜手术器械腹腔内跟踪是一个常见的研究领域。然而，跟踪只能针对腹腔镜图像中实际可见的器械进行。通过使用腹腔外摄像机来检测套管针并对其占用状态进行分类，可以获得有关器械位置的更多信息，例如器械是否仍在腹部内。这可以增强对腹腔镜手术流程的理解，并丰富现有的腹腔内解决方案。
生成了一个由四个腹腔镜手术记录组成的数据集，这些手术记录使用两个时间同步的腹腔外 2D 摄像机进行记录。预处理和注释的数据用于训练一个基于深度学习的网络架构，该架构包括套管针检测、质心跟踪器和时间模型，以提供手术过程中所有套管针的占用状态。
套管针检测模型的 F1 得分为 。对占用状态的预测得到了 F1 得分为 ，这是朝着增强手术流程理解迈出的第一步。
当前的方法在腹腔外跟踪套管针及其占用状态方面显示出有前景的结果。未来的发展包括扩大数据集并纳入腹腔内成像，以方便将器械准确分配给套管针。
