Lee Eung-Joo, Plishker William, Liu Xinyang, Bhattacharyya Shuvra S, Shekhar Raj
Department of Electrical and Computer Engineering and the Institute for Advanced Computer Studies, University of Maryland, College Park, MD, USA.
IGI Technologies, Inc., College Park, MD, USA.
Healthc Technol Lett. 2019 Nov 26;6(6):231-236. doi: 10.1049/htl.2019.0083. eCollection 2019 Dec.
Surgical tool tracking has a variety of applications in different surgical scenarios. Electromagnetic (EM) tracking can be utilised for tool tracking, but the accuracy is often limited by magnetic interference. Vision-based methods have also been suggested; however, tracking robustness is limited by specular reflection, occlusions, and blurriness observed in the endoscopic image. Recently, deep learning-based methods have shown competitive performance on segmentation and tracking of surgical tools. The main bottleneck of these methods lies in acquiring a sufficient amount of pixel-wise, annotated training data, which demands substantial labour costs. To tackle this issue, the authors propose a weakly supervised method for surgical tool segmentation and tracking based on hybrid sensor systems. They first generate semantic labellings using EM tracking and laparoscopic image processing concurrently. They then train a light-weight deep segmentation network to obtain a binary segmentation mask that enables tool tracking. To the authors' knowledge, the proposed method is the first to integrate EM tracking and laparoscopic image processing for generation of training labels. They demonstrate that their framework achieves accurate, automatic tool segmentation (i.e. without any manual labelling of the surgical tool to be tracked) and robust tool tracking in laparoscopic image sequences.
手术工具跟踪在不同的手术场景中有多种应用。电磁（EM）跟踪可用于工具跟踪，但精度常常受到磁干扰的限制。也有人提出基于视觉的方法；然而，跟踪的鲁棒性受到内窥镜图像中观察到的镜面反射、遮挡和模糊的限制。最近，基于深度学习的方法在手术工具的分割和跟踪方面表现出了有竞争力的性能。这些方法的主要瓶颈在于获取足够数量的逐像素标注的训练数据，这需要大量的人力成本。为了解决这个问题，作者提出了一种基于混合传感器系统的用于手术工具分割和跟踪的弱监督方法。他们首先同时使用EM跟踪和腹腔镜图像处理生成语义标签。然后他们训练一个轻量级的深度分割网络以获得能够实现工具跟踪的二进制分割掩码。据作者所知，所提出的方法是首次将EM跟踪和腹腔镜图像处理集成用于生成训练标签。他们证明了他们的框架在腹腔镜图像序列中实现了准确、自动的工具分割（即无需对要跟踪的手术工具进行任何手动标注）以及鲁棒的工具跟踪。
