Sestini Luca, Rosa Benoit, De Momi Elena, Ferrigno Giancarlo, Padoy Nicolas
ICube, University of Strasbourg, CNRS, France; Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milano, Italy.
ICube, University of Strasbourg, CNRS, France.
Med Image Anal. 2025 Apr;101:103471. doi: 10.1016/j.media.2025.103471. Epub 2025 Jan 22.
Instance segmentation of surgical instruments is a long-standing research problem, crucial for the development of many applications for computer-assisted surgery. This problem is commonly tackled via fully-supervised training of deep learning models, requiring expensive pixel-level annotations to train. In this work, we develop a framework for instance segmentation not relying on spatial annotations for training. Instead, our solution only requires binary tool masks, obtainable using recent unsupervised approaches, and tool presence labels, freely obtainable in robot-assisted surgery. Based on the binary mask information, our solution learns to extract individual tool instances from single frames, and to encode each instance into a compact vector representation, capturing its semantic features. Such representations guide the automatic selection of a tiny number of instances (8 only in our experiments), displayed to a human operator for tool-type labelling. The gathered information is finally used to match each training instance with a tool presence label, providing an effective supervision signal to train a tool instance classifier. We validate our framework on the EndoVis 2017 and 2018 segmentation datasets. We provide results using binary masks obtained either by manual annotation or as predictions of an unsupervised binary segmentation model. The latter solution yields an instance segmentation approach completely free from spatial annotations, outperforming several state-of-the-art fully-supervised segmentation approaches.
手术器械的实例分割是一个长期存在的研究问题，对于许多计算机辅助手术应用的开发至关重要。这个问题通常通过深度学习模型的全监督训练来解决，这需要昂贵的像素级注释来进行训练。在这项工作中，我们开发了一个实例分割框架，该框架不依赖于用于训练的空间注释。相反，我们的解决方案只需要二进制工具掩码（可使用最近的无监督方法获得）和工具存在标签（在机器人辅助手术中可免费获得）。基于二进制掩码信息，我们的解决方案学会从单帧中提取单个工具实例，并将每个实例编码为紧凑的向量表示，以捕获其语义特征。这样的表示指导自动选择少量实例（在我们的实验中仅为8个），显示给人类操作员进行工具类型标注。最终，收集到的信息用于将每个训练实例与工具存在标签进行匹配，提供一个有效的监督信号来训练工具实例分类器。我们在EndoVis 2017和2018分割数据集上验证了我们的框架。我们提供了使用通过手动注释获得的二进制掩码或作为无监督二进制分割模型预测的二进制掩码得到的结果。后一种解决方案产生了一种完全不依赖空间注释的实例分割方法，其性能优于几种先进的全监督分割方法。
