Systems and Control Group, Indian Institute of Technology Bombay, Mumbai, Maharastra, India.
Int J Med Robot. 2023 Aug;19(4):e2514. doi: 10.1002/rcs.2514. Epub 2023 Mar 28.
Image segmentation of instruments in the raw surgical videos is a critical component of intraoperative assistance softwares. Challenges include addressing rendered overlays occluding the instrument while providing pivotal input to instrument tracking frameworks and, train the segmentation process with limited labelled data available from surgical videos.
The proposed adversarial network, InstruSegNet uses unpaired training (eliminating need for massive paired data) for automated multi-class surgical instrument segmentation in raw surgical videos with complex backgrounds. The proposed method is applied for single/multiple robotic and rigid instruments and optimised on least square Generative Adversarial Networks loss.
Promising validation has been conducted on the publicly available dataset. Proposed approach for multi-class segmentation of robotic and rigid instruments meets outstanding performance in terms of accuracy and surpasses the existing methods.
This work facilitates segmenting instrument information without manual interventions from raw videos providing means to code surgeon's actions for developing intelligent assistance software.
在原始手术视频中对器械进行图像分割是手术辅助软件的关键组成部分。挑战包括解决渲染覆盖物遮挡器械的问题，同时为器械跟踪框架提供关键输入，以及利用从手术视频中获得的有限标记数据来训练分割过程。
所提出的对抗网络 InstruSegNet 使用非配对训练（消除对大量配对数据的需求），可在具有复杂背景的原始手术视频中自动对多类手术器械进行分割。该方法适用于单/多机器人和刚性器械，并在最小二乘生成对抗网络损失上进行了优化。
在公开数据集上进行了有前途的验证。所提出的用于机器人和刚性器械的多类分割方法在准确性方面表现出色，超过了现有方法。
这项工作无需人工干预即可从原始视频中分割器械信息，为开发智能辅助软件提供了对医生操作进行编码的手段。
