Huang Baoru, Nguyen Anh, Wang Siyao, Wang Ziyang, Mayer Erik, Tuch David, Vyas Kunal, Giannarou Stamatia, Elson Daniel S
The Hamlyn Centre for Robotic Surgery, Imperial College London, SW7 2AZ, UK.
Department of Surgery & Cancer, Imperial College London, SW7 2AZ, UK.
IEEE Trans Med Robot Bionics. 2022 May;4(2):335-338. doi: 10.1109/TMRB.2022.3170215.
Surgical instrument segmentation and depth estimation are crucial steps to improve autonomy in robotic surgery. Most recent works treat these problems separately, making the deployment challenging. In this paper, we propose a unified framework for depth estimation and surgical tool segmentation in laparoscopic images. The network has an encoder-decoder architecture and comprises two branches for simultaneously performing depth estimation and segmentation. To train the network end to end, we propose a new multi-task loss function that effectively learns to estimate depth in an unsupervised manner, while requiring only semi-ground truth for surgical tool segmentation. We conducted extensive experiments on different datasets to validate these findings. The results showed that the end-to-end network successfully improved the state-of-the-art for both tasks while reducing the complexity during their deployment.
手术器械分割和深度估计是提高机器人手术自主性的关键步骤。最近的工作分别处理这些问题，这使得部署具有挑战性。在本文中，我们提出了一个用于腹腔镜图像深度估计和手术工具分割的统一框架。该网络具有编码器-解码器架构，由两个分支组成，用于同时执行深度估计和分割。为了端到端地训练网络，我们提出了一种新的多任务损失函数，该函数有效地以无监督方式学习估计深度，同时仅需要手术工具分割的半地面真值。我们在不同数据集上进行了广泛的实验以验证这些发现。结果表明，端到端网络在这两项任务上均成功改进了现有技术，同时降低了部署过程中的复杂性。
