State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin, China.
J Digit Imaging. 2022 Aug;35(4):923-937. doi: 10.1007/s10278-022-00616-9. Epub 2022 Mar 9.
Vision-based detection and tracking of surgical instrument are attractive because it relies purely on surgical instrument already in the operating scenario. The vision knowledge of the surgical instruments is a crucial piece of topic for surgical task understanding, autonomous robot control and human-robot collaborative surgeries to enhance surgical outcomes. In this work, a novel method has been demonstrated by developing a multitask lightweight deep neural network framework to explore surgical instrument articulated joint detection. The model has an end-to-end architecture with two branches, which share the same high-level visual features provided by a lightweight backbone while holding respective layers targeting for specific tasks. We have designed a novel subnetwork with joint detection branch and an instrument classification branch to sufficiently take advantage of the relatedness of surgical instrument presence detection and surgical instrument articulated joint detection tasks. The lightweight joint detection branch has been employed to efficiently locate the articulated joint position with simultaneously holding low computational cost. Moreover, the surgical instrument classification branch is introduced to boost the performance of joint detection. The two branches are merged to output the articulated joint location with respective instrument type. Extensive validation has been conducted to evaluate the proposed method. The results demonstrate promising performance of our proposed method. The work represents the feasibility to perform real-time surgical instrument articulated joint detection by taking advantage of the components of surgical robot system, contributing to the reference for further surgical intelligence.
基于视觉的手术器械检测和跟踪很有吸引力，因为它纯粹依赖于手术场景中已经存在的手术器械。手术器械的视觉知识是手术任务理解、自主机器人控制和人机协作手术的关键部分，以提高手术效果。在这项工作中，通过开发一种多任务轻量级深度神经网络框架，展示了一种新的方法，用于探索手术器械铰接关节检测。该模型具有端到端架构，有两个分支，它们共享由轻量级主干提供的相同高级视觉特征，同时保持各自针对特定任务的层。我们设计了一个具有关节检测分支和器械分类分支的新子网，以充分利用手术器械存在检测和手术器械铰接关节检测任务的相关性。轻量级关节检测分支被用于高效定位铰接关节位置，同时保持低计算成本。此外，引入了手术器械分类分支来提高关节检测的性能。这两个分支合并输出具有各自器械类型的铰接关节位置。已经进行了广泛的验证来评估所提出的方法。结果表明，我们提出的方法具有很有前景的性能。这项工作展示了通过利用手术机器人系统的组件进行实时手术器械铰接关节检测的可行性，为进一步的手术智能提供了参考。
