Department of Electronics Information and Bioengineering, Politecnico di Milano, P.zza L. Da Vinci, 32, Milano 20133, Italy; Department of Advanced Robotics, Istituto Italiano di Tecnologia, via Morego, 30, Genova, 16163, Italy.
Centre for Medical Image Computing, Department of Computer Science, University College London, United Kingdom.
Med Image Anal. 2018 Apr;45:13-23. doi: 10.1016/j.media.2017.12.010. Epub 2017 Dec 22.
Despite the benefits introduced by robotic systems in abdominal Minimally Invasive Surgery (MIS), major complications can still affect the outcome of the procedure, such as intra-operative bleeding. One of the causes is attributed to accidental damages to arteries or veins by the surgical tools, and some of the possible risk factors are related to the lack of sub-surface visibilty. Assistive tools guiding the surgical gestures to prevent these kind of injuries would represent a relevant step towards safer clinical procedures. However, it is still challenging to develop computer vision systems able to fulfill the main requirements: (i) long term robustness, (ii) adaptation to environment/object variation and (iii) real time processing. The purpose of this paper is to develop computer vision algorithms to robustly track soft tissue areas (Safety Area, SA), defined intra-operatively by the surgeon based on the real-time endoscopic images, or registered from a pre-operative surgical plan. We propose a framework to combine an optical flow algorithm with a tracking-by-detection approach in order to be robust against failures caused by: (i) partial occlusion, (ii) total occlusion, (iii) SA out of the field of view, (iv) deformation, (v) illumination changes, (vi) abrupt camera motion, (vii), blur and (viii) smoke. A Bayesian inference-based approach is used to detect the failure of the tracker, based on online context information. A Model Update Strategy (MUpS) is also proposed to improve the SA re-detection after failures, taking into account the changes of appearance of the SA model due to contact with instruments or image noise. The performance of the algorithm was assessed on two datasets, representing ex-vivo organs and in-vivo surgical scenarios. Results show that the proposed framework, enhanced with MUpS, is capable of maintain high tracking performance for extended periods of time ( ≃ 4 min - containing the aforementioned events) with high precision (0.7) and recall (0.8) values, and with a recovery time after a failure between 1 and 8 frames in the worst case.
尽管机器人系统在腹部微创手术（MIS）中带来了诸多益处，但仍可能出现严重并发症，影响手术效果，如术中出血。导致这些并发症的原因之一是手术器械意外损伤动脉或静脉，而一些可能的风险因素与亚表面可视性不足有关。辅助工具可以引导手术动作，防止此类损伤，这将是迈向更安全临床手术的重要一步。然而，开发能够满足以下主要要求的计算机视觉系统仍然具有挑战性：（i）长期鲁棒性，（ii）适应环境/物体变化，以及（iii）实时处理。本文旨在开发计算机视觉算法，以稳健地跟踪软组织区域（安全区域，SA），这些区域由外科医生根据实时内窥镜图像或从术前手术计划中进行定义。我们提出了一种框架，将光流算法与基于检测的跟踪方法相结合，以抵抗以下因素导致的跟踪失败：（i）部分遮挡，（ii）完全遮挡，（iii）SA 超出视场，（iv）变形，（v）光照变化，（vi）相机剧烈运动，（vii）模糊，（viii）烟雾。基于在线上下文信息，采用贝叶斯推理方法来检测跟踪器的失败。还提出了一种模型更新策略（MUpS），以在失败后改进 SA 的重新检测，考虑到由于与器械接触或图像噪声导致的 SA 模型外观变化。该算法的性能在两个数据集上进行了评估，代表了离体器官和体内手术场景。结果表明，增强了 MUpS 的所提出的框架能够长时间（包含上述事件）保持高精度（0.7）和召回率（0.8）的高跟踪性能，在最坏情况下，故障后的恢复时间在 1 到 8 帧之间。
