Annu Int Conf IEEE Eng Med Biol Soc. 2022 Jul;2022:4839-4842. doi: 10.1109/EMBC48229.2022.9871588.
In image-guided surgery, endoscope tracking and surgical scene reconstruction are critical, yet equally challenging tasks. We present a hybrid visual odometry and reconstruction framework for stereo endoscopy that leverages unsupervised learning-based and traditional optical flow methods to enable concurrent endoscope tracking and dense scene reconstruction. More specifically, to reconstruct texture-less tissue surfaces, we use an unsupervised learning-based optical flow method to estimate dense depth maps from stereo images. Robust 3D landmarks are selected from the dense depth maps and tracked via the Kanade-Lucas-Tomasi tracking algorithm. The hybrid visual odometry also benefits from traditional visual odometry modules, such as keyframe insertion and local bundle adjustment. We evaluate the proposed framework on endoscopic video sequences openly available via the SCARED dataset against both ground truth data, as well as two other state-of-the-art methods - ORB-SLAM2 and Endo-depth. Our proposed method achieved comparable results in terms of both RMS Absolute Trajectory Error and Cloud-to-Mesh RMS Error, suggesting its potential to enable accurate endoscope tracking and scene reconstruction.
在图像引导手术中，内窥镜跟踪和手术场景重建是至关重要的，但同样具有挑战性的任务。我们提出了一种用于立体内窥镜的混合视觉里程计和重建框架，该框架利用基于无监督学习的和传统的光流方法来实现内窥镜的跟踪和密集场景的重建。更具体地说，为了重建无纹理的组织表面，我们使用基于无监督学习的光流方法从立体图像中估计密集的深度图。从密集深度图中选择鲁棒的 3D 地标，并通过 Kanade-Lucas-Tomasi 跟踪算法进行跟踪。混合视觉里程计还受益于传统的视觉里程计模块，例如关键帧插入和局部束调整。我们在 SCARED 数据集上的内窥镜视频序列上评估了所提出的框架，该数据集既可以与地面实况数据进行比较，也可以与其他两种最先进的方法（ORB-SLAM2 和 Endo-depth）进行比较。我们提出的方法在 RMS 绝对轨迹误差和云到网格 RMS 误差方面都取得了相当的结果，这表明它有可能实现准确的内窥镜跟踪和场景重建。
