Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) and Department of Computer Science, University College London, London, UK.
Department of Mechanical Engineering, KU Leuven University, Leuven, Belgium.
Int J Comput Assist Radiol Surg. 2020 Nov;15(11):1807-1816. doi: 10.1007/s11548-020-02242-8. Epub 2020 Aug 17.
Fetoscopic laser photocoagulation is a minimally invasive surgical procedure used to treat twin-to-twin transfusion syndrome (TTTS), which involves localization and ablation of abnormal vascular connections on the placenta to regulate the blood flow in both fetuses. This procedure is particularly challenging due to the limited field of view, poor visibility, occasional bleeding, and poor image quality. Fetoscopic mosaicking can help in creating an image with the expanded field of view which could facilitate the clinicians during the TTTS procedure.
We propose a deep learning-based mosaicking framework for diverse fetoscopic videos captured from different settings such as simulation, phantoms, ex vivo, and in vivo environments. The proposed mosaicking framework extends an existing deep image homography model to handle video data by introducing the controlled data generation and consistent homography estimation modules. Training is performed on a small subset of fetoscopic images which are independent of the testing videos.
We perform both quantitative and qualitative evaluations on 5 diverse fetoscopic videos (2400 frames) that captured different environments. To demonstrate the robustness of the proposed framework, a comparison is performed with the existing feature-based and deep image homography methods.
The proposed mosaicking framework outperformed existing methods and generated meaningful mosaic, while reducing the accumulated drift, even in the presence of visual challenges such as specular highlights, reflection, texture paucity, and low video resolution.
羊膜镜激光光凝术是一种微创外科手术，用于治疗双胎输血综合征（TTTS），该手术涉及定位和消融胎盘上的异常血管连接，以调节两个胎儿的血流。由于视场有限、能见度差、偶尔出血和图像质量差，该手术极具挑战性。羊膜镜拼图可以帮助创建具有扩展视场的图像，这可以在 TTTS 手术期间为临床医生提供便利。
我们提出了一种基于深度学习的拼图框架，用于处理来自不同环境（如模拟、体模、离体和体内环境）的各种羊膜镜视频。所提出的拼图框架通过引入受控数据生成和一致的单应性估计模块，扩展了现有的深度图像单应性模型，以处理视频数据。训练是在一小部分独立于测试视频的羊膜镜图像上进行的。
我们对 5 个不同的羊膜镜视频（2400 帧）进行了定量和定性评估，这些视频捕捉了不同的环境。为了证明所提出框架的鲁棒性，我们与现有的基于特征和深度图像单应性方法进行了比较。
所提出的拼图框架表现优于现有方法，并生成了有意义的拼图，同时减少了累积漂移，即使在存在镜面反射、反射、纹理不足和低视频分辨率等视觉挑战的情况下也是如此。
