Mathew Shawn, Nadeem Saad, Kaufman Arie
Department of Computer Science, Stony Brook University.
Department of Medical Physics, Memorial Sloan Kettering Cancer Center.
Med Image Comput Comput Assist Interv. 2021 Sep-Oct;12903:221-230. doi: 10.1007/978-3-030-87199-4_21. Epub 2021 Sep 21.
Haustral folds are colon wall protrusions implicated for high polyp miss rate during optical colonoscopy procedures. If segmented accurately, haustral folds can allow for better estimation of missed surface and can also serve as valuable landmarks for registering pre-treatment virtual (CT) and optical colonoscopies, to guide navigation towards the anomalies found in pre-treatment scans. We present a novel generative adversarial network, FoldIt, for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays. A new transitive loss is introduced in order to leverage ground truth information between haustral fold annotations and virtual colonoscopy renderings. We demonstrate the effectiveness of our model on real challenging optical colonoscopy videos as well as on textured virtual colonoscopy videos with clinician-verified haustral fold annotations. All code and scripts to reproduce the experiments of this paper will be made available via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.
袋状皱襞是结肠壁的突出部分，在光学结肠镜检查过程中与较高的息肉漏检率有关。如果能准确分割，袋状皱襞可以更好地估计漏检表面，还可以作为宝贵的地标，用于配准治疗前的虚拟（CT）和光学结肠镜检查，以引导朝向治疗前扫描中发现的异常部位导航。我们提出了一种新颖的生成对抗网络FoldIt，用于将光学结肠镜检查视频进行特征一致的图像转换，生成带有袋状皱襞叠加的虚拟结肠镜渲染图。引入了一种新的传递损失，以便利用袋状皱襞注释和虚拟结肠镜渲染图之间的真实信息。我们在具有挑战性的真实光学结肠镜检查视频以及带有经临床医生验证的袋状皱襞注释的纹理虚拟结肠镜检查视频上展示了我们模型的有效性。本文所有用于重现实验的代码和脚本将通过我们的计算内镜平台（https://github.com/nadeemlab/CEP）提供。
