Daher Rema, Vasconcelos Francisco, Stoyanov Danail
Department of Computer Science, UCL Hawkes Institute, University College London, Gower Street, London, WC1E 6BT, UK.
Int J Comput Assist Radiol Surg. 2025 Jun;20(6):1255-1263. doi: 10.1007/s11548-025-03371-8. Epub 2025 May 13.
Visual 3D scene reconstruction can support colonoscopy navigation. It can help in recognising which portions of the colon have been visualised and characterising the size and shape of polyps. This is still a very challenging problem due to complex illumination variations, including abundant specular reflections. We investigate how to effectively decouple light and depth in this problem.
We introduce a self-supervised model that simultaneously characterises the shape and lighting of the visualised colonoscopy scene. Our model estimates shading, albedo, depth, and specularities (SHADeS) from single images. Unlike previous approaches (IID (Li et al. IEEE J Biomed Health Inform https://doi.org/10.1109/JBHI.2024.3400804 , 2024)), we use a non-Lambertian model that treats specular reflections as a separate light component. The implementation of our method is available at https://github.com/RemaDaher/SHADeS .
We demonstrate on real colonoscopy images (Hyper Kvasir) that previous models for light decomposition (IID) and depth estimation (MonoViT, ModoDepth2) are negatively affected by specularities. In contrast, SHADeS can simultaneously produce light decomposition and depth maps that are robust to specular regions. We also perform a quantitative comparison on phantom data (C3VD) where we further demonstrate the robustness of our model.
Modelling specular reflections improves depth estimation in colonoscopy. We propose an effective self-supervised approach that uses this insight to jointly estimate light decomposition and depth. Light decomposition has the potential to help with other problems, such as place recognition within the colon.
可视化三维场景重建可支持结肠镜检查导航。它有助于识别结肠镜已观察到的结肠部分，并确定息肉的大小和形状。由于复杂的光照变化，包括大量的镜面反射，这仍然是一个极具挑战性的问题。我们研究如何在这个问题中有效地分离光线和深度。
我们引入了一种自监督模型，该模型同时对可视化结肠镜检查场景的形状和光照进行表征。我们的模型从单张图像中估计阴影、反照率、深度和镜面反射（SHADeS）。与之前的方法（IID（Li等人，《IEEE生物医学与健康信息学杂志》https://doi.org/10.1109/JBHI.2024.3400804，2024年））不同，我们使用一种非朗伯模型，将镜面反射视为一个单独的光分量。我们方法的实现可在https://github.com/RemaDaher/SHADeS获取。
我们在真实的结肠镜检查图像（Hyper Kvasir）上证明，之前用于光分解（IID）和深度估计（MonoViT、ModoDepth2）的模型会受到镜面反射的负面影响。相比之下，SHADeS可以同时生成对镜面反射区域具有鲁棒性的光分解图和深度图。我们还对体模数据（C3VD）进行了定量比较，进一步证明了我们模型的鲁棒性。
对镜面反射进行建模可改善结肠镜检查中的深度估计。我们提出了一种有效的自监督方法，利用这一见解联合估计光分解和深度。光分解有潜力帮助解决其他问题，例如结肠内的位置识别。
