Cha Junghun, Cho Doohyun, Lee SeungJae
Annu Int Conf IEEE Eng Med Biol Soc. 2025 Jul;2025:1-7. doi: 10.1109/EMBC58623.2025.11254375.
3D reconstruction in endoscopic imaging can significantly improve diagnostic accuracy, enhance surgical planning, and provide realistic training environments for clinicians. However, existing reconstruction methods like Neural Radiance Fields and standard 3D Gaussian Splatting (3DGS) struggle in gastrointestinal (GI) environments, where narrow viewpoints and uniform textures hinder reliable 3D modeling. To overcome these challenges, we propose an advanced 3DGS-based framework that integrates a deep learning-based Structure-from-Motion (SfM) technique with sophisticated depth regularization. Our method leverages Super-point and Superglue within the SfM process to extract and match features robustly from GI tract scenes, facilitating accurate camera pose estimation and effective initialization of 3D Gaussians. We further refine the reconstruction by aligning monocular depth predictions from a pre-trained Depth-Anything-V2 model with SfM-derived depth using a scale-offset adjustment, enforced by an L loss. In addition, hard depth regularization, imposed via a Huber loss, ensures precise placement of 3D Gaussians, while globallocal depth normalization preserves both fine local details and overall structural consistency. Extensive experiments on multiple endoscopic datasets demonstrate that our approach delivers enhanced reconstruction quality and reduced artifacts.
内镜成像中的三维重建可以显著提高诊断准确性,加强手术规划,并为临床医生提供逼真的训练环境。然而,现有的重建方法,如神经辐射场和标准三维高斯喷绘(3DGS),在胃肠道(GI)环境中存在困难,因为狭窄的视角和均匀的纹理阻碍了可靠的三维建模。为了克服这些挑战,我们提出了一个基于3DGS的先进框架,该框架将基于深度学习的运动结构(SfM)技术与复杂的深度正则化相结合。我们的方法在SfM过程中利用超级点和超级胶水从胃肠道场景中稳健地提取和匹配特征,便于精确的相机姿态估计和三维高斯的有效初始化。我们通过使用L损失强制的比例偏移调整,将预训练的深度任意V2模型的单目深度预测与SfM派生的深度对齐,进一步优化重建。此外,通过Huber损失进行的硬深度正则化确保了三维高斯的精确放置,而全局-局部深度归一化则保留了精细的局部细节和整体结构一致性。在多个内镜数据集上进行的广泛实验表明,我们的方法提供了更高的重建质量并减少了伪影。