Richter Alexander, Steinmann Till, Reichenbach Andreas, Rupitsch Stefan J
Electrical Instrumentation and Embedded Systems, Department of Microsystems Engineering, Albert-Ludwigs-Universität Freiburg, Georges-Köhler-Allee 106, 79110 Freiburg, Germany.
Fraunhofer Institute for High-Speed Dynamics, Ernst-Mach-Institut (EMI), Ernst-Zermelo-Straße 4, 79104 Freiburg, Germany.
Sensors (Basel). 2025 Jul 2;25(13):4124. doi: 10.3390/s25134124.
Real-time 3D reconstruction in minimally invasive surgery improves depth perception and supports intraoperative decision-making and navigation. However, endoscopic imaging presents significant challenges, such as specular reflections, low-texture surfaces, and tissue deformation. We present a novel, deterministic and iterative stereo-matching method based on adaptive support weights that is tailored to these constraints. The algorithm is implemented in CUDA and C++ to enable real-time performance. We evaluated our method on the Stereo Correspondence and Reconstruction of Endoscopic Data (SCARED) dataset and a custom synthetic dataset using the mean absolute error (MAE), root mean square error (RMSE), and frame rate as metrics. On SCARED datasets 8 and 9, our method achieves MAEs of 3.79 mm and 3.61 mm, achieving 24.9 FPS on a system with an AMD Ryzen 9 5950X and NVIDIA RTX 3090. To the best of our knowledge, these results are on par with or surpass existing deterministic stereo-matching approaches. On synthetic data, which eliminates real-world imaging errors, the method achieves an MAE of 140.06 μm and an RMSE of 251.9 μm, highlighting its performance ceiling under noise-free, idealized conditions. Our method focuses on single-shot 3D reconstruction as a basis for stereo frame stitching and full-scene modeling. It provides accurate, deterministic, real-time depth estimation under clinically relevant conditions and has the potential to be integrated into surgical navigation, robotic assistance, and augmented reality workflows.
微创手术中的实时三维重建可改善深度感知,并支持术中决策和导航。然而,内镜成像存在重大挑战,如镜面反射、低纹理表面和组织变形。我们提出了一种基于自适应支持权重的新颖、确定性和迭代立体匹配方法,该方法针对这些限制进行了定制。该算法用CUDA和C++实现,以实现实时性能。我们使用平均绝对误差(MAE)、均方根误差(RMSE)和帧率作为指标,在立体内镜数据对应与重建(SCARED)数据集和自定义合成数据集上评估了我们的方法。在SCARED数据集8和9上,我们的方法实现了3.79毫米和3.61毫米的平均绝对误差,在配备AMD锐龙9 5950X和NVIDIA RTX 3090的系统上实现了24.9帧每秒。据我们所知,这些结果与现有的确定性立体匹配方法相当或更优。在消除了现实世界成像误差的合成数据上,该方法实现了140.06微米的平均绝对误差和251.9微米的均方根误差,突出了其在无噪声、理想化条件下的性能上限。我们的方法专注于单次三维重建,作为立体帧拼接和全场景建模的基础。它在临床相关条件下提供准确、确定性的实时深度估计,并且有潜力集成到手术导航、机器人辅助和增强现实工作流程中。