School of Future Technology, University of Chinese Academy of Sciences, Beijing 100039, China; School of Aerospace Science And Technology, Xidian University, Xian 710071, China.
Brussel Photonics, Department of Applied Physics and Photonics, Vrije Universiteit Brussel and Flanders Make, 1050 Brussels, Belgium.
Comput Biol Med. 2023 Sep;164:107305. doi: 10.1016/j.compbiomed.2023.107305. Epub 2023 Aug 1.
During invasive surgery, the use of deep learning techniques to acquire depth information from lesion sites in real-time is hindered by the lack of endoscopic environmental datasets. This work aims to develop a high-accuracy three-dimensional (3D) simulation model for generating image datasets and acquiring depth information in real-time. Here, we proposed an end-to-end multi-scale supervisory depth estimation network (MMDENet) model for the depth estimation of pairs of binocular images. The proposed MMDENet highlights a multi-scale feature extraction module incorporating contextual information to enhance the correspondence precision of poorly exposed regions. A multi-dimensional information-guidance refinement module is also proposed to refine the initial coarse disparity map. Statistical experimentation demonstrated a 3.14% reduction in endpoint error compared to state-of-the-art methods. With a processing time of approximately 30fps, satisfying the requirements of real-time operation applications. In order to validate the performance of the trained MMDENet in actual endoscopic images, we conduct both qualitative and quantitative analysis with 93.38% high precision, which holds great promise for applications in surgical navigation.
在侵入性手术中，由于缺乏内镜环境数据集，深度学习技术难以实时从病变部位获取深度信息。本研究旨在开发一种高精度的三维（3D）模拟模型，用于生成图像数据集并实时获取深度信息。为此，我们提出了一种用于双目图像对深度估计的端到端多尺度监督深度估计网络（MMDENet）模型。所提出的 MMDENet 突出了一个多尺度特征提取模块，该模块结合了上下文信息，以提高曝光不良区域的对应精度。还提出了一个多维信息引导细化模块，以细化初始的粗视差图。统计实验表明，与最先进的方法相比，端点误差降低了 3.14%。处理时间约为 30fps，满足实时操作应用的要求。为了验证训练后的 MMDENet 在实际内窥镜图像中的性能，我们进行了定性和定量分析，精度高达 93.38%，这为手术导航应用提供了很大的潜力。
