Zhang Ziyan, Liu Yan, Tian Jiawei, Liu Shan, Yang Bo, Xiang Longhai, Yin Lirong, Zheng Wenfeng
School of Innovation and Entrepreneurship, Xi'an Fanyi University, Xi'an 710105, China.
School of Automation, University of Electronic Science and Technology of China, Chengdu 610054, China.
Sensors (Basel). 2021 Nov 14;21(22):7570. doi: 10.3390/s21227570.
At present, feature-based 3D reconstruction and tracking technology is widely applied in the medical field. In minimally invasive surgery, the surgeon can achieve three-dimensional reconstruction through the images obtained by the endoscope in the human body, restore the three-dimensional scene of the area to be operated on, and track the motion of the soft tissue surface. This enables doctors to have a clearer understanding of the location depth of the surgical area, greatly reducing the negative impact of 2D image defects and ensuring smooth operation. In this study, firstly, the 3D coordinates of each feature point are calculated by using the parameters of the parallel binocular endoscope and the spatial geometric constraints. At the same time, the discrete feature points are divided into multiple triangles using the Delaunay triangulation method. Then, the 3D coordinates of feature points and the division results of each triangle are combined to complete the 3D surface reconstruction. Combined with the feature matching method based on convolutional neural network, feature tracking is realized by calculating the three-dimensional coordinate changes of the same feature point in different frames. Finally, experiments are carried out on the endoscope image to complete the 3D surface reconstruction and feature tracking.
目前，基于特征的三维重建与跟踪技术在医学领域得到了广泛应用。在微创手术中，外科医生可以通过人体内部内窥镜获取的图像实现三维重建，还原手术区域的三维场景，并跟踪软组织表面的运动。这使医生能够更清楚地了解手术区域的位置深度，极大地减少二维图像缺陷的负面影响，确保手术顺利进行。在本研究中，首先利用平行双目内窥镜的参数和空间几何约束计算每个特征点的三维坐标。同时，采用德劳内三角剖分方法将离散的特征点划分为多个三角形。然后，将特征点的三维坐标与每个三角形的划分结果相结合，完成三维表面重建。结合基于卷积神经网络的特征匹配方法，通过计算同一特征点在不同帧中的三维坐标变化来实现特征跟踪。最后，在内窥镜图像上进行实验，完成三维表面重建和特征跟踪。
