Medical Physics Laboratory Simulation Centre, School of Medicine, University of Athens, Greece.
Int J Med Robot. 2013 Dec;9(4):e34-51. doi: 10.1002/rcs.1485. Epub 2013 Jan 25.
Despite the popular use of virtual and physical reality simulators in laparoscopic training, the educational potential of augmented reality (AR) has not received much attention. A major challenge is the robust tracking and three-dimensional (3D) pose estimation of the endoscopic instrument, which are essential for achieving interaction with the virtual world and for realistic rendering when the virtual scene is occluded by the instrument. In this paper we propose a method that addresses these issues, based solely on visual information obtained from the endoscopic camera.
Two different tracking algorithms are combined for estimating the 3D pose of the surgical instrument with respect to the camera. The first tracker creates an adaptive model of a colour strip attached to the distal part of the tool (close to the tip). The second algorithm tracks the endoscopic shaft, using a combined Hough-Kalman approach. The 3D pose is estimated with perspective geometry, using appropriate measurements extracted by the two trackers.
The method has been validated on several complex image sequences for its tracking efficiency, pose estimation accuracy and applicability in AR-based training. Using a standard endoscopic camera, the absolute average error of the tip position was 2.5 mm for working distances commonly found in laparoscopic training. The average error of the instrument's angle with respect to the camera plane was approximately 2°. The results are also supplemented by video segments of laparoscopic training tasks performed in a physical and an AR environment.
The experiments yielded promising results regarding the potential of applying AR technologies for laparoscopic skills training, based on a computer vision framework. The issue of occlusion handling was adequately addressed. The estimated trajectory of the instruments may also be used for surgical gesture interpretation and assessment.
尽管虚拟和物理现实模拟器在腹腔镜培训中得到了广泛应用，但增强现实（AR）的教育潜力尚未得到太多关注。一个主要的挑战是内窥镜器械的强大跟踪和三维（3D）姿态估计，这对于实现与虚拟世界的交互以及在器械遮挡虚拟场景时实现真实渲染至关重要。在本文中，我们提出了一种仅基于从内窥镜相机获得的视觉信息来解决这些问题的方法。
两种不同的跟踪算法结合使用，以估计相对于相机的手术器械的 3D 姿态。第一个跟踪器创建附着在工具远端（靠近尖端）的彩色条纹的自适应模型。第二个算法使用组合的霍夫-卡尔曼方法跟踪内窥镜轴。使用由两个跟踪器提取的适当测量值，通过透视几何来估计 3D 姿态。
该方法已在几个复杂的图像序列上进行了验证，以评估其跟踪效率、姿态估计精度以及在基于 AR 的培训中的适用性。使用标准内窥镜相机，在腹腔镜培训中常见的工作距离下，尖端位置的绝对平均误差为 2.5 毫米。器械相对于相机平面的角度的平均误差约为 2°。实验结果还补充了在物理和 AR 环境中进行的腹腔镜培训任务的视频片段。
基于计算机视觉框架，该实验在将 AR 技术应用于腹腔镜技能培训方面取得了有前景的结果。器械遮挡处理问题得到了充分解决。估计的器械轨迹也可用于手术手势解释和评估。
