Shi Haochen, Xu Jiangchang, Li Haitao, Jiang Shuanglin, Lei Chaoyu, Zhou Huifang, Li Yinwei, Chen Xiaojun
Institute of Biomedical Manufacturing and Life Quality Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200241, China.
Institute of Biomedical Manufacturing and Life Quality Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200241, China; Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China.
Med Image Anal. 2025 Jul;103:103609. doi: 10.1016/j.media.2025.103609. Epub 2025 May 1.
Augmented reality (AR) has significant potential to enhance the identification of critical locations during endoscopic surgeries, where accurate endoscope calibration is essential for ensuring the quality of augmented images. In optical-based surgical navigation systems, asynchrony between the optical tracker and the endoscope can cause the augmented scene to diverge from reality during rapid movements, potentially misleading the surgeon-a challenge that remains unresolved. In this paper, we propose a novel spatial-temporal endoscope calibration method that simultaneously determines the spatial transformation from the image to the optical marker and the temporal latency between the tracking and image acquisition systems. To estimate temporal latency, we utilize a Monte Carlo method to estimate the intrinsic parameters of the endoscope's imaging system, leveraging a dataset of thousands of calibration samples. This dataset is larger than those typically employed in conventional camera calibration routines, rendering traditional algorithms computationally infeasible within a reasonable timeframe. By introducing latency as an independent variable into the principal equation of hand-eye calibration, we developed a weighted algorithm to iteratively solve the equation. This approach eliminates the need for a fixture to stabilize the endoscope during calibration, allowing for quicker calibration through handheld flexible movement. Experimental results demonstrate that our method achieves an average 2D error of 7±3 pixels and a pseudo-3D error of 1.2±0.4mm for stable scenes within 82.4±16.6 seconds-approximately 68% faster in operation time than conventional methods. In dynamic scenes, our method compensates for the virtual-to-reality latency of 11±2ms, which is shorter than a single frame interval and 5.7 times shorter than the uncompensated conventional method. Finally, we successfully integrated the proposed method into our surgical navigation system and validated its feasibility in clinical trials for transnasal optic canal decompression surgery. Our method has the potential to improve the safety and efficacy of endoscopic surgeries, leading to better patient outcomes.
增强现实（AR）在增强内镜手术中关键位置的识别方面具有巨大潜力，在内镜手术中，精确的内镜校准对于确保增强图像的质量至关重要。在基于光学的手术导航系统中，光学跟踪器与内镜之间的异步性可能导致在快速移动过程中增强场景与现实脱节，从而可能误导外科医生——这一挑战至今仍未解决。在本文中，我们提出了一种新颖的时空内镜校准方法，该方法同时确定从图像到光学标记的空间变换以及跟踪系统与图像采集系统之间的时间延迟。为了估计时间延迟，我们利用蒙特卡罗方法估计内镜成像系统的固有参数，利用包含数千个校准样本的数据集。这个数据集比传统相机校准程序中通常使用的数据集更大，使得传统算法在合理的时间范围内计算上不可行。通过将延迟作为一个独立变量引入手眼校准的主方程，我们开发了一种加权算法来迭代求解该方程。这种方法在校准过程中无需固定装置来稳定内镜，通过手持灵活移动可以更快地进行校准。实验结果表明，对于稳定场景，我们的方法在82.4±16.6秒内实现了平均2D误差为7±3像素，伪3D误差为1.2±0.4毫米，操作时间比传统方法快约68%。在动态场景中，我们的方法补偿了11±2毫秒的虚拟到现实延迟，这比单个帧间隔短，比未补偿的传统方法短5.7倍。最后，我们成功地将所提出的方法集成到我们的手术导航系统中，并在经鼻视神经管减压手术的临床试验中验证了其可行性。我们的方法有潜力提高内镜手术的安全性和有效性，从而为患者带来更好的治疗效果。
