Allen Daniel, Peters Terry, Chen Elvis C S
School of Biomedical Engineering, Western University, London, Ontario, Canada.
Robarts Research Institute, Western University, London, Ontario, Canada.
Int J Comput Assist Radiol Surg. 2025 Jan;20(1):137-146. doi: 10.1007/s11548-024-03250-8. Epub 2024 Sep 11.
Optical-see-through head-mounted displays have the ability to seamlessly integrate virtual content with the real world through a transparent lens and an optical combiner. Although their potential for use in surgical settings has been explored, their clinical translation is sparse in the current literature, largely due to their limited tracking capabilities and the need for manual alignment of virtual representations of objects with their real-world counterparts.
We propose a simple and robust hand-eye calibration process for the depth camera of the Microsoft HoloLens 2, utilizing a tracked surgical stylus fitted with infrared reflective spheres as the calibration tool.
Using a Monte Carlo simulation and a paired-fiducial registration algorithm, we show that a calibration accuracy of 1.65 mm can be achieved with as little as 6 fiducial points. We also present heuristics for optimizing the accuracy of the calibration. The ability to use our calibration method in a clinical setting is validated through a user study, with users achieving a mean calibration accuracy of 1.67 mm in an average time of 42 s.
This work enables real-time hand-eye calibration for the Microsoft HoloLens 2, without any need for a manual alignment process. Using this framework, existing surgical navigation systems employing optical or electromagnetic tracking can easily be incorporated into an augmented reality environment with a high degree of accuracy.
光学透视头戴式显示器能够通过透明镜片和光学组合器将虚拟内容与现实世界无缝集成。尽管其在手术场景中的应用潜力已得到探索，但在当前文献中其临床转化情况稀少，这主要归因于其有限的跟踪能力以及需要手动将物体的虚拟表示与其现实世界对应物对齐。
我们为微软HoloLens 2的深度相机提出了一种简单且稳健的手眼校准方法，利用配备红外反射球的跟踪手术笔作为校准工具。
使用蒙特卡洛模拟和配对基准配准算法，我们表明仅用6个基准点就能实现1.65毫米的校准精度。我们还提出了优化校准精度的启发式方法。通过一项用户研究验证了在临床环境中使用我们校准方法的能力，用户在平均42秒的时间内实现了1.67毫米的平均校准精度。
这项工作实现了微软HoloLens 2的实时手眼校准，无需任何手动对齐过程。使用此框架，现有的采用光学或电磁跟踪的手术导航系统可以轻松地高精度整合到增强现实环境中。
