Kerkhof Enzo, Thabit Abdullah, Benmahdjoub Mohamed, Ambrosini Pierre, van Ginhoven Tessa, Wolvius Eppo B, van Walsum Theo
Department of Radiology & Nuclear Medicine, Biomedical Imaging Group Rotterdam, Erasmus MC, Rotterdam, The Netherlands.
Department of Surgical Oncology, Netherlands Cancer Institute, 1066 CX, Amsterdam, The Netherlands.
Int J Comput Assist Radiol Surg. 2025 May;20(5):901-912. doi: 10.1007/s11548-025-03328-x. Epub 2025 Mar 14.
In augmented reality (AR) surgical navigation, a registration step is required to align the preoperative data with the patient. This work investigates the use of the depth sensor of HoloLens 2 for registration in surgical navigation.
An AR depth-based registration framework was developed. The framework aligns preoperative and intraoperative point clouds and overlays the preoperative model on the patient. For evaluation, three experiments were conducted. First, the accuracy of the HoloLens's depth sensor was evaluated for both Long-Throw (LT) and Articulated Hand Tracking (AHAT) modes. Second, the overall registration accuracy was assessed with different alignment approaches. The accuracy and success rate of each approach were evaluated. Finally, a qualitative assessment of the framework was performed on various objects.
The depth accuracy experiment showed mean overestimation errors of 5.7 mm for AHAT and 9.0 mm for LT. For the overall alignment, the mean translation errors of the different methods ranged from 12.5 to 17.0 mm, while rotation errors ranged from 0.9 to 1.1 degrees.
The results show that the depth sensor on the HoloLens 2 can be used for image-to-patient alignment with 1-2 cm accuracy and within 4 s, indicating that with further improvement in the accuracy, this approach can offer a convenient alternative to other time-consuming marker-based approaches. This work provides a generic marker-less registration framework using the depth sensor of the HoloLens 2, with extensive analysis of the sensor's reconstruction and registration accuracy. It supports advancing the research of marker-less registration in surgical navigation.
在增强现实（AR）手术导航中，需要进行配准步骤以将术前数据与患者对齐。本研究探讨了使用HoloLens 2的深度传感器进行手术导航配准的情况。
开发了一种基于AR深度的配准框架。该框架对齐术前和术中的点云，并将术前模型叠加在患者身上。为进行评估，开展了三项实验。首先，针对长距离（LT）和关节手部跟踪（AHAT）模式评估了HoloLens深度传感器的精度。其次，使用不同的对齐方法评估整体配准精度。评估了每种方法的精度和成功率。最后，对各种物体对该框架进行了定性评估。
深度精度实验显示，AHAT模式的平均高估误差为5.7毫米，LT模式为9.0毫米。对于整体对齐，不同方法的平均平移误差在12.5至17.0毫米之间，而旋转误差在0.9至1.1度之间。
结果表明，HoloLens 2上的深度传感器可用于图像与患者的对齐，精度为1 - 2厘米，时间在4秒以内，这表明随着精度的进一步提高，该方法可以为其他耗时的基于标记的方法提供一种便捷的替代方案。本研究提供了一个使用HoloLens 2深度传感器的通用无标记配准框架，并对传感器的重建和配准精度进行了广泛分析。它支持推进手术导航中无标记配准的研究。
