Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing, 100081, China.
School of Medical Technology, Beijing Institute of Technology, Beijing, 100081, China.
Comput Biol Med. 2022 Sep;148:105826. doi: 10.1016/j.compbiomed.2022.105826. Epub 2022 Jul 6.
Marker-based augmented reality (AR) calibration methods for surgical navigation often require a second computed tomography scan of the patient, and their clinical application is limited due to high manufacturing costs and low accuracy.
This work introduces a novel type of AR calibration framework that combines a Microsoft HoloLens device with a single camera registration module for surgical navigation. A camera is used to gather multi-view images of a patient for reconstruction in this framework. A shape feature matching-based search method is proposed to adjust the size of the reconstructed model. The double clustering-based 3D point cloud segmentation method and 3D line segment detection method are also proposed to extract the corner points of the image marker. The corner points are the registration data of the image marker. A feature triangulation iteration-based registration method is proposed to quickly and accurately calibrate the pose relationship between the image marker and the patient in the virtual and real space. The patient model after registration is wirelessly transmitted to the HoloLens device to display the AR scene.
The proposed approach was used to conduct accuracy verification experiments on the phantoms and volunteers, which were compared with six advanced AR calibration methods. The proposed method obtained average fusion errors of 0.70 ± 0.16 and 0.91 ± 0.13 mm in phantom and volunteer experiments, respectively. The fusion accuracy of the proposed method is the highest among all comparison methods. A volunteer liver puncture clinical simulation experiment was also conducted to show the clinical feasibility.
Our experiments proved the effectiveness of the proposed AR calibration method, and revealed a considerable potential for improving surgical performance.
基于标记的增强现实（AR）校准方法常用于手术导航，通常需要对患者进行第二次计算机断层扫描，由于制造成本高和精度低，其临床应用受到限制。
本研究引入了一种新的 AR 校准框架，该框架将 Microsoft HoloLens 设备与用于手术导航的单相机注册模块相结合。该框架使用相机采集患者的多视图图像进行重建。提出了一种基于形状特征匹配的搜索方法来调整重建模型的大小。还提出了基于双聚类的 3D 点云分割方法和 3D 线段检测方法，以提取图像标记的角点。角点是图像标记的注册数据。提出了一种基于特征三角迭代的注册方法，可快速准确地校准虚拟和真实空间中图像标记和患者之间的位姿关系。注册后患者模型通过无线传输到 HoloLens 设备以显示 AR 场景。
该方法用于在体模和志愿者上进行精度验证实验，并与六种先进的 AR 校准方法进行了比较。该方法在体模和志愿者实验中分别获得了 0.70±0.16mm 和 0.91±0.13mm 的平均融合误差。在所有比较方法中，该方法的融合精度最高。还进行了志愿者肝脏穿刺临床模拟实验，以展示其临床可行性。
我们的实验证明了所提出的 AR 校准方法的有效性，并揭示了提高手术性能的巨大潜力。
