Departamento de Bioingeniería e Ingeniería Aeroespacial, Universidad Carlos III de Madrid, Avda. Universidad 30, 28911, Leganés, Spain.
Institute for Robotics and Cognitive Systems, University of Lübeck, Ratzeburger Allee 160, 23562, Lübeck, Schleswig-Holstein, Germany.
Int J Comput Assist Radiol Surg. 2022 Nov;17(11):2081-2091. doi: 10.1007/s11548-022-02695-z. Epub 2022 Jul 1.
Augmented Reality (AR) has the potential to simplify ultrasound (US) examinations which usually require a skilled and experienced sonographer to mentally align narrow 2D cross-sectional US images in the 3D anatomy of the patient. This work describes and evaluates a novel approach to track retroreflective spheres attached to the US probe using an inside-out technique with the AR glasses HoloLens 2. Finally, live US images are displayed in situ on the imaged anatomy.
The Unity application UltrARsound performs spatial tracking of the US probe and attached retroreflective markers using the depth camera integrated into the AR glasses-thus eliminating the need for an external tracking system. Additionally, a Kalman filter is implemented to improve the noisy measurements of the camera. US images are streamed wirelessly via the PLUS toolkit to HoloLens 2. The technical evaluation comprises static and dynamic tracking accuracy, frequency and latency of displayed images.
Tracking is performed with a median accuracy of 1.98 mm/1.81[Formula: see text] for the static setting when using the Kalman filter. In a dynamic scenario, the median error was 2.81 mm/1.70[Formula: see text]. The tracking frequency is currently limited to 20 Hz. 83% of the displayed US images had a latency lower than 16 ms.
In this work, we showed that spatial tracking of retroreflective spheres with the depth camera of HoloLens 2 is feasible, achieving a promising accuracy for in situ visualization of live US images. For tracking, no additional hardware nor modifications to HoloLens 2 are required making it a cheap and easy-to-use approach. Moreover, a minimal latency of displayed images enables a real-time perception for the sonographer.
增强现实（AR）有潜力简化超声（US）检查，通常需要熟练且经验丰富的超声医师在患者的 3D 解剖结构中通过心理对齐狭窄的 2D 横截面 US 图像。本工作描述并评估了一种使用 HoloLens 2 的内向外技术跟踪附着在 US 探头的反射球的新方法。最后，实时 US 图像在成像的解剖结构上原位显示。
Unity 应用程序 UltrARsound 使用集成在 AR 眼镜中的深度相机执行 US 探头和附着的反射标记的空间跟踪，从而无需外部跟踪系统。此外，实现了卡尔曼滤波器以改善相机的噪声测量。US 图像通过 PLUS 工具包无线流式传输到 HoloLens 2。技术评估包括静态和动态跟踪精度、显示图像的频率和延迟。
使用卡尔曼滤波器时，静态设置下的跟踪精度中位数为 1.98mm/1.81[公式：见正文]。在动态场景中，中位数误差为 2.81mm/1.70[公式：见正文]。跟踪频率目前限于 20Hz。83%的显示 US 图像的延迟低于 16ms。
在这项工作中，我们表明使用 HoloLens 2 的深度相机进行反射球的空间跟踪是可行的，为实时可视化实时 US 图像实现了有希望的准确性。对于跟踪，不需要额外的硬件，也不需要对 HoloLens 2 进行修改，使其成为一种廉价且易于使用的方法。此外，显示图像的最小延迟能够实现超声医师的实时感知。
