Chien Jong-Chih, Tsai Yao-Ren, Wu Chieh-Tsai, Lee Jiann-Der
Degree Program of Digital Space and Product Design, Kainan University, Taoyuan 33857, Taiwan.
Department of Electrical Engineering, Chang Gung University, Taoyuan 333, Taiwan.
Sensors (Basel). 2019 Aug 15;19(16):3555. doi: 10.3390/s19163555.
By the standard of today's image-guided surgery (IGS) technology, in order to check and verify the progress of the surgery, the surgeons still require divert their attention from the patients occasionally to check against the display. In this paper, a mixed-reality system for medical use is proposed that combines an Intel RealSense sensor with Microsoft's Hololens head-mounted display system, for superimposing medical data onto the physical surface of a patient, so the surgeons do not need to divert their attention from their patients. The main idea of our proposed system is to display the 3D medical images of the patients on the actual patients themselves by placing the medical images and the patients in the same coordinate space. However, the virtual medical data may contain noises and outliers, so the transformation mapping function must be able to handle these problems. The transform function in our system is performed by the use of our proposed Denoised-Resampled-Weighted-and-Perturbed-Iterative Closest Points (DRWP-ICP) algorithm, which performs denoising and removal of outliers before aligning the pre-operative medical image data points to the patient's physical surface position before displaying the result using the Microsoft HoloLens display system. The experimental results shows that our proposed mixed-reality system using DRWP-ICP is capable of performing accurate and robust mapping despite the presence of noise and outliers.
以当今的图像引导手术（IGS）技术标准来看，为了检查和核实手术进展，外科医生仍需偶尔将注意力从患者身上转移开去查看显示屏。本文提出了一种医疗用混合现实系统，该系统将英特尔实感传感器与微软的HoloLens头戴式显示系统相结合，用于将医疗数据叠加到患者的身体表面，这样外科医生就无需将注意力从患者身上转移开。我们所提出系统的主要理念是通过将医学图像和患者置于同一坐标空间，从而在实际患者自身身上显示患者的3D医学图像。然而，虚拟医学数据可能包含噪声和离群值，因此变换映射函数必须能够处理这些问题。我们系统中的变换函数通过使用我们提出的去噪-重采样-加权-扰动迭代最近点（DRWP-ICP）算法来执行，该算法在将术前医学图像数据点与患者身体表面位置对齐之前，先进行去噪和去除离群值操作，然后使用微软HoloLens显示系统显示结果。实验结果表明，我们提出的使用DRWP-ICP的混合现实系统尽管存在噪声和离群值，仍能够执行准确且稳健的映射。
