Computer Science, Kent State University, 800 E Summit St, Kent, OH 44240, USA.
College of Nursing, Kent State University, 800 E Summit St, Kent, OH 44240, USA.
Sensors (Basel). 2023 Jul 26;23(15):6697. doi: 10.3390/s23156697.
In this study, we developed a new haptic-mixed reality intravenous (HMR-IV) needle insertion simulation system, providing a bimanual haptic interface integrated into a mixed reality system with programmable variabilities considering real clinical environments. The system was designed for nursing students or healthcare professionals to practice IV needle insertion into a virtual arm with unlimited attempts under various changing insertion conditions (e.g., skin: color, texture, stiffness, friction; vein: size, shape, location depth, stiffness, friction). To achieve accurate hand-eye coordination under dynamic mixed reality scenarios, two different haptic devices (Dexmo and Geomagic Touch) and a standalone mixed reality system (HoloLens 2) were integrated and synchronized through multistep calibration for different coordinate systems (real world, virtual world, mixed reality world, haptic interface world, HoloLens camera). In addition, force-profile-based haptic rendering proposed in this study was able to successfully mimic the real tactile feeling of IV needle insertion. Further, a global hand-tracking method, combining two depth sensors (HoloLens and Leap Motion), was developed to accurately track a haptic glove and simulate grasping a virtual hand with force feedback. We conducted an evaluation study with 20 participants (9 experts and 11 novices) to measure the usability of the HMR-IV simulation system with user performance under various insertion conditions. The quantitative results from our own metric and qualitative results from the NASA Task Load Index demonstrate the usability of our system.
在这项研究中，我们开发了一种新的触觉混合现实静脉（HMR-IV）针插入模拟系统，提供了一种双手触觉界面，集成到具有可编程可变性的混合现实系统中，考虑到真实的临床环境。该系统旨在为护理学生或医疗保健专业人员提供实践，他们可以在各种变化的插入条件下（例如，皮肤：颜色、质地、硬度、摩擦力；静脉：大小、形状、位置深度、硬度、摩擦力），将 IV 针插入虚拟手臂，并且可以进行无限次尝试。为了在动态混合现实场景下实现精确的手眼协调，我们集成并同步了两种不同的触觉设备（Dexmo 和 Geomagic Touch）和一个独立的混合现实系统（HoloLens 2），通过多步校准针对不同的坐标系（现实世界、虚拟世界、混合现实世界、触觉界面世界、HoloLens 相机）。此外，本研究中提出的基于力廓线的触觉渲染能够成功地模拟 IV 针插入的真实触觉感觉。此外，我们开发了一种全局手部跟踪方法，结合两个深度传感器（HoloLens 和 Leap Motion），以准确跟踪触觉手套并模拟用力反馈抓取虚拟手。我们对 20 名参与者（9 名专家和 11 名新手）进行了评估研究，以测量在各种插入条件下，HMR-IV 模拟系统的用户性能的可用性。我们自己的度量标准的定量结果和 NASA 任务负荷指数的定性结果都证明了我们系统的可用性。
