Krieger Kathrin, Egger Jan, Kleesiek Jens, Gunzer Matthias, Chen Jianxu
Biospectroscopy, Leibniz-Institut for Analytical Science-ISAS-e.V., Bunsen-Kirchhoff-Str. 11, Dortmund, 44139, NRW, Germany.
Neuroinformatics Group, Faculity of Technology, Bielefeld University, Inspiration 1, Bielefeld, 33619, NRW, Germany.
J Imaging Inform Med. 2025 Feb;38(1):646-655. doi: 10.1007/s10278-024-01094-x. Epub 2024 Jun 11.
3D data from high-resolution volumetric imaging is a central resource for diagnosis and treatment in modern medicine. While the fast development of AI enhances imaging and analysis, commonly used visualization methods lag far behind. Recent research used extended reality (XR) for perceiving 3D images with visual depth perception and touch but used restrictive haptic devices. While unrestricted touch benefits volumetric data examination, implementing natural haptic interaction with XR is challenging. The research question is whether a multisensory XR application with intuitive haptic interaction adds value and should be pursued. In a study, 24 experts for biomedical images in research and medicine explored 3D medical shapes with 3 applications: a multisensory virtual reality (VR) prototype using haptic gloves, a simple VR prototype using controllers, and a standard PC application. Results of standardized questionnaires showed no significant differences between all application types regarding usability and no significant difference between both VR applications regarding presence. Participants agreed to statements that VR visualizations provide better depth information, using the hands instead of controllers simplifies data exploration, the multisensory VR prototype allows intuitive data exploration, and it is beneficial over traditional data examination methods. While most participants mentioned manual interaction as the best aspect, they also found it the most improvable. We conclude that a multisensory XR application with improved manual interaction adds value for volumetric biomedical data examination. We will proceed with our open-source research project ISH3DE (Intuitive Stereoptic Haptic 3D Data Exploration) to serve medical education, therapeutic decisions, surgery preparations, or research data analysis.
来自高分辨率容积成像的3D数据是现代医学诊断和治疗的核心资源。虽然人工智能的快速发展增强了成像和分析能力,但常用的可视化方法却远远落后。最近的研究使用扩展现实(XR)来感知具有视觉深度感知和触觉的3D图像,但使用的是受限的触觉设备。虽然无限制的触摸有利于容积数据检查,但实现与XR的自然触觉交互具有挑战性。研究问题是具有直观触觉交互的多感官XR应用是否具有价值以及是否值得追求。在一项研究中,24位生物医学图像研究和医学领域的专家使用3种应用程序探索了3D医学形状:一种使用触觉手套的多感官虚拟现实(VR)原型、一种使用控制器的简单VR原型和一个标准的PC应用程序。标准化问卷的结果显示,在可用性方面,所有应用类型之间没有显著差异,在临场感方面,两种VR应用之间也没有显著差异。参与者同意以下观点:VR可视化提供了更好的深度信息,使用手而不是控制器简化了数据探索,多感官VR原型允许直观的数据探索,并且它比传统的数据检查方法更有益。虽然大多数参与者提到手动交互是最好的方面,但他们也发现这是最需要改进的方面。我们得出结论,具有改进的手动交互的多感官XR应用为容积生物医学数据检查增加了价值。我们将继续我们的开源研究项目ISH3DE(直观立体触觉3D数据探索),以服务于医学教育、治疗决策、手术准备或研究数据分析。
Biomed Eng Online. 2023-7-31
IEEE Trans Vis Comput Graph. 2022-2
Sensors (Basel). 2023-2-1
Cell Rep Med. 2021-7-20
Biomed Tech (Berl). 2024-12-30
Radiol Artif Intell. 2023-7-5
Life (Basel). 2023-1-31
Int J Comput Assist Radiol Surg. 2020-11
IEEE Trans Haptics. 2018-11-19
Int J Mol Sci. 2015-7-14
Exp Brain Res. 2007-6
Nat Rev Neurosci. 2005-4