Schreiter Josefine, Heinrich Florian, Hatscher Benjamin, Schott Danny, Hansen Christian
Faculty of Computer Science and Research Campus STIMULATE, University of Magdeburg, Magdeburg, Germany.
Siemens Healthineers, Forchheim, Germany.
Int J Comput Assist Radiol Surg. 2025 Apr;20(4):807-816. doi: 10.1007/s11548-024-03263-3. Epub 2024 Oct 28.
As technology advances, more research dedicated to medical interactive systems emphasizes the integration of touchless and multimodal interaction (MMI). Particularly in surgical and interventional settings, this approach is advantageous because it maintains sterility and promotes a natural interaction. Past reviews have focused on investigating MMI in terms of technology and interaction with robots. However, none has put particular emphasis on analyzing these kind of interactions for surgical and interventional scenarios.
Two databases were included in the query to search for relevant publications within the past 10 years. After identification, two screening steps followed which included eligibility criteria. A forward/backward search was added to identify more relevant publications. The analysis incorporated the clustering of references in terms of addressed medical field, input and output modalities, and challenges regarding the development and evaluation.
A sample of 31 references was obtained (16 journal articles, 15 conference papers). MMI was predominantly developed for laparoscopy and radiology and interaction with image viewers. The majority implemented two input modalities, with voice-hand interaction being the most common combination-voice for discrete and hand for continuous navigation tasks. The application of gaze, body, and facial control is minimal, primarily because of ergonomic concerns. Feedback was included in 81% publications, of which visual cues were most often applied.
This work systematically reviews MMI for surgical and interventional scenarios over the past decade. In future research endeavors, we propose an enhanced focus on conducting in-depth analyses of the considered use cases and the application of standardized evaluation methods. Moreover, insights from various sectors, including but not limited to the gaming sector, should be exploited.
随着技术的进步，越来越多致力于医学交互系统的研究强调非接触式和多模态交互（MMI）的整合。特别是在手术和介入环境中，这种方法具有优势，因为它能保持无菌并促进自然交互。过去的综述主要集中在从技术以及与机器人交互的角度研究MMI。然而，没有一篇综述特别强调分析手术和介入场景中的这类交互。
查询了两个数据库，以搜索过去10年内的相关出版物。在识别之后，进行了两个筛选步骤，包括纳入标准。还增加了向前/向后搜索以识别更多相关出版物。分析包括根据所涉及的医学领域、输入和输出模态以及开发和评估方面的挑战对参考文献进行聚类。
获得了31篇参考文献样本（16篇期刊文章，15篇会议论文）。MMI主要是为腹腔镜检查和放射学以及与图像查看器的交互而开发的。大多数实现了两种输入模态，语音-手部交互是最常见的组合——语音用于离散任务，手部用于连续导航任务。注视、身体和面部控制的应用很少，主要是出于人体工程学方面的考虑。81%的出版物包含反馈，其中视觉提示应用最为频繁。
这项工作系统地回顾了过去十年中手术和介入场景中的MMI。在未来的研究中，我们建议更加注重对所考虑的用例进行深入分析以及应用标准化评估方法。此外，应该利用包括但不限于游戏领域等各个领域的见解。
