Suppr超能文献

集成深度学习的增强现实手术导航系统

Augmented Reality Surgical Navigation System Integrated with Deep Learning.

作者信息

Chiou Shin-Yan, Liu Li-Sheng, Lee Chia-Wei, Kim Dong-Hyun, Al-Masni Mohammed A, Liu Hao-Li, Wei Kuo-Chen, Yan Jiun-Lin, Chen Pin-Yuan

机构信息

Department of Electrical Engineering, College of Engineering, Chang Gung University, Kwei-Shan, Taoyuan 333, Taiwan.

Department of Nuclear Medicine, Linkou Chang Gung Memorial Hospital, Taoyuan 333, Taiwan.

出版信息

Bioengineering (Basel). 2023 May 20;10(5):617. doi: 10.3390/bioengineering10050617.

Abstract

Most current surgical navigation methods rely on optical navigators with images displayed on an external screen. However, minimizing distractions during surgery is critical and the spatial information displayed in this arrangement is non-intuitive. Previous studies have proposed combining optical navigation systems with augmented reality (AR) to provide surgeons with intuitive imaging during surgery, through the use of planar and three-dimensional imagery. However, these studies have mainly focused on visual aids and have paid relatively little attention to real surgical guidance aids. Moreover, the use of augmented reality reduces system stability and accuracy, and optical navigation systems are costly. Therefore, this paper proposed an augmented reality surgical navigation system based on image positioning that achieves the desired system advantages with low cost, high stability, and high accuracy. This system also provides intuitive guidance for the surgical target point, entry point, and trajectory. Once the surgeon uses the navigation stick to indicate the position of the surgical entry point, the connection between the surgical target and the surgical entry point is immediately displayed on the AR device (tablet or HoloLens glasses), and a dynamic auxiliary line is shown to assist with incision angle and depth. Clinical trials were conducted for EVD (extra-ventricular drainage) surgery, and surgeons confirmed the system's overall benefit. A "virtual object automatic scanning" method is proposed to achieve a high accuracy of 1 ± 0.1 mm for the AR-based system. Furthermore, a deep learning-based U-Net segmentation network is incorporated to enable automatic identification of the hydrocephalus location by the system. The system achieves improved recognition accuracy, sensitivity, and specificity of 99.93%, 93.85%, and 95.73%, respectively, representing a significant improvement from previous studies.

摘要

目前大多数手术导航方法依赖于光学导航仪,图像显示在外部屏幕上。然而,在手术过程中尽量减少干扰至关重要,并且这种布置中显示的空间信息不直观。先前的研究提出将光学导航系统与增强现实(AR)相结合,通过使用平面和三维图像,在手术过程中为外科医生提供直观的成像。然而,这些研究主要集中在视觉辅助方面,相对较少关注实际的手术引导辅助工具。此外,增强现实的使用会降低系统的稳定性和准确性,并且光学导航系统成本高昂。因此,本文提出了一种基于图像定位的增强现实手术导航系统,该系统以低成本、高稳定性和高精度实现了所需的系统优势。该系统还为手术靶点、入点和轨迹提供直观的引导。一旦外科医生使用导航棒指示手术入点的位置,手术靶点与手术入点之间的连接会立即显示在AR设备(平板电脑或HoloLens眼镜)上,并显示一条动态辅助线以辅助切口角度和深度。针对脑室外引流(EVD)手术进行了临床试验,外科医生确认了该系统的整体益处。提出了一种“虚拟物体自动扫描”方法,以使基于AR的系统实现1±0.1毫米的高精度。此外,并入了基于深度学习的U-Net分割网络,以使系统能够自动识别脑积水位置。该系统分别实现了99.93%、93.85%和95.73%的更高识别准确率、灵敏度和特异性,与先前的研究相比有显著提高。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/aa62/10215407/bfb6763155d6/bioengineering-10-00617-g001.jpg

相似文献

1
Augmented Reality Surgical Navigation System Integrated with Deep Learning.
Bioengineering (Basel). 2023 May 20;10(5):617. doi: 10.3390/bioengineering10050617.
2
Augmented Reality Surgical Navigation System for External Ventricular Drain.
Healthcare (Basel). 2022 Sep 21;10(10):1815. doi: 10.3390/healthcare10101815.
4
Design and Validation of a Spinal Surgical Navigation System Based on Spatial Augmented Reality.
Spine (Phila Pa 1976). 2020 Dec 1;45(23):E1627-E1633. doi: 10.1097/BRS.0000000000003666.
6
Research on augmented reality navigation of in vitro fenestration of stent-graft based on deep learning and virtual-real registration.
Comput Assist Surg (Abingdon). 2023 Dec;28(1):2289339. doi: 10.1080/24699322.2023.2289339. Epub 2023 Dec 7.
7
Augmented Reality Visualization for Image-Guided Surgery: A Validation Study Using a Three-Dimensional Printed Phantom.
J Oral Maxillofac Surg. 2021 Sep;79(9):1943.e1-1943.e10. doi: 10.1016/j.joms.2021.04.001. Epub 2021 Apr 15.
8
Towards quantitative and intuitive percutaneous tumor puncture via augmented virtual reality.
Comput Med Imaging Graph. 2021 Jun;90:101905. doi: 10.1016/j.compmedimag.2021.101905. Epub 2021 Mar 31.
9
Navigate biopsy with ultrasound under augmented reality device: Towards higher system performance.
Comput Biol Med. 2024 May;174:108453. doi: 10.1016/j.compbiomed.2024.108453. Epub 2024 Apr 16.

引用本文的文献

1
Deep learning in neurosurgery: a systematic literature review with a structured analysis of applications across subspecialties.
Front Neurol. 2025 Apr 16;16:1532398. doi: 10.3389/fneur.2025.1532398. eCollection 2025.
2
Augmented Reality in Neurosurgery.
Adv Exp Med Biol. 2024;1462:351-374. doi: 10.1007/978-3-031-64892-2_21.
3
The future of artificial intelligence in thoracic surgery for non-small cell lung cancer treatment a narrative review.
Front Oncol. 2024 Feb 13;14:1347464. doi: 10.3389/fonc.2024.1347464. eCollection 2024.

本文引用的文献

2
CMM-Net: Contextual multi-scale multi-level network for efficient biomedical image segmentation.
Sci Rep. 2021 May 13;11(1):10191. doi: 10.1038/s41598-021-89686-3.
3
Augmented reality based navigation for distal interlocking of intramedullary nails utilizing Microsoft HoloLens 2.
Comput Biol Med. 2021 Jun;133:104402. doi: 10.1016/j.compbiomed.2021.104402. Epub 2021 Apr 15.
5
Automatic segmentation and location learning of neonatal cerebral ventricles in 3D ultrasound data combining CNN and CPPN.
Comput Biol Med. 2021 Apr;131:104268. doi: 10.1016/j.compbiomed.2021.104268. Epub 2021 Feb 17.
6
SVM-Based Normal Pressure Hydrocephalus Detection.
Clin Neuroradiol. 2021 Dec;31(4):1029-1035. doi: 10.1007/s00062-020-00993-0. Epub 2021 Jan 26.
7
Automated detection of cerebral microbleeds in MR images: A two-stage deep learning approach.
Neuroimage Clin. 2020;28:102464. doi: 10.1016/j.nicl.2020.102464. Epub 2020 Oct 13.
8
Evaluation of an artificial intelligent hydrocephalus diagnosis model based on transfer learning.
Medicine (Baltimore). 2020 Jul 17;99(29):e21229. doi: 10.1097/MD.0000000000021229.
9
HoloInjection: augmented reality support for CT-guided spinal needle injections.
Healthc Technol Lett. 2019 Nov 26;6(6):165-171. doi: 10.1049/htl.2019.0062. eCollection 2019 Dec.
10
Comparison of morphometric parameters in prediction of hydrocephalus using random forests.
Comput Biol Med. 2020 Jan;116:103547. doi: 10.1016/j.compbiomed.2019.103547. Epub 2019 Nov 20.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验