Suppr超能文献

基于深度混合模型的用于COVID-19肺活检的智能术中触觉增强现实导航

Intelligent Intraoperative Haptic-AR Navigation for COVID-19 Lung Biopsy Using Deep Hybrid Model.

作者信息

Tai Yonghang, Qian Kai, Huang Xiaoqiao, Zhang Jun, Jan Mian Ahmad, Yu Zhengtao

机构信息

Yunnan Key Laboratory of Opto-Electronic Information TechnologyYunnan Normal University Kunming 650500 China.

Department of Thoracic SurgeryYunnan First People's Hospital Kunming 650000 China.

出版信息

IEEE Trans Industr Inform. 2021 Jan 19;17(9):6519-6527. doi: 10.1109/TII.2021.3052788. eCollection 2021 Sep.

Abstract

A novel intelligent navigation technique for accurate image-guided COVID-19 lung biopsy is addressed, which systematically combines augmented reality (AR), customized haptic-enabled surgical tools, and deep neural network to achieve customized surgical navigation. Clinic data from 341 COVID-19 positive patients, with 1598 negative control group, have collected for the model synergy and evaluation. Biomechanics force data from the experiment are applied a WPD-CNN-LSTM (WCL) to learn a new patient-specific COVID-19 surgical model, and the ResNet was employed for the intraoperative force classification. To boost the user immersion and promote the user experience, intro-operational guiding images have combined with the haptic-AR navigational view. Furthermore, a 3-D user interface (3DUI), including all requisite surgical details, was developed with a real-time response guaranteed. Twenty-four thoracic surgeons were invited to the objective and subjective experiments for performance evaluation. The root-mean-square error results of our proposed WCL model is 0.0128, and the classification accuracy is 97%, which demonstrated that the innovative AR with deep learning (DL) intelligent model outperforms the existing perception navigation techniques with significantly higher performance. This article shows a novel framework in the interventional surgical integration for COVID-19 and opens the new research about the integration of AR, haptic rendering, and deep learning for surgical navigation.

摘要

本文提出了一种用于精确图像引导的新冠肺炎肺活检的新型智能导航技术,该技术系统地结合了增强现实(AR)、定制的触觉手术工具和深度神经网络,以实现定制化手术导航。已收集了341例新冠肺炎阳性患者的临床数据以及1598例阴性对照组数据,用于模型协同和评估。将实验中的生物力学力数据应用于WPD-CNN-LSTM(WCL),以学习新的针对特定患者的新冠肺炎手术模型,并采用ResNet进行术中力分类。为了增强用户沉浸感并提升用户体验,术中引导图像已与触觉AR导航视图相结合。此外,还开发了一个包含所有必要手术细节的3D用户界面(3DUI),并保证实时响应。邀请了24名胸外科医生参与客观和主观实验以进行性能评估。我们提出的WCL模型的均方根误差结果为0.0128,分类准确率为97%,这表明创新的带有深度学习(DL)的智能AR模型在性能上明显优于现有的感知导航技术。本文展示了一种用于新冠肺炎介入手术整合的新型框架,并开启了关于将AR、触觉渲染和深度学习整合用于手术导航的新研究。

相似文献

1
Intelligent Intraoperative Haptic-AR Navigation for COVID-19 Lung Biopsy Using Deep Hybrid Model.
IEEE Trans Industr Inform. 2021 Jan 19;17(9):6519-6527. doi: 10.1109/TII.2021.3052788. eCollection 2021 Sep.
2
Trustworthy and Intelligent COVID-19 Diagnostic IoMT Through XR and Deep-Learning-Based Clinic Data Access.
IEEE Internet Things J. 2021 Feb 1;8(21):15965-15976. doi: 10.1109/JIOT.2021.3055804. eCollection 2021 Nov 1.
3
Intelligent HMI in Orthopedic Navigation.
Adv Exp Med Biol. 2018;1093:207-224. doi: 10.1007/978-981-13-1396-7_17.
4
Deep Learning-Based Human Activity Real-Time Recognition for Pedestrian Navigation.
Sensors (Basel). 2020 Apr 30;20(9):2574. doi: 10.3390/s20092574.
5
A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals.
Comput Biol Med. 2022 Apr;143:105288. doi: 10.1016/j.compbiomed.2022.105288. Epub 2022 Feb 10.
8
An effective visualization technique for depth perception in augmented reality-based surgical navigation.
Int J Med Robot. 2016 Mar;12(1):62-72. doi: 10.1002/rcs.1657. Epub 2015 May 5.
9
Evaluation of the 3D Augmented Reality-Guided Intraoperative Positioning of Dental Implants in Edentulous Mandibular Models.
Int J Oral Maxillofac Implants. 2018 Nov/Dec;33(6):1219-1228. doi: 10.11607/jomi.6638.
10
Towards Virtual VATS, Face, and Construct Evaluation for Peg Transfer Training of Box, VR, AR, and MR Trainer.
J Healthc Eng. 2019 Jan 6;2019:6813719. doi: 10.1155/2019/6813719. eCollection 2019.

引用本文的文献

1
Exploring Augmented Reality Integration in Diagnostic Imaging: Myth or Reality?
Diagnostics (Basel). 2024 Jun 23;14(13):1333. doi: 10.3390/diagnostics14131333.
4
A Novel ABRM Model for Predicting Coal Moisture Content.
J Intell Robot Syst. 2022;104(2):30. doi: 10.1007/s10846-021-01552-6. Epub 2022 Feb 3.

本文引用的文献

1
Deep Learning Enables Accurate Diagnosis of Novel Coronavirus (COVID-19) With CT Images.
IEEE/ACM Trans Comput Biol Bioinform. 2021 Nov-Dec;18(6):2775-2780. doi: 10.1109/TCBB.2021.3065361. Epub 2021 Dec 8.
2
FSS-2019-nCov: A deep learning architecture for semi-supervised few-shot segmentation of COVID-19 infection.
Knowl Based Syst. 2021 Jan 5;212:106647. doi: 10.1016/j.knosys.2020.106647. Epub 2020 Dec 4.
4
Automated detection of COVID-19 cases using deep neural networks with X-ray images.
Comput Biol Med. 2020 Jun;121:103792. doi: 10.1016/j.compbiomed.2020.103792. Epub 2020 Apr 28.
5
A fully automatic deep learning system for COVID-19 diagnostic and prognostic analysis.
Eur Respir J. 2020 Aug 6;56(2). doi: 10.1183/13993003.00775-2020. Print 2020 Aug.
8
First Case of 2019 Novel Coronavirus in the United States.
N Engl J Med. 2020 Mar 5;382(10):929-936. doi: 10.1056/NEJMoa2001191. Epub 2020 Jan 31.
10
Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China.
Lancet. 2020 Feb 15;395(10223):497-506. doi: 10.1016/S0140-6736(20)30183-5. Epub 2020 Jan 24.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验