Arefinia E, Jayender J, Patel R V
Department of Electrical and Computer Engineering, Western University, London, ON, Canada, and Canadian Surgical Technologies and Advanced Robotics (CSTAR), University Hospital, LHSC, London, ON, Canada.
Department of Radiology at Brigham and Women's Hospital, and the Harvard Medical School, Boston, MA 02115, USA.
IEEE Trans Med Robot Bionics. 2024 Aug;6(3):1004-1016. doi: 10.1109/tmrb.2024.3407590. Epub 2024 May 31.
Catheter-based cardiac ablation is a minimally invasive procedure for treating atrial fibrillation (AF). Electrophysiologists perform the procedure under image guidance during which the contact force between the heart tissue and the catheter tip determines the quality of lesions created. This paper describes a novel multi-modal contact force estimator based on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). The estimator takes the shape and optical flow of the deflectable distal section as two modalities since frames and motion between frames complement each other to capture the long context in the video frames of the catheter. The angle between the tissue and the catheter tip is considered a complement of the extracted shape. The data acquisition platform measures the two-degrees-of-freedom contact force and video data as the catheter motion is constrained in the imaging plane. The images are captured via a camera that simulates single-view fluoroscopy for experimental purposes. In this sensor-free procedure, the features of the images and optical flow modalities are extracted through transfer learning. Long Short-Term Memory Networks (LSTMs) with a memory fusion network (MFN) are implemented to consider time dependency and hysteresis due to friction. The architecture integrates spatial and temporal networks. Late fusion with the concatenation of LSTMs, transformer decoders, and Gated Recurrent Units (GRUs) are implemented to verify the feasibility of the proposed network-based approach and its superiority over single-modality networks. The resulting mean absolute error, which accounted for only 2.84% of the total magnitude, was obtained by collecting data under more realistic circumstances in contrast to previous research studies. The decrease in error is considerably better than that achieved by individual modalities and late fusion with concatenation. These results emphasize the practicality and relevance of utilizing a multimodal network in real-world scenarios.
基于导管的心脏消融术是一种用于治疗心房颤动（AF）的微创手术。电生理学家在图像引导下进行该手术，在此过程中，心脏组织与导管尖端之间的接触力决定了所形成损伤的质量。本文描述了一种基于卷积神经网络（CNN）和循环神经网络（RNN）的新型多模态接触力估计器。该估计器将可弯曲远端部分的形状和光流作为两种模态，因为帧与帧之间的运动相互补充，以捕捉导管视频帧中的长上下文信息。组织与导管尖端之间的角度被视为所提取形状的补充。数据采集平台在导管运动受限于成像平面时，测量二维接触力和视频数据。图像通过一台模拟单视角荧光透视的相机采集，用于实验目的。在这个无传感器的手术过程中，通过迁移学习提取图像和光流模态的特征。实现了带有记忆融合网络（MFN）的长短期记忆网络（LSTM），以考虑由于摩擦导致的时间依赖性和滞后现象。该架构集成了空间和时间网络。通过将LSTM、Transformer解码器和门控循环单元（GRU）进行串联的后期融合，以验证所提出的基于网络的方法的可行性及其相对于单模态网络的优越性。与先前的研究相比，在更现实的情况下收集数据得到的结果平均绝对误差仅占总量级的2.84%。误差的降低明显优于单模态和串联后期融合所达到的效果。这些结果强调了在现实场景中使用多模态网络的实用性和相关性。
