Department of Radiological Sciences, University of California Los Angeles, 300 UCLA Medical Plaza, Suite B119, Los Angeles, CA, 90095, USA.
Department of Bioengineering, University of California Los Angeles, Los Angeles, CA, USA.
Int J Comput Assist Radiol Surg. 2024 Nov;19(11):2227-2237. doi: 10.1007/s11548-024-03077-3. Epub 2024 Mar 23.
Accurate and rapid needle localization on 3D magnetic resonance imaging (MRI) is critical for MRI-guided percutaneous interventions. The current workflow requires manual needle localization on 3D MRI, which is time-consuming and cumbersome. Automatic methods using 2D deep learning networks for needle segmentation require manual image plane localization, while 3D networks are challenged by the need for sufficient training datasets. This work aimed to develop an automatic deep learning-based pipeline for accurate and rapid 3D needle localization on in vivo intra-procedural 3D MRI using a limited training dataset.
The proposed automatic pipeline adopted Shifted Window (Swin) Transformers and employed a coarse-to-fine segmentation strategy: (1) initial 3D needle feature segmentation with 3D Swin UNEt TRansfomer (UNETR); (2) generation of a 2D reformatted image containing the needle feature; (3) fine 2D needle feature segmentation with 2D Swin Transformer and calculation of 3D needle tip position and axis orientation. Pre-training and data augmentation were performed to improve network training. The pipeline was evaluated via cross-validation with 49 in vivo intra-procedural 3D MR images from preclinical pig experiments. The needle tip and axis localization errors were compared with human intra-reader variation using the Wilcoxon signed rank test, with p < 0.05 considered significant.
The average end-to-end computational time for the pipeline was 6 s per 3D volume. The median Dice scores of the 3D Swin UNETR and 2D Swin Transformer in the pipeline were 0.80 and 0.93, respectively. The median 3D needle tip and axis localization errors were 1.48 mm (1.09 pixels) and 0.98°, respectively. Needle tip localization errors were significantly smaller than human intra-reader variation (median 1.70 mm; p < 0.01).
The proposed automatic pipeline achieved rapid pixel-level 3D needle localization on intra-procedural 3D MRI without requiring a large 3D training dataset and has the potential to assist MRI-guided percutaneous interventions.
在 3D 磁共振成像（MRI）上进行准确、快速的针定位对于 MRI 引导的经皮介入至关重要。目前的工作流程需要在 3D MRI 上手动进行针定位，这既耗时又繁琐。使用二维深度学习网络进行针分割的自动方法需要手动进行图像平面定位，而三维网络则需要足够的训练数据集。本研究旨在开发一种基于自动深度学习的流水线，以便在使用有限的训练数据集的情况下，对体内术中的 3D MRI 进行准确、快速的 3D 针定位。
所提出的自动流水线采用了 Shifte Window（Swin）Transformers，并采用了一种从粗到细的分割策略：（1）使用 3D Swin UNETR 转换器进行初始的 3D 针特征分割；（2）生成包含针特征的 2D 重构图；（3）使用 2D Swin Transformer 进行精细的 2D 针特征分割，并计算 3D 针尖位置和轴方向。进行了预训练和数据增强，以提高网络训练效果。使用来自临床前猪实验的 49 个体内术中 3D MRI 图像进行了交叉验证，评估了该流水线。使用 Wilcoxon 符号秩检验比较了针尖和轴定位误差与人类内部读者变异性，p<0.05 被认为具有统计学意义。
该流水线的平均端到端计算时间为每个 3D 体积 6 秒。流水线中 3D Swin UNETR 和 2D Swin Transformer 的平均 Dice 评分分别为 0.80 和 0.93。3D 针尖和轴定位误差的中位数分别为 1.48mm（1.09 像素）和 0.98°。针尖定位误差明显小于人类内部读者变异性（中位数 1.70mm；p<0.01）。
所提出的自动流水线实现了在术中 3D MRI 上快速的像素级 3D 针定位，而无需大型 3D 训练数据集，并且有可能辅助 MRI 引导的经皮介入。
