School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, People's Republic of China.
School of Medicine, Shanghai General Hospital, Shanghai Jiao Tong University, Shanghai, People's Republic of China.
Int J Comput Assist Radiol Surg. 2022 Feb;17(2):295-303. doi: 10.1007/s11548-021-02519-6. Epub 2021 Oct 22.
Robot-assisted needle insertion guided by 2D ultrasound (US) can effectively improve the accuracy and success rate of clinical puncture. To this end, automatic and accurate needle-tracking methods are important for monitoring puncture processes, avoiding the needle deviating from the intended path, and reducing the risk of injury to surrounding tissues. This work aims to develop a framework for automatic and accurate detection of an inserted needle in 2D US image during the insertion process.
We propose a novel convolutional neural network architecture comprising of a two-channel encoder and single-channel decoder for needle segmentation using needle motion information extracted from two adjacent US image frames. Based on the novel network, we further propose an automatic needle detection framework. According to the prediction result of the previous frame, a region of interest of the needle in the US image was extracted and fed into the proposed network to achieve finer and faster continuous needle localization.
The performance of our method was evaluated based on 1000 pairs of US images extracted from robot-assisted needle insertions on freshly excised bovine and porcine tissues. The needle segmentation network achieved 99.7% accuracy, 86.2% precision, 89.1% recall, and an F-score of 0.87. The needle detection framework successfully localized the needle with a mean tip error of 0.45 ± 0.33 mm and a mean orientation error of 0.42° ± 0.34° and achieved a total processing time of 50 ms per image.
The proposed framework demonstrated the capability to realize robust, accurate, and real-time needle localization during robot-assisted needle insertion processes. It has a promising application in tracking the needle and ensuring the safety of robotic-assisted automatic puncture during challenging US-guided minimally invasive procedures.
在二维超声（US）引导下，机器人辅助的针插入可以有效地提高临床穿刺的准确性和成功率。为此，自动和准确的针跟踪方法对于监测穿刺过程、避免针偏离预定路径以及减少周围组织损伤的风险非常重要。这项工作旨在开发一种用于在插入过程中自动和准确地检测二维 US 图像中插入针的框架。
我们提出了一种新的卷积神经网络架构，该架构由两个通道的编码器和单个通道的解码器组成，用于使用从两个相邻 US 图像帧中提取的针运动信息进行针分割。基于新的网络，我们进一步提出了一种自动针检测框架。根据前一帧的预测结果，从 US 图像中提取针的感兴趣区域，并将其输入到所提出的网络中，以实现更精细和更快的连续针定位。
我们的方法的性能是基于从新鲜切除的牛和猪组织上的机器人辅助针插入中提取的 1000 对 US 图像进行评估的。针分割网络的准确率达到 99.7%，精度为 86.2%，召回率为 89.1%，F1 评分为 0.87。针检测框架成功地定位了针，尖端误差平均为 0.45 ± 0.33mm，方向误差平均为 0.42° ± 0.34°，每张图像的总处理时间为 50ms。
所提出的框架展示了在机器人辅助针插入过程中实现稳健、准确和实时针定位的能力。它有望在跟踪针和确保机器人辅助自动穿刺在具有挑战性的超声引导微创过程中的安全性方面得到应用。
