School of Information Science and Engineering, Shandong University, Qingdao 266237, China.
School of Software, Shandong University, Jinan 250101, China.
Comput Biol Med. 2023 Sep;163:107091. doi: 10.1016/j.compbiomed.2023.107091. Epub 2023 Jun 7.
The accurate segmentation of carotid plaques in ultrasound videos will provide evidence for clinicians to evaluate the properties of plaques and treat patients effectively. However, the confusing background, blurry boundaries and plaque movement in ultrasound videos make accurate plaque segmentation challenging. To address the above challenges, we propose the Refined Feature-based Multi-frame and Multi-scale Fusing Gate Network (RMFG_Net), which captures spatial and temporal features in consecutive video frames for high-quality segmentation results and no manual annotation of the first frame. A spatial-temporal feature filter is proposed to suppress the noise of low-level CNN features and promote the detailed target area. To obtain a more accurate plaque position, we propose a transformer-based cross-scale spatial location algorithm, which models the relationship between adjacent layers of consecutive video frames to achieve stable positioning. To make full use of more detailed and semantic information, multi-layer gated computing is applied to fuse features of different layers, ensuring sufficient useful feature map aggregation for segmentation. Experiments on two clinical datasets demonstrate that the proposed method outperforms other state-of-the-art methods under different evaluation metrics, and it processes images with a speed of 68 frames per second which is suitable for real-time segmentation. A large number of ablation experiments were conducted to demonstrate the effectiveness of each component and experimental setting, as well as the potential of the proposed method in ultrasound video plaque segmentation tasks. The codes can be publicly available from https://github.com/xifengHuu/RMFG_Net.git.
颈动脉斑块的准确分割为临床医生评估斑块性质和有效治疗患者提供了依据。然而，超声视频中混杂的背景、模糊的边界和斑块运动使得准确的斑块分割具有挑战性。针对上述挑战，我们提出了基于精炼特征的多帧多尺度融合门网络（RMFG_Net），该网络能够在连续视频帧中捕获空间和时间特征，从而获得高质量的分割结果，并且无需对第一帧进行手动标注。提出了一种时空特征滤波器，以抑制低层 CNN 特征的噪声，并促进目标区域的细节化。为了更准确地定位斑块位置，我们提出了一种基于变形器的跨尺度空间位置算法，该算法可以对连续视频帧的相邻层之间的关系进行建模，从而实现稳定的定位。为了充分利用更详细和语义信息，应用多层门控计算来融合不同层的特征，从而确保分割时充分聚合有用的特征图。在两个临床数据集上的实验表明，该方法在不同的评估指标下优于其他最先进的方法，并且处理图像的速度为 68 帧/秒，适用于实时分割。进行了大量的消融实验，以证明每个组件和实验设置的有效性，以及该方法在超声视频斑块分割任务中的潜力。代码可在 https://github.com/xifengHuu/RMFG_Net.git 上公开获取。
