IEEE J Biomed Health Inform. 2024 Mar;28(3):1228-1239. doi: 10.1109/JBHI.2023.3273728. Epub 2024 Mar 6.
Colonoscopy is considered the best diagnostic tool for early detection and resection of polyps, which can effectively prevent consequential colorectal cancer. In clinical practice, segmenting and classifying polyps from colonoscopic images have a great significance since they provide precious information for diagnosis and treatment. In this study, we propose an efficient multi-task synergetic network (EMTS-Net) for concurrent polyp segmentation and classification, and we introduce a polyp classification benchmark for exploring the potential correlations of the above-mentioned two tasks. This framework is composed of an enhanced multi-scale network (EMS-Net) for coarse-grained polyp segmentation, an EMTS-Net (Class) for accurate polyp classification, and an EMTS-Net (Seg) for fine-grained polyp segmentation. Specifically, we first obtain coarse segmentation masks by using EMS-Net. Then, we concatenate these rough masks with colonoscopic images to assist EMTS-Net (Class) in locating and classifying polyps precisely. To further enhance the segmentation performance of polyps, we propose a random multi-scale (RMS) training strategy to eliminate the interference caused by redundant information. In addition, we design an offline dynamic class activation mapping (OFLD CAM) generated by the combined effect of EMTS-Net (Class) and RMS strategy, which optimizes bottlenecks between multi-task networks efficiently and elegantly and helps EMTS-Net (Seg) to perform more accurate polyp segmentation. We evaluate the proposed EMTS-Net on the polyp segmentation and classification benchmarks, and it achieves an average mDice of 0.864 in polyp segmentation and an average AUC of 0.913 with an average accuracy of 0.924 in polyp classification. Quantitative and qualitative evaluations on the polyp segmentation and classification benchmarks demonstrate that our EMTS-Net achieves the best performance and outperforms previous state-of-the-art methods in terms of both efficiency and generalization.
结肠镜检查被认为是早期发现和切除息肉的最佳诊断工具，这可以有效地预防随后发生的结直肠癌。在临床实践中，从结肠镜图像中分割和分类息肉具有重要意义，因为它们为诊断和治疗提供了宝贵的信息。在本研究中，我们提出了一种高效的多任务协同网络（EMTS-Net），用于同时进行息肉分割和分类，并引入了一个息肉分类基准，以探索上述两个任务的潜在相关性。该框架由一个用于粗粒度息肉分割的增强多尺度网络（EMS-Net）、一个用于准确息肉分类的 EMTS-Net（Class）和一个用于细粒度息肉分割的 EMTS-Net（Seg）组成。具体来说，我们首先使用 EMS-Net 获得粗分割掩模。然后，我们将这些粗略的掩模与结肠镜图像连接起来，以帮助 EMTS-Net（Class）精确定位和分类息肉。为了进一步增强息肉的分割性能，我们提出了一种随机多尺度（RMS）训练策略，以消除冗余信息造成的干扰。此外，我们设计了一种由 EMTS-Net（Class）和 RMS 策略的联合作用生成的离线动态类激活映射（OFLD CAM），它有效地、优雅地优化了多任务网络之间的瓶颈，帮助 EMTS-Net（Seg）进行更准确的息肉分割。我们在息肉分割和分类基准上评估了所提出的 EMTS-Net，它在息肉分割方面的平均 mDice 为 0.864，在息肉分类方面的平均 AUC 为 0.913，平均准确率为 0.924。在息肉分割和分类基准上的定量和定性评估表明，我们的 EMTS-Net 达到了最佳性能，并在效率和泛化方面优于以前的最先进方法。
