Ren Xinzhen, Zhou Wenju, Yuan Naitong, Li Fang, Ruan Yetian, Zhou Huiyu
Shanghai Key Laboratory of Power Station Automation Technology, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, CO 200444, China.
Shanghai Key Laboratory of Power Station Automation Technology, School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, CO 200444, China.
Med Image Anal. 2025 May;102:103510. doi: 10.1016/j.media.2025.103510. Epub 2025 Feb 28.
Accurate judgment and identification of polyp size is crucial in endoscopic diagnosis. However, the indistinct boundaries of polyps lead to missegmentation and missed cancer diagnoses. In this paper, a prompt-based polyp segmentation method (PPSM) is proposed to assist in early-stage cancer diagnosis during endoscopy. It combines endoscopists' experience and artificial intelligence technology. Firstly, a prompt-based polyp segmentation network (PPSN) is presented, which contains the prompt encoding module (PEM), the feature extraction encoding module (FEEM), and the mask decoding module (MDM). The PEM encodes prompts to guide the FEEM for feature extracting and the MDM for mask generating. So that PPSN can segment polyps efficiently. Secondly, endoscopists' ocular attention data (gazes) are used as prompts, which can enhance PPSN's accuracy for segmenting polyps and obtain prompt data effectively in real-world. To reinforce the PPSN's stability, non-uniform dot matrix prompts are generated to compensate for frame loss during the eye-tracking. Moreover, a data augmentation method based on the segment anything model (SAM) is introduced to enrich the prompt dataset and improve the PPSN's adaptability. Experiments demonstrate the PPSM's accuracy and real-time capability. The results from cross-training and cross-testing on four datasets show the PPSM's generalization. Based on the research results, a disposable electronic endoscope with the real-time auxiliary diagnosis function for early cancer and an image processor have been developed. Part of the code and the method for generating the prompts dataset are available at https://github.com/XinZhenRen/PPSM.
在内镜诊断中，准确判断和识别息肉大小至关重要。然而，息肉边界不清晰会导致分割错误和癌症漏诊。本文提出了一种基于提示的息肉分割方法（PPSM），以辅助内镜检查中的早期癌症诊断。它结合了内镜医师的经验和人工智能技术。首先，提出了一种基于提示的息肉分割网络（PPSN），其包含提示编码模块（PEM）、特征提取编码模块（FEEM）和掩码解码模块（MDM）。PEM对提示进行编码，以指导FEEM进行特征提取和MDM进行掩码生成。从而使PPSN能够高效地分割息肉。其次，将内镜医师的眼部注意力数据（注视）用作提示，这可以提高PPSN分割息肉的准确性，并在现实世界中有效地获取提示数据。为了增强PPSN的稳定性，生成了非均匀点阵提示，以补偿眼动追踪过程中的帧丢失。此外，引入了一种基于分割一切模型（SAM）的数据增强方法，以丰富提示数据集并提高PPSN的适应性。实验证明了PPSM的准确性和实时能力。在四个数据集上进行交叉训练和交叉测试的结果显示了PPSM的泛化能力。基于研究结果，开发了一种具有早期癌症实时辅助诊断功能的一次性电子内镜和图像处理器。部分代码和生成提示数据集的方法可在https://github.com/XinZhenRen/PPSM上获取。
