Guo Zhenghao, Hu Yanyan, Ge Peixuan, Chan In Neng, Yan Tao, Wong Pak Kin, Xu Shaoyong, Li Zheng, Gao Shan
School of Mechanical Engineering, Hubei University of Arts and Science, Xiangyang, China.
Xiangyang Central Hospital, Affiliated Hospital of Hubei University of Arts and Science, Xiangyang, China.
PeerJ Comput Sci. 2025 Mar 25;11:e2780. doi: 10.7717/peerj-cs.2780. eCollection 2025.
Colorectal polyps are potential precursor lesions of colorectal cancer. Accurate classification of colorectal polyps during endoscopy is crucial for early diagnosis and effective treatment. Automatic and accurate classification of colorectal polyps based on convolutional neural networks (CNNs) during endoscopy is vital for assisting endoscopists in diagnosis and treatment. However, this task remains challenging due to difficulties in the data acquisition and annotation processes, the poor interpretability of the data output, and the lack of widespread acceptance of the CNN models by clinicians. This study proposes an innovative approach that utilizes gaze attention information from endoscopists as an auxiliary supervisory signal to train a CNN-based model for the classification of colorectal polyps. Gaze information from the reading of endoscopic images was first recorded through an eye-tracker. Then, the gaze information was processed and applied to supervise the CNN model's attention via an . Comprehensive experiments were conducted on a dataset that contained three types of colorectal polyps. The results showed that EfficientNet_b1 with supervised gaze information achieved an overall test accuracy of 86.96%, a precision of 87.92%, a recall of 88.41%, an F1 score of 88.16%, the area under the receiver operating characteristic (ROC) curve (AUC) is 0.9022. All evaluation metrics surpassed those of EfficientNet_b1 without gaze information supervision. The class activation maps generated by the proposed network also indicate that the endoscopist's gaze-attention information, as auxiliary prior knowledge, increases the accuracy of colorectal polyp classification, offering a new solution to the field of medical image analysis.
结直肠息肉是结直肠癌潜在的前驱病变。在内镜检查过程中对结直肠息肉进行准确分类对于早期诊断和有效治疗至关重要。基于卷积神经网络（CNN）在内镜检查期间对结直肠息肉进行自动准确分类对于辅助内镜医师进行诊断和治疗至关重要。然而，由于数据采集和标注过程存在困难、数据输出的可解释性差以及临床医生对CNN模型缺乏广泛接受度，这项任务仍然具有挑战性。本研究提出了一种创新方法，该方法利用内镜医师的注视注意力信息作为辅助监督信号来训练基于CNN的结直肠息肉分类模型。首先通过眼动仪记录内镜图像阅读过程中的注视信息。然后，对注视信息进行处理，并通过一个……应用于监督CNN模型的注意力。在一个包含三种类型结直肠息肉的数据集上进行了全面实验。结果表明，具有监督注视信息的EfficientNet_b1总体测试准确率为86.96%，精确率为87.92%，召回率为88.41%，F1分数为88.16%，受试者操作特征（ROC）曲线下面积（AUC）为0.9022。所有评估指标均超过了无注视信息监督的EfficientNet_b1。所提出网络生成的类激活映射还表明，内镜医师的注视注意力信息作为辅助先验知识提高了结直肠息肉分类的准确性，为医学图像分析领域提供了一种新的解决方案。
