Qi Jing, Ruan Guangcong, Liu Jia, Yang Yi, Cao Qian, Wei Yanling, Nian Yongjian
Department of Digital Medicine, School of Biomedical Engineering and Imaging Medicine, Army Medical University (Third Military Medical University), Chongqing 400038, China.
Department of Gastroenterology, Daping Hospital, Army Medical University (Third Military Medical University), Chongqing 400042, China.
Bioengineering (Basel). 2022 Nov 1;9(11):632. doi: 10.3390/bioengineering9110632.
Evaluating the severity of ulcerative colitis (UC) through the Mayo endoscopic subscore (MES) is crucial for understanding patient conditions and providing effective treatment. However, UC lesions present different characteristics in endoscopic images, exacerbating interclass similarities and intraclass differences in MES classification. In addition, inexperience and review fatigue in endoscopists introduces nontrivial challenges to the reliability and repeatability of MES evaluations. In this paper, we propose a pyramid hybrid feature fusion framework (PHF3) as an auxiliary diagnostic tool for clinical UC severity classification. Specifically, the PHF3 model has a dual-branch hybrid architecture with ResNet50 and a pyramid vision Transformer (PvT), where the local features extracted by ResNet50 represent the relationship between the intestinal wall at the near-shot point and its depth, and the global representations modeled by the PvT capture similar information in the cross-section of the intestinal cavity. Furthermore, a feature fusion module (FFM) is designed to combine local features with global representations, while second-order pooling (SOP) is applied to enhance discriminative information in the classification process. The experimental results show that, compared with existing methods, the proposed PHF3 model has competitive performance. The area under the receiver operating characteristic curve (AUC) of MES 0, MES 1, MES 2, and MES 3 reached 0.996, 0.972, 0.967, and 0.990, respectively, and the overall accuracy reached 88.91%. Thus, our proposed method is valuable for developing an auxiliary assessment system for UC severity.
通过梅奥内镜亚评分（MES）评估溃疡性结肠炎（UC）的严重程度对于了解患者病情和提供有效治疗至关重要。然而，UC病变在内镜图像中呈现出不同的特征，加剧了MES分类中的类间相似性和类内差异。此外，内镜医师缺乏经验和审查疲劳给MES评估的可靠性和可重复性带来了不小的挑战。在本文中，我们提出了一种金字塔混合特征融合框架（PHF3）作为临床UC严重程度分类的辅助诊断工具。具体而言，PHF3模型具有一个带有ResNet50和金字塔视觉Transformer（PvT）的双分支混合架构，其中ResNet50提取的局部特征代表近景点处肠壁与其深度之间的关系，而PvT建模的全局表示捕获肠腔横截面中的相似信息。此外，设计了一个特征融合模块（FFM）将局部特征与全局表示相结合，同时应用二阶池化（SOP）在分类过程中增强判别信息。实验结果表明，与现有方法相比，所提出的PHF3模型具有有竞争力的性能。MES 0、MES 1、MES 2和MES 3的受试者工作特征曲线下面积（AUC）分别达到0.996、0.972、0.967和0.990，总体准确率达到88.91%。因此，我们提出的方法对于开发UC严重程度的辅助评估系统具有重要价值。
