Zhang Hanchen, Yi Hang, Qin Si, Liu Xiaoyin, Liu Guangjian
Department of Medical Ultrasonics, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, China.
Department of Nuclear Medicine, The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, China.
PLoS One. 2024 Dec 11;19(12):e0315339. doi: 10.1371/journal.pone.0315339. eCollection 2024.
Forecasting the patient's response to neoadjuvant chemoradiotherapy (nCRT) is crucial for managing locally advanced rectal cancer (LARC). This study investigates whether a predictive model using image-text features extracted from endorectal ultrasound (ERUS) via Contrastive Language-Image Pretraining (CLIP) can predict tumor regression grade (TRG) before nCRT.
A retrospective analysis of 577 LARC patients who received nCRT followed by surgery was conducted from January 2018 to December 2023. ERUS scans and TRG were used to assess nCRT response, categorizing patients into good (TRG 0) and poor (TRG 1-3) responders. Image and text features were extracted using the ResNet50+RBT3 (RN50) and ViT-B/16+RoBERTa-wwm (VB16) components of the Chinese-CLIP model. LightGBM was used for model construction and comparison. A subset of 100 patients from each responder group was used to compare the CLIP method with manual radiomics methods (logistic regression, support vector machines, and random forest). SHapley Additive exPlanations (SHAP) technique was used to analyze feature contributions.
The RN50 and VB16 models achieved AUROC scores of 0.928 (95% CI: 0.90-0.96) and 0.900 (95% CI: 0.86-0.93), respectively, outperforming manual radiomics methods. SHAP analysis indicated that image features dominated the RN50 model, while both image and text features were significant in the VB16 model.
The CLIP-based predictive model using ERUS image-text features and LightGBM showed potential for improving personalized treatment strategies. However, this study is limited by its retrospective design and single-center data.
预测患者对新辅助放化疗（nCRT）的反应对于局部晚期直肠癌（LARC）的治疗管理至关重要。本研究调查了一种使用通过对比语言-图像预训练（CLIP）从直肠内超声（ERUS）中提取的图像-文本特征的预测模型是否能够在nCRT之前预测肿瘤退缩分级（TRG）。
对2018年1月至2023年12月期间接受nCRT后进行手术的577例LARC患者进行回顾性分析。使用ERUS扫描和TRG来评估nCRT反应，将患者分为反应良好（TRG 0）和反应不佳（TRG 1-3）的患者。使用中文CLIP模型的ResNet50+RBT3（RN50）和ViT-B/16+RoBERTa-wwm（VB16）组件提取图像和文本特征。使用LightGBM进行模型构建和比较。从每个反应组中选取100例患者的子集，将CLIP方法与手动放射组学方法（逻辑回归、支持向量机和随机森林）进行比较。使用SHapley加性解释（SHAP）技术分析特征贡献。
RN50和VB16模型的曲线下面积（AUROC）得分分别为0.928（95%CI：0.90-0.96）和0.900（95%CI：0.86-0.93），优于手动放射组学方法。SHAP分析表明，图像特征在RN50模型中占主导地位，而图像和文本特征在VB16模型中均具有显著性。
使用ERUS图像-文本特征和LightGBM的基于CLIP的预测模型显示出改善个性化治疗策略的潜力。然而，本研究受到其回顾性设计和单中心数据的限制。
