School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200040, China.
Shanghai Institute of Hematology, State Key Laboratory of Medical Genomics, National Research Center for Translational Medicine at Shanghai, Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China.
Eur Radiol. 2023 Jan;33(1):77-88. doi: 10.1007/s00330-022-09031-8. Epub 2022 Aug 27.
The prediction of primary treatment failure (PTF) is necessary for patients with diffuse large B-cell lymphoma (DLBCL) since it serves as a prominent means for improving front-line outcomes. Using interim F-fluoro-2-deoxyglucose ([F]FDG) positron emission tomography/computed tomography (PET/CT) imaging data, we aimed to construct multimodal deep learning (MDL) models to predict possible PTF in low-risk DLBCL.
Initially, 205 DLBCL patients undergoing interim [F]FDG PET/CT scans and the front-line standard of care were included in the primary dataset for model development. Then, 44 other patients were included in the external dataset for generalization evaluation. Based on the powerful backbone of the Conv-LSTM network, we incorporated five different multimodal fusion strategies (pixel intermixing, separate channel, separate branch, quantitative weighting, and hybrid learning) to make full use of PET/CT features and built five corresponding MDL models. Moreover, we found the best model, that is, the hybrid learning model, and optimized it by integrating the contrastive training objective to further improve its prediction performance.
The final model with contrastive objective optimization, named the contrastive hybrid learning model, performed best, with an accuracy of 91.22% and an area under the receiver operating characteristic curve (AUC) of 0.926, in the primary dataset. In the external dataset, its accuracy and AUC remained at 88.64% and 0.925, respectively, indicating its good generalization ability.
The proposed model achieved good performance, validated the predictive value of interim PET/CT, and holds promise for directing individualized clinical treatment.
• The proposed multimodal models achieved accurate prediction of primary treatment failure in DLBCL patients. • Using an appropriate feature-level fusion strategy can make the same class close to each other regardless of the modal heterogeneity of the data source domain and positively impact the prediction performance. • Deep learning validated the predictive value of interim PET/CT in a way that exceeded human capabilities.
对于弥漫性大 B 细胞淋巴瘤（DLBCL）患者，预测原发性治疗失败（PTF）是必要的，因为它是改善一线治疗结果的重要手段。我们使用中期 F-氟-2-脱氧葡萄糖（[F]FDG）正电子发射断层扫描/计算机断层扫描（PET/CT）成像数据构建多模态深度学习（MDL）模型，旨在预测低危 DLBCL 患者可能发生的 PTF。
最初，纳入 205 例接受中期[F]FDG PET/CT 扫描和一线标准治疗的 DLBCL 患者，用于模型开发的原始数据集。然后，纳入另外 44 例患者作为外部数据集用于泛化评估。基于 Conv-LSTM 网络的强大主干，我们采用五种不同的多模态融合策略（像素混合、单独通道、单独分支、定量加权和混合学习），充分利用 PET/CT 特征，构建了五个相应的 MDL 模型。此外，我们发现了最好的模型，即混合学习模型，并通过整合对比训练目标对其进行优化，以进一步提高其预测性能。
最终模型（带有对比目标优化的对比混合学习模型）在原始数据集中表现最佳，其准确率为 91.22%，接收者操作特征曲线（ROC）下面积（AUC）为 0.926。在外部数据集中，其准确率和 AUC 分别为 88.64%和 0.925，表明其具有良好的泛化能力。
所提出的模型取得了良好的性能，验证了中期 PET/CT 的预测价值，并有望指导个体化临床治疗。
• 所提出的多模态模型在 DLBCL 患者中实现了对原发性治疗失败的准确预测。• 使用适当的特征级融合策略可以使同类数据无论来自源域模态的异质性如何都彼此靠近，并对预测性能产生积极影响。• 深度学习以超越人类能力的方式验证了中期 PET/CT 的预测价值。
