Elhaminia Behnaz, Gilbert Alexandra, Scarsbrook Andrew, Lilley John, Appelt Ane, Gooya Ali
Centre for Computational Imaging and Simulation Technologies in Biomedicine (CISTIB), Schools of Computing and Medicine, University of Leeds, Leeds, UK.
Leeds Institute of Medical Research at St James's University Hospital, University of Leeds, Leeds, UK.
Phys Imaging Radiat Oncol. 2025 Feb 1;33:100710. doi: 10.1016/j.phro.2025.100710. eCollection 2025 Jan.
A comprehensive understanding of radiotherapy toxicity requires analysis of multimodal data. However, it is challenging to develop a model that can analyse both 3D imaging and clinical data simultaneously. In this study, a deep learning model is proposed for simultaneously analysing computed tomography scans, dose distributions, and clinical metadata to predict toxicity, and identify the impact of clinical risk factors and anatomical regions.
: A deep model based on multiple instance learning with feature-level fusion and attention was developed. The study used a dataset of 313 patients treated with 3D conformal radiation therapy and volumetric modulated arc therapy, with heterogeneous cohorts varying in dose, volume, fractionation, concomitant therapies, and follow-up periods. The dataset included 3D computed tomography scans, planned dose distributions to the bowel cavity, and patient clinical data. The model was trained on patient-reported data on late bowel toxicity.
Results showed that the network can identify potential risk factors and critical anatomical regions. Analysis of clinical data jointly with imaging and dose for bowel urgency and faecal incontinence improved performance (area under receiver operating characteristic curve [AUC] of 88% and 78%, respectively) while best performance for diarrhoea was when analysing clinical features alone (68% AUC).
Results demonstrated that feature-level fusion along with attention enables the network to analyse multimodal data. This method also provides explanations for each input's contribution to the final result and detects spatial associations of toxicity.
全面了解放射治疗毒性需要对多模态数据进行分析。然而，开发一个能够同时分析3D成像和临床数据的模型具有挑战性。在本研究中，我们提出了一种深度学习模型，用于同时分析计算机断层扫描、剂量分布和临床元数据，以预测毒性，并确定临床风险因素和解剖区域的影响。
开发了一种基于多实例学习并具有特征级融合和注意力机制的深度模型。该研究使用了一个包含313例接受三维适形放射治疗和容积调强弧形治疗患者的数据集，这些患者群体在剂量、体积、分割方式、同步治疗和随访时间等方面存在差异。数据集包括三维计算机断层扫描、肠道腔的计划剂量分布以及患者临床数据。该模型基于患者报告的晚期肠道毒性数据进行训练。
结果表明，该网络能够识别潜在风险因素和关键解剖区域。联合分析临床数据与成像和剂量数据，对于肠道急迫感和大便失禁的预测性能有所提高（受试者操作特征曲线下面积[AUC]分别为88%和78%），而对于腹泻，单独分析临床特征时性能最佳（AUC为68%）。
结果表明，特征级融合与注意力机制使网络能够分析多模态数据。该方法还为每个输入对最终结果的贡献提供了解释，并检测了毒性的空间关联。
