Faculty of Industrial Engineering and Systems Management, Amirkabir University of Technology, Tehran, Iran.
Faculty of Industrial Engineering and Systems Management, Amirkabir University of Technology, Tehran, Iran.
J Biomed Inform. 2022 Mar;127:104006. doi: 10.1016/j.jbi.2022.104006. Epub 2022 Jan 30.
Glioblastoma multiforme (GBM) is the most common and deadly type of primary cancers of the brain and central nervous system in adults. Despite the importance of designing a personalized treatment regimen for the patient, clinical trials prescribe a set of conventional regimens for GBM patients. We propose a computerized framework for designing chemo-radiation therapy (CRT) regimen based on patient characteristics.
An intelligent agent, based on deep reinforcement learning, interacts with a virtual personalized GBM. The proposed deep Q network (DQN) uses a deep neural network to estimate the state - action value function. The algorithm stores agent experiences in a replay memory to be used for training of the deep neural network. Also, the proliferation-invasion model is used to simulate spatiotemporal dynamics of GBM growth and its response to therapeutic agents.
Assuming tumor size at the end of the treatment course as a measure of the quality of the treatment regimen, experiments show that the proposed DQN is superior to the Q learning. Also, while the quality of the protocols obtained by the Q learning as well as its convergence speed decreases sharply with the increase in the dimensions of the state-action value function, the DQN is relatively robust against increasing the initial tumor size or lengthening the treatment period.
Our results suggest that the optimal personalized treatment regimen may differ from the conventional regimens suggested by clinical trials. Given the scalability of the proposed DQN in designing treatment regimen for real size tumors, as well as its superiority over previous models, it is a suitable tool for designing personalized CRT regimen for GBM patients.
多形性胶质母细胞瘤(GBM)是成人脑和中枢神经系统中最常见和最致命的原发性癌症。尽管为患者设计个性化治疗方案非常重要,但临床试验为 GBM 患者规定了一套常规方案。我们提出了一种基于患者特征设计化疗-放疗(CRT)方案的计算机框架。
基于深度强化学习的智能体与虚拟个性化 GBM 相互作用。所提出的深度 Q 网络(DQN)使用深度神经网络来估计状态-动作值函数。该算法将代理经验存储在回放存储器中,用于深度神经网络的训练。此外,增殖-侵袭模型用于模拟 GBM 生长及其对治疗剂的时空动力学。
假设治疗过程结束时的肿瘤大小作为治疗方案质量的衡量标准,实验表明,所提出的 DQN 优于 Q 学习。此外,虽然 Q 学习获得的方案质量及其收敛速度随着状态-动作值函数维度的增加而急剧下降,但 DQN 对增加初始肿瘤大小或延长治疗时间具有相对鲁棒性。
我们的结果表明,最佳个性化治疗方案可能与临床试验建议的常规方案不同。鉴于所提出的 DQN 在为实际大小的肿瘤设计治疗方案方面的可扩展性,以及其优于以前模型的优势,它是为 GBM 患者设计个性化 CRT 方案的合适工具。