Goldberg Yair, Kosorok Michael R
Department of Biostatistics, The University of North Carolina At Chapel Hill, Chapel Hill, NC 27599, U.S.A.
Ann Stat. 2012 Feb 1;40(1):529-560. doi: 10.1214/12-AOS968.
We develop methodology for a multistage-decision problem with flexible number of stages in which the rewards are survival times that are subject to censoring. We present a novel Q-learning algorithm that is adjusted for censored data and allows a flexible number of stages. We provide finite sample bounds on the generalization error of the policy learned by the algorithm, and show that when the optimal Q-function belongs to the approximation space, the expected survival time for policies obtained by the algorithm converges to that of the optimal policy. We simulate a multistage clinical trial with flexible number of stages and apply the proposed censored-Q-learning algorithm to find individualized treatment regimens. The methodology presented in this paper has implications in the design of personalized medicine trials in cancer and in other life-threatening diseases.
我们针对具有灵活阶段数的多阶段决策问题开发了一种方法，其中奖励是受删失影响的生存时间。我们提出了一种新颖的Q学习算法，该算法针对删失数据进行了调整，并允许灵活的阶段数。我们给出了算法学习到的策略的泛化误差的有限样本界，并表明当最优Q函数属于逼近空间时，算法得到的策略的预期生存时间收敛到最优策略的预期生存时间。我们模拟了一个具有灵活阶段数的多阶段临床试验，并应用所提出的删失Q学习算法来寻找个性化治疗方案。本文提出的方法对癌症和其他危及生命疾病的个性化医学试验设计具有重要意义。
