Chakraborty Bibhas, Laber Eric B, Zhao Ying-Qi
Centre for Quantitative Medicine, Duke-NUS Graduate Medical School, Singapore, Singapore Department of Biostatistics, Columbia University, New York, NY, USA
Department of Statistics, North Carolina State University, Raleigh, NC, USA.
Clin Trials. 2014 Aug;11(4):408-417. doi: 10.1177/1740774514537727. Epub 2014 Jun 12.
A dynamic treatment regime (DTR) comprises a sequence of decision rules, one per stage of intervention, that recommends how to individualize treatment to patients based on evolving treatment and covariate history. These regimes are useful for managing chronic disorders, and fit into the larger paradigm of personalized medicine. The Value of a DTR is the expected outcome when the DTR is used to assign treatments to a population of interest.
The Value of a data-driven DTR, estimated using data from a Sequential Multiple Assignment Randomized Trial, is both a data-dependent parameter and a non-smooth function of the underlying generative distribution. These features introduce additional variability that is not accounted for by standard methods for conducting statistical inference, for example, the bootstrap or normal approximations, if applied without adjustment. Our purpose is to provide a feasible method for constructing valid confidence intervals (CIs) for this quantity of practical interest.
We propose a conceptually simple and computationally feasible method for constructing valid CIs for the Value of an estimated DTR based on subsampling. The method is self-tuning by virtue of an approach called the double bootstrap. We demonstrate the proposed method using a series of simulated experiments.
The proposed method offers considerable improvement in terms of coverage rates of the CIs over the standard bootstrap approach.
In this article, we have restricted our attention to Q-learning for estimating the optimal DTR. However, other methods can be employed for this purpose; to keep the discussion focused, we have not explored these alternatives.
Subsampling-based CIs provide much better performance compared to standard bootstrap for the Value of an estimated DTR.
动态治疗方案（DTR）由一系列决策规则组成，每个干预阶段一个，根据不断变化的治疗和协变量历史，推荐如何对患者进行个体化治疗。这些方案对于管理慢性疾病很有用，并且符合个性化医疗的更大范式。DTR的价值是当DTR用于为感兴趣的人群分配治疗时的预期结果。
使用序贯多重分配随机试验的数据估计的数据驱动DTR的价值，既是一个依赖于数据的参数，也是潜在生成分布的非光滑函数。这些特征引入了额外的变异性，如果不进行调整就应用标准的统计推断方法（例如，自助法或正态近似），则无法考虑这些变异性。我们的目的是提供一种可行的方法，为这个具有实际意义的量构建有效的置信区间（CI）。
我们提出了一种概念上简单且计算可行的方法，用于基于子采样为估计的DTR的价值构建有效的CI。该方法通过一种称为双重自助法的方法进行自我调整。我们使用一系列模拟实验展示了所提出的方法。
与标准自助法相比，所提出的方法在CI覆盖率方面有显著改进。
在本文中，我们将注意力限制在用于估计最优DTR的Q学习上。然而，也可以为此目的采用其他方法；为了保持讨论的重点，我们没有探讨这些替代方法。
与标准自助法相比，基于子采样的CI为估计的DTR的价值提供了更好的性能。
