Sun Jian, Fu Bo, Su Li
School of Data Science, Fudan University, Shanghai 200433, China.
MRC Biostatistics Unit, School of Clinical Medicine, University of Cambridge, Cambridge, CB2 0SR, United Kingdom.
Biometrics. 2025 Jan 7;81(1). doi: 10.1093/biomtc/ujae161.
Dynamic treatment regimes (DTRs) formalize medical decision-making as a sequence of rules for different stages, mapping patient-level information to recommended treatments. In practice, estimating an optimal DTR using observational data from electronic medical record (EMR) databases can be complicated by nonignorable missing covariates resulting from informative monitoring of patients. Since complete case analysis can provide consistent estimation of outcome model parameters under the assumption of outcome-independent missingness, Q-learning is a natural approach to accommodating nonignorable missing covariates. However, the backward induction algorithm used in Q-learning can introduce challenges, as nonignorable missing covariates at later stages can result in nonignorable missing pseudo-outcomes at earlier stages, leading to suboptimal DTRs, even if the longitudinal outcome variables are fully observed. To address this unique missing data problem in DTR settings, we propose 2 weighted Q-learning approaches where inverse probability weights for missingness of the pseudo-outcomes are obtained through estimating equations with valid nonresponse instrumental variables or sensitivity analysis. The asymptotic properties of the weighted Q-learning estimators are derived, and the finite-sample performance of the proposed methods is evaluated and compared with alternative methods through extensive simulation studies. Using EMR data from the Medical Information Mart for Intensive Care database, we apply the proposed methods to investigate the optimal fluid strategy for sepsis patients in intensive care units.
动态治疗方案（DTRs）将医疗决策形式化为针对不同阶段的一系列规则，将患者层面的信息映射到推荐的治疗方案。在实践中，使用电子病历（EMR）数据库中的观察数据估计最优DTR可能会因对患者进行信息性监测导致的不可忽略的协变量缺失而变得复杂。由于在结果独立缺失的假设下，完整病例分析可以提供结果模型参数的一致估计，Q学习是一种适应不可忽略的协变量缺失的自然方法。然而，Q学习中使用的反向归纳算法可能会带来挑战，因为后期阶段不可忽略的协变量缺失会导致早期阶段不可忽略的伪结果缺失，从而导致次优的DTR，即使纵向结果变量被完全观察到。为了解决DTR设置中这个独特的缺失数据问题，我们提出了两种加权Q学习方法，其中通过使用有效的无应答工具变量的估计方程或敏感性分析来获得伪结果缺失的逆概率权重。推导了加权Q学习估计器的渐近性质，并通过广泛的模拟研究评估了所提出方法的有限样本性能，并与其他方法进行了比较。使用重症监护医学信息库（Medical Information Mart for Intensive Care）的EMR数据，我们应用所提出的方法来研究重症监护病房中脓毒症患者的最优液体策略。
