Department of Human Development, Teachers College, Columbia University, 525 West 120th Street, New York, NY, 10027, USA.
Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, 265 South 37th Street, Philadelphia, USA.
Psychometrika. 2023 Dec;88(4):1171-1196. doi: 10.1007/s11336-023-09937-2. Epub 2023 Oct 24.
Optimal treatment regimes (OTRs) have been widely employed in computer science and personalized medicine to provide data-driven, optimal recommendations to individuals. However, previous research on OTRs has primarily focused on settings that are independent and identically distributed, with little attention given to the unique characteristics of educational settings, where students are nested within schools and there are hierarchical dependencies. The goal of this study is to propose a framework for designing OTRs from multisite randomized trials, a commonly used experimental design in education and psychology to evaluate educational programs. We investigate modifications to popular OTR methods, specifically Q-learning and weighting methods, in order to improve their performance in multisite randomized trials. A total of 12 modifications, 6 for Q-learning and 6 for weighting, are proposed by utilizing different multilevel models, moderators, and augmentations. Simulation studies reveal that all Q-learning modifications improve performance in multisite randomized trials and the modifications that incorporate random treatment effects show the most promise in handling cluster-level moderators. Among weighting methods, the modification that incorporates cluster dummies into moderator variables and augmentation terms performs best across simulation conditions. The proposed modifications are demonstrated through an application to estimate an OTR of conditional cash transfer programs using a multisite randomized trial in Colombia to maximize educational attainment.
优化治疗方案（OTRs）已广泛应用于计算机科学和个性化医学领域，为个人提供数据驱动的最佳推荐。然而，之前关于 OTRs 的研究主要集中在独立同分布的环境中，很少关注教育环境的独特特征，在教育环境中，学生嵌套在学校中，存在层次依赖关系。本研究的目的是提出一种从多站点随机试验中设计 OTRs 的框架，这是教育和心理学中常用的实验设计，用于评估教育计划。我们研究了对流行的 OTR 方法（特别是 Q-学习和加权方法）的修改，以提高它们在多站点随机试验中的性能。通过利用不同的多层次模型、调节剂和增强剂，总共提出了 12 种修改，其中 6 种用于 Q-学习，6 种用于加权。模拟研究表明，所有 Q-学习修改都提高了多站点随机试验的性能，并且纳入随机治疗效果的修改在处理集群级调节剂方面最有前途。在加权方法中，将集群虚拟变量和增强项纳入调节剂变量的修改在所有模拟条件下表现最佳。通过在哥伦比亚的一项多站点随机试验中应用于估计条件现金转移计划的 OTR 来最大化教育程度，展示了所提出的修改。
