Applied Statistics Research Unit, Faculty of Mathematics and Geoinformation, TU Wien, Vienna, Austria.
Harvard-MIT Center for Regulatory Science, Harvard Medical School, Boston, MA.
JCO Precis Oncol. 2023 Feb;7:e2200606. doi: 10.1200/PO.22.00606.
Adaptive clinical trials use algorithms to predict, during the study, patient outcomes and final study results. These predictions trigger interim decisions, such as early discontinuation of the trial, and can change the course of the study. Poor selection of the Prediction Analyses and Interim Decisions (PAID) plan in an adaptive clinical trial can have negative consequences, including the risk of exposing patients to ineffective or toxic treatments.
We present an approach that leverages data sets from completed trials to evaluate and compare candidate PAIDs using interpretable validation metrics. The goal is to determine whether and how to incorporate predictions into major interim decisions in a clinical trial. Candidate PAIDs can differ in several aspects, such as the prediction models used, timing of interim analyses, and potential use of external data sets. To illustrate our approach, we considered a randomized clinical trial in glioblastoma. The study design includes interim futility analyses on the basis of the predictive probability that the final analysis, at the completion of the study, will provide significant evidence of treatment effects. We examined various PAIDs with different levels of complexity to investigate if the use of biomarkers, external data, or novel algorithms improved interim decisions in the glioblastoma clinical trial.
Validation analyses on the basis of completed trials and electronic health records support the selection of algorithms, predictive models, and other aspects of PAIDs for use in adaptive clinical trials. By contrast, PAID evaluations on the basis of arbitrarily defined ad hoc simulation scenarios, which are not tailored to previous clinical data and experience, tend to overvalue complex prediction procedures and produce poor estimates of trial operating characteristics such as power and the number of enrolled patients.
Validation analyses on the basis of completed trials and real world data support the selection of predictive models, interim analysis rules, and other aspects of PAIDs in future clinical trials.
适应性临床试验使用算法在研究期间预测患者的结局和最终研究结果。这些预测会触发中期决策，如提前终止试验，并且可能改变研究进程。适应性临床试验中预测分析和中期决策（PAID）计划选择不当可能会产生负面影响，包括使患者面临无效或有毒治疗的风险。
我们提出了一种方法，利用已完成试验的数据来评估和比较候选 PAID，使用可解释的验证指标。目标是确定是否以及如何将预测纳入临床试验中的主要中期决策。候选 PAID 在多个方面可能存在差异，例如使用的预测模型、中期分析的时间以及潜在使用外部数据集。为了说明我们的方法，我们考虑了一项胶质母细胞瘤的随机临床试验。该研究设计包括基于预测概率的中期无效性分析，该概率是最终分析在研究完成时将提供治疗效果显著证据的概率。我们检查了各种具有不同复杂程度的 PAID，以调查是否使用生物标志物、外部数据或新算法改善了胶质母细胞瘤临床试验中的中期决策。
基于已完成试验和电子健康记录的验证分析支持选择算法、预测模型和 PAID 的其他方面，用于适应性临床试验。相比之下，基于任意定义的特定模拟场景的 PAID 评估，这些场景没有针对先前的临床数据和经验进行定制，往往会高估复杂的预测程序，并对试验操作特征（如功效和入组患者数量）产生较差的估计。
基于已完成试验和真实世界数据的验证分析支持在未来临床试验中选择预测模型、中期分析规则和 PAID 的其他方面。
