The Alan Turing Institute, British Library, 96 Euston Road, London, NW1 2DB, UK.
University College London, Institute of Health Informatics, 222 Euston Rd, London, NW1 2DA, UK.
BMC Med Inform Decis Mak. 2023 Jun 16;23(1):110. doi: 10.1186/s12911-023-02207-2.
Precision medicine requires reliable identification of variation in patient-level outcomes with different available treatments, often termed treatment effect heterogeneity. We aimed to evaluate the comparative utility of individualized treatment selection strategies based on predicted individual-level treatment effects from a causal forest machine learning algorithm and a penalized regression model.
Cohort study characterizing individual-level glucose-lowering response (6 month reduction in HbA1c) in people with type 2 diabetes initiating SGLT2-inhibitor or DPP4-inhibitor therapy. Model development set comprised 1,428 participants in the CANTATA-D and CANTATA-D2 randomised clinical trials of SGLT2-inhibitors versus DPP4-inhibitors. For external validation, calibration of observed versus predicted differences in HbA1c in patient strata defined by size of predicted HbA1c benefit was evaluated in 18,741 patients in UK primary care (Clinical Practice Research Datalink).
Heterogeneity in treatment effects was detected in clinical trial participants with both approaches (proportion predicted to have a benefit on SGLT2-inhibitor therapy over DPP4-inhibitor therapy: causal forest: 98.6%; penalized regression: 81.7%). In validation, calibration was good with penalized regression but sub-optimal with causal forest. A strata with an HbA1c benefit > 10 mmol/mol with SGLT2-inhibitors (3.7% of patients, observed benefit 11.0 mmol/mol [95%CI 8.0-14.0]) was identified using penalized regression but not causal forest, and a much larger strata with an HbA1c benefit 5-10 mmol with SGLT2-inhibitors was identified with penalized regression (regression: 20.9% of patients, observed benefit 7.8 mmol/mol (95%CI 6.7-8.9); causal forest 11.6%, observed benefit 8.7 mmol/mol (95%CI 7.4-10.1).
Consistent with recent results for outcome prediction with clinical data, when evaluating treatment effect heterogeneity researchers should not rely on causal forest or other similar machine learning algorithms alone, and must compare outputs with standard regression, which in this evaluation was superior.
精准医学需要可靠地识别不同治疗方案下患者结局的差异，通常称为治疗效果异质性。本研究旨在评估基于因果森林机器学习算法和惩罚回归模型预测的个体治疗效果，对个体化治疗选择策略的比较效用。
本队列研究描述了 2 型糖尿病患者起始 SGLT2 抑制剂或 DPP4 抑制剂治疗后 6 个月时的血糖降低反应（HbA1c 降低值）的个体水平。模型开发集包括 CANTATA-D 和 CANTATA-D2 随机临床试验中 SGLT2 抑制剂与 DPP4 抑制剂的 1428 名参与者。为了外部验证，在英国初级保健的临床实践研究数据链接（Clinical Practice Research Datalink）中，根据预测 HbA1c 获益大小定义的患者亚组，评估了观察到的 HbA1c 差异与预测 HbA1c 差异之间的校准。
两种方法均检测到临床试验参与者的治疗效果存在异质性（预测在 SGLT2 抑制剂治疗中获益的比例高于 DPP4 抑制剂治疗：因果森林：98.6%；惩罚回归：81.7%）。在验证中，惩罚回归的校准效果较好，但因果森林的校准效果较差。使用惩罚回归识别出 SGLT2 抑制剂治疗 HbA1c 获益>10mmol/mol 的亚组（患者 3.7%，观察到获益 11.0mmol/mol[95%CI 8.0-14.0]），但因果森林未识别出该亚组，而惩罚回归还识别出了一个 HbA1c 获益 5-10mmol/mol 的更大亚组（回归：20.9%的患者，观察到获益 7.8mmol/mol[95%CI 6.7-8.9]；因果森林 11.6%，观察到获益 8.7mmol/mol[95%CI 7.4-10.1]）。
与最近关于临床数据预测结局的结果一致，当评估治疗效果异质性时，研究人员不应仅依赖因果森林或其他类似的机器学习算法，而必须将输出与标准回归进行比较，在本评估中，标准回归更优。
