Cross-trial prediction in psychotherapy: External validation of the Personalized Advantage Index using machine learning in two Dutch randomized trials comparing CBT versus IPT for depression.

Optimizing treatment selection may improve treatment outcomes in depression. A promising approach is the Personalized Advantage Index (PAI), which predicts the optimal treatment for a given individual. To determine the generalizability of the PAI, models needs to be externally validated, which has rarely been done. PAI models were developed within each of two independent trials, with substantial between-study differences, that both compared CBT and IPT for depression (STEPd:  = 151 and FreqMech:  = 200). Subsequently, both PAI models were tested in the other dataset. In the STEPd study, post-treatment depression was significantly different between individuals assigned to their PAI-indicated treatment versus those assigned to their non-indicated treatment ( = .57). In the FreqMech study, post-treatment depression was not significantly different between patients receiving their indicated treatment versus those receiving their non-indicated treatment ( = .20). Cross-trial predictions indicated that post-treatment depression was not significantly different between those receiving their indicated treatment and those receiving their non-indicated treatment ( = .16 and  = .27). Sensitivity analyses indicated that cross-trial prediction based on only overlapping variables didn't improve the results. External validation of the PAI has modest results and emphasizes between-study differences and many other challenges.