Department of Psychology, Clinical Psychology Unit, University of Sheffield, Sheffield, UK.
Cumbria, Northumberland, Tyne and Wear NHS Foundation Trust, Penrith, UK.
Lancet Digit Health. 2021 Apr;3(4):e231-e240. doi: 10.1016/S2589-7500(21)00018-2.
Common mental disorders can be effectively treated with psychotherapy, but some patients do not respond well and require timely identification to prevent treatment failure. We aimed to develop and validate a dynamic model to predict psychological treatment outcomes, and to compare the model with currently used methods, including expected treatment response models and machine learning models.
In this prediction model development and validation study, we obtained data from two UK studies including patients who had accessed therapy via Improving Access to Psychological Therapies (IAPT) services managed by ten UK National Health Service (NHS) Trusts between March, 2012, and June, 2018, to predict treatment outcomes. In study 1, we used data on patient-reported depression (Patient Health Questionnaire 9 [PHQ-9]) and anxiety (Generalised Anxiety Disorder 7 [GAD-7]) symptom measures obtained on a session-by-session basis (Leeds Community Healthcare NHS Trust dataset; n=2317) to train the Oracle dynamic prediction model using iterative logistic regression analysis. The outcome of interest was reliable and clinically significant improvement in depression (PHQ-9) and anxiety (GAD-7) symptoms. The predictive accuracy of the model was assessed in an external test sample (Cumbria Northumberland Tyne and Wear NHS Foundation Trust dataset; n=2036) using the area under the curve (AUC), positive predictive values (PPVs), and negative predictive values (NPVs). In study 2, we retrained the Oracle algorithm using a multiservice sample (South West Yorkshire Partnership NHS Foundation Trust, North East London NHS Foundation Trust, Cheshire and Wirral Partnership NHS Foundation Trust, and Cambridgeshire and Peterborough NHS Foundation Trust; n=42 992) and compared its performance with an expected treatment response model and five machine learning models (Bayesian updating algorithm, elastic net regularisation, extreme gradient boosting, support vector machine, and neural networks based on a multilayer perceptron algorithm) in an external test sample (Whittington Health NHS Trust; Barnet Enfield and Haringey Mental Health Trust; Pennine Care NHS Foundation Trust; and Humber NHS Foundation Trust; n=30 026).
The Oracle algorithm trained using iterative logistic regressions generalised well to external test samples, explaining up to 47·3% of variability in treatment outcomes. Prediction accuracy was modest at session one (AUC 0·59 [95% CI 0·55-0·62], PPV 0·63, NPV 0·61), but improved over time, reaching high prediction accuracy (AUC 0·81 [0·77-0·86], PPV 0·79, NPV 0·69) as early as session seven. The performance of the Oracle model was similar to complex (eg, including patient profiling variables) and computationally intensive machine learning models (eg, neural networks based on a multilayer perceptron algorithm, extreme gradient boosting). Furthermore, the predictive accuracy of a more simple dynamic algorithm including only baseline and index-session scores was comparable to more complex algorithms that included additional predictors modelling sample-level and individual-level variability. Overall, the Oracle algorithm significantly outperformed the expected treatment response model (mean AUC 0·80 vs 0·70, p<0·0001]).
Dynamic prediction models using sparse and readily available symptom measures are capable of predicting psychotherapy outcomes with high accuracy.
University of Sheffield.
常见的精神障碍可以通过心理治疗有效地治疗，但有些患者反应不佳，需要及时识别，以防止治疗失败。我们旨在开发和验证一种动态模型来预测心理治疗结果，并将该模型与目前使用的方法进行比较，包括预期治疗反应模型和机器学习模型。
在这项预测模型开发和验证研究中，我们从两项英国研究中获取了数据，这些研究的数据来源于 2012 年 3 月至 2018 年 6 月期间通过英国国家医疗服务体系（NHS）的 10 家信托机构管理的改善心理治疗获取途径（IAPT）服务获得的患者，以预测治疗结果。在研究 1 中，我们使用了关于患者报告的抑郁（患者健康问卷 9 [PHQ-9]）和焦虑（广泛性焦虑症 7 [GAD-7]）症状的基于会话的逐会话数据（利兹社区保健 NHS 信托数据集；n=2317），使用迭代逻辑回归分析训练 Oracle 动态预测模型。感兴趣的结果是抑郁（PHQ-9）和焦虑（GAD-7）症状的可靠和临床显著改善。该模型的预测准确性在外部测试样本（坎布里亚诺森伯兰泰恩和惠尔港 NHS 基金会信托数据集；n=2036）中使用曲线下面积（AUC）、阳性预测值（PPV）和阴性预测值（NPV）进行评估。在研究 2 中，我们使用来自多个服务的样本（南约克郡西部合作伙伴 NHS 基金会信托、东北伦敦 NHS 基金会信托、柴郡和威尔尔比 NHS 基金会信托和剑桥和彼得伯勒 NHS 基金会信托；n=42992）重新训练 Oracle 算法，并将其与预期治疗反应模型和五种机器学习模型（贝叶斯更新算法、弹性网正则化、极端梯度提升、支持向量机和基于多层感知器算法的神经网络）在外部测试样本（惠廷顿健康 NHS 信托；巴内特恩菲尔德和哈林盖心理健康信托；彭宁山 NHS 基金会信托和亨伯 NHS 基金会信托；n=30026）中进行比较。
使用迭代逻辑回归训练的 Oracle 算法很好地推广到了外部测试样本，解释了治疗结果变化的 47.3%。在第一会话时预测精度较低（AUC 0.59[95%CI 0.55-0.62]，PPV 0.63，NPV 0.61），但随着时间的推移逐渐提高，在第七会话时达到了较高的预测精度（AUC 0.81[0.77-0.86]，PPV 0.79，NPV 0.69）。Oracle 模型的性能与复杂（例如，包括患者分析变量）和计算密集型机器学习模型（例如，基于多层感知器算法的神经网络，极端梯度提升）相似。此外，仅包括基线和指数会话分数的更简单动态算法的预测准确性与包括额外预测因子的更复杂算法相当，这些预测因子可以建模样本级和个体级的变异性。总的来说，Oracle 算法显著优于预期治疗反应模型（平均 AUC 0.80 与 0.70，p<0.0001）。
使用稀疏且易于获得的症状测量值的动态预测模型能够以高准确度预测心理治疗结果。
谢菲尔德大学。
