Franken Katinka, Ten Klooster Peter, Bohlmeijer Ernst, Westerhof Gerben, Kraiss Jannis
Department of Psychology, Health and Technology, Faculty of Behavioural, Management and Social Sciences, University of Twente, Enschede, Netherlands.
Front Psychiatry. 2023 Sep 25;14:1236551. doi: 10.3389/fpsyt.2023.1236551. eCollection 2023.
Anxiety and mood disorders greatly affect the quality of life for individuals worldwide. A substantial proportion of patients do not sufficiently improve during evidence-based treatments in mental healthcare. It remains challenging to predict which patients will or will not benefit. Moreover, the limited research available on predictors of treatment outcomes comes from efficacy RCTs with strict selection criteria which may limit generalizability to a real-world context. The current study evaluates the performance of different machine learning (ML) models in predicting non-improvement in an observational sample of patients treated in routine specialized mental healthcare.
In the current longitudinal exploratory prediction study diagnosis-related, sociodemographic, clinical and routinely collected patient-reported quantitative outcome measures were acquired during treatment as usual of 755 patients with a primary anxiety, depressive, obsessive compulsive or trauma-related disorder in a specialized outpatient mental healthcare center. ML algorithms were trained to predict non-response (< 0.5 standard deviation improvement) in symptomatic distress 6 months after baseline. Different models were trained, including models with and without early change scores in psychopathology and well-being and models with a trimmed set of predictor variables. Performance of trained models was evaluated in a hold-out sample (30%) as a proxy for unseen data.
ML models without early change scores performed poorly in predicting six-month non-response in the hold-out sample with Area Under the Curves (AUCs) < 0.63. Including early change scores slightly improved the models' performance (AUC range: 0.68-0.73). Computationally-intensive ML models did not significantly outperform logistic regression (AUC: 0.69). Reduced prediction models performed similar to the full prediction models in both the models without (AUC: 0.58-0.62 vs. 0.58-0.63) and models with early change scores (AUC: 0.69-0.73 vs. 0.68-0.71). Across different ML algorithms, early change scores in psychopathology and well-being consistently emerged as important predictors for non-improvement.
Accurately predicting treatment outcomes in a mental healthcare context remains challenging. While advanced ML algorithms offer flexibility, they showed limited additional value compared to traditional logistic regression in this study. The current study confirmed the importance of taking early change scores in both psychopathology and well-being into account for predicting longer-term outcomes in symptomatic distress.
焦虑和情绪障碍极大地影响着全球个人的生活质量。相当一部分患者在精神卫生保健的循证治疗期间改善不充分。预测哪些患者会受益或不会受益仍然具有挑战性。此外，关于治疗结果预测因素的现有研究有限，这些研究来自具有严格选择标准的疗效随机对照试验，这可能会限制其在现实世界背景中的普遍性。本研究评估了不同机器学习（ML）模型在预测常规专科精神卫生保健中接受治疗的患者观察样本中未改善情况的性能。
在当前的纵向探索性预测研究中，在一家专科门诊精神卫生保健中心对755例患有原发性焦虑、抑郁、强迫或创伤相关障碍的患者进行常规治疗期间，获取了与诊断相关、社会人口统计学、临床和常规收集的患者报告的定量结果指标。训练ML算法以预测基线后6个月症状困扰方面无反应（改善<0.5标准差）的情况。训练了不同的模型，包括有和没有精神病理学和幸福感早期变化分数的模型，以及具有精简预测变量集的模型。在一个留出样本（30%）中评估训练模型的性能，作为未见过数据的代理。
在留出样本中，没有早期变化分数的ML模型在预测6个月无反应方面表现不佳，曲线下面积（AUC）<0.63。纳入早期变化分数略微改善了模型的性能（AUC范围：0.68 - 0.73）。计算密集型ML模型没有显著优于逻辑回归（AUC：0.69）。在没有早期变化分数的模型（AUC：0.58 - 0.62对0.58 - 0.63）和有早期变化分数的模型（AUC：0.69 - 0.73对0.68 - 0.71）中，简化预测模型的表现与完整预测模型相似。在不同的ML算法中，精神病理学和幸福感方面的早期变化分数始终是未改善的重要预测因素。
在精神卫生保健背景下准确预测治疗结果仍然具有挑战性。虽然先进的ML算法提供了灵活性，但在本研究中与传统逻辑回归相比，它们显示出有限的附加价值。本研究证实了在预测症状困扰的长期结果时，考虑精神病理学和幸福感方面的早期变化分数的重要性。
