Alcohol and Drug Assessment Unit, Princess Alexandra Hospital, Wooloongabba, Brisbane, Queensland 4102, Australia; Discipline of Psychiatry, The University of Queensland, K Floor, Mental Health Centre, Royal Brisbane and Women's Hospital, Herston, Brisbane, Queensland 4029, Australia; Telethon Kids Institute, West Perth, Western Australia 6872, Australia.
Alcohol and Drug Assessment Unit, Princess Alexandra Hospital, Wooloongabba, Brisbane, Queensland 4102, Australia; Centre for Youth Substance Abuse Research, The University of Queensland, Upland Road, St Lucia, Brisbane, Queensland 4072, Australia.
J Subst Abuse Treat. 2019 Apr;99:156-162. doi: 10.1016/j.jsat.2019.01.020. Epub 2019 Jan 30.
Clinical staff providing addiction treatment predict patient outcome poorly. Prognoses based on linear statistics are rarely replicated. Addiction is a complex non-linear behavior. Incorporating non-linear models, Machine Learning (ML) has successfully predicted treatment outcome when applied in other areas of medicine. Using identical assessment data across the two groups, this study compares the accuracy of ML models versus clinical staff to predict alcohol dependence treatment outcome in behavior therapy using patient data only.
Machine learning models (n = 28) were constructed ('trained') using demographic and psychometric assessment data from 780 previously treated patients who had undertaken a 12 week, abstinence-based Cognitive Behavioral Therapy program for alcohol dependence. Independent predictions applying assessment data for an additional 50 consecutive patients were obtained from 10 experienced addiction therapists and the 28 trained ML models. The predictive accuracy of the ML models and the addiction therapists was then compared with further investigation of the 10 best models selected by cross-validated accuracy on the training-set. Variables selected as important for prediction by staff and the most accurate ML model were examined.
The most accurate ML model (Fuzzy Unordered Rule Induction Algorithm, 74%) was significantly more accurate than the four least accurate clinical staff (51%-40%). However, the robustness of this finding may be limited by the moderate area under the receiver operator curve (AUC = 0.49). There was no significant difference in mean aggregate predictive accuracy between 10 clinical staff (56.1%) and the 28 best models (58.57%). Addiction therapists favoured demographic and consumption variables compared with the ML model using more questionnaire subscales.
The majority of staff and ML models were not more accurate than suggested by chance. However, the best performing prediction models may provide useful adjunctive information to standard clinically available prognostic data to more effectively target treatment approaches in clinical settings.
提供成瘾治疗的临床医务人员对患者预后的预测能力较差。基于线性统计的预测结果很少被复制。成瘾是一种复杂的非线性行为。将非线性模型与机器学习（ML）结合使用，在其他医学领域成功地预测了治疗结果。本研究使用相同的评估数据在两组之间进行比较，比较了 ML 模型与临床医务人员仅使用患者数据预测行为治疗酒精依赖治疗结果的准确性。
使用以前接受过为期 12 周、以戒酒为基础的认知行为治疗酒精依赖的 780 名患者的人口统计学和心理测量评估数据，构建（“训练”）了机器学习模型（n=28）。从 10 名经验丰富的成瘾治疗师和 28 个经过训练的 ML 模型中获得了另外 50 名连续患者的独立预测评估数据。然后，比较了 ML 模型和成瘾治疗师的预测准确性，并进一步研究了通过在训练集上进行交叉验证准确性选择的 10 个最佳模型。检查了作为预测重要性的变量以及员工和最准确的 ML 模型选择的变量。
最准确的 ML 模型（模糊无序规则归纳算法，74%）的准确性明显高于 4 名最不准确的临床人员（51%-40%）。然而，由于接受者操作特征曲线下的面积（AUC=0.49）适中，这一发现的稳健性可能受到限制。10 名临床人员（56.1%）和 28 个最佳模型（58.57%）的平均综合预测准确性之间没有显著差异。成瘾治疗师倾向于使用人口统计学和消费变量，而不是 ML 模型，使用更多的问卷子量表。
大多数医务人员和 ML 模型的准确性并不高于随机预测。然而，表现最佳的预测模型可能为标准临床可用预后数据提供有用的辅助信息，以更有效地针对临床环境中的治疗方法。
