Kelly Anthony, Jensen Esben Kjems, Grua Eoin Martino, Mathiasen Kim, Van de Ven Pepijn
Department of Electronic and Computer Engineering, University of Limerick, Limerick, Ireland.
Health Research Institute, University of Limerick, Limerick, Ireland.
JMIR Med Inform. 2025 Mar 26;13:e64617. doi: 10.2196/64617.
Machine learning (ML) systems in health care have the potential to enhance decision-making but often fail to address critical issues such as prediction explainability, confidence, and robustness in a context-based and easily interpretable manner.
This study aimed to design and evaluate an ML model for a future decision support system for clinical psychopathological treatment assessments. The novel ML model is inherently interpretable and transparent. It aims to enhance clinical explainability and trust through a transparent, hierarchical model structure that progresses from questions to scores to classification predictions. The model confidence and robustness were addressed by applying Monte Carlo dropout, a probabilistic method that reveals model uncertainty and confidence.
A model for clinical psychopathological treatment assessments was developed, incorporating a novel ML model structure. The model aimed at enhancing the graphical interpretation of the model outputs and addressing issues of prediction explainability, confidence, and robustness. The proposed ML model was trained and validated using patient questionnaire answers and demographics from a web-based treatment service in Denmark (N=1088).
The balanced accuracy score on the test set was 0.79. The precision was ≥0.71 for all 4 prediction classes (depression, panic, social phobia, and specific phobia). The area under the curve for the 4 classes was 0.93, 0.92, 0.91, and 0.98, respectively.
We have demonstrated a mental health treatment ML model that supported a graphical interpretation of prediction class probability distributions. Their spread and overlap can inform clinicians of competing treatment possibilities for patients and uncertainty in treatment predictions. With the ML model achieving 79% balanced accuracy, we expect that the model will be clinically useful in both screening new patients and informing clinical interviews.
医疗保健中的机器学习（ML）系统有潜力改善决策，但往往未能以基于上下文且易于解释的方式解决诸如预测可解释性、置信度和稳健性等关键问题。
本研究旨在为临床精神病理学治疗评估的未来决策支持系统设计并评估一个ML模型。这个新颖的ML模型本质上是可解释且透明的。它旨在通过一个从问题到分数再到分类预测的透明分层模型结构来提高临床可解释性和可信度。通过应用蒙特卡洛随机失活（一种揭示模型不确定性和置信度的概率方法）来解决模型的置信度和稳健性问题。
开发了一个用于临床精神病理学治疗评估的模型，纳入了一种新颖的ML模型结构。该模型旨在增强模型输出的图形化解释，并解决预测可解释性、置信度和稳健性问题。使用来自丹麦一个基于网络的治疗服务机构的患者问卷答案和人口统计学数据（N = 1088）对所提出的ML模型进行训练和验证。
测试集上的平衡准确率得分是0.79。所有4个预测类别（抑郁症、恐慌症、社交恐惧症和特定恐惧症）的精确率均≥0.71。这4个类别的曲线下面积分别为0.93、0.92、0.91和0.98。
我们展示了一个心理健康治疗ML模型，它支持对预测类别概率分布进行图形化解释。它们的分布和重叠可以让临床医生了解患者的多种治疗可能性以及治疗预测中的不确定性。鉴于该ML模型达到了79%的平衡准确率，我们预计该模型在筛查新患者和为临床访谈提供信息方面都将具有临床实用性。
