Perlis Roy H, Verhaak Pilar F, Goldberg Joseph, Cusin Cristina, Ostacher Michael, Malhi Gin S, Zarate Carlos A, Shelton Richard C, Iosifescu Dan V, Tohen Mauricio, Jha Manish Kumar, Sajatovic Martha, Berk Michael
Center for Quantitative Health and Department of Psychiatry, Massachusetts General Hospital, Boston, MA, United States.
Department of Psychiatry, Harvard Medical School, Boston, MA, United States.
J Mood Anxiety Disord. 2025 Jul 14;12:100142. doi: 10.1016/j.xjmad.2025.100142. eCollection 2025 Dec.
We evaluated whether a large language model could assist in selecting psychopharmacological treatments for adults with treatment-resistant depression.
We generated 20 clinical vignettes reflecting treatment-resistant depression among adults based on distributions drawn from electronic health records. Each vignette was evaluated by 2 expert psychopharmacologists to determine and rank the 5 best next-step pharmacologic interventions, as well as contraindicated or poor next-step treatments. Vignettes were then presented in random order, permuting gender and race, to a large language model (Qwen 2.5:7B), augmented with a synopsis of published treatment guidelines. Model output was compared to expert rankings, as well as to those of a convenience sample of community clinicians and an additional group of expert clinicians.
The augmented model prioritized the expert-designated optimal choice for 114/320 vignettes (35.6 %, 95 % CI 30.6 %-41.0 %; Cohen's kappa = 0.34, 95 % CI 0.28-0.39). There were no vignettes for which any of the model choices were among the poor or contraindicated treatments. Results were not meaningfully different when gender or race of the vignette was permuted to examine risk for bias. A sample of community clinicians identified the optimal treatment choice for 12/91 vignettes (13.2 %, 95 % CI: 7.7-21.6 %; Cohen's kappa = 0.10, 95 % CI 0.03-0.18), while an additional group of expert psychopharmacologists identified optimal treatment for 9/140 (6.4 %, 95 %CI: 3.4-11.8 %; Cohen's kappa = 0.03, 95 % CI 0.01-0.08).
An augmented language model demonstrated moderate agreement with expert recommendations and avoided contraindicated treatments, suggesting potential as a tool for supporting complex psychopharmacologic decision-making in treatment-resistant depression.
我们评估了一个大语言模型是否能够协助为患有难治性抑郁症的成年人选择心理药物治疗方案。
我们根据从电子健康记录中提取的分布情况生成了20个反映成年人难治性抑郁症的临床病例。每一个病例都由2名专业精神药理学家进行评估，以确定并排列出5种最佳的下一步药物干预措施，以及禁忌或不佳的下一步治疗方法。然后，将病例以随机顺序呈现，对性别和种族进行排列，交给一个大语言模型（文心一言2.5：7B），并附上已发表治疗指南的概要。将模型的输出结果与专家的排名进行比较，同时也与一组社区临床医生的便利样本以及另一组专家临床医生的排名进行比较。
增强后的模型在320个病例中的114个病例（35.6%，95%置信区间30.6%-41.0%；科恩卡方系数=0.34，95%置信区间0.28-0.39）中优先选择了专家指定的最佳选择。没有任何一个病例的模型选择属于不佳或禁忌治疗方法。当对病例的性别或种族进行排列以检查偏倚风险时，结果没有显著差异。一组社区临床医生在91个病例中的12个病例（13.2%，95%置信区间：7.7-21.6%；科恩卡方系数=0.10，95%置信区间0.03-0.18）中识别出了最佳治疗选择，而另一组专家精神药理学家在140个病例中的9个病例（6.4%，95%置信区间：3.4-11.8%；科恩卡方系数=0.03，95%置信区间0.01-0.08）中识别出了最佳治疗选择。
一个增强后的语言模型与专家建议表现出中等程度的一致性，并且避免了禁忌治疗方法，这表明它有潜力作为一种工具来支持难治性抑郁症中复杂的心理药物治疗决策。
