Elyospeh Zohar, Nobile Bénédicte, Levkovich Inbar, Chancel Raphael, Courtet Philippe, Levi-Belz Yossi
https://ror.org/02f009v59University of Haifa, Mount Carmel, Haifa, Israel.
Department of Emergency Psychiatry and Acute Care, CHU Montpellier, Montpellier, France.
Eur Psychiatry. 2025 May 27;68(1):e81. doi: 10.1192/j.eurpsy.2025.10037.
Improving media adherence to World Health Organization (WHO) guidelines is crucial for preventing suicidal behaviors in the general population. However, there is currently no valid, rapid, and effective method to evaluate the adherence to these guidelines.
This comparative effectiveness study (January-August 2024) evaluated the ability of two artificial intelligence (AI) models (Claude Opus 3 and GPT-4O) to assess the adherence of media reports to WHO suicide-reporting guidelines. A total of 120 suicide-related articles (40 in English, 40 in Hebrew, and 40 in French) published within the past 5 years were sourced from prominent newspapers. Six trained human raters (two per language) independently evaluated articles based on a WHO guideline-based questionnaire addressing aspects, such as prominence, sensationalism, and prevention. The same articles were also processed using AI models. Intraclass correlation coefficients (ICCs) and Spearman correlations were calculated to assess agreement between human raters and AI models.
Overall adherence to WHO guidelines was ~50% across all languages. Both AI models demonstrated strong agreement with human raters, with GPT-4O showing the highest agreement (ICC = 0.793 [0.702; 0.855]). The combined evaluations of GPT-4O and Claude Opus 3 yielded the highest reliability (ICC = 0.812 [0.731; 0.869]).
AI models can replicate human judgment in evaluating media adherence to WHO guidelines. However, they have limitations and should be used alongside human oversight. These findings may suggest that AI tools have the potential to enhance and promote responsible reporting practices among journalists and, thus, may support suicide prevention efforts globally.
提高媒体对世界卫生组织（WHO）指南的遵循程度对于预防普通人群的自杀行为至关重要。然而，目前尚无有效、快速且高效的方法来评估对这些指南的遵循情况。
这项比较有效性研究（2024年1月至8月）评估了两种人工智能（AI）模型（Claude Opus 3和GPT - 4O）评估媒体报道对WHO自杀报告指南遵循情况的能力。从知名报纸中选取了过去5年内发表的总共120篇与自杀相关的文章（40篇英文、40篇希伯来文和40篇法文）。六名经过培训的人类评分员（每种语言两名）根据一份基于WHO指南的问卷对文章进行独立评估，该问卷涉及突出性、轰动性和预防等方面。同样的文章也使用AI模型进行处理。计算组内相关系数（ICC）和斯皮尔曼相关性以评估人类评分员与AI模型之间的一致性。
所有语言对WHO指南的总体遵循率约为50%。两种AI模型都与人类评分员表现出高度一致性，GPT - 4O的一致性最高（ICC = 0.793 [0.702; 0.855]）。GPT - 4O和Claude Opus 3的综合评估产生了最高的可靠性（ICC = 0.812 [0.731; 0.869]）。
AI模型在评估媒体对WHO指南的遵循情况时能够复制人类的判断。然而，它们存在局限性，应在人类监督下使用。这些发现可能表明AI工具具有增强并促进记者进行负责任报道实践的潜力，从而可能支持全球的自杀预防工作。
