School of Public Health, Faculty of Medicine, The University of Queensland, Herston, QLD, 4006, Australia.
Queensland Centre for Mental Health Research, The Park Centre for Mental Health, Archerfield, QLD, 4108, Australia.
Syst Rev. 2018 Nov 15;7(1):194. doi: 10.1186/s13643-018-0853-z.
This review assesses the utility of applying an automated content analysis method to the field of mental health policy development. We considered the possibility of using the Wordscores algorithm to assess research and policy texts in ways that facilitate the uptake of research into mental health policy.
The PRISMA framework and the McMaster appraisal tools were used to systematically review and report on the strengths and limitations of the Wordscores algorithm. Nine electronic databases were searched for peer-reviewed journal articles published between 2003 and 2016. Inclusion criteria were (1) articles had to be published in public health, political science, social science or health services disciplines; (2) articles had to be research articles or opinion pieces that used Wordscores; and (3) articles had to discuss both strengths and limitations of using Wordscores for content analysis.
The literature search returned 118 results. Twelve articles met the inclusion criteria. These articles explored a range of policy questions and appraised different aspects of the Wordscores method.
Following synthesis of the material, we identified the following as potential strengths of Wordscores: (1) the Wordscores algorithm can be used at all stages of policy development; (2) it is valid and reliable; (3) it can be used to determine the alignment of health policy drafts with research evidence; (4) it enables existing policies to be revised in the light of research; and (5) it can determine whether changes in policy over time were supported by the evidence. Potential limitations identified were (1) decreased accuracy with short documents, (2) words constitute the unit of analysis and (3) expertise is needed to choose 'reference texts'.
Automated content analysis may be useful in assessing and improving the use of evidence in mental health policies. Wordscores is an automated content analysis option for comparing policy and research texts that could be used by both researchers and policymakers.
本综述评估了将自动化内容分析方法应用于精神卫生政策制定领域的效用。我们考虑了使用 Wordscores 算法评估研究和政策文本的可能性，以便将研究成果更有效地应用于精神卫生政策。
采用 PRISMA 框架和 McMaster 评价工具系统地回顾和报告了 Wordscores 算法的优缺点。9 个电子数据库检索了 2003 年至 2016 年间发表的同行评议期刊文章。纳入标准为：(1)文章必须发表在公共卫生、政治学、社会科学或卫生服务学科领域；(2)文章必须是使用 Wordscores 的研究文章或观点文章；(3)文章必须讨论使用 Wordscores 进行内容分析的优缺点。
文献检索返回 118 个结果。12 篇文章符合纳入标准。这些文章探讨了一系列政策问题，并评价了 Wordscores 方法的不同方面。
对材料进行综合分析后，我们确定了 Wordscores 的以下潜在优势：(1)Wordscores 算法可用于政策制定的所有阶段；(2)它具有有效性和可靠性；(3)可用于确定卫生政策草案与研究证据的一致性；(4)可根据研究结果修改现有政策；(5)可确定政策随时间变化是否有证据支持。确定的潜在局限性包括：(1)短文件的准确性降低；(2)单词构成分析单位；(3)选择“参考文本”需要专业知识。
自动化内容分析可用于评估和提高精神卫生政策中证据的使用。Wordscores 是一种用于比较政策和研究文本的自动化内容分析方法，研究人员和决策者均可使用。
