Neary Martha, Bunyi John, Palomares Kristina, Mohr David C, Powell Adam, Ruzek Josef, Williams Leanne M, Wykes Til, Schueller Stephen M
Department of Psychological Science, University of California, University of California, Irvine, CA, USA.
Center for Behavioral Intervention Technologies, Department of Preventive Medicine, Northwestern University, Chicago, IL, USA.
Digit Health. 2021 Oct 29;7:20552076211053690. doi: 10.1177/20552076211053690. eCollection 2021 Jan-Dec.
Given the increasing number of publicly available mental health apps, we need independent advice to guide adoption. This paper discusses the challenges and opportunities of current mental health app rating systems and describes the refinement process of one prominent system, the One Mind PsyberGuide Credibility Rating Scale (PGCRS).
PGCRS Version 1 was developed in 2013 and deployed for 7 years, during which time a number of limitations were identified. Version 2 was created through multiple stages, including a review of evaluation guidelines and consumer research, input from scientific experts, testing, and evaluation of face validity. We then re-reviewed 161 mental health apps using the updated rating scale, investigated the reliability and discrepancy of initial scores, and updated ratings on the One Mind PsyberGuide public app guide.
Reliabilities across the scale's 9 items ranged from -0.10 to 1.00, demonstrating that some characteristics of apps are more difficult to rate consistently. The average overall score of the 161 reviewed mental health apps was 2.51/5.00 (range 0.33-5.00). Ratings were not strongly correlated with app store star ratings, suggesting that credibility scores provide different information to what is contained in star ratings.
PGCRS summarizes and weights available information in 4 domains: intervention specificity, consumer ratings, research, and development. Final scores are created through an iterative process of initial rating and consensus review. The process of updating this rating scale and integrating it into a procedure for evaluating apps demonstrates one method for determining app quality.
鉴于公开可用的心理健康应用程序数量不断增加，我们需要独立的建议来指导其应用。本文讨论了当前心理健康应用程序评级系统面临的挑战和机遇，并描述了一个著名系统——一心 PsyberGuide 可信度评级量表（PGCRS）的完善过程。
PGCRS 第 1 版于 2013 年开发并使用了 7 年，在此期间发现了一些局限性。第 2 版通过多个阶段创建，包括审查评估指南和消费者研究、科学专家的意见、测试以及表面效度评估。然后，我们使用更新后的评级量表重新审查了 161 款心理健康应用程序，调查了初始分数的可靠性和差异，并在一心 PsyberGuide 公共应用程序指南上更新了评级。
该量表 9 个项目的可靠性范围为 -0.10 至 1.00，表明应用程序的某些特征更难进行一致评级。161 款经审查的心理健康应用程序的平均总分为 2.51/5.00（范围为 0.33 - 5.00）。评级与应用商店星级评分的相关性不强，这表明可信度分数提供的信息与星级评分中的信息不同。
PGCRS 在四个领域对可用信息进行汇总和加权：干预特异性、消费者评级、研究和开发。最终分数通过初始评级和共识审查的迭代过程得出。更新此评级量表并将其整合到应用程序评估程序中的过程展示了一种确定应用程序质量的方法。
