Lo Brian, Shi Jenny, Hollenberg Elisa, Abi-Jaoudé Alexxa, Johnson Andrew, Wiljer David
Office of Education, Centre for Addiction and Mental Health, Toronto, ON, Canada.
Information Management Group, Centre for Addiction and Mental Health, Toronto, ON, Canada.
JMIR Ment Health. 2020 Jun 25;7(6):e15942. doi: 10.2196/15942.
Consumer-facing digital health interventions provide a promising avenue to bridge gaps in mental health care delivery. To evaluate these interventions, understanding how the target population uses a solution is critical to the overall validity and reliability of the evaluation. As a result, usage data (analytics) can provide a proxy for evaluating the engagement of a solution. However, there is paucity of guidance on how usage data or analytics should be used to assess and evaluate digital mental health interventions.
This review aimed to examine how usage data are collected and analyzed in evaluations of mental health mobile apps for transition-aged youth (15-29 years).
A scoping review was conducted using the Arksey and O'Malley framework. A systematic search was conducted on 5 journal databases using keywords related to usage and engagement, mental health apps, and evaluation. A total of 1784 papers from 2008 to 2019 were identified and screened to ensure that they included analytics and evaluated a mental health app for transition-aged youth. After full-text screening, 49 papers were included in the analysis.
Of the 49 papers included in the analysis, 40 unique digital mental health innovations were evaluated, and about 80% (39/49) of the papers were published over the past 6 years. About 80% involved a randomized controlled trial and evaluated apps with information delivery features. There were heterogeneous findings in the concept that analytics was ascribed to, with the top 3 being engagement, adherence, and acceptability. There was also a significant spread in the number of metrics collected by each study, with 35% (17/49) of the papers collecting only 1 metric and 29% (14/49) collecting 4 or more analytic metrics. The number of modules completed, the session duration, and the number of log ins were the most common usage metrics collected.
This review of current literature identified significant variability and heterogeneity in using analytics to evaluate digital mental health interventions for transition-aged youth. The large proportion of publications from the last 6 years suggests that user analytics is increasingly being integrated into the evaluation of these apps. Numerous gaps related to selecting appropriate and relevant metrics and defining successful or high levels of engagement have been identified for future exploration. Although long-term use or adoption is an important precursor to realizing the expected benefits of an app, few studies have examined this issue. Researchers would benefit from clarification and guidance on how to measure and analyze app usage in terms of evaluating digital mental health interventions for transition-aged youth. Given the established role of adoption in the success of health information technologies, understanding how to abstract and analyze user adoption for consumer digital mental health apps is also an emerging priority.
面向消费者的数字健康干预措施为弥合心理健康护理服务中的差距提供了一条有前景的途径。为了评估这些干预措施，了解目标人群如何使用某种解决方案对于评估的整体有效性和可靠性至关重要。因此，使用数据（分析）可以为评估解决方案的参与度提供一个代理指标。然而，关于如何使用使用数据或分析来评估数字心理健康干预措施的指导却很匮乏。
本综述旨在研究在针对过渡年龄青年（15 - 29岁）的心理健康移动应用程序评估中如何收集和分析使用数据。
使用阿克西和奥马利框架进行了一项范围综述。使用与使用和参与度、心理健康应用程序以及评估相关的关键词，在5个期刊数据库中进行了系统检索。共识别并筛选了2008年至2019年的1784篇论文，以确保它们包括分析内容并评估了一款针对过渡年龄青年的心理健康应用程序。经过全文筛选，49篇论文被纳入分析。
在纳入分析的49篇论文中，评估了40项独特的数字心理健康创新，约80%（39/49）的论文是在过去6年发表的。约80%涉及随机对照试验，并评估了具有信息传递功能的应用程序。在赋予分析的概念方面存在异质性结果，排在前三位的是参与度、依从性和可接受性。每项研究收集的指标数量也有很大差异，35%（17/49）的论文仅收集1项指标，29%（14/49）的论文收集4项或更多分析指标。完成的模块数量、会话时长和登录次数是最常收集的使用指标。
对当前文献的这项综述表明，在使用分析来评估针对过渡年龄青年的数字心理健康干预措施方面存在显著的变异性和异质性。过去6年中大量的出版物表明，用户分析越来越多地被纳入这些应用程序的评估中。已经确定了许多与选择合适和相关指标以及定义成功或高参与度水平相关的差距，以供未来探索。虽然长期使用或采用是实现应用程序预期效益的重要前提，但很少有研究探讨这个问题。在评估针对过渡年龄青年的数字心理健康干预措施时，研究人员将受益于关于如何测量和分析应用程序使用情况的澄清和指导。鉴于采用在健康信息技术成功中的既定作用，了解如何提取和分析消费者数字心理健康应用程序的用户采用情况也是一个新出现的优先事项。
