Hightow-Weidman Lisa B, Bauermeister Jose A
Institute of Global Health and Infectious Diseases, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
School of Nursing, University of Pennsylvania, Philadelphia, PA, USA.
Mhealth. 2020 Jan 5;6:7. doi: 10.21037/mhealth.2019.10.01. eCollection 2020.
Engagement is the primary metric by which researchers can assess whether participants in a mHealth intervention used and interacted with the intervention's content as intended over a pre-specified period to result in behavior change. Paradata, defined as the process data documenting users' access, participation, and navigation through a mHealth intervention, have been associated with differential treatment outcomes in mHealth interventions. Within behavioral mHealth interventions, there has been an increase in the number of studies addressing the HIV prevention and care continuum in recent years, yet few have presented engagement metrics or examined how these data could inform design modifications, promote continued engagement, and supplement primary intervention efficacy and scale-up efforts.
We review common paradata metrics in mHealth interventions (e.g., amount, frequency, duration and depth of use), using case studies from four technology-driven HIV interventions to illustrate their utility in evaluating mHealth behavioral interventions for HIV prevention and care. Across the four case studies, participants' ages ranged between 15 and 30 years and included a racially and ethnically diverse sample of youth. The four case studies had different approaches for engaging young men who have sex with men: a tailored brief intervention, an interactive modular program, a daily tool to monitor and self-regulate treatment adherence, and an online platform promoting social engagement and social support. Each focused on key outcomes across the HIV prevention and care continuum [e.g., safer sex behaviors, HIV testing, antiretroviral therapy (ART) adherence] and collected paradata metrics systematically.
Across the four interventions, paradata was utilized to identify patterns of use, create user profiles, and determine a minimum engagement threshold for future randomized trials based on initial pilot trial data. Evidence of treatment differences based on paradata analyses were also observed in between-arm and within-arm analyses, indicating that intervention exposure and dosage might influence the strength of the observed intervention effects. Paradata reflecting participants' engagement with intervention content was used to suggest modifications to intervention design and navigation, to understand what theoretically-driven content participants chose to engage with in an intervention, and to illustrate how engagement was linked to HIV-related outcomes.
Paradata monitoring and reporting can enhance the rigor of mHealth trials. Metrics of engagement must be systematically collected, analyzed and interpreted to meaningfully understand a mHealth intervention's efficacy. Future mHealth trials should work to identify suitable engagement metrics during intervention development, ensure their collection throughout the trial, and evaluate their impact on trial outcomes.
参与度是研究人员评估移动健康干预措施的参与者在预先设定的时间段内是否按预期使用并与干预内容进行互动，从而实现行为改变的主要指标。辅助数据被定义为记录用户通过移动健康干预措施进行访问、参与和导航的过程数据，它与移动健康干预措施中的不同治疗结果相关。在行为移动健康干预措施中，近年来关注艾滋病毒预防和护理连续统一体的研究数量有所增加，但很少有研究提出参与度指标，或研究这些数据如何为设计修改提供信息、促进持续参与，并补充主要干预措施的疗效和扩大规模的努力。
我们回顾了移动健康干预措施中常见的辅助数据指标（例如，使用量、频率、持续时间和使用深度），通过四项技术驱动的艾滋病毒干预措施的案例研究来说明它们在评估艾滋病毒预防和护理的移动健康行为干预措施中的作用。在这四项案例研究中，参与者的年龄在15岁至30岁之间，包括一个种族和民族多样化的青年样本。这四项案例研究采用了不同的方法来吸引男男性行为者：一种量身定制的简短干预措施、一个交互式模块化项目、一个用于监测和自我调节治疗依从性的日常工具，以及一个促进社会参与和社会支持的在线平台。每项研究都关注艾滋病毒预防和护理连续统一体的关键结果[例如，更安全的性行为、艾滋病毒检测、抗逆转录病毒疗法（ART）依从性]，并系统地收集辅助数据指标。
在这四项干预措施中，辅助数据被用于识别使用模式、创建用户档案，并根据初步试点试验数据确定未来随机试验的最低参与门槛。在组间和组内分析中也观察到基于辅助数据分析的治疗差异证据，表明干预暴露和剂量可能会影响观察到的干预效果的强度。反映参与者与干预内容互动情况的辅助数据被用于建议对干预设计和导航进行修改，以了解参与者在干预中选择参与哪些理论驱动的内容，并说明参与度如何与艾滋病毒相关结果相关联。
辅助数据监测和报告可以提高移动健康试验的严谨性。必须系统地收集、分析和解释参与度指标，以便有意义地理解移动健康干预措施的疗效。未来的移动健康试验应努力在干预开发过程中确定合适的参与度指标，确保在整个试验过程中收集这些指标，并评估它们对试验结果的影响。
