Imperial Clinical Trials Unit, Imperial College London, White City Campus, Stadium House, 68 Wood Lane, London, W12 7RH, UK.
University of Oxford, Oxford, UK.
BMC Med Res Methodol. 2024 Aug 24;24(1):184. doi: 10.1186/s12874-024-02308-0.
Digital mental health interventions (DMHIs) overcome traditional barriers enabling wider access to mental health support and allowing individuals to manage their treatment. How individuals engage with DMHIs impacts the intervention effect. This review determined whether the impact of user engagement was assessed in the intervention effect in Randomised Controlled Trials (RCTs) evaluating DMHIs targeting common mental disorders (CMDs).
This systematic review was registered on Prospero (CRD42021249503). RCTs published between 01/01/2016 and 17/09/2021 were included if evaluated DMHIs were delivered by app or website; targeted patients with a CMD without non-CMD comorbidities (e.g., diabetes); and were self-guided. Databases searched: Medline; PsycInfo; Embase; and CENTRAL. All data was double extracted. A meta-analysis compared intervention effect estimates when accounting for engagement and when engagement was ignored.
We identified 184 articles randomising 43,529 participants. Interventions were delivered predominantly via websites (145, 78.8%) and 140 (76.1%) articles reported engagement data. All primary analyses adopted treatment policy strategies, ignoring engagement levels. Only 19 (10.3%) articles provided additional intervention effect estimates accounting for user engagement: 2 (10.5%) conducted a complier-average-causal effect (CACE) analysis (principal stratum strategy) and 17 (89.5%) used a less-preferred per-protocol (PP) population excluding individuals failing to meet engagement criteria (estimand strategies unclear). Meta-analysis for PP estimates, when accounting for user engagement, changed the standardised effect to -0.18 95% CI (-0.32, -0.04) from - 0.14 95% CI (-0.24, -0.03) and sample sizes reduced by 33% decreasing precision, whereas meta-analysis for CACE estimates were - 0.19 95% CI (-0.42, 0.03) from - 0.16 95% CI (-0.38, 0.06) with no sample size decrease and less impact on precision. DISCUSSION: Many articles report user engagement metrics but few assessed the impact on the intervention effect missing opportunities to answer important patient centred questions for how well DMHIs work for engaged users. Defining engagement in this area is complex, more research is needed to obtain ways to categorise this into groups. However, the majority that considered engagement in analysis used approaches most likely to induce bias.
数字心理健康干预（DMHI）克服了传统障碍，使更多人能够获得心理健康支持，并允许个人管理自己的治疗。个人与 DMHI 的互动方式会影响干预效果。本综述旨在确定在评估针对常见精神障碍（CMD）的 DMHI 的随机对照试验（RCT）中，是否评估了用户参与对干预效果的影响。
本系统评价已在 Prospero（CRD42021249503）上注册。如果评估的 DMHI 通过应用程序或网站提供、针对无非 CMD 合并症（如糖尿病）的 CMD 患者、且为自我指导，则纳入 RCT。检索数据库：Medline；PsycInfo；Embase；和 CENTRAL。所有数据均由两人进行双重提取。元分析比较了在考虑参与度和忽略参与度时干预效果估计值的差异。
我们确定了 184 篇随机分配 43529 名参与者的文章。干预主要通过网站（145，78.8%）提供，140 篇（76.1%）文章报告了参与数据。所有主要分析均采用治疗政策策略，忽略了参与水平。只有 19 篇（10.3%）文章提供了额外的干预效果估计值，考虑了用户参与度：2 篇（10.5%）进行了遵从平均因果效应（CACE）分析（主要分层策略），17 篇（89.5%）使用了不太理想的符合方案（PP）人群排除不符合参与标准的个体（估计目标策略不明确）。当考虑用户参与度时，PP 估计值的元分析将标准化效应从-0.14（95%CI：-0.24，-0.03）变为-0.18（95%CI：-0.32，-0.04），样本量减少了 33%，降低了精度，而 CACE 估计值的元分析为-0.19（95%CI：-0.42，0.03）与-0.16（95%CI：-0.38，0.06）相比，样本量没有减少，对精度的影响也较小。讨论：许多文章报告了用户参与度指标，但很少有文章评估其对干预效果的影响，从而错失了回答有关 DMHI 对参与用户效果如何的重要以患者为中心的问题的机会。在这一领域定义参与度非常复杂，需要进一步研究以找到将其分类为不同组的方法。然而，大多数在分析中考虑参与度的方法都很可能会引起偏差。
