Readdressing the Ongoing Challenge of Missing Data in Youth Ecological Momentary Assessment Studies: Meta-Analysis Update.

BACKGROUND

Ecological momentary assessment (EMA) is pivotal in longitudinal health research in youth, but potential bias associated with nonparticipation, omitted reports, or dropout threatens its clinical validity. Previous meta-analytic evidence is inconsistent regarding specific determinants of missing data.

OBJECTIVE

This meta-analysis aimed to update and expand upon previous research by examining key participation metrics-acceptance, compliance, and retention-in youth EMA studies. In addition, it sought to identify potential moderators among sample and design characteristics, with the goal of better understanding and mitigating the impact of missing data.

METHODS

We used a bibliographic database search to identify EMA studies involving children and adolescents published from 2001 to November 2023. Eligible studies used mobile-delivered EMA protocols in samples with an average age up to 18 years. We conducted separate meta-analyses for acceptance, compliance, and retention rates, and performed meta-regressions to address sample and design characteristics. Furthermore, we extracted and pooled sample-level effect sizes related to correlates of response compliance. Risk of publication bias was assessed using funnel plots, regression tests, and sensitivity analyses targeting inflated compliance rates.

RESULTS

We identified 285 samples, including 17,441 participants aged 5 to 17.96 years (mean age 14.22, SD 2.24 years; mean percentage of female participants 55.7%). Pooled estimates were 67.27% (k=88, 95% CI 62.39-71.96) for acceptance, 71.97% (k=216, 95% CI 69.83-74.11) for compliance, and 96.57% (k=169, 95% CI 95.42-97.56) for retention. Despite overall poor moderation of participation metrics, acceptance rates decreased as the number of EMA items increased (log-transformed b=-0.115, SE 0.036; 95% CI -0.185 to -0.045; P=.001; R=19.98), compliance rates declined by 0.8% per year of publication (SE 0.25, 95% CI -1.3 to -0.3; P=.002; R=4.17), and retention rates dropped with increasing study duration (log-transformed b=-0.061, SE 0.015; 95% CI -0.091 to 0.032; P<.001; R=10.06). The benefits of monetary incentives on response compliance diminished as the proportion of female participants increased (b=-0.002, SE 0.001; 95% CI -0.003 to -0.001; P=.003; R=9.47). Within-sample analyses showed a small but significant effect indicating higher compliance in girls compared to boys (k=25; g=0.18; 95% CI 0.06-0.31; P=.003), but no significant age-related effects were found (k=14; z score=0.05; 95% CI -0.01 to 0.16).

CONCLUSIONS

Despite a 5-fold increase in included effect sizes compared to the initial review, the variability in rates of missing data that one can expect based on specific sample and design characteristics remains substantial. The inconsistency in identifying robust moderators highlights the need for greater attention to missing data and its impact on study results. To eradicate any health-related bias in EMA studies, researchers should collectively increase transparent reporting practices, intensify primary methodological research, and involve participants' perspectives on missing data.

TRIAL REGISTRATION

PROSPERO CRD42022376948; https://www.crd.york.ac.uk/PROSPERO/view/CRD42022376948.