Small Study Effects in Diagnostic Imaging Accuracy: A Meta-Analysis.

IMPORTANCE

Small study effects are the phenomena that studies with smaller sample sizes tend to report larger and more favorable effect estimates than studies with larger sample sizes.

OBJECTIVE

To evaluate the presence and extent of small study effects in diagnostic imaging accuracy meta-analyses.

DATA SOURCES

A search was conducted in the PubMed database for diagnostic imaging accuracy meta-analyses published between 2010 and 2019.

STUDY SELECTION

Meta-analyses with 10 or more studies of medical imaging diagnostic accuracy, assessing a single imaging modality, and providing 2 × 2 contingency data were included. Studies that did not assess diagnostic accuracy of medical imaging techniques, compared 2 or more imaging modalities or different methods of 1 imaging modality, were cost analyses, used predictive or prognostic tests, did not provide individual patient data, or were network meta-analyses were excluded.

DATA EXTRACTION AND SYNTHESIS

Data extraction was performed in accordance with the PRISMA guidelines.

MAIN OUTCOMES AND MEASURES

The diagnostic odds ratio (DOR) was calculated for each primary study using 2 × 2 contingency data. Regression analysis was used to examine the association between effect size estimate and precision across meta-analyses.

RESULTS

A total of 31 meta-analyses involving 668 primary studies and 80 206 patients were included. Fixed effects analysis produced a regression coefficient for the natural log of DOR against the SE of the natural log of DOR of 2.19 (95% CI, 1.49-2.90; P < .001), with computed tomography as the reference modality. Interaction test for modality and SE of the natural log of DOR did not depend on modality (Wald statistic P = .50). Taken together, this analysis found an inverse association between effect size estimate and precision that was independent of imaging modality. Of 26 meta-analyses that formally assessed for publication bias using funnel plots and statistical tests for funnel plot asymmetry, 21 found no evidence for such bias.

CONCLUSIONS AND RELEVANCE

This meta-analysis found evidence of widespread prevalence of small study effects in the diagnostic imaging accuracy literature. One likely contributor to the observed effects is publication bias, which can undermine the results of many meta-analyses. Conventional methods for detecting funnel plot asymmetry conducted by included studies appeared to underestimate the presence of small study effects. Further studies are required to elucidate the various factors that contribute to small study effects.