Trial-Adjusted Versus Generic Simulated Comparator Trial (SCT) Settings for Selection Bias Appraisal Using the I2 Test.

AIM

The aim was to test two null hypotheses: that I testing with trial-adjusted simulated comparator trial (SCT) settings does not change the odds of identifying selection bias in clinical trials compared to I testing with generic SCT settings, and that I testing with trial-adjusted SCT settings does not change the odds of identifying selection bias in smaller trials (with sample size (n) = 100-199 per treatment group) compared to larger trials (n > 200 per group).

METHODS

Baseline data from 67 randomized controlled trials previously tested for selection bias using the I test with generic SCT settings were extracted. The generic settings were: SCT sample size N = 200 (100 for each of Groups A and B), minimum-maximum range of random values (R) = 67 (minimum = 18, maximum = 85), number of generated SCTs used in all meta-analyses (SCT) = 2. The trials were re-tested with trial-adjusted SCT settings. Additionally, the SCT sample sizes were further increased stepwise to N = 400, 800, and 1200, and the resulting I point estimates were recorded. Positive test results (I > 0%) were assigned a score of 1, while negative test results (I = 0%) were assigned a score of 0. From the resulting 0 and 1 scores of both types of SCT settings, odds ratios (ORs) with 95% confidence intervals (CIs) and p-values were computed. The alpha level was set at 5%.

RESULTS

The original I testing with generic SCT settings yielded four positive and 63 negative results. In contrast, testing with trial-adjusted SCT settings of the same trials revealed 13 positive and 54 negative results (OR: 3.79; 95% CI: 1.17 - 12.32; p = 0.03). When the SCT sample size was increased with trial-adjusted SCT settings, the number of positive results rose from 13 to 16 (OR: 1.30; 95% CI: 0.57 - 2.98; p = 0.53). Consequently, only the first null hypothesis was rejected.

CONCLUSION

I testing with trial-adjusted SCT settings increased the odds of identifying selection bias in clinical trials and did not significantly alter the odds in smaller trials with fewer than 200 patients per intervention group.