Does treatment effect depend on control event rate? Revisiting a meta-analysis of suicidality and antidepressant use in children.

BACKGROUND

Weisberg et al. (Clin Trials 2009, 6: 109-18) illustrate that entry criteria in randomized trials can lead to results that are biased for the target population. Presenting data from a previously published meta-analysis of the effect of antidepressants in children on suicidality, they suggest a relationship between the control event rate and relative risk.

PURPOSE

Our purpose is threefold: (1) to show that natural methods of demonstrating a relationship between the control event rate and relative treatment benefit are fraught with danger, (2) to develop better methods, and (3) to assess whether there is evidence of such a relationship in the suicidality meta-analysis.

METHODS

We propose an improved graphical method and an exact and approximate test of whether the treatment effect increases (or decreases) with the control event rate.

RESULTS

The apparent relationship in the naive plot of relative risk against control rate is actually no stronger than what would be expected by chance. Results of our test do not support the conclusion that the odds ratio for suicidality varies with the control rate (one-sided p = 0.39).

LIMITATIONS

We are not able to apply the exact test to this data set. Simulation results indicate that the relationship would have to be very strong to detect it with these sample sizes and control event rates.

CONCLUSIONS

The difficulty in showing that the treatment effect in clinical trials differs by control rate is caused by 3 factors: (1) artificial correlation between a relative risk and control rate, (2) regression dilution bias because the independent variable - the control proportion - is subject to random variability, and (3) low power because we are trying to detect an interaction. Our graph and test are useful tools. Using them, we found no support for a relationship between the control event rate and treatment effect on suicidality.