The inclusion of historical control data may reduce the power of a confirmatory study.

Large sample sizes in clinical trials increase the cost of clinical research and delay the availability of new treatments. Fewer patients could be recruited into clinical trials if historical data on the comparator could be used reliably in a trial's analysis. However, old trials may bias rather than augment data from a new trial if, for example, the standard of care has improved over time. A hierarchical model for the data from the current and historical trials decreases the weight given to the historical data in line with the discrepancy between the results from the different trials. This reduces the risk of substantial bias. This paper shows that this down-weighting is not sufficiently sensitive to differences in the response rates between trials. Motivated by recent trials in HIV, this paper proposes and examines a more conservative weighting of historical data. Simulation showed that both the standard hierarchical and the proposed weighting of historical data led to Type II error rates worse than those attained by ignoring the historical data completely. This underlines the risks of including historical data in the primary analysis of a trial for which strict control of error rates is paramount.