Evaluating methodological assumptions in comparative effectiveness research: overcoming pitfalls.

The scope of comparative effectiveness research (CER) is wide and therefore requires the application of complex statistical tools and nonstandard procedures. The commonly used methods presuppose the realization of important, and often untestable, assumptions pertaining to the underlying distribution, study heterogeneity and targeted population. Accordingly, the value of the results obtained based on such tools is in large part dependent on the validity of the underlying assumptions relating to the operating characteristics of the procedures. In this article, we elucidate some of the pitfalls that may arise with use of the most commonly used techniques, including those that are applied in network meta-analysis, observational data analysis and patient-reported outcome evaluation. In addition, reference is made to the impact of data quality and database heterogeneity on the performance of commonly used CER tools and the need for standards in order to inform researchers engaged in CER.