"Lies, damned lies ..." and observational studies in comparative effectiveness research.

A new federal initiative has allocated $1.1 billion to comparative effectiveness research, and many have emphasized the importance of including observational studies in this effort. The rationale for using observational studies to assess comparative effectiveness is based on concerns that randomized controlled trials (RCTs) are not "real world" because they enroll homogeneous patient populations, measure study outcomes that are not important to patients, use protocols that are overly complex, are conducted in specialized centers, and use study treatments that are not consistent with usual care, and that RCTs are not always feasible because of a lack of equipoise, the need to assess delayed endpoints, and concerns that they take years to complete and are expensive. This essay questions the validity of each of these proposed limitations, summarizes concerns raised about the accuracy of results generated by observational studies, provides some examples of discrepancies between results of observational studies and RCTs that pertain to pulmonary and critical care, and suggests that using observational studies for comparative effectiveness research may increase rather than decrease the cost of health care and may harm patients.