Opportunities and Challenges of Observational Studies and Randomized Controlled Trials for Evaluating the Therapeutic Efficacy of COVID-19 Convalescent Plasma.

Large randomized controlled trials (RCTs) remain the gold standard for evaluating treatment efficacy. However, observational studies, including non-randomized cohort studies, as well as small RCTs have gained increasing attention especially during the SARS-CoV-2 pandemic where critical evaluation of limited therapeutic options are sought to improve patient care while awaiting results for subsequent RCTs. As the authors have previously discussed, RCTs and observational studies are complementary approaches which often appear synergistic with one another. While not all real-world studies are the same, the results of observational studies are notoriously subject to both known and unknown confounding factors. The utilization of COVID-19 Convalescent Plasma is a timely illustration of evaluating the efficacy and safety of a COVID-19 therapy given the dangerous and often lethal effects of the virus and the limited approved therapeutic options for the disease. While awaiting the results of large RCTS of convalescent plasma, serval observational cohorts and small RCTs have attempted to assess the efficacy and safety of this approach with very mixed results. Among the likely reasons for this failure to provide a definitive answer concerning the value of convalescent plasma are the many limitations inherent to addressing treatment efficacy in non-randomized studies. While such studies are often able to capture information on large numbers of individuals rapidly, it is important to understand that although larger numbers may enhance the precision of estimates provided, larger numbers, in and of themselves, do not increase the accuracy of estimates due to patient selection and other biases. At the same time, both observational studies and small RCTS are at risk for publication bias due to investigator, reviewer and editorial bias toward positive studies. In this commentary we discuss the advantages and limitations of these methodologic approaches when addressing urgently needed evidence on the effectiveness and safety of therapies in a crisis such as the COVID-19 pandemic.