Cross-sectional evaluation of medical reversals among National Institute of Health guideline practices implemented during the COVID-19 pandemic: how often did experts err in a time of crisis?

OBJECTIVE

The COVID-19 pandemic required the rapid and often widespread implementation of medical practices without robust data. Many of these practices have since been tested in large, randomised trials and were found to be in error. We sought to identify incorrect recommendations, or reversals, among National Institute of Health COVID-19 guidelines and Food and Drug Administration (FDA) approvals and authorisations.

DESIGN

Retrospective cross-sectional study.

PARTICIPANTS

Recommended medical practices and FDA authorisations or approvals for COVID-19 prevention, treatment and/or management.

MAIN OUTCOME MEASURES

The frequency and characteristics of COVID-19 medical reversals, defined as practices that were implemented and/or recommended during the pandemic, but were later tested in randomised trials that failed to find benefit.

RESULTS

We found 332 COVID-19 recommendations. 85 (25.6%) opposed a medical practice, 23 (6.9%) were to continue a pre-COVID standard of care without deviation and 224 (67.5%) reccommended a new medical practice. We found randomised trials assessing 72 of these practices (32.1%), among which 25 (35%) were found to be in error and deemed medical reversals. Among medical reversals, 21 (84%) were prescription medications and 1 (4%) was convalescent plasma. 17 (68%) were repurposed medications. Two (8%) were procedures or mechanical interventions and one (4%) was a device. 16 (64%) reversals pertained to the hospital setting (4 to intensive care units), 4 (16%) were non-specific (ie, applicable to any setting), 4 (16%) pertained to a non-hospital setting and 1 pertained to healthcare workers.

CONCLUSION

When faced with a novel pandemic, policymakers rapidly made hundreds of specific medical recommendations. More than two out of three were never robustly tested. Among practices tested in a randomised fashion, one in three was made in error. Pandemic recommendation errors were substantial. Early and coordinated efforts to initiate randomised trials, even during dire situations, may mitigate the perpetuation of ineffective practices.