Tracking vaccine effectiveness in an evolving pandemic, countering misleading hot takes and epidemiologic fallacies.

With the emergence of Omicron during the pandemic and the establishment of antibody waning over time, vaccine effectiveness, especially against infection, declined sharply from the original levels seen after the initial rollout. However, studies have demonstrated that they still provided substantial protection vs severe/fatal disease even with Omicron and after waning. Social media has been rife with reports claiming vaccines provided no benefit and some even claiming they made things worse, often driven by simple presentations of raw observational data using erroneous arguments involving epidemiologic fallacies including the base rate fallacy, Simpson's paradox, and the ecological fallacy and ignoring the extensive bias especially from confounding that is an inherent feature of these data. Similar fallacious arguments have been made by some in promoting vaccination policies, as well. Generally, vaccine effectiveness cannot be accurately estimated from raw population summaries but instead require rigorous, careful studies using epidemiologic designs and statistical analysis tools attempting to adjust for key confounders and sources of bias. This article summarizes what aggregated evidence across studies reveals about effectiveness of the mRNA vaccines as the pandemic has evolved, chronologically summarized with emerging variants and highlighting some of the fallacies and flawed arguments feeding social media-based claims that have obscured society's collective understanding.