Evaluating the Test-Negative Design for COVID-19 Vaccine Effectiveness Using Randomized Trial Data: A Secondary Cross-Protocol Analysis of 5 Randomized Clinical Trials.

IMPORTANCE

The test-negative design (TND) has been widely used to assess postmarketing COVID-19 vaccine effectiveness but requires further evaluation for this application.

OBJECTIVE

To determine whether the TND reliably evaluates vaccine effectiveness against symptomatic COVID-19 using placebo-controlled vaccine efficacy randomized clinical trials (RCTs).

DESIGN, SETTING, AND PARTICIPANTS: This secondary cross-protocol analysis constructed TND study datasets from study sites in 16 countries across 5 continents using the blinded phase cohorts of 5 harmonized phase 3 COVID-19 Prevention Network RCTs: COVE (Coronavirus Vaccine Efficacy and Safety), AZD1222, ENSEMBLE, PREVENT-19 (Prefusion Protein Subunit Vaccine Efficacy Novavax Trial COVID-19), and VAT00008. Participants included adults who received the intended number of doses, experienced COVID-19-like symptoms, and obtained SARS-CoV-2 testing. Start dates ranged from July 27, 2020, to October 19, 2021; data cutoff dates ranged from March 26, 2021, to March 15, 2022. Statistical analysis was performed from May 11, 2023, to February 25, 2025.

INTERVENTIONS

Participants received vaccines consisting of messenger RNA-1273 (COVE; 2 doses 28 days apart), ChAdOx1 nCoV-19 (AZD1222; 2 doses 28 days apart), Ad26.COV2.S (ENSEMBLE; 1 dose), NVX-CoV2373 (PREVENT-19; 2 doses 21 days apart), CoV2 preS dTM-AS03 (VAT00008; D614) (2 doses 21 days apart), or CoV2 preS dTM-AS03 (D614 plus B.1.351) (VAT00008; 2 doses 21 days apart) or placebo.

MAIN OUTCOMES AND MEASURES

Main outcomes were symptomatic COVID-19 according to each trial's primary efficacy definition and the Centers for Disease Control and Prevention definition. Vaccine effectiveness was estimated using targeted maximum likelihood estimation under a semiparametric logistic regression model and ordinary logistic regression. Noncase exchangeability, a core TND assumption for unbiased estimation, was also assessed by estimating vaccine efficacy against non-COVID-19 illness.

RESULTS

Among the 12 157 participants included in the analysis, mean (SD) age was 45 (15) years, 6414 were female (53%), 5858 were vaccinated (48%), 2835 experienced primary COVID-19 (23%), and 2992 experienced Centers for Disease Control and Prevention-defined COVID-19 (25%). TND vaccine effectiveness estimates were concordant with RCT vaccine efficacy estimates (concordance correlation coefficient, 0.86 [95% CI, 0.58-0.96] for both outcomes). The semiparametric method had 48% smaller variance estimates than ordinary logistic regression. Noncase exchangeability was generally supported with a median vaccine efficacy against non-COVID-19 illness of 7.7% (IQR, 2.7%-16.8%) across trial cohorts and most 95% CIs including 0.

CONCLUSIONS AND RELEVANCE

In this cross-protocol analysis, the TND provided reliable inferences on COVID-19 vaccine effectiveness in health care-seeking populations for multiple vaccines and symptom definitions when confounding and selection bias were absent. A machine-learning approach for robust confounding control in postmarketing TND studies was also introduced.