Evaluating the Test-Negative Design for COVID-19 Vaccine Effectiveness Using Randomized Trial Data: A Secondary Cross-Protocol Analysis of 5 Randomized Clinical Trials

Abstract

Importance: The test-negative design (TND) has been widely used to assess postmarketing COVID-19 vaccine effectiveness but requires further evaluation for this application.

Objective: To determine whether the TND reliably evaluates vaccine effectiveness against symptomatic COVID-19 using placebo-controlled vaccine efficacy randomized clinical trials (RCTs).

Design, setting, and participants: This secondary cross-protocol analysis constructed TND study datasets from study sites in 16 countries across 5 continents using the blinded phase cohorts of 5 harmonized phase 3 COVID-19 Prevention Network RCTs: COVE (Coronavirus Vaccine Efficacy and Safety), AZD1222, ENSEMBLE, PREVENT-19 (Prefusion Protein Subunit Vaccine Efficacy Novavax Trial COVID-19), and VAT00008. Participants included adults who received the intended number of doses, experienced COVID-19-like symptoms, and obtained SARS-CoV-2 testing. Start dates ranged from July 27, 2020, to October 19, 2021; data cutoff dates ranged from March 26, 2021, to March 15, 2022. Statistical analysis was performed from May 11, 2023, to February 25, 2025.

Interventions: Participants received vaccines consisting of messenger RNA-1273 (COVE; 2 doses 28 days apart), ChAdOx1 nCoV-19 (AZD1222; 2 doses 28 days apart), Ad26.COV2.S (ENSEMBLE; 1 dose), NVX-CoV2373 (PREVENT-19; 2 doses 21 days apart), CoV2 preS dTM-AS03 (VAT00008; D614) (2 doses 21 days apart), or CoV2 preS dTM-AS03 (D614 plus B.1.351) (VAT00008; 2 doses 21 days apart) or placebo.

Main outcomes and measures: Main outcomes were symptomatic COVID-19 according to each trial's primary efficacy definition and the Centers for Disease Control and Prevention definition. Vaccine effectiveness was estimated using targeted maximum likelihood estimation under a semiparametric logistic regression model and ordinary logistic regression. Noncase exchangeability, a core TND assumption for unbiased estimation, was also assessed by estimating vaccine efficacy against non-COVID-19 illness.

Results: Among the 12 157 participants included in the analysis, mean (SD) age was 45 (15) years, 6414 were female (53%), 5858 were vaccinated (48%), 2835 experienced primary COVID-19 (23%), and 2992 experienced Centers for Disease Control and Prevention-defined COVID-19 (25%). TND vaccine effectiveness estimates were concordant with RCT vaccine efficacy estimates (concordance correlation coefficient, 0.86 [95% CI, 0.58-0.96] for both outcomes). The semiparametric method had 48% smaller variance estimates than ordinary logistic regression. Noncase exchangeability was generally supported with a median vaccine efficacy against non-COVID-19 illness of 7.7% (IQR, 2.7%-16.8%) across trial cohorts and most 95% CIs including 0.

Conclusions and relevance: In this cross-protocol analysis, the TND provided reliable inferences on COVID-19 vaccine effectiveness in health care-seeking populations for multiple vaccines and symptom definitions when confounding and selection bias were absent. A machine-learning approach for robust confounding control in postmarketing TND studies was also introduced.