(137) Assessing The Robustness of COVID-19 Vaccine Effectiveness Studies to Bias

Publication Number: 468

Background: Policy responses to the COVID-19 pandemic have relied heavily on observational studies of vaccine effectiveness (VE), particularly against new variants of the virus.

Objectives: To assess the robustness of a test-negative design (TND) COVID-19 VE study to a range of biases.

Methods: Design
Test-negative case-control study
Data sources
A questionnaire was sent to 23,963 adults aged ≥70 years with a first positive COVID-19 polymerase chain reaction (PCR) test and symptoms of COVID-19 between 1 and 19 February 2021, to collect information on sources of bias in a TND VE study by Bernal 2021 which used national vaccination and PCR testing data in England.
Setting
Community in England.
Exposures or interventions
Vaccination with BNT162b2 or ChAdOx1-S
Main outcome measures
VE estimate against COVID-19 infection
Statistical analysis
The original TND study used logistic regression to estimate VE adjusting for age, gender, ethnicity, geography, index of multiple deprivation, care home status and week of onset. Sources of bias were described using additional questionnaire data including self-reported vaccination and infection onset dates, COVID-19 symptoms, co-morbidities and extreme vulnerability to COVID-19, care home status, household size and type, vaccine delay and reasons, and mixing behaviours after vaccination. The original analysis was updated to adjust for potential biases.

Results: 8,648 individuals (response 36.1%) were included. There was minimal evidence of exposure and outcome misclassification, confounding by comorbidities, riskier behaviour after vaccination or vaccination itself associated with COVID-19. Additional adjustment for potential confounding by household size, type and clinical vulnerability decreased the final reported VE estimate from 88% (95%CI: 79-94%) to 87% (95%CI: 78-93%) after a second dose of BNT162b2. The follow-up for patients with ChAdOx1-S was too short to assess. Health-seeking behaviour markers were not available to assess potential confounding or collider bias. Assessment of healthy vaccinee bias from vaccine delay was limited since it was difficult to determine an accurate COVID-19 onset date.

Conclusions: Early TND COVID-19 VE studies appear robust to several biases, however, the current study was limited by questionnaire responses and data available. Further research is required to assess the robustness of TND and other observational studies to confounding by health-seeking behaviour, collider bias, and healthy vaccinee bias from vaccine delay.