(263) Overlap Weighting Vs Inverse Probability of Treatment Weighting: An Empirical Evaluation of Propensity Score Methods to Minimise Confounding in COVID-19 Vaccine Effectiveness Studies

Publication Number: 937

Background: While several methods to minimise confounding were used in previous COVID-19 vaccine effectiveness studies, a rigorous assessment of their ability to resolve confounding was not yet completed.

Objectives: To empirically evaluate the comparative performance of two propensity score (PS) methods, overlap weighting (OW) and inverse probability of treatment weighting (IPTW) with trimming at [0.05-0.95], to minimise confounding in COVID-19 vaccine effectiveness studies.

Methods: We used UK primary care records (CPRD AURUM), mapped to the OMOP CDM. All people aged ≥ 75 years, who were not previously infected with or vaccinated against SARS-CoV-2 as of January 4th 2021, were included.
Subsequently, individuals were assigned to the vaccinated and unvaccinated cohorts based on administered COVID-19 vaccination between 4th-28th January 2021. Index date for vaccinated people was the vaccination date. For the unvaccinated cohort, index dates were randomly assigned following the distribution of the vaccinated cohort’s index dates.
Lasso regression was used to identify relevant covariates (e.g. conditions, drug exposures) for inclusion to large-scale PS. In addition, location, age, prior observation time, regional vaccination rates, diagnostic effort and COVID-19 incidence rates at index date were forced into the PS equation. We estimated three different PS, including different levels of location granularity: (1) no location, (2) regions or (3) GP practice.
Following PS weighting, standardized mean differences (SMD) were used to assess covariate imbalance and compare OW and IPTW performance in minimising observed confounding. Negative Control Outcomes (NCO) were used to detect residual confounding.

Results: Vaccinated and unvaccinated cohorts comprised 583,813 and 332,315 individuals, respectively. Prior to weighting, baseline characteristics were largely unbalanced between the cohorts, particularly age [SMD 0.31], location (region [0.14] and GP practice [0.74]) and GP visit counts. Overall, OW showed lower minimum SMD (median: 0.002; max: 0.025) for all covariates compared to IPTW (median: 0.004; max: 0.135). Assessing NCO, OW showed lower systematic error (26% NCOs associated with exposure) than IPTW in most scenarios (56%). Unweighted analyses showed one-sided systematic error, with 74% NCOs being positively associated with vaccination.

Conclusions: Our study found OW to be preferable over IPTW in minimising observed and unobserved confounding in COVID-19 vaccine effectiveness research. Additionally, our findings illustrate the need to incorporate patient location and related variables (e.g. testing and transmission rates) to minimize community-level confounding.