Comparative Evaluation of Automated Approaches for Confounder Selection in Ultra High-dimensional Data with Rare Outcomes

Background: Ultra high-dimensional settings occur when the number of baseline variables available for confounder selection exceeds the sample size. Large-scale confounder selection in ultra high-dimensional data remains challenging, particularly when outcome events are rare. These situations are increasing in drug-safety and effectiveness research as linked electronic health records with claims data are becoming more available.

Objectives: We aimed to investigate the performance of several lasso-based methods for large-scale confounder selection in ultra high-dimensional data with rare outcomes using simulated datasets.

Methods: We simulated datasets with a sample size of 5,000, outcome incidence of 2%, treatment prevalence of 30%, and 10,000 binary baseline covariates (100 confounders, 10 risk factors, and 10 instrumental and 9,880 spurious variables). We estimated the propensity score (PS) using 4 lasso-based models: 1) treatment lasso, 2) outcome-adaptive lasso, 3) collaborative-controlled lasso, 4) collaborative-controlled outcome-adaptive lasso. Model 1 chooses variables to optimize treatment prediction while Model 2 prioritizes variables based on the strength of estimated associations with the outcome. Model 3 utilizes the ‘collaborative learning' framework to consider variable associations with both treatment and outcome with the goal to minimize a bias-variance tradeoff in the estimated treatment effect. Finally, Model 4 extends Model 3 by placing a higher weight on predictors of the outcome prior to running Model 3. We fit each model using traditional lasso estimation and again using the relaxed lasso fit which runs lasso twice—once on the full covariate set and again on the selected variables—to reduce ‘regularization bias'. For each PS model, we estimated the treatment effect using targeted maximum likelihood estimation that included initial predictions from an outcome lasso model.

Results: In terms of cross-validated predictive performance for treatment, model 1 performed best followed by models 2, 3, then 4 (C-statistics ranging from 0.7 to 0.75). However, in terms of reducing bias in the estimated treatment effect, model 4 consistently performed best (bias reduction of 72% relative to the unadjusted estimate). After re-running all models using the relaxed-lasso fit, the final models were more parsimonious with less shrinkage, which resulted in a further reduction in bias (bias reduction of 89%).

Conclusions: The collaborative-controlled outcome-adaptive lasso with a relaxed fit consistently performed best in terms of reducing measured confounding and regularization bias in simulations. This approach may be promising for improved bias reduction in ultra high-dimensional data with rare outcomes.