(268) Propensity Score Estimation Strategies for Inclusion of Surgeon Confounders Under Different Surgeon Impact on Treatment Allocation for Medical Device Epidemiology: A Simulation Study

Publication Number: 942

Background: Surgeon-related factors such as years of experience or number of surgeries performed can act as confounders in observational studies involving surgeon-level information. This creates challenges in propensity score (PS) estimation due to the clustered nature of the data.

Objectives: To estimate the accuracy and precision of random effect modelling (REM) compared to logistic regression (LR) for PS estimation for a range of different surgeon-level confounders impact on treatment allocation scenarios from negligible to strong impact.

Methods: Multi-level Carlo simulations (1,000 iterations) with 10,000 patients nested under 500 different surgeons (ratio 20:1) and sample size of n=10,000 were conducted. Five patient confounders, one instrumental variable and one risk factor were generated, fixed true treatment effect at odd ratio (OR) 1.5. Two surgeon-level confounders were generated, one binary and one continuous with OR = 1.5 for effect on treatment outcome and OR = 1.01, 1.25, 1.5 and 2.5 association with treatment allocation. Six different PS estimation strategies were tested i) logistic regression PS excluding surgeon-level confounders; ii) logistic regression PS including surgeon-level confounders; iii) same as ii) but including cross-level interactions; iv), v) and vi), similar to i), ii) and iii) respectively but using REM instead of logistic regression PS. Then the treatment effects were estimated using REM as outcome model weighted with stabilized inverse probability weighting calculated from the PS model (i to vi). The average relative bias, mean square error (MSE) and empirical standard error of the treatment effect estimates for i to iv were calculated by comparing with the true effect.

Results: The relative bias, MSE and empirical standard error of the treatment estimates increased in the strong surgeon effect on treatment allocation scenario (OR = 2.5) regardless of the PS strategy used. For example, PS strategy ii) the relative bias, MSE and empirical standard error were 14.3%, 0.005 and 0.058 for OR =2.5 scenarios compared to 9.78%, 0.002, 0.050 for OR =1.5 scenarios.

Conclusions: The accuracy and precision of the estimates decrease as the impact of surgeon confounders on treatment allocation increases for all propensity score weighting strategies tested. This suggests propensity score weighting may not be suitable for medical device studies with strong surgeon impact on treatment allocation.