MP18-08: Novel Machine Learning Algorithms are Accurate and Noninferior to Human Experts in Predicting Subjective Patient Outcomes Following Third Line Therapy for Overactive Bladder

Glenn T. Werneburg*, Cleveland, OH, Eric Werneburg, Stony Brook, NY, Howard Goldman, Cleveland, OH, Andrew Mullhaupt, Stony Brook, NY, Sandip Vasavada, Cleveland, OH

Introduction: We demonstrated machine learning (ML) algorithms accurately predict objective outcomes following third-line treatment for overactive bladder (OAB).  Importantly, objective improvements (e.g. leak number) do not necessarily translate to subjective improvements, and thus here we developed ML algorithms to predict patient-reported subjective improvement in bladder leakage and function following OnabotulinumtoxinA (OBTX-A) or sacral neuromodulation (SNM) for OAB.  We hypothesized algorithms would exhibit noninferiority to human experts.

Methods: ROSETTA datasets for OAB patients randomized to OBTX-A or SNM were obtained from NICHD.  Novel ML algorithms, using reproducing kernel techniques, were trained and tasked to predict improvement in bladder leakage and function measured by Patient Global Impression of Improvement (PGI-I), a robust and valid patient-reported outcome measure of improvement following OAB treatment in a subset of patients in which algorithms were blinded to outcomes.  Blinded expert FPMRS urologists given the same data also predicted outcomes.  ROC curves with AUC and 95% CI were generated.

Results: Top algorithms exhibited excellent predictive accuracy for bladder function improvement in patients who underwent both OBTX-A (yellow; AUC 0.86, CI: 0.77–0.99, figure left) and SNM (AUC 0.96, CI: 0.88–1.0, figure right), and were noninferior to expert FPMRS urologists (purple and blue; Figure left: OBTX-A; Figure right: SNM).  Top algorithms also exhibited excellent predictive accuracy for bladder leakage improvement in patients who underwent both OBTX-A (AUC 0.75, CI: 0.58–0.90) and SNM (AUC 0.80, CI: 0.62–0.98), and were noninferior to experts.

Conclusions: Novel ML algorithms were accurate and noninferior to experts in prediction of patient-reported subjective improvements in bladder function and leakage in OAB.  Objective outcomes do not necessarily translate to subjective improvement, and thus patient-reported outcomes are an optimal outcome measure.  ML algorithms generated using the described novel techniques may complement clinical judgement in patient counseling, and their generalizability to other scenarios within and outside urology is promising and warrants investigation.

Source of Funding: N/A