(PS7-B26) Predicting Treatment Outcomes in Youth with Irritability Undergoing Exposure-based Cognitive Behavioral Therapy: preliminary Application of Machine Learning

Predicting likelihood of acute and long-term treatment efficacy on an individual level has profound clinical implications for precision medicine (Insel, 2012). Machine learning (ML) techniques applied on clinical data is an emerging field, as understanding the demographic, socioeconomic, and clinical characteristics that are associated with the greatest benefit has the potential for profound clinical impact. Previous studies have utilized machine learning (ML) models to predict responses to psychotherapy in adults with MDD, such as whether patients will experience more relief from CBT or antidepressants (Chekroud et al., 2021; DeRubeis et al., 2014; Wallace et al., 2013). Additionally, ML models have been utilized to predict remission from depression following psychotherapy (Chekroud et al., 2021; Wallert et al., 2018; Ewbank et al, 2021). However, ML models have yet to be applied systematically in pediatric populations with severe and impairing irritability (Brotman et al., 2017). 

Here, leveraging existing data from a completed novel exposure-based treatment for N=38 youth with irritability (e.g., disruptive mood dysregulation disorder, DMDD; M_age= 11.28 years, SD_age = 1.85, 39.5% female, 10.5% Black or African American, 2.6% Asian, 7.9% multiple races, 5.3% Latinx, 76.3% taking medication) we found a significant improvement in symptoms post-treatment (p< 0.001).  Comorbidities were common: 73.7% met criteria for attention deficit hyperactivity disorder (ADHD) and 44.7% for an anxiety disorder. ML modeling was used to predict treatment outcomes from patient baseline demographics; the Clinical Global Impression Scale (CGI-S) were used to identify responders and non-responders.

The specific ML model created utilizes random forest analysis using the Scikit-learn library in R, with the dependent variable of responders vs.  non-responders. All these independent variables are included in one model where the random forest package in R was used.

We compared the accuracy of random forest predicting treatment outcomes measures using age, sex, race, ethnicity, psychiatric medication status, ADHD, and anxiety diagnoses. The cross-validation performance was used to tune the model (N = 30), and a separated test (N = 8) was used to evaluate its accuracy. The test classification accuracy of the random forest model was 62.5 %. The model was able to predict the treatment outcome in out-of-bag (OOB) sample at 73% accuracy. The model classifier struggled to predict the treatment responders (yes) about 17% of the time. The model’s sensitivity was estimated to be 71% and the specificity to be 83%. The three most important predictors of treatment outcome were age, race, and anxiety diagnosis. As these main predictors are identified, more specific models will be run on these main predictors going forward.

While the findings must be validated, analyses provide first steps in the applying of ML in clinical treatment trials in youth.