Symposia
Research Methods and Statistics
Alessandro S. De Nadai, Ph.D.
Assistant Professor
Texas State University
San Marcos, Texas
While unsupervised machine learning (ML) provides great opportunities to contribute to cognitive behavioral research and treatment, there are many under-recognized barriers to its successful application. These issues often emerge because ML methods have been designed for areas where measurement is more precise than in most mental health settings (e.g., computer vision). As a result, many ML approaches are not fully adapted to behavioral research and implementation, which can lead to problems that are consistent with other causes of the replication crisis that has been affecting clinical psychology (Tackett et al., 2019).
To address this situation, the proposed presentation has two parts. The first part will provide a brief overview of unsupervised machine learning. Unsupervised ML is particularly useful for identifying patients who are more likely to respond to treatment. To illustrate this, I will show an example of how unsupervised ML can be used to personalize treatment programs in response to the opioid crisis, with focus on results from a large sample of participants who underwent semistructured assessment (N=36,309; De Nadai et al., 2019). Unsupervised ML is also useful for resolving debates that stem from competing findings, especially in situations where simultaneous positive and negative results might cancel each other out within a sample. To show how this process can work, I will show an example using preliminary results from work that addresses changes in mental health and substance abuse among college students in response to COVID-19 quarantines (N=4,749; Vincent, De Nadai, et al., in preparation). In this research, symptoms worsened in some individuals but not in others. Unsupervised ML is able to separate individuals into different subgroups and provide recommendations that are tailored to each group. When using traditional methodological approaches, these conflicting effects would cancel each other out, resulting in null or inaccurate findings.
The second part of the presentation will highlight best practices and unsupervised ML implementations to avoid. These insights will draw upon new developments in the methodological literature. They will also be informed by experience in related NIH-funded research and experience in peer review of manuscripts that employ unsupervised ML, where several patterns of strengths and preventable weaknesses are apparent. Overall, this presentation will aid ongoing research and clarify interpretation of ML results, which are increasingly being used to inform clinical applications.