Symposia
Dissemination & Implementation Science
Torrey A. Creed, Ph.D.
Assistant Professor
Perelman School of Medicine at the University of Pennsylvania
Philadelphia, Pennsylvania
To capitalize on investments being made in implementing evidence-based practices, technology is needed to scale up tools that assess therapeutic skills quickly and efficiently. There is tremendous promise in the use of machine learning (ML) technology to support the evaluation of psychotherapy. However, new innovations from university-based research do not automatically –nor often –translate into use in practice. Integration of community stakeholder feedback regarding feasibility and applicability of these tools has the potential to facilitate more rapid adoption. This study examined provider perceptions of an automated fidelity-rating tool for use in the supervision of Cognitive Behavioral Therapy. We gathered quantitative (surveys) and qualitative (focus group) data from community mental health therapists (n=18) and clinical leadership (n=12). Discussions centered on typical supervision practices followed by feedback about the fidelity tool. Transcripts were analyzed using a grounded theory approach. Standard supervision was described as collaboratively guided, either scheduled or spontaneous, and focused on clinical content, self-care, and documentation. Participants highlighted the tool’s utility for supervision, training, and professional growth, but queried its ability to rate skills related to rapport, cultural diversity, and non-verbal communication. Concerns were raised about privacy risks and the impact of low scores on therapist confidence. Desired features included labeling of interventions used and transparency about how fidelity scores related to specific point in the session. Opportunities for asynchronous, remote, and targeted supervision were of particular value. Participants indicated that the tool was highly acceptable (therapist m=4.28, SD=0.56; leadership m=4.19, SD=0.62), appropriate (therapist m=4.07, SD=0.73; leadership m=4.04, SD=0.72), and feasible (therapist m=4.06, SD=0.59; leadership m=4.10, SD=0.69) on a scale of 1-5 where 5 indicated positive perceptions (Weiner et al., 2017. Therapists and clinical leadership also indicated that they had access to the technology needed to use these tools (94.44% and 75.00% respectively), but that they would want additional training to use them (50.00%, 75.00% respectively).ML technology may present an opportunity for an acceptable, appropriate, and feasible approach to large-scale EBP implementation. Continued partnership with community stakeholders will be key for designing such tools in a way that increases likelihood of uptake.