Quantifying the Association Between Psychotherapy Content and Clinical Outcomes Using Deep Learning.
ABSTRACT: Importance:Compared with the treatment of physical conditions, the quality of care of mental health disorders remains poor and the rate of improvement in treatment is slow, a primary reason being the lack of objective and systematic methods for measuring the delivery of psychotherapy. Objective:To use a deep learning model applied to a large-scale clinical data set of cognitive behavioral therapy (CBT) session transcripts to generate a quantifiable measure of treatment delivered and to determine the association between the quantity of each aspect of therapy delivered and clinical outcomes. Design, Setting, and Participants:All data were obtained from patients receiving internet-enabled CBT for the treatment of a mental health disorder between June 2012 and March 2018 in England. Cognitive behavioral therapy was delivered in a secure online therapy room via instant synchronous messaging. The initial sample comprised a total of 17 572 patients (90 934 therapy session transcripts). Patients self-referred or were referred by a primary health care worker directly to the service. Exposures:All patients received National Institute for Heath and Care Excellence-approved disorder-specific CBT treatment protocols delivered by a qualified CBT therapist. Main Outcomes and Measures:Clinical outcomes were measured in terms of reliable improvement in patient symptoms and treatment engagement. Reliable improvement was calculated based on 2 severity measures: Patient Health Questionnaire (PHQ-9) and Generalized Anxiety Disorder 7-item scale (GAD-7), corresponding to depressive and anxiety symptoms respectively, completed by the patient at initial assessment and before every therapy session (see eMethods in the Supplement for details). Results:Treatment sessions from a total of 14 899 patients (10 882 women) aged between 18 and 94 years (median age, 34.8 years) were included in the final analysis. We trained a deep learning model to automatically categorize therapist utterances into 1 or more of 24 feature categories. The trained model was applied to our data set to obtain quantifiable measures of each feature of treatment delivered. A logistic regression revealed that increased quantities of a number of session features, including change methods (cognitive and behavioral techniques used in CBT), were associated with greater odds of reliable improvement in patient symptoms (odds ratio, 1.11; 95% CI, 1.06-1.17) and patient engagement (odds ratio, 1.20, 95% CI = 1.12-1.27). The quantity of nontherapy-related content was associated with reduced odds of symptom improvement (odds ratio, 0.89; 95% CI, 0.85-0.92) and patient engagement (odds ratio, 0.88, 95% CI, 0.84-0.92). Conclusions and Relevance:This work demonstrates an association between clinical outcomes in psychotherapy and the content of therapist utterances. These findings support the principle that CBT change methods help produce improvements in patients' presenting symptoms. The application of deep learning to large clinical data sets can provide valuable insights into psychotherapy, informing the development of new treatments and helping standardize clinical practice.
Project description:Transdiagnostic Cognitive Behavior Therapy (TCBT) manuals delivered in individual format have been reported to be just as effective as traditional diagnosis specific CBT manuals. We have translated and modified the "The Unified Protocol for Transdiagnostic Treatment of Emotional Disorders" (UP-CBT) for group delivery in Mental Health Service (MHS), and shown effects comparable to traditional CBT in a naturalistic study. As the use of one manual instead of several diagnosis-specific manuals could simplify logistics, reduce waiting time, and increase therapist expertise compared to diagnosis specific CBT, we aim to test the relative efficacy of group UP-CBT and diagnosis specific group CBT.The study is a partially blinded, pragmatic, non-inferiority, parallel, multi-center randomized controlled trial (RCT) of UP-CBT vs diagnosis specific CBT for Unipolar Depression, Social Anxiety Disorder and Agoraphobia/Panic Disorder. In total, 248 patients are recruited from three regional MHS centers across Denmark and included in two intervention arms. The primary outcome is patient-ratings of well-being (WHO Well-being Index, WHO-5), secondary outcomes include level of depressive and anxious symptoms, personality variables, emotion regulation, reflective functioning, and social adjustment. Assessments are conducted before and after therapy and at 6 months follow-up. Weekly patient-rated outcomes and group evaluations are collected for every session. Outcome assessors, blind to treatment allocation, will perform the observer-based symptom ratings, and fidelity assessors will monitor manual adherence.The current study will be the first RCT investigating the dissemination of the UP in a MHS setting, the UP delivered in groups, and with depressive patients included. Hence the results are expected to add substantially to the evidence base for rational group psychotherapy in MHS. The planned moderator and mediator analyses could spur new hypotheses about mechanisms of change in psychotherapy and the association between patient characteristics and treatment effect.Clinicaltrials.gov NCT02954731 . Registered 25 October 2016.
Project description:OBJECTIVE:To improve success rates in psychotherapy, we developed and evaluated an alliance-focused training (AFT) protocol with regard to patient-therapist interpersonal behavior in a 30-session protocol of cognitive-behavioral therapy (CBT) for outpatients comorbid with Axis I and II conditions. METHOD:Participants included 40 patients treated by 40 therapists in a multiple baseline design in which novice therapists trained to fidelity standards in CBT were introduced to AFT at different time intervals (after either 8 or 16 sessions) during a 30-session CBT protocol. Interpersonal behaviors were assessed with a simplified version of the Structural Analysis of Social Behavior (SASB) on videotaped sessions sampled from the early (between Sessions 6 through 8), mid (Sessions 14 through 16), and late (Sessions 22 through 24) phases of therapy. RESULTS:As predicted, several significant interactions were observed between within-subject interpersonal change and between-groups differences in exposure to AFT. Specifically, there were decreases in patient dependence and in therapist control (including criticism), plus increases in patient expressiveness and in therapist affirmation and expressiveness, all of which could be attributed to AFT. The predictive relationship of several of these variables to session-level and overall treatment outcome was also demonstrated. CONCLUSIONS:This study demonstrates that novice CBT therapists can be trained to improve their interpersonal process with patients who present with comorbid diagnoses, including a personality disorder. (PsycINFO Database Record
Project description:OBJECTIVE:Despite a growing body of research supporting the efficacy of cognitive-behavioral therapy (CBT) for depressed adolescents, few studies have investigated the role of the acquisition and use of CBT skills in accounting for symptom improvement. The present study examined the role of cognitive versus behavioral skills in predicting symptom improvement in depressed youth. Analyses considered different raters of patient skills (patient vs. therapist) as well as disaggregated between-patient versus within-patient effects. METHOD:Data were derived from a 12-week clinical trial of CBT for depressed adolescent females (N = 33; ages 13-18 years; 69.7% White). Both therapist-report and patient-report measures of CBT skills (skills of cognitive therapy) were acquired at 5 time points throughout therapy: Sessions 1, 3, 6, 9, and 12. Depressive symptoms (Beck Depression Inventory-II) were assessed at every session. RESULTS:Therapist and patient ratings of CBT skills showed small to moderate associations (rs = .20-.38). Intraclass correlation coefficients indicated that the majority of the variance in skills scores (61-90%) was attributable to within-patient variance from session to session, rather than due to between-patient differences. When disaggregating within-patient and between-patient effects, and consistent with a causal relationship, within-patient variability in both patient-rated (b = -2.55; p = .025) and therapist-rated (b = -2.41; p = .033) behavioral skills predicted subsequent symptom change. CONCLUSIONS:Analyses highlight the importance of the acquisition and use of behavioral skills in CBT for depressed adolescents. Findings also underscore the importance of disentangling within-patient from between-patient effects in future studies, an approach infrequently used in process-outcome research. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Project description:BACKGROUND:Bona fide psychotherapy approaches are effective treatments for generalized anxiety disorder (GAD) compared to no-treatment conditions. Treatment manuals and protocols allow a relatively high degree of freedom for the way therapists implement these overall treatment packages and there is a systematic lack of knowledge on how therapists should customize these treatments. The present study experimentally examines two implementation strategies of customizing a bona fide psychotherapy approach based on a 16 session time-limited cognitive-behavioral therapy (CBT) protocol and their relation to the post-session and ultimate treatment outcomes. METHODS:This trial contrasts two different implementation strategies of how to customize the in-session structure of a manual-based CBT-protocol for GAD. The patients will be randomly assigned to two implementation conditions: (1) a systematic focus on subtle changes lasting from 7 to 20 min at the check-in phase of every psychotherapy session and (2) a state-of-the-art (SOTA) check-in phase lasting several minutes mainly focused on the session goals. Potential therapist effects will be examined based on an ABAB crossed-therapist design. Treatment outcomes will be assessed at the following times: post-session outcomes, treatment outcome at post assessment and 6- as well as 12-month follow-up. DISCUSSION:The proposed randomized clinical implementation trial addresses the clinically relevant question of how to customize a bona fide psychotherapy protocol experimentally contrasting two implementation strategies. Through the development and testing of the proposed implementation design, this trial has the potential to inform therapists about efficacious implementation strategies of how to customize a manual-based treatment protocol in respect to the timing of the in-session structure. TRIAL REGISTRATION:This trial was registered at ClinicalTrials.gov ( NCT03079336 ) at March 14, 2017.
Project description:This randomized trial will compare three methods of assessing fidelity to cognitive-behavioral therapy (CBT) for youth to identify the most accurate and cost-effective method. The three methods include self-report (i.e., therapist completes a self-report measure on the CBT interventions used in session while circumventing some of the typical barriers to self-report), chart-stimulated recall (i.e., therapist reports on the CBT interventions used in session via an interview with a trained rater, and with the chart to assist him/her) and behavioral rehearsal (i.e., therapist demonstrates the CBT interventions used in session via a role-play with a trained rater). Direct observation will be used as the gold-standard comparison for each of the three methods.This trial will recruit 135 therapists in approximately 12 community agencies in the City of Philadelphia. Therapists will be randomized to one of the three conditions. Each therapist will provide data from three unique sessions, for a total of 405 sessions. All sessions will be audio-recorded and coded using the Therapy Process Observational Coding System for Child Psychotherapy-Revised Strategies scale. This will enable comparison of each measurement approach to direct observation of therapist session behavior to determine which most accurately assesses fidelity. Cost data associated with each method will be gathered. To gather stakeholder perspectives of each measurement method, we will use purposive sampling to recruit 12 therapists from each condition (total of 36 therapists) and 12 supervisors to participate in semi-structured qualitative interviews.Results will provide needed information on how to accurately and cost-effectively measure therapist fidelity to CBT for youth, as well as important information about stakeholder perspectives with regard to each measurement method. Findings will inform fidelity measurement practices in future implementation studies as well as in clinical practice.NCT02820623 , June 3rd, 2016.
Project description:Importance:Cognitive behavioral therapy (CBT), the best-studied treatment for body dysmorphic disorder (BDD), has to date not been compared with therapist-delivered supportive psychotherapy, the most commonly received psychosocial treatment for BDD. Objective:To determine whether CBT for BDD (CBT-BDD) is superior to supportive psychotherapy in reducing BDD symptom severity and associated BDD-related insight, depressive symptoms, functional impairment, and quality of life, and whether these effects are durable. Design, Setting, and Participants:This randomized clinical trial conducted at Massachusetts General Hospital and Rhode Island Hospital recruited adults with BDD between October 24, 2011, and July 7, 2016. Participants (n?=?120) were randomized to the CBT-BDD arm (n?=?61) or the supportive psychotherapy arm (n?=?59). Weekly treatments were administered at either hospital for 24 weeks, followed by 3- and 6-month follow-up assessments. Measures were administered by blinded independent raters. Intention-to-treat statistical analyses were performed from February 9, 2017, to September 22, 2018. Interventions:Cognitive behavioral therapy for BDD, a modular skills-based treatment, addresses the unique symptoms of the disorder. Supportive psychotherapy is a nondirective therapy that emphasizes the therapeutic relationship and self-esteem; supportive psychotherapy was enhanced with BDD-specific psychoeducation and treatment rationale. Main Outcomes and Measures:The primary outcome was BDD symptom severity measured by the change in score on the Yale-Brown Obsessive-Compulsive Scale Modified for BDD from baseline to end of treatment. Secondary outcomes were the associated symptoms and these were assessed using the Brown Assessment of Beliefs Scale, Beck Depression Inventory-Second Edition, Sheehan Disability Scale, and Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form. Results:Of the 120 participants, 92 (76.7%) were women, with a mean (SD) age of 34.0 (13.1) years. The difference in effectiveness between CBT-BDD and supportive psychotherapy was site specific: at 1 site, no difference was detected (estimated mean [SE] slopes, -18.6?[1.9] vs -16.7?[1.9]; P?=?.48; d growth-modeling analysis change, -0.25), whereas at the other site, CBT-BDD led to greater reductions in BDD symptom severity, compared with supportive psychotherapy (estimated mean [SE] slopes, -18.6?[2.2] vs -7.6?[2.0]; P?<?.001; d growth-modeling analysis change,?-1.36). No posttreatment symptom changes were observed throughout the 6 -months of follow-up (all slope P???.10). Conclusions and Relevance:Body dysmorphic disorder severity and associated symptoms appeared to improve with both CBT-BDD and supportive psychotherapy, although CBT-BDD was associated with more consistent improvement in symptom severity and quality of life. Trial Registration:ClinicalTrials.gov identifier: NCT01453439.
Project description:OBJECTIVE:Two decades of empirical research suggest that changes in symptoms are not linear, and many patients gain much of their symptom reduction in one between-sessions interval. Theoretically, such gains are expected to be manifested in the working alliance as well, following a rupture session; however, no study to date has directly examined between-sessions sudden gains in the alliance. In the present study we examined whether ruptures predict subsequent sudden gains in the alliance, which in turn show an effect on outcome that is specific to the treatment in which the alliance is conceptualized as an active mechanism of change. METHOD:In a sample of 241 patient-therapist dyads, patients received either brief relational therapy (BRT), in which the alliance is conceptualized as an active mechanism of change, or cognitive-behavioral therapy (CBT), in which it is not. We examined whether patient and therapist reports of ruptures predicted sudden gains in alliance in the subsequent session, and whether early sudden gains in alliance were significantly associated with treatment outcome in BRT versus CBT. RESULTS:Rupture sessions, as reported by therapists but not by patients, predicted a sudden gain in both patient and therapist-reported alliance in the subsequent session. Findings revealed a moderating effect of treatment condition on the association between sudden gains and treatment outcome, in which gains in alliance were associated with better treatment outcome in BRT than in CBT. CONCLUSIONS:The findings support the potential role of gains in alliance as a specific mechanism of change in BRT versus CBT. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Project description:OBJECTIVE:Randomized comparisons of acceptance-based treatments with traditional cognitive behavioral therapy (CBT) for anxiety disorders are lacking. To address this gap, we compared acceptance and commitment therapy (ACT) to CBT for heterogeneous anxiety disorders. METHOD:One hundred twenty-eight individuals (52% female, mean age = 38, 33% minority) with 1 or more DSM-IV anxiety disorders began treatment following randomization to CBT or ACT; both treatments included behavioral exposure. Assessments at pre-treatment, post-treatment, and 6- and 12-month follow-up measured anxiety-specific (principal disorder Clinical Severity Ratings [CSRs], Anxiety Sensitivity Index, Penn State Worry Questionnaire, Fear Questionnaire avoidance) and non-anxiety-specific (Quality of Life Index [QOLI], Acceptance and Action Questionnaire-16 [AAQ]) outcomes. Treatment adherence, therapist competency ratings, treatment credibility, and co-occurring mood and anxiety disorders were investigated. RESULTS:CBT and ACT improved similarly across all outcomes from pre- to post-treatment. During follow-up, ACT showed steeper linear CSR improvements than CBT (p < .05, d = 1.26), and at 12-month follow-up, ACT showed lower CSRs than CBT among completers (p < .05, d = 1.10). At 12-month follow-up, ACT reported higher AAQ than CBT (p = .08, d = 0.42; completers: p < .05, d = 0.56), whereas CBT reported higher QOLI than ACT (p < .05, d = 0.42). Attrition and comorbidity improvements were similar; ACT used more non-study psychotherapy at 6-month follow-up. Therapist adherence and competency were good; treatment credibility was higher in CBT. CONCLUSIONS:Overall improvement was similar between ACT and CBT, indicating that ACT is a highly viable treatment for anxiety disorders.
Project description:<b>Background:</b> Written and online information about cognitive-behavioral therapy (CBT) prioritizes the role of specific techniques (e.g., cognitive restructuring) and typically omits discussion of "common factors" (e.g., the working alliance, or therapist empathy). However, according to extensive psychotherapy process research the common factors may be important mediators of client improvement. <b>Objectives:</b> This study aimed to assess lay opinions about the role of specific and common factors in CBT for depression. We also aimed to determine how different client disclosure processes might affect lay opinions about the relative importance of specific and common factors in CBT. <b>Methods:</b> We conducted a web-based experiment involving a sample of US participants who had never undergone psychotherapy. All participants were presented with similar vignettes describing an individual suffering from depression whose doctor recommends CBT. Participants were randomized to read one of six vignettes created in a 2 × 3 factorial design that crossed client gender with type of informed consent (Standard CBT Disclosure vs. Common Factors and CBT Disclosure vs. No Disclosure). <b>Results:</b> Disclosure type had a significant effect on participants' ratings of Common and Specific factors in psychotherapy. As compared to the CBT disclosure, participants allocated to the Common Factors disclosure rated Empathy and Positive Regard as significantly more important to treatment outcome, and rated the Specific factors of CBT as significantly less important to outcome. There were no significant differences between No Disclosure and Standard CBT Disclosure, and these participants rated Specific factors of CBT and the Working Alliance as more important components in treatment, and Empathy and Positive Regard as less important. <b>Conclusions:</b> The content of information disclosures influences lay opinions about the importance of specific and common factors in CBT. Further research should investigate ethically acceptable disclosures to CBT and other forms of psychotherapy, including whether disclosure practices affect treatment outcome.
Project description:<h4>Background</h4>Major depressive disorder (MDD) is a prevalent psychiatric condition associated with significant disability, mortality and economic burden. Cognitive behavioral therapy (CBT) and psychodynamic psychotherapy (PDT) are found to be equally effective for patients with depression. However, many patients do not respond sufficiently to either treatment. To offer individualized treatment, we need to know if some patients benefit more from one of the two therapies. At present little is known about what patient characteristics (moderators) may be associated with differential outcomes of CBT and PDT, and through what therapeutic processes and mechanisms (mediators) improvements occur in each therapy mode. Presently only theoretical assumptions, sparsely supported by research findings, describe what potentially moderates and mediates the treatment effects of CBT and PDT. The overall aim of this study is to examine theoretically derived putative moderators and mediators in CBT and PDT and strengthen the evidence base about for whom and how these treatments works in a representative sample of patients with MDD.<h4>Methods</h4>One hundred patients with a diagnosis of MDD will be randomized to either CBT or PDT. Patients will be treated over 28 weeks with either CBT (one weekly session over 16 weeks and three monthly booster sessions) or PDT (one weekly session over 28 weeks). The patients will be evaluated at baseline, during the course of therapy, at the end of therapy, and at follow-up investigations 1 and 3 years post treatment. A large range of patient and observer rated questionnaires (specific preselected putative moderators and mediators) are included.<h4>Discussion</h4>The clinical outcome of this study may better guide clinicians when deciding what kind of treatment any individual patient should be offered. Moreover, the study aims to further our knowledge of what mechanisms lead to symptom improvement and increased psychosocial functioning.<h4>Trial registration</h4>ClinicalTrials.gov Identifier: NCT03022071.