Can implementation support help community-based settings better deliver evidence-based sexual health promotion programs? A randomized trial of Getting To Outcomes®.
ABSTRACT: BACKGROUND:Research is needed to evaluate the impact of implementation support interventions over and above typical efforts by community settings to deploy evidence-based prevention programs. METHODS:Enhancing Quality Interventions Promoting Healthy Sexuality is a randomized controlled trial testing Getting To Outcomes (GTO), a 2-year implementation support intervention. It compares 16 Boys and Girls Club sites implementing Making Proud Choices (MPC, control group), a structured teen pregnancy prevention evidence-based program with 16 similar sites implementing MPC augmented with GTO (intervention group). All sites received training and manuals typical for MPC. GTO has its own manuals, training, and onsite technical assistance (TA) to help practitioners complete key programming practices specified by GTO. During the first year, TA providers helped the intervention group adopt, plan, and deliver MPC. This group then received training on the evaluation and quality improvement steps of GTO, including feedback reports summarizing their data, which were used in a TA-facilitated quality improvement process that yielded revised plans for the second MPC implementation. This paper presents results regarding GTO's impact on performance of the sites (i.e., how well key programming practices were carried out), fidelity of MPC implementation, and the relationship between amount of TA support, performance, and fidelity. Performance was measured using ratings made from a standardized, structured interview conducted with participating staff at all 32 Boys and Girls Clubs sites after the first and second years of MPC implementation. Multiple elements of fidelity (adherence, classroom delivery, dosage) were assessed at all sites by observer ratings and attendance logs. RESULTS:After 2 years, the intervention sites had higher ratings of performance, adherence, and classroom delivery (dosage remained similar). Higher performance predicted greater adherence in both years. CONCLUSIONS:These findings suggest that in typical community-based settings, manuals and training common to structured EBPs may be sufficient to yield low levels of performance and moderate levels of fidelity but that systematic implementation support is needed to achieve high levels of performance and fidelity. TRIAL REGISTRATION:ClinicalTrials.gov, NCT01818791.
Project description:BACKGROUND:Community organizations can have difficulty implementing evidence-based prevention programs. More research is needed on implementation support interventions designed to help these organizations implement programs with quality. METHODS:Preparing to Run Effective Programs (PREP) is a randomized controlled trial testing Getting To Outcomes (GTO), a 2-year implementation support intervention. It compares 15 Boys and Girls Club sites implementing CHOICE (control group), a five-session evidence-based alcohol and drug prevention program, with 14 similar sites implementing CHOICE supported by GTO (intervention group). PREP replicates a previous GTO study that had the same design, but featured a teen pregnancy prevention program instead. All sites received typical CHOICE training. Fourteen intervention sites received GTO manuals, training, and onsite technical assistance to help practitioners complete implementation best practices specified by GTO (i.e., GTO steps). During the first year, technical assistance providers helped the intervention group adopt, plan, and deliver CHOICE. Then, this group was trained on evaluation and quality improvement steps of GTO using feedback reports summarizing their own data, which yielded revised plans for subsequent implementation of CHOICE. This paper presents results regarding GTO's impact on CHOICE fidelity (adherence, quality of delivery, dosage) and the proximal outcomes of the youth participants (aged 10-14)-attitudes and intentions regarding cigarettes, alcohol, and marijuana use. Fidelity was assessed at all sites by observer ratings and attendance logs. Proximal outcomes were assessed via survey at baseline, 3, and 6 months. RESULTS:After 1 year, fidelity and proximal outcomes were similar between Intervention and control groups. After 2 years (which included GTO quality improvement activities that took place between years 1 and 2), intervention sites had higher ratings of CHOICE adherence and quality of delivery (dosage remained similar). Proximal outcomes did not differ between groups in either year, although there was universally high endorsement of prosocial responses to those outcomes from the start. CONCLUSIONS:Findings suggest that systematic implementation support provided by GTO can help community organizations achieve better fidelity. Findings replicate the implementation results from a previous GTO study using the same design, but with a different evidence-based program and different fidelity measures. Although proximal outcomes did not change, in large part due to ceiling effects, the implementation findings suggest GTO can support a variety of programs. TRIAL REGISTRATION:This project is registered at ClinicalTrials.gov with number NCT02135991 . The trial was first registered on May 12, 2014.
Project description:Implementation support interventions have helped organizations implement programs with quality and obtain intended outcomes. For example, a recent randomized controlled trial called Preparing to Run Effective Programs (PREP) showed that an implementation support intervention called Getting To Outcomes (GTO) improved implementation of an evidence-based substance use prevention program (CHOICE) run in community-based settings. However, more information is needed on how these interventions affect organizational barriers and facilitators of implementation. This paper aims to identify differences in implementation facilitators and barriers in sites conducting a substance use prevention program with and without GTO. PREP is a cluster-randomized controlled trial testing GTO, a two-year implementation support intervention, in Boys & Girls Clubs. The trial compares 15 Boys & Girls Club sites implementing CHOICE (control group), a five-session evidence-based alcohol and drug prevention program, with 14 Boys & Girls Club sites implementing CHOICE supported by GTO (intervention group). All sites received CHOICE training. Intervention sites also received GTO manuals, training, and onsite technical assistance to help practitioners complete implementation best practices specified by GTO (i.e., GTO steps). During the first year, technical assistance providers helped the intervention group adopt, plan, and deliver CHOICE, and then evaluate and make quality improvements to CHOICE implementation using feedback reports summarizing their data. Following the second year of CHOICE and GTO implementation, all sites participated in semi-structured interviews to identify barriers and facilitators to CHOICE implementation using the Consolidated Framework for Implementation Research (CFIR). This paper assesses the extent to which these facilitators and barriers differed between intervention and control group. Intervention sites had significantly higher average ratings than control sites for two constructs from the CFIR process domain: planning and reflecting and evaluating. At the same time, intervention sites had significantly lower ratings on the culture and available resources constructs. Findings suggest that strong planning, evaluation, and reflection-likely improved with GTO support-can facilitate implementation even in the face of perceptions of a less desirable implementation climate. These findings highlight that implementation support, such as GTO, is likely to help low-resourced community-based organizations improve program delivery through a focus on implementation processes. TRIAL REGISTRATION: This project is registered at ClinicalTrials.gov with number NCT02135991 (URL: https://clinicaltrials.gov/show/NCT02135991). The trial was first registered May 12, 2014.
Project description:The Housing and Urban Development-Veterans Affairs Supportive Housing (HUD-VASH) program is one of the largest initiatives to end Veteran homelessness. However, mental health and substance use disorders continue to reduce client stability and impede program success. HUD-VASH programs do not consistently employ evidence-based practices that address co-occurring mental health and substance use disorders. This paper presents a study protocol to evaluate the implementation of an evidence-based, co-occurring disorder treatment called Maintaining Independence and Sobriety Through Systems Integration, Outreach, and Networking-Veterans Edition (MISSION-Vet) in HUD-VASH using an implementation strategy called Getting To Outcomes (GTO).In three large VA Medical Centers, this Hybrid Type III trial will randomize case managers and their clients by HUD-VASH sub-teams to receive either MISSION-Vet Implementation as Usual (IU-standard training and access to the MISSION-Vet treatment manuals) or MISSION-Vet implementation augmented by GTO. In addition to testing GTO, effectiveness of the treatment (MISSION-Vet) will be assessed using existing Veteran-level data from the HUD-VASH data monitoring system. This project will compare GTO and IU case managers and their clients on the following variables: (1) fidelity to the MISSION-Vet intervention; (2) proportion of time the Veteran is housed; (3) mental health, substance use, and functional outcomes among Veterans; and (4) factors key to the successful deployment of a new treatment as specified by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) model.This project is an important step for developing an implementation strategy to increase adoption of evidence-based practice use in VA homeless programs, and to further examine efficacy of MISSION-Vet in HUD-VASH. This project has important implications for program managers, policy makers, and researchers within the homelessness field. VA Central IRB approval for this study was granted in October 2011. The three sites were trained on MISSION-Vet and GTO in the first half of 2013. The first GTO planning meetings began after training occurred, between January 2013 and November 2013, across the three sites. The data collection-via a fidelity measure embedded into the VA Computerized Patient Record System-began as each site initiated MISSION-Vet, between April 2013 and January 2014.ClinicalTrials.gov: NCT01430741.
Project description:The USA has high teen pregnancy rates compared to other developed nations. Many community-based organizations need assistance conducting evidence-based teen pregnancy prevention programs (EBPs) appropriately. This study evaluated the impact of an implementation support intervention called Getting To Outcomes (GTO) designed to help such organizations. This cluster randomized controlled trial compared 16 Boys and Girls Clubs (BGCs) implementing a teen pregnancy prevention EBP called Making Proud Choices for two years, with 16 BGCs implementing MPC augmented with GTO training, tools, and technical assistance. Participating middle school youth were compared on proximal outcomes (knowledge, attitudes, and intentions about sex and condoms from baseline to post) and sexual behaviors (frequency of sex and condom use, from baseline to 6-month follow-up). In year 1, there were no significant effects of GTO for any proximal outcome. After GTO-stimulated quality improvement in year 2, the GTO group improved significantly more on condom attitudes and use intentions. Frequency of sex and condom use did not differ between the two groups in either year; however, base rates of these behaviors in the sample were very low. Findings suggest that in typical community-based settings, detailed manuals and training common to structured EBPs may be sufficient to yield some improvement in key proximal outcomes, but that more systematic implementation support is needed to achieve greater improvement in these outcomes. Using GTO with many communities, as currently supported by various federal agencies, could yield public health impact via improvements in condom attitudes and use intentions.
Project description:Incorporating evidence-based integrated treatment for dual disorders into typical care settings has been challenging, especially among those serving Veterans who are homeless. This paper presents an evaluation of an effort to incorporate an evidence-based, dual disorder treatment called Maintaining Independence and Sobriety Through Systems Integration, Outreach, and Networking-Veterans Edition (MISSION-Vet) into case management teams serving Veterans who are homeless, using an implementation strategy called Getting To Outcomes (GTO).This Hybrid Type III, cluster-randomized controlled trial assessed the impact of GTO over and above MISSION-Vet Implementation as Usual (IU). Both conditions received standard MISSION-Vet training and manuals. The GTO group received an implementation manual, training, technical assistance, and data feedback. The study occurred in teams at three large VA Medical Centers over 2 years. Within each team, existing sub-teams (case managers and Veterans they serve) were the clusters randomly assigned. The trial assessed MISSION-Vet services delivered and collected via administrative data and implementation barriers and facilitators, via semi-structured interview.No case managers in the IU group initiated MISSION-Vet while 68% in the GTO group did. Seven percent of Veterans with case managers in the GTO group received at least one MISSION-Vet session. Most case managers appreciated the MISSION-Vet materials and felt the GTO planning meetings supported using MISSION-Vet. Case manager interviews also showed that MISSION-Vet could be confusing; there was little involvement from leadership after their initial agreement to participate; the data feedback system had a number of difficulties; and case managers did not have the resources to implement all aspects of MISSION-Vet.This project shows that GTO-like support can help launch new practices but that multiple implementation facilitators are needed for successful execution of a complex evidence-based program like MISSION-Vet.ClinicalTrials.gov NCT01430741.
Project description:BACKGROUND:Evidence-based treatment for co-occurring disorders is needed within programs that serve homeless Veterans to assist with increasing engagement in care and to prevent future housing loss. A specialized co-occurring disorders treatment engagement intervention called Maintaining Independence and Sobriety Through Systems Integration, Outreach and Networking - Veterans Edition (MISSION-Vet) was implemented within the Housing and Urban Development - Veterans Affairs Supportive Housing (HUD-VASH) Programs with and without an implementation strategy called Getting To Outcomes (GTO). While implementation was modest for the GTO group, no one adopted MISSION in the non-GTO group. This paper reports Veteran level outcome data on treatment engagement and select behavioral health outcomes for Veterans exposed to the MISSION-Vet model compared to Veterans without access to MISSION-Vet. METHODS:This hybrid Type III trial compared 81 Veterans in the GTO group to a similar group of 87 Veterans with mental health and substance use disorders from the caseload of staff in the non-GTO group. Comparisons were made on treatment engagement, negative housing exits, drug and alcohol abuse, inpatient hospitalizations, emergency department visits and income level over time, using mixed-effect or Cox regression models. RESULTS:Treatment engagement, as measured by the overall number of case manager contacts with Veterans and others (e.g. family members, health providers), was significantly higher among Veterans in the GTO group (B?=?2.30, p?=?.04). Supplemental exploratory analyses between Veterans who received "higher" and "lower" intensity MISSION-Vet services in the GTO group failed to show differences in alcohol and drug use, inpatient hospitalization and emergency department use. CONCLUSIONS:Despite modest MISSION-Vet fidelity among staff treating Veterans in the GTO group, differences were found in treatment engagement. However, this study failed to show differences in alcohol use, drug use, mental health hospitalizations and negative housing exits over time among those Veterans receiving higher intensity MISSION-Vet services versus low intensity services. This project suggests that MISSION-Vet could be used in HUD-VASH to increase engagement among Veterans struggling with homelessness, a group often disconnected from care. TRIAL REGISTRATION:Clinicaltrials.gov, registration number: NCT01430741 , registered July 26, 2011.
Project description:Effectiveness of evidence-based behaviour change interventions is likely to be undermined by failure to deliver interventions as planned. Behavioural support for smoking cessation can be a highly cost-effective, life-saving intervention. However, in practice, outcomes are highly variable. Part of this may be due to variability in fidelity of intervention implementation. To date, there have been no published studies on this. The present study aimed to: evaluate a method for assessing fidelity of behavioural support; assess fidelity of delivery in two English Stop-Smoking Services; and compare the extent of fidelity according to session types, duration, individual practitioners, and component behaviour change techniques (BCTs).Treatment manuals and transcripts of 34 audio-recorded behavioural support sessions were obtained from two Stop-Smoking Services and coded into component BCTs using a taxonomy of 43 BCTs. Inter-rater reliability was assessed using percentage agreement. Fidelity was assessed by examining the proportion of BCTs specified in the manuals that were delivered in individual sessions. This was assessed by session type (i.e., pre-quit, quit, post-quit), duration, individual practitioner, and BCT.Inter-coder reliability was high (87.1%). On average, 66% of manual-specified BCTs were delivered per session (SD 15.3, range: 35% to 90%). In Service 1, average fidelity was highest for post-quit sessions (69%) and lowest for pre-quit (58%). In Service 2, fidelity was highest for quit-day (81%) and lowest for post-quit sessions (56%). Session duration was not significantly correlated with fidelity. Individual practitioner fidelity ranged from 55% to 78%. Individual manual-specified BCTs were delivered on average 63% of the time (SD 28.5, range: 0 to 100%).The extent to which smoking cessation behavioural support is delivered as specified in treatment manuals can be reliably assessed using transcripts of audiotaped sessions. This allows the investigation of the implementation of evidence-based practice in relation to smoking cessation, a first step in designing interventions to improve it. There are grounds for believing that fidelity in the English Stop-Smoking Services may be low and that routine monitoring is warranted.
Project description:Despite the availability of many evidence-based prevention interventions (EBIs), gaps exist in bringing these programs into widespread practice. Technical assistance (TA) is a strategy for enhancing the readiness of practitioners to implement EBIs. Although many millions of dollars are spent on TA each year, there is little consensus about what the essential features of TA are and how to provide TA with quality. A broad-based research synthesis methodology was used for analyzing the current evidence base for TA using three frames: (1) applying the Getting To Outcomes (GTO) model for categorizing evidence on TA that specifies tasks for planning, implementing, and evaluating TA; (2) understanding the relevance of a successful relationship between the TA provider and TA recipient; and (3) considering the extent to which TA fits the life cycle needs of the preventive intervention. Results indicated that an explicit model or organizing framework is rarely used to plan, implement, and/or evaluate TA; specific TA tasks performed vary widely across studies; TA is rarely delivered to recipients who are seeking to sustain innovations subsequent to adoption and implementation; however, there is systematic attention to relationships and relationship-building. Overall, this synthesis indicates that the extent to which TA is being delivered systematically is limited. We suggest that funders and other stakeholders develop and implement standards for TA quality in order to ensure that many of these limitations are addressed.
Project description:BACKGROUND:Little is known about how to build leadership capacity to support implementation of evidence-based practices within health systems. We observed substantial variability across sites in uptake and sustainability of a peer-led educational outreach intervention for lay health workers (LHWs) providing tuberculosis care in Malawi. Feedback from peer-trainers (PTs) suggested that leadership may have contributed to the variation. We sought to assess the impact of PT leadership style on implementation, and to identify leadership traits of more successful PTs, to inform future implementation planning and to identify targets for leadership capacity building. METHODS:Qualitative study employing interviews with PTs and LHWs at high and low implementation sites, and review of study team and quarterly PT meeting notes. High implementation sites achieved high uptake, sustainability and fidelity of implementation including: close adherence to training content and process, high levels of coverage (training most or all eligible LHWs at their site), and outcomes were achieved with high levels of self reported competence with the intervention among both PTs and LHWs. Low implementation sites achieved limited coverage (<= 50% of LHWs trained), and intervention fidelity. RESULTS:Eight PTs and 10 LHWs from eight high and 10 low implementation sites participated in interviews. Leadership traits of more successful PTs included: flexibility in their approach to training, role modeling and provision of supportive supervision to support learning; addressing challenges proactively and as they occurred; collaborative planning; knowledgeable; and availability to support implementation. Traits unique to less successful PTs included: a poor attitude toward their role as PT and a passive-avoidant approach to challenges. CONCLUSION:This study identified leadership traits more common among unit level leaders at sites with higher uptake, sustainability, and fidelity of implementation. These findings provide a starting point for development and evaluation of a leadership capacity building intervention for unit level leaders to support implementation.