To promote the availability of evidence-based psychotherapies for Veterans and realize the full potential of these treatments, the U.S. Department of Veterans Affairs (VA) Health Care System has implemented a national initiative to disseminate and implement 16 evidence-based psychotherapies for a variety of mental and behavioral health conditions throughout the VA health care system (Karlin & Cross, 2014). This effort has been guided by a multi-level model accounting for barriers and facilitators documented in the implementation science literature. This national, multi-level model includes specific implementation strategies at the policy, provider, local systems, patient, and accountability levels. At the policy level, VA developed national policies requiring that facilities make specific EBPs available for particular conditions, with other services also available. At the provider level, VA developed national competency-based training programs, with rating and feedback on actual implementation of therapy, for each of the EBPs being nationally disseminated. VA has, to date, provided training to more than 10,000 staff in the delivery of one or more EBPs. A number of other strategies have been implemented at local systems and patient levels to promote local clinical infrastructures to support the delivery of EBPs and to increase patient awareness of and engagement in these treatments. National program evaluation results published in more than 20 journal articles have demonstrated that EBP training and implementation in the VA health care system has resulted in robust therapist and patient outcomes, including significant increases in therapist competencies and significant clinical improvements among Veterans. Beyond improvements in clinical symptoms, EBPs have often led to significant improvements in quality of life and in the therapeutic alliance among Veterans treated in routine practice settings. Furthermore, recent evaluation findings have shown that Veteran receiving several EBPs demonstrated significant reductions in suicidal ideation.

Descriptive Paper:

Karlin, B. E., & Cross, G. (2014). From the laboratory to the therapy room: national dissemination and implementation of evidence-based psychotherapies in the US Department of Veterans Affairs Health Care System. American Psychologist, 69(1), 19-33.

Evaluation/Outcomes Papers:

Eftekhari, A., Ruzek, J. I., Crowley, J., Rosen, C., Greenbaum, M. A., & Karlin, B. E. (2013). Effectiveness of national implementation of Prolonged Exposure Therapy in VA care. JAMA Psychiatry, 70, 949-955.

Karlin, B. E., Brown, G. B., Trockel, M., Cunning, D., Zeiss, A. M., & Taylor, C. B. (2012). National dissemination of Cognitive Behavioral Therapy for depression in the Department of Veterans Affairs health care system: Therapist and patient-level outcomes. Journal of Consulting and Clinical Psychology, 80, 707-718.

Karlin, B. E., Trockel, M., Brown, G. K., Gordienko, M., Yesavage, J., & Taylor, C. B. (2015). Comparison of the effectiveness of Cognitive Behavioral Therapy for depression among older versus younger veterans: Results of a national evaluation. Journals of Gerontology, Series B: Psychological Sciences and Social Sciences, 70, 3-12.

Stewart, M. O., Raffa, S. D., Steele, J. L., Miller, S. A., Clougherty, K. F., Hinrichsen, G. A., & Karlin, B. E. (2014). National dissemination of Interpersonal Psychotherapy for depression in Veterans: Therapist and patient-level outcomes. Journal of Consulting and Clinical Psychology, 82, 1201-1206.

Trockel, M., Karlin, B. E., Taylor, C. B., Brown, G. K., & Manber, R. (2015). Effects of Cognitive Behavioral Therapy for insomnia on suicidal ideation in Veterans. SLEEP, 38, 259-265.

The United Kingdom launched the Improving Access for Psychological Therapy (IAPT) program, which expanded the provision of EBPs (most prominently, Cognitive Behavioral Therapy, although Interpersonal Psychotherapy, Couples Therapy, and Brief Psychodynamic Therapy are also supported) within the National Health Service, with the aim of making therapy for depression and anxiety available on a large scale. System-wide, a guideline was put into place that specified EBPs as front-line treatments for depression and anxiety. The program began with a demonstration project, which was evaluated to examine patient outcomes and costs. The national rollout included funding for training and a standardized national training curriculum, a service model that outlines general standards but allows for some degree of local autonomy, and monitoring of clinical outcomes. The decision was made to train and support a small number of new services at full capacity and gradually roll out services throughout the country. To address capacity, a stepped-care model was implemented, in which clients are triaged into high (e.g., face-to- face treatment) or low intensity work (e.g., telephone and internet-based guided self-help) based on severity and treatment response. Outcome data from every IAPT service in the country are made publicly available on a quarterly basis.

Descriptive Paper:

Clark, D. M. (2011). Implementing NICE guidelines for the treatment of depression and anxiety disorders: the IAPT experience. International Review of Psychiatry, 23(4), 318-327.

Evaluation/Outcomes Papers:

Gyani, A., Shafran, R., Layard, R., & Clark, D. M. (2013). Enhancing recovery rates: Lessons from year one of IAPT. Behaviour Research and Therapy,51(9), 597-606.

Radhakrishnan, M., Hammond, G., Jones, P. B., Watson, A., McMillan-Shields, F., & Lafortune, L. (2013). Cost of improving Access to Psychological Therapies (IAPT) programme: an analysis of cost of session, treatment and recovery in selected Primary Care Trusts in the East of England region.Behaviour research and therapy, 51(1), 37-45.

In 2004, the State of Texas undertook a comprehensive redesign of the service system focused based on a standardized assessment process to identify the EBPs and supports that would be most likely to meet individuals’ recovery goals. As part of a larger system initiative to redesign state-funded mental health services, community mental health centers were selected receive CBT training and supervision to examine the role that CBT could play in the treatment of adults with Major Depressive Disorder (MDD). This implementation plan was funded by grant funding from the National Institute of Mental Health. During an initial feasibility phase, 7 clinicians from 4 publicly- funded community mental health centers (MCHCs) received 36 hours of didactic training plus 5 months of supervision by a CBT expert, during which time their skills approached competency levels of therapists in randomized controlled trials. Fourteen additional therapists were trained in a second phase (with didactics plus 6-9 months of supervision), during which effectiveness of CBT for MDD was studied in keeping with the Deployment Focused Model. Adults who enter services receive a diagnostic interview, a symptom-based assessment, and a measure of functioning across multiple domains. Those diagnosed with depression receive an algorithm-based medication treatment plus case coordination services as a first-line services. If they do not achieve full remission of depression after two trials of medication, they became eligible for CBT. The implementation team noted that although therapist feedback indicated that they appreciated the frequency of the weekly supervision, organizations may be challenged to support this level of training without external funding. Recommendations included the development of internal expertise to provide on-going supervision.

Descriptive/Evaluation Paper:

Lopez, M. A., & Basco, M. R. (2011). Feasibility of dissemination of cognitive behavioral therapy to Texas community mental health centers. The journal of behavioral health services & research, 38(1), 91-104.

Evaluation/Outcomes Paper:

Lopez, M. A., & Basco, M. A. (2015). Effectiveness of cognitive behavioral therapy in public mental health: Comparison to treatment as usual for treatment-resistant depression. Administration and Policy in Mental Health AND Mental Health Services Research, 42(1), 87-98. doi:http://dx.doi.org/10.1007/s10488-014- 0546-4

Trauma-focused cognitive behavioral therapy (TF-CBT) is an evidence-based practice (EBP) for children. TF-CBT is being disseminated and implemented in New York State through a variety of strategies, including distance learning/ Internet training, live training + ongoing phone consultation, a learning collaborative model, and mixed models. Examples are provided of how the different models have been used to spread the TF-CBT model among community clinicians treating traumatized children. Quality and quantity of data from these dissemination/ implementation models varies, but overall they support the effectiveness of both the TF-CBT model in treating traumatized children and a variety of dissemination and implementation models.

Descriptive/Evaluation Paper:

Cohen, J., & Mannarino, A. P. (2008). Disseminating and implementing trauma-focused CBT in community settings. Trauma, Violence, & Abuse,9(4), 214-226.

The Child and Adolescent Mental Health Division of the Hawaii Department of Health established a Task Force in 1999, to provide an interdisciplinary evaluation of interventions for common youth disorders based on evidence from controlled treatment studies. This committee initially focused on identifying evidence-based treatments for 5 disorders, and disseminated its initial findings locally through various technical and national reports. Procedures were developed to review the treatment outcome literature biannually to monitor the evolving state of the science. The methodology of the ongoing review expanded to focus on problem areas rather than specific disorders, as well as a multi-axial system of examining treatment outcomes. The committee focused on common elements, rather than specific protocols, to identify practices or techniques related to treatment outcomes. Coding of common elements provided the committee with practical information from the scientific literature to inform selection of EBPs when possible, and selection of evidence-supported practices when no appropriate EBP was available. Coding resulted in a detailed online report published biannually and a briefer summary updated every 6 months, which are now available from a privatized corporation. The committee functions as a community of practice wherein members with diverse experience interact frequently and use a collective (rather than directive) and data-driven decision-making process to ensure that all stakeholders remain engaged. Community members are encouraged to participate in the committee through participation in the meetings and strategies focused on members sharing how EBP impacted their professional or personal lives. An online reporting system was developed to monitor the use of specific practice elements in youth services, and compare those practices to the best-practices literature. These data are then used to drive state-sponsored direct-service training initiatives for practice elements. The systems developed in Hawaii have spread to other systems including Western Australia, states of Minnesota and California, and the American Academy of Pediatrics.

Descriptive Paper:

Nakamura, B.J., Chorpita, B.F., Hirsch, M., Daleiden, E., Slavin, L., Amundwon, M.J., Rocco, S., Mueller, C., Osiecki, S., Southam-Gerow, M.A., Stern, K., & Virsino, W.M. (2011). Large‐scale implementation of evidence‐based treatments for children 10 years later: Hawaii’s evidence‐based services initiative in children’s mental health. Clinical Psychology: Science and Practice, 18, 24-35.

Outcomes/Evaluation Papers:

Lim, A., Nakamura, B. J., Higa-McMillan, C. K., Shimabukuro, S., & Slavin, L. (2012). Effects of workshop trainings on evidence-based practice knowledge and attitudes among youth community mental health providers. Behaviour Research & Therapy, 50(6), 397-406. doi:10.1016/j.brat.2012.03.008

Nakamura, B. J., Selbo-Bruns, A., Okamura, K., Chang, J., Slavin, L., & Shimabukuro, S. (2014). Developing a systematic evaluation approach for training programs within a train-the- trainer model for youth cognitive behavior therapy. Behaviour Research & Therapy, 5310-19. doi:10.1016/j.brat.2013.12.001

An evidence-informed model of care, Managing and Adapting Practice (MAP) has been implemented in Los Angeles County, California. MAP complemented a variety of EBPs that were selected by the county as part of a large system reform effort designed to improve care for children and adolescents. The county’s program includes a fiscal mandate, a decision support dashboard, training therapists using either a national trainer model or an agency supervisor (train the trainer) model, and evaluation of utilization and clinical outcomes.

Descriptive/Evaluation/Outcomes Paper:

Southam-Gerow, M. A., Daleiden, E. L., Chorpita, B. F., Bae, C., Mitchell, C., Faye, M., & Alba, M. (2014). MAPping Los Angeles County: Taking an evidence-informed model of mental health care to scale. Journal of Clinical Child & Adolescent Psychology, 43(2),
190-200.

Facing a fiscal mandate to transition the vast majority of clinical services to EBPs, an urban children's mental health center in LA County successfully implemented rapid change. Steps taken included intensive training in seven treatment models, funded by donor support, grant funding, and California Mental Health Services Act funding. Leadership worked to help staff understand the rationale for change and to identify ways to make EBPs as flexible as needed to serve their clinical population. Staff could choose the EBPs for which they wished to pursue training, and training was made available to interdisciplinary staff to support collaborative care. Training included supervision and review of session videos. The intake process was changed to facilitate recommendations for specific EBPs based on the initial assessment. Clinicians involved the diverse clientele in treatment decision-making and explained the rationale behind the transformation. The agency tracked training and provision of EBPs. Ongoing communication between leadership and staff allowed concerns to be discussed and barriers to be problem-solved.

Descriptive Paper:

Williams, M.E., Rogers, K.C., Carson, M.C., Sherer, S., & Hudson, B.O. (2012). Opportunities arising from transformation from treatment as usual to evidence-based practice. Professional Psychology: Research and Practice, 43, 9-16. dx.doi.org/10.1037/a0025003

In response to a policy shift toward EBPs in a large, urban mental health system, CBT was implemented in 30 CMHCs in Philadelphia's Department of Behavioral Health and Intellectual disAbility Services over a period of 7 years. Participating CMHCs served a diverse group of populations (e.g., children, individuals experiencing chronic homelessness, addictions) across many levels of care (e.g., outpatient, inpatient, residential). Using the ACCESS model framework, early stages with each CMHC included assessing the characteristics of each center and adapting the content and process of training for each site, tailored workshops to build clinicians’ knowledge bases, 6 months of consultation with review of audio recordings, assessment of competency using a gold-standard measure of CBT, evaluation of outcomes. Strategies to support sustained practice over time included a web-based training to replenish capacity lost to turn-over and promotion, ongoing CBT consultation groups run by each CMHC to reduce drift and support competency development for web-based trainees, and bi-monthly support from the implementation team.

Descriptive Papers:

Creed, T. A., Stirman, S. W., & Evans, A. C. (2014). A model for implementation of cognitive therapy in community mental health: The Beck Initiative. The Behavior Therapist, 37(3), 56-64.

Stirman, S. W., Buchhofer, R., J Bryce McLaulin, M. D., Evans, A. C., & Beck, A. T. (2009). Public-academic partnerships: The Beck initiative: A partnership to implement cognitive therapy in a community behavioral health system. Psychiatric Services.

Riggs, S. & Creed, T.A. (2016) A model to transform treatment for psychosis using CBT informed interventions for the mental health milieu. Cognitive and Behavioral Practice. http://dx.doi.org/10.1016/j.cbpra.2016.08.001

Evaluation/Outcomes Papers:

Creed, T.A., Frankel, S.A., German, R.,Green, K.L.,Jager-Hyman, S.,Pontoski, K., Adler, A.,Wolk, C.B., Stirman, S.W., Waltman, S.H., Williston, M.A., Sherrill, R., Evans, A.C., & Beck. A.T. (2016). Implementation of transdiagnostic cognitive therapy in diverse community settings: The Beck Community Initiative. Journal of Consulting and Community Psychology.

Stirman, S. W., Pontoski, K., Creed, T., Xhezo, R., Evans, A. C., Beck, A. T., & Crits-Christoph, P. (2015). A Non-randomized Comparison of Strategies for Consultation in a Community-Academic Training Program to Implement an Evidence-Based Psychotherapy. Administration and Policy in Mental Health and Mental Health Services Research, 1-12.

Stirman, S. W., Matza, A., Gamarra, J., Toder, K., Xhezo, R., Evans, A. C., ... & Creed, T. (2015). System-level influences on the sustainability of a cognitive therapy program in a community behavioral health network.Psychiatric Services.

Pontoski, K., Jager-Hyman, S., Cunningham, A., Sposato, R., Schultz, L., Evans, A.C., Beck, A.T., & Creed, T.A. (2016). Using a Cognitive Behavioral framework to train staff serving individuals who experience chronic homelessness. Journal of Community Psychology, 44,674-680.

Creed, T.A., Jager-Hyman, S., Pontoski, K., Feinberg, B., Rosenberg, Z., Evans, A.C., Hurford, M.O., & Beck, A.T. (2013). The Beck Initiative: A strength-based approach to training school-based mental health staff in cognitive therapy. International Journal of Emotional Education, 5, 49-66.

The CFIR model was used to guide implementation of EBPs in 13 school-based health centers (SBHCs) in a large, urban public school district. Assessment of the inner context in the SBHCs indicated that competing responsibilities, poor parent engagement, and logistical barriers posed challenges to implementing EBPs, Modular psychotherapies were identified as having characteristics that could be successfully adapted for the setting, including flexibility in length and frequency of sessions. The Managing and Adapting Practice (MAP) system was adapted to maximize appropriateness and acceptability of the EBP, and the fit of the intervention to aspects of the inner context. Training workshops were followed by bi-weekly consultation for the remained of the academic year, during which therapists identified and treated training cases. Differences in implementation success were observed based on organizational affiliation, underscoring the need for organizational-level pre-training supports.

Descriptive / Evalution / Outcome Papers:

Lyon, A., Charlesworth-Attie, S., Vander Stoep, A., & McCauley, E. (2011). Modular psychotherapy for youth with internalizing problems: Implementation with therapists in school-based health centers. School Psychology Review, 40, 569-581.

Lyon, A. R., Ludwig, K., Romano, E., Koltracht, J., Vander Stoep, A., & McCauley, E. (2014). Using modular psychotherapy in school mental health: Provider perspectives on intervention-setting fit. Journal of Clinical Child and Adolescent Psychology, 43(6), 890-901.

Cook, C. R., Lyon, A. R., Kubergovic, D., Wright, D. B., & Zhang, Y. (2015). A supportive beliefs intervention to facilitate the implementation of evidence-based practices within a multi-tiered system of supports. School Mental Health, 7(1), 49-60.

Lyon, A. R., Frazier, S. L., Mehta, T., Atkins, M. S., & Weisbach, J. (2011). Easier said than done: Intervention sustainability in an urban after-school program. Administration and Policy in Mental Health and Mental Health Services Research, 38(6), 504-517.

Trauma-informed programming was implemented in two youth residential treatment settings, based on the Attachment, Self-regulation and Competency (ARC) intervention framework. The implementation strategy was based on the 6-step framework suggested by Fixsen and colleagues. Both top down (e.g. by agency administrators) and bottom up (e.g., clients, families, program staff) approaches were used to assess the needs of
each program, identifying five core in need of pre-implementation readiness interventions. Implementation teams were identified within each setting to spearhead the process. Didactic and experiential training was delivered for all staff, in modules appropriate for their jobs and logistical considerations. Specific goals and objectives chosen to enhance the milieu were selected by the implementation teams at each site based on the organizational needs assessment. Adjunctive individual and group therapy in the ARC model was implemented for a subset of clients. Four key areas were identified as crucial to sustaining the EBP: 1) trauma team leadership and focus; 2) policy and procedure; 3) orientation and ongoing training; and 4) ongoing evaluation

Descriptive / Evalution / Outcome Paper:

Hodgdon, H.B., Kinniburgh, K.J., Gabowitz, D., Blaustein, M.E., & Spinazzola, J. (2013). Development and implementation of trauma-informed programming in youth residential treatment centers using the ARC framework. Journal of Family Violence, 28, 679-692.

CBT was integrated into a large-group practice mental health services delivery organization over a period of several years. Leadership devoted substantial time and monetary resources to ensure implementation success. They identified a psychologist who had demonstrated previous experience in developing clinical guidelines to lead the effort, and formed a committee, comprising both clinicians and managers from the various clinics to provide input into the plan to assure that both organizational and individual needs of clients and clinicians were taken into account in the development of the training program. The training program included didactics and experiential exercises on CBT principles, plus ongoing consultation. Efforts were made to integrate CBT into the culture of the agency through peer consultation that integrated CBT into case reviews, chart audits focusing on the use of EBPs, and hiring of staff with CBT backgrounds.

Description and Program Evaluation:

Steinfeld, B.I., Coffman, S.J., & Keyes, J.A. (2009). Implementation of an evidence- based practice in a clinical setting: What happens when you get there? Professional Psychology: Research and Practice, 40, 410-416.