Ebp in the real world
This presentation is the property of its rightful owner.
Sponsored Links
1 / 65

EBP in the Real World PowerPoint PPT Presentation


  • 77 Views
  • Uploaded on
  • Presentation posted in: General

EBP in the Real World. A Provider’s Perspective on the Use of Evidence-Based Programs and Practices. Operation PAR, Inc. Mission : To strengthen our communities by caring for families and individuals impacted by substance abuse and mental illness.

Download Presentation

EBP in the Real World

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Ebp in the real world

EBP in the Real World

A Provider’s Perspective on the Use of Evidence-Based Programs and Practices


Operation par inc

Operation PAR, Inc.

  • Mission: To strengthen our communities by caring for families and individuals impacted by substance abuse and mental illness.

  • Vision: Operation PAR, a beacon of hope in our communities—helping people be aware, be responsible, be healthy and be happy.


Learning objectives

Learning Objectives

  • To encourage audience discussion of “Evidence-Based Programs and Practices” including a review of the literature and practical definitions and applications.

  • To discuss the importance of training and certification in Evidence-Based Programs and Practices to ensure fidelity and adherence to the model.

  • To share Operation PAR’s experience implementing SAMHSA/CSAT projects involving the use of two evidence based practices (A-CRA/ACC and the GAIN) over a 5 year time span.


The research perspective

The Research Perspective

What constitutes an evidence-based practice

Why it matters


Defining evidence based practice

Defining “Evidence-based practice”

  • Core elements:

    • Scientifically tested

    • Results are replicable

    • Clearly defined training protocol

    • Clearly defined adherence protocol

    • Measurable outcomes*


Key definitions

Key Definitions

  • Diffusion: The “grape vine.”

  • Dissemination: Methods by which therapist competency or skillfulness in a particular evidence-based approach are taught and strengthened.

  • Implementation: Agency-level methods for adopting and integrating an evidence-based practice into clinical programs.


Issues in translating research to practice

Issues in translating research to practice

  • Why does a gap exist between research and the “real world”?

  • The impact of the “grape-vine” effect

    • What we don’t know we don’t know

  • Barriers to individual clinician buy-in

    • The “cookie-cutter” or “cookbook” effect

  • Organizational or systemic barriers

    • Changing business as usual


Ebp in the real world

The Great Divide:

Why a gap exists between research and the real world

The Great Divide:

Why a gap exists between research and the real world


Ebp in the real world

The Disconnect:

Researchers versus Clinicians


Ebp in the real world

Individual Clinician Resistance:

Fact or Fiction?


Ebp in the real world

The

Agency


Key findings in literature

Key Findings in Literature

  • Traditional dissemination methods (e.g., manuals, workshops) are ineffective in building competencies and instilling intrinsic motivations to adopt evidence-based approaches, yet they are the most common.

  • Over the last decade, the field has been literally flooded with “evidence-based” practices, BUT:

    • Qualifiers for what constitutes “evidence-based” varies

    • Valuable information on potentially beneficial EBP’s flows through the wrong channels

    • Clinicians and upper management are too busy with day-to-day operations to keep up


Key findings continued

Key Findings Continued…

  • We are witnessing a shift in the methods by which researchers and policy-makers determine which treatment should be translated into practice.

    • Treatment Improvement Protocols have been widely used and are reflective of a consensus of experts in the field

    • Others blend research findings with expert opinion to offer guidelines

    • More recently, there has been an increasing focus on summary reviews and meta-analyses of published research, often with the goal of producing a listing of treatments which could be defined as “evidence-based”


Nrepp

NREPP

  • National Registry of Evidence-based Programs and Practices

    • Dates back to the 1990’s, when an effort was launched to collect information on effective treatment interventions.

    • A report by SAMHSA Science to Service Coordinator, Kevin Hennessey noted (see reference below) that by 2003:

      • 1,100 interventions had been submitted for NREPP review

      • Of these, 150 were listed on NREPP

    • NREPP initially targeted prevention projects, since it was born under a CSAP initiative.

Source http://www.jbassoc.com/reports/documents/panels/panels/panel%208/panel8hennessy.pdf


Nrepp1

NREPP

  • By 2003, SAMHSA realized the need to expand NREPP reviews to include:

    • Substance abuse interventions

    • Mental health interventions

    • Interventions which treat co-occurring disorders

  • In response to a solicitation for input, the public asked for NREPP changes to:

    • Provide summaries that were easy to understand and were “transparent.”

    • Prohibited agencies from mandating NREPP interventions

Source http://www.jbassoc.com/reports/documents/panels/panels/panel%208/panel8hennessy.pdf


Nrepp helps by providing

NREPP helps by providing:

A publicly available list of interventions reviewed by NREPP

Results of independent reviews which are posted for consideration

Strict, standardized criteria used across all types of interventions to ensure uniformity


Ebp in the real world

Source: http://www.nrepp.samhsa.gov/AboutNREPP.aspx


How can this benefit

How can this benefit…


Nrepp review process

NREPP Review Process


Nrepp review process1

NREPP Review Process

Both Quality of Research and Readiness for Dissemination are rated on a scale of 1.0 to 4.0


Nrepp ratings

NOTE:

Though one of the program examples utilized Motivational Enhancement Therapy in conjunction with Cognitive Behavioral Therapy, CBT has not formally undergone the NREPP review process and thus is excluded from the following slides

NREPP Ratings

  • Let’s compare the NREPP ratings for the EBP’s utilized in the Operation PAR programs under discussion today:

    • Motivational Enhancement Therapy (MET)

    • Family Support Network (FSN)

    • Adolescent Community Reinforcement Approach (A-CRA)


How is quality of research rated

How is Quality of Research Rated?

  • The dimensions of “Quality of Research” discussed previously (e.g., Reliability of measure, Validity of measure, etc.) are rated according to the outcomes reported in the published research provided as evidence of the program’s effectiveness.

  • This will vary across interventions.


Met quality of research

MET: Quality of Research

Table 1. Quality of Research Criteria for Motivational Enhancement Therapy (MET)


Fsn quality of research

FSN: Quality of Research

Table 2. Quality of Research Criteria for Family Support Network (FSN)


A cra quality of research part 1

A-CRA: Quality of Research (part 1)

Table 3. Quality of Research Criteria for A-CRA


A cra quality of research part 2

A-CRA: Quality of Research (part 2)

Table 4. Quality of Research Criteria for A-CRA


Cross comparison quality of research

Cross Comparison: Quality of Research


Readiness for dissemination

Readiness for Dissemination

Table 5. Comparison of Readiness for Dissemination Ratings


The clinical perspective

The Clinical Perspective

The importance of training and certification

Why fidelity matters


Evidence based practices included

Each EBP includes a specific training and certification process

Evidence-based practices included:

  • Global Appraisal of Individual Needs (GAIN)

  • Adolescent Community Reinforcement Approach

  • Assertive Continuing Care

  • Motivational Enhancement Therapy/Cognitive Behavioral Therapy

  • Family Support Network


Gain training and certification

GAIN Training and Certification

  • Different types of certification

    • GAIN Administration

    • GAIN Clinical Interpretation

  • Different levels of certification

    • Site Interviewer

    • Local Trainer

    • Site Interpreter

    • GAIN Clinical Local Trainer

Source http://www.chestnut.org/LI/gain/GAIN%20Training/index.html


A cra acc training and certification

A-CRA/ACC Training and Certification

  • Different types of certification

    • A-CRA

    • ACC

  • Different levels of certification for both A-CRA and ACC

    • Clinician certification

    • Supervisor certification

Source http://www.chestnut.org/LI/acra-acc/index.html


Met cbt fsn training and certification

MET/CBT & FSN Training and Certification

  • Different types of certification

  • Different levels of certification

    • Clinician certification

    • Supervisor certification

Source http://www.chestnut.org/LI/cyt/products/index.html#treatment


Common elements of training

Common Elements of Training

  • Structured training manual

  • Standardized training process

  • Training includes variety of teaching methods:

    • Lecture style

    • Demonstration

    • Hands on practice


Common elements of certification

Common Elements of Certification

  • Requirement to audio or video record sessions (treatment or assessment/interview)

  • Submission of audio recordings for quality assurance review

    • Each EBP has a corresponding standardized procedure for evaluating the quality of sessions or interviews

    • Written feedback is provided to candidate

    • Process repeats until certification is achieved

  • Certification Deadlines


How does this really work

How does this really work?


Challenges

  • Supervisor Complaints:

  • Time investment for staff

  • Handling staff complaints

  • Doubts efficacy of “new” model

  • Program-level change is complex

  • Often insufficient/non-existing funding for new intervention

  • Clinician Complaints:

  • Perceived threat to competency

  • Not engaging in certification

  • Disdain for “cookbook” therapy

  • Feel clinical skills are lost

  • Issues with audio recordings

Challenges


Prochaska diclemente s stages of change model

Prochaska & DiClemente’s Stages of Change Model


Rogers 2003 application of stages of change to individual clinicians

Rogers’ (2003) application of Stages of Change to individual clinicians


Benefits

Benefits

  • Supervisors:

  • Supervision simplified yet thorough

  • Positive program outcomes

  • Increased attendance

  • Increased engagement

  • Clinicians:

  • Clinicians are more marketable

  • Achieve certification

  • Become an “expert” in the model

  • Learning the model enables flexibility in clinical style


Speaking a common language

Speaking a Common Language

  • Researchers and Clinicians

    • Understanding what we’re both talking about

  • Clinicians and Clinicians

    • Differences in clinical studies and clinical programs


Reconciling paperwork requirements

Reconciling Paperwork Requirements

  • Differences in clinical charting

    • Agency requirements

    • A-CRA requirements


The program manager s perspective

The Program Manager’s Perspective

How it works in the real world

Challenges, successes, and hopes for the future


Bay area young offender reentry program yorp 1

Bay Area Young Offender Reentry Program (YORP) #1

  • CSAT Grant (TI16928) funded from 2005 through 2009

  • Targeted sentenced, substance abusing, adolescents (age 13 to 18) who were within 60 days of being released from a Florida Department of Juvenile Justice (DJJ) residential commitment program.

  • EBP’s utilized:

    • Global Appraisal of Individual Needs (GAIN)

    • Motivational Enhancement Therapy/Cognitive Behavioral Therapy - 12 sessions (MET/CBT 12)

    • Family Support Network (FSN)


Yorp 1 implementation challenges

YORP #1 Implementation Challenges

  • Extreme delays in IRB approval resulted in extreme delays in recruitment/enrollment

  • IRB approval was received, however:

    • Removed our permission to recruit youth while in commitment

    • Reduced referral sources by half

  • Staff turnover

  • Clinicians had virtually no buy-in to EBP’s


Impact on treatment engagement

Impact on Treatment Engagement


Impact on treatment outcomes

Impact on Treatment Outcomes


Yorp 1 lessons learned

YORP #1 Lessons Learned

  • Start IRB review process as early as possible

    • Contact IRB for review guidelines or requirements prior to submission, if possible

  • Have a back-up plan for additional referral sources

  • Staff clinicians who can embrace and fully adopt the EBP

  • Schedule regular meetings

  • Use data regularly to inform program adaptations

    • The data and corresponding adaptations must be timely to be effective


Par adolescent recovery intervention services paris 1

PAR Adolescent Recovery Intervention Services (PARIS) #1

  • CSAT Grant (TI17761) funded from 2006 through 2009

  • Targeted substance abusing adolescents (age 13 to 17) who were referred for substance abuse treatment from a variety of agencies (school, probation, community, etc.).

  • EBP’s utilized:

    • Global Appraisal of Individual Needs (GAIN)

    • Adolescent Community Reinforcement Approach (A-CRA)

    • Assertive Continuing Care (ACC)


Paris 1 implementation challenges

PARIS #1 Implementation Challenges

  • First research grant and program at Operation PAR to utilize A-CRA/ACC

  • Clinical staff turnover

    • 1 was a poor fit and 1 was fired for performance issues

  • Issues with clinical buy-in were still present, but not nearly as pronounced as seen in YORP #1

  • The treatment was provided in-home, which was a shift for most clinicians who started on the grant

  • The treatment site was located farther off site than any program before

    • Site was a 40 mile round trip drive


Paris 1 implementation

PARIS #1 Implementation

  • Start-up

    • Positives

      • Full training and support with procedure updates and weekly clinical supervision to certification.

      • Continuing advanced training opportunities during the grant timeframe.

      • A clear, concise therapeutic protocol that can be taught and adopted rather quickly. Especially to staff new in the field of addictions.


Paris 1 implementation1

PARIS #1 Implementation

  • Program Challenges

    • Extensive certification process (3 to 6 months)

    • Some difficulty in staff adoption of A-CRA/ACC, resulting in staff turnover (especially in more seasoned staff)

  • Systemic Challenges

    • Conflict between treatment philosophy and outcomes expected by other systems of care

      • Probation: Abstinence vs. Sobriety

      • Interagency: Different paperwork requirements


Impact on treatment engagement1

Impact on Treatment Engagement


Impact on treatment outcomes1

Impact on Treatment Outcomes


Impact on treatment outcomes2

Impact on Treatment Outcomes


Paris 1 lessons learned

PARIS #1 Lessons Learned

  • Have a back-up plan for additional referral sources

  • Handle staff issues with buy-in early and quickly

  • Adapt regularly scheduled meetings to meet the needs of clinical staff


Bay area young offender reentry program yorp 2

Bay Area Young Offender Reentry Program (YORP) #2

  • CSAT Grant (TI21580) funded in 2009 through 2012

  • Targeted sentenced, substance abusing, adolescents (age 13 to 18) who were residing within or recently released from a Florida Department of Juvenile Justice (DJJ) residential commitment program.

  • EBP’s utilized:

    • Global Appraisal of Individual Needs (GAIN)

    • Adolescent Community Reinforcement Approach (A-CRA)

    • Assertive Continuing Care (ACC)


Yorp 2 implementation

YORP #2 Implementation

  • How the implementation of YORP #2 differed so significantly from YORP #1

    • Built on lessons learned from both YORP #1 and PARIS #1

  • Experience in changing the evidence-based treatment from MET/FSN to A-CRA/ACC

    • Effect on clinician buy-in and certification

    • Still required population adaptations


Bay area young offender reentry program yorp

Bay Area Young Offender Reentry Program (YORP)

  • Second program at Operation PAR to utilize A-CRA/ACC

    • Instead of spending time on training/certification, time and resources were used for implementation and further staff development activities (including GAIN training).

    • With previous relationships and contacts in the community setting up referral relationships was faster and easier.

    • As all staff were already employed by Operation PAR the need for interviewing, hiring, and background checks was not an issue.


Benefitting from lessons learned

Benefitting from Lessons Learned

  • Start-up

    • Positives

      • Fully-trained and certified staff at the start of the grant.

      • Time to train in GAIN administration and certification to support other grant activities.

      • Easy and precise description and explanation of services to referral sources and clientele to aid in their decision making process.


Still some bumps in the road

Still some bumps in the road

  • New challenges and issues affecting implementation:

    • GAIN administration conducted by clinicians not research assistant.

    • Limited access to continued support and supervision for new procedures or changes to existing procedures.

  • Challenges of using A-CRA/ACC with justice-involved youth:

    • Clinical staff had to adapt procedures within the parameters of the grant to meet the needs of the youth.

    • Parameters of FL DJJ conditional release programs offered some challenges in terms of service delivery and length of stay.


Par adolescent recovery intervention services paris 2

PAR Adolescent Recovery Intervention Services (PARIS) #2

  • CSAT Grant (TI23247) funded in 2010

  • Targets substance abusing adolescents (age 13 to 17) who are enrolled in residential substance abuse treatment.

  • EBP’s utilized:

    • Global Appraisal of Individual Needs (GAIN)

    • Adolescent Community Reinforcement Approach (A-CRA)

    • Assertive Continuing Care (ACC)


Paris 2 implementation

PARIS #2 Implementation

  • Building on the lessons learned from three previous grant projects

  • How this impacted implementation?


Thank you

Thank You!

  • For more information on Implementation Research see:

    • http://www.fpg.unc.edu/~nirn/

  • For more information on Evidence-Based Practices see:.

    • http://www.chestnut.org/LI/cyt/products/index.html#treatment

    • http://www.nrepp.samhsa.gov

  • For more information about this presentation or Operation PAR, Inc., please see:

    • http://www.operationpar.org

    • Contact: [email protected]


References

References

Glasgow, R. E., Lichtenstein, E., & Marcus, A. C. (2003). Why Don’t We See More Translation of Health Promotion Research to Practice? Rethinking the Efficacy-to-Effectiveness Transition. American Journal of Public Health, Vol. 93, No. 8, 1261-1267.

Klein, K. J., & Sorroa, J. (2005). The Challenge of Innovation Implementation. The Academy of Management Review, Vol. 21, No. 4,1055-1080.

Klein, K. J., & Sorroa, J. (2005). The Challenge of Innovation Implementation. The Academy of Management Review, Vol. 21, No. 4,1055-1080.

Miller, W. R., Sorensen, J. L., Selzer, J. A., & Brigham, G. S. (2006). Disseminating evidence-based practices in substance abuse treatment: A review with suggestions. Journal of Substance Abuse Treatment, 31, 25-39.

Prochaska, J. O., & DiClemente, C. C. The Transtheoretical Approach: Crossing the Traditional Boundaries of Therapy. Homewood, IL: Dorsey/Dow Jones-Irwin, 1984.

Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York. Free Press.

Rogers, E. M. (2002). Diffusion of preventive innovations. Addictive Behaviors, 27, 989-993.

Schoenwald, S. K. & Henggeler, S. W. (2003). Current strategies for moving evidence-based interventions into clinical practice. Cognitive and Behavioral Practice, 10, 275-277.

Simpson, D. D. (2002). A conceptual framework for transferring research to practice. Journal of Substance Abuse Treatment, 22, 171-182.

Willenbring, M. L., Kivlahan, D., Kenny, M., Grillo, M., Hagedorn, H., & Postier, A. (2004). Journal of Substance Abuse Treatment, 26, 79-85.


  • Login