Evidence-Based Practices: Challenges and Opportunities

1. Evidence-Based Practices: Challenges and Opportunities Michael DeBernardi, PsyD, MS The Life Link May 23, 2019

2. If what I am doing is already working, why do I need to try something different?

3. Treatment As Usual Why do you work the way you do? Training Agency/program philosophies Peer influence Clinical experience “It’s how we’ve always done it” “We just know that it works”

4. evidence-based practice healing traditions best practices scientifically valid practice-based evidence promising practices emerging practices clinical orientation

5. Definitions

6. Efficacy vs. Effectiveness • Efficacy refers to interventions which produce positive results in a controlled experimental research trial. • Effectiveness refers to interventions that produce positive results in a usual or routine care setting (i.e. the real world) • Efficacy in research does not necessarily equate with real world effectiveness.

7. Levels of Scientific Evidence CSAT 2006

8. Evidence-Based Thinking • A process by which diverse sources of information (research, theory, practice principles, practice guidelines, clinical experience) are synthesized by a clinician, expert, or group of experts in order to identify or choose the optimal clinical approach for a given clinical situation. • These identified components can be integrated in to a meaningful feedback loop that enables continued refinement of the true “evidence base”

9. Evidence-Based Thinking CSAT 2006

10. Evidence-Based Thinking Evidence-based thinking requires that practitioners: • Continue to expand knowledge over the course of a career • Use a scientific approach to decision-making • Take into account the individual needs and circumstances of the people they serve to get the best results for their clients and programs

11. What are EBPs? EBPs are practices which, based on expert or consensus opinion about available evidence, are expected to produce a specific clinical outcome (i.e. measurable change in client status). They are an integration of best research evidence with clinical expertise and patient values.

12. Components of EBPs

13. Why use EBPs? • Improved quality of care • Increased likelihood that desired outcomes will be attained • Availability of manuals • More efficient use of available resources. • External demands (VBP) • Consumer choice

14. Recovery

15. Recovery • Treatment and support services are more effective when consumers are given information about: • the types of service available • which services are most likely to help them meet their goals (e.g. symptom reduction, improved functioning, housing, employment, etc.) • relative roles both they and their service providers should play in that process.

16. Recovery • Practitioner needs to engage the client in clarifying wishes and discuss available options through which an intervention is most likely to accomplish the client’s goals • Practitioner does not know what is best for the client, who may be seeking assistance with a range of needs (e.g. housing, employment, symptom reduction, social skills training) • Practitioner needs to know the literature and provide the consumer with the information to make informed choices about service options.

17. Recovery What if the consumer does not want to participate in an evidence-based practice? • Most often, provided that the practitioner has understood the consumer’s wishes, good information about evidence-based assistance will lead to a good choice • At times, however, practitioner may need to abandon what he/she knows works, at least temporarily, and respect the consumer’s choice.

18. Cultural Considerations How do we know that a particular practice that was researched in a different place with a different population will work with my clients? Most EBPs have been verified with limited populations, so they are not necessarily culturally-specific to our clients’ needs. EBPs should be applied in a manner that is individually and culturally sensitive.

19. Cultural Considerations • Consensus opinion is that EBPs do help minority individuals as well as Anglo populations. • Latest research shows that EBPs are, in general, cross-culturally effective, but modifications may need to occur. • Don’t be afraid to stretch the parameters of an EBP to fit your needs, just do so mindfully and document modifications.

20. Where to Find EBPs • Specific online searches can provide a wealth of options for selection of EBPs • In the past, SAMHSA has taken the lead on evaluating and disseminating EBPs through the NREPP database. • As of January 2018, this database is no longer available

21. NREPP The National Registry of Evidence-based Programs and Practices (NREPP) was launched by SAMHSA in 1997 to provide a comprehensive, searchable database for EBPs Vetted clinical practices and evaluated strength of research for a wide range of topics, and provided scientific and clinical resources. By early 2018, had listing for 453 EBPs

22. NREPP • In January 2018, the current federal administration cut funding for NREPP, halted all maintenance of the project and eventually pulled the website down. • HHS administration said that they planned “to reconfigure its approach to identifying and disseminating evidence-based practice and programs.”

23. NREPP • NREPP closure followed shortly after administration guidance to CDC and other federal agencies to avoid using seven terms in budget documents including: • Advised that instead of using the term “evidence-based”, should say that the CDC “bases its recommendations on science in consideration with community standards and wishes” • Evidence-based • Science-based • Diversity • Vulnerable • Entitlement • Fetus • Transgender

24. NREPP • In late 2018, SAMHSA launched the Evidence-Based Practices Resource Center • www.samhsa.gov/ebp-resource-center • Provides links to publications and some rudimentary search features. • Lists 148 practices

25. Examples of EBPs Assertive Community Treatment (ACT) Cognitive-Behavioral Therapy (CBT) Matrix Model Illness Management and Recovery (IMR) Family Psychoeducation Psychopharmacology Integrated Dual Disorders Treatment (IDDT) Dialectical Behavior Therapy (DBT) Motivational Interviewing Motivational Incentives Multisystemic Therapy (MST) Functional Family Training (FFT)

26. Identify organizational changes that will need to occur Create an Action Plan, with timelines, regarding training, supervision, rollout, etc. Monitor outcomes and fidelity to model Modify Continually revise Action Plan based on findings Sustain! Implementation Steps

27. Workforce issues Organizational readiness Stakeholder support Financial commitment Need for training and supervision Organizational culture not amenable to change. Systemic bureaucracy Financial barriers Cultural/Community barriers Political forces Implementation Challenges

28. Action Plan • As with Service Planning, an Action Plan for EBP implementation should identify long-term goals and short-term objectives, responsible individuals, time-frames, etc. • Should cover the range of activities from beginning to end. • Can address the 10-steps in a structured manner.

29. ActionPlan • Identify specific goals in the Action Plan, as well as activities and strategies needed for implementation. • Important areas to focus might include: • Getting support at all levels of staff • Educating everyone about the purpose and benefits • Visiting other programs • Providing needed training • Outcomes and fidelity.

30. A 10-Step Guide to Adopting and Sustaining Evidence-Based Practices Metz, 2007 • Prioritize Outcomes: Identify the Need for an Intervention • Conduct Readiness Assessments • Garner Support of Critical Stakeholders • Research and Identify a “Good Fit” • Decide on Necessary Program Adaptations • Seek Information, Technical Assistance, and Program Consultation • Provide Staff Training and Consultation • Begin Initial Implementation • Conduct Ongoing Evaluation and Fidelity Assessments • Create Feedback Loops and Ongoing Opportunities for Learning and Reflection

31. 10-Steps • Planning and Assessing Readiness • 1. Prioritize Outcomes: Identify the Need for an Intervention • Prioritize program outcomes and desires • Assess how gaps in resources are interfering with desired outcomes. • 2. Conduct Readiness Assessment(s) • Evaluate program’s capacity and motivation for change • Use information to implement strategies that will enhance success of change. • Useful tools include EBPAS, GOI, Stages of Change model. • 3. Garner the Support of Critical Stakeholders • Need buy-in at all levels of agency, internally and externally • Obtain necessary resources (systemic/financial/staffing) needed to implement

32. Assessing Readiness • Successful implementation begins with a thorough assessment of systemic readiness to change. • Areas to assess include: • Staff/Practitioners • Consumers and Families • Other Stakeholders • The Program and Larger Organization • Finances and Funding • Policies/Procedures/Documentation • Politics

33. Assessing Readiness • Think about your clients, your staff, and your organization • Determine what type of outcomes you hope to achieve. • Gather advice from a variety of stakeholders as to what they see as a priority (i.e. surveys, focus groups) • Review the literature to determine what models might help with these goals. • Determine which EBP would be the most useful for your program.

34. Leadership is critical to success Attitude sets the tone and can model flexible responses to challenges Prioritizing change will demonstrate to everyone that effort is constructive Action related to hiring right staff, supporting training and supervision, policies and procedures, etc. Critical Factor

35. 10-Steps • Creating The Perfect Fit • 4. Research and Identify a “Good Fit” • Identify EBP that meets client needs, can be incorporated into existing program, and help achieve outcomes. • Consider issues related to desired client outcomes, community needs, organizational culture, financial and human resources. • 5. Decide on Necessary Program Adaptations • May need to modify program to meet local needs, but cannot sacrifice fidelity. • Consult with program developer or other experts if possible. • Document any significant changes to the model. • 6. Seek Information, Technical Assistance, and Program Consultation • Gather as much information and support as possible. • Seek local ‘mentors’, if possible.

36. Selection Considerations Choosing an EBP to implement is dependent on your outcome goals. What does the population need most? What do your clients say that they want as outcomes? Housing Employment Medication Management Addiction treatment Symptom management Improved relationships

37. SelectionConsiderations Client characteristics, including culture, socioeconomics and other health or social issues that might complicate service delivery Staff attributes and skills required by the EBP Facilities and resources required by the EBP Agency policies and administrative procedures needed to support the EBP Community and interagency linkages

38. Implementation of EBPs challenges an organization on many levels. Significant requirements include: Organizational restructuring Changes to practice Ongoing staff training and supervision Commitment of money and resources Sustained effort to support and maintain the model Implementation

39. Financial commitment Initial and regular training is necessary Supervision is non-negotiable, but expensive EBPs require non-billable time Staff turnover is a reality and major challenge Potential for bundled rates Critical Factor

40. EBP “culture” micro- and macro- systems need to understand rationale, benefits, and consequences of using EBPs need to be integrated into existing array of services (not something being done separately) need to be considered in management and QA functions Critical Factor

41. 10-Steps • Training Staff and Getting Going! • 7. Provide Staff Training and Coaching • Adequate time and resources must be directed to staff training. • Training needed for not only frontline staff, but for supervisors, managers, QA. • Focus training on both knowledge acquisition and skill development • Ongoing supervision and consultation is high priority. • 8. Begin Initial Implementation • Implementation occurs at all levels of the organization. • May require changes to policies and procedures, resource allocation, staffing, operations, etc.

42. Consultation and training infrastructure Clinical supervisors and staff require depth and breadth of information about the practice Education of consumers and family members about advantages/disadvantages of EBP, input during implementation process Consultation for initiation and ongoing implementation Training alone does increase knowledge, but has limited impact on practice Critical Factor

43. Staff Challenges Practitioners have a sense of what works from the multiple clients they have helped and their years of practice. They have experienced success with clients using the techniques familiar to them and do not necessarily believe that client care needs to be improved. They also don’t like to be told their approaches don’t work, especially by non-clinical bureaucrats!

44. Staff Challenges • Clinicians are creatures of habit, and in order to seriously consider making a change, they have to believe that: • the new practice is better than the current one • that it will make the lives of their clients better • and that it will improve their ability to be clinically successful.

45. Staff Resistance Resistance should be viewed as a problem with the manner in which new practices are being implemented. Resistance should not be seen as failure, but rather as an opportunity to assess readiness and provide support and encouragement. Strategies can and should be tailored to individual and systemic stages of readiness. Leaders should work with staff to identify and eliminate barriers to, and help establish positive influences for, the adoption of EBPs.

46. Staff Change Strategies • 1. Approach and Engagement • The manner in which staff is approached and the message that they receive about EBPs will impact their degree of willingness to embrace a new model. • Being critical of the “old ways” may generate defensiveness. • Staff should understand why the changes will benefit the clients, the clinicians, and the agency as a whole. • The message-bearers are critical - identify champions among leadership, line staff, consumers and family.

47. Staff Change Strategies • 2. Training and Access to Information About EBPs • It is a common mistake to think that simply attending a training workshop is enough to change professional attitudes toward or delivery of care. • In order to “sink in”, effective training requires knowledge acquisition, skills training, and attitude change. • Modalities may include literature review, “grand rounds”, formal and informal discussions, visiting other programs, consulting, and supervision.

48. Staff Change Strategies • 3. Simplifying Practices and Easing Difficulty of • Change • Break the new practice into parts and discuss and implement in stages. • Involve all levels of staff and consumers/families to problem-solve and brainstorm solutions. • Replace current activities with new ones to lessen the sense of added work. • If possible, allow practitioners to choose to participate in new program.

49. Staff Change Strategies • 4. Providing Supports and Removing Barriers • Allow staff time to learn and discuss new practices, participate in planning and development, and the real opportunity to learn and practice necessary skills. • Reinforce new behaviors through encouragement, feedback, supervision, and emphasis on fidelity to model. • Provide technological support if possible. • Follow-up in a critical and consistent way.

50. Staff Change Strategies • 5. Use of Incentives and/or Sanctions • Sometimes staff needs a little external motivation. • Rewards such as workload reduction, public recognition, travel for training, extra pay, etc., can all motivate participation. • Try not to create a sense that the program is an “extra” that requiring more work. It is simply a change in focus. • May need to utilize consequences for lack of cooperation, but first try to understand and address barriers