Implementing Evidence-based Practice (EBP) in Social Work: A Pilot Study

1. Implementing Evidence-based Practice (EBP) in Social Work: A Pilot Study Edward J Mullen Jennifer Bellamy Sarah Bledsoe Columbia University Musher Center Supported in part by National Institute of Health Doctoral Training Program in Mental Health Services Research #5 T32 MH14623-24/25 & the Willma & Albert Musher Center at Columbia University

2. Pilot Study • Pilot study aim: • to design, test & refine a strategy for implementing EBP in social work agencies • Uses a social intervention research methodology • Multi-site to permit comparison • Conducted: • with support of the Columbia University Musher Center • in collaboration with the National Institute of Mental Health/Columbia University School of Social Work Pre-doctoral Training Program in Mental Health Services Research (#5 T32 MH14623-24/25)

3. Main Research Question: • We want to learn how social work organizations & practitioners can be engaged & helped to adopt & implement evidence-based practice

4. Social Intervention Research Design: Phase 1 • Review literature pertaining what is known about dissemination & implementation strategies targeting EBP in social work • Talk to experienced researchers who have studied EBP and implementation of EBP to get their insights & experiences • Talk to agency administrators to explore interest in partnering • Talk to practitioners to find out what would motivate them to do EBP, how they would like to learn EBP, & perceived barriers to EBP

5. Social Intervention Research Design: Phase 2 • Select one or more agencies as partners in the pilot study • Design the pilot study with the agency representatives so as to: • use the information gained in step 1 • shape the intervention to the context of the specific agency • Secure funding for implementation of the pilot intervention • Apply for Institutional Review Board Approval

6. Social Intervention Research Design: Phase 3 • Implement the intervention • Monitor implementation • Modify intervention during implementation as experience dictates • Evaluate process & outcomes

7. Social Intervention Research Design: Phase 4 • Use the information coming from the evaluation to redesign the implementation strategy & the evaluation design • Report experiences and findings • Consider new pilot or larger field test if the strategy merits

8. Immediate Question • What strategy is most effective to achieve enduring, effective, & efficient implementation of EBP in a social work agency? • What are the options? • The phase 1 literature review and interviews with EBP researchers were designed to help answer these questions

9. Results of Literature Review & Interviews Reported: • Bellamy, J. L., Traube, D. E., & Bledsoe, S. E. (2004). Strategies and barriers in sharing knowledge: Caring for clients through methods of best practice. Unpublished manuscript, Quebec City, 4th International Conference on Social Work in Health and Mental Health • Bellamy, J., Bledsoe, S. E., & Traube, D. (in press). The current state of evidence based practice in social work: a review of the literature and qualitative analysis of expert interviews. Journal of Evidence-Based Social Work

10. Interviews • To provide a comprehensive and unified approach to EBP social work should… • Network and share resources between agencies and practitioners • Increase buy-in and ownership at all levels of stakeholders • Provide more and higher quality continuing education • Translate research into user-friendly, digestible, and specific approaches • Provide tools such as toolkits, guidelines, and technical support to both support • Attach evidence based practices to meaningful signposts • Protect caseworker, team, or supervisor time for research, development, and training toward EBPs

11. Literature Review • Search of electronic databases: • Social Work Abstracts • PsychLit • MedLine • Research Unit for Research Utilization (RURU) • Major topics: • barriers to the process of using evidence/EBP • frameworks & strategies for dissemination & implementation of research/EBP

12. Barriers to Implementation • Lack of Knowledge & Skills for EBP • Lack of Knowledge Fit to Agency Practice • Suspicion of Researchers & EBP • Limited Resource for Doing EBP

13. Lack of Knowledge & Skills for EBP • General lack of awareness of available evidence based practices and the difficulty in processing or understanding identified research: • How to best access, evaluate, and translate evidence (Anderson, Cosby, Swan, Moore, & Broekhoven, 1999; Mullen & Bacon, In press). • Misunderstanding of what constitutes an evidence based practice such as (Gibbs & Gambrill, 2002) : • We’re already using evidence • It’s a matter of opinion • There is no evidence

14. Lack of Knowledge Fit to Needs of Agency Practice • Evidence or research that is available or accessible is not often particularly helpful: • Cookbook/stilted approaches • Overly broad • Ignores contextual or cultural issues • Not realistic in practice • Mismatch with current policy and technology • The culture of knowledge transmission within social work has been historically unsupportive of the use of research evidence in practice (Barratt, 2003 Anderson, Cosby, Swan, Moore, & Broekhoven, 1999; Gibbs & Gambrill, 2002 ).

15. Suspicion of Researchers & EBP • Basic distrust for evidence, based on objections related to political, ethical, or control issues: • Politically motivated, guided by efficiency, researcher preferences, or simply a cost-cutting tool (something other than client’s best interest) • Devalues the practitioner’s professional judgment or clients preferences • Trendy, more about research reputations, and malleable evidence (Gibbs & Gambrill, 2002; Mullen & Bacon, 2003)

16. Limited Resource for Doing EBP • Even for those practitioners and agencies that understand, appreciate, and want to use evidence in their work they may find themselves incapable due to a lack of resources including: • Lack of training, materials, time and staff dedicated to researching the evidence • Poor technology resources (computers and internet access) • Inconsistent funding routed everywhere else first (Anderson, Cosby, Swan, Moore, & Broekhoven, 1999; Barratt, 2003; Mullen & Bacon, 2003)

17. Current Strategies for EBP Implementation • Bottom-up: • Teaching professionals to be evidence-based, lifetime learners (Sackett, 2000; Gibbs & Gambrill, 2002; Gibbs, 2003; Gray, 2001) • Top-down • Tool kits/application kits/manuals/guidelines (Mueser, Torrey, Lynde, Singer and Drake, 2003) • Top-down/bottom-up • Combining evidence & consensus (Cook, 2004) • Combine focus on practitioner training & organizational development • Interactive staff training (McCracken & Corrigan, 2004) • Outcomes & objectives orientation (Rosen, Proctor, Morrow-Howell, Auslander, & Staudt, 1993) • Targeting the Social Work Profession • Objectives-Focused Multilevel Strategy (Proctor, 2004)

18. 1. Bottom-upTeaching Professionals to be Evidence-based, Lifetime Learners • Oldest approach to disseminating & implementing EBP • Focus on teaching students & practitioners evidence-based processes so they can be lifetime learners • Clinical learning strategy developed at McMaster Medical School, Canada - 1980’s, 1991 & 1992 • Adapted in healthcare (Gray, 2001) & social work (Gibbs & Gambrill, 2002)

19. What is Taught? • Motivation for EBP • How to convert information need into a search question • Evidence search tools • Evidence appraisal skills • Information integration skills (next slide) • Self-evaluation • Teaching EBP Gibbs, 2003

20. Client State & Circumstances Professional Expertise Client Preferences & Actions Research Evidence Integration Skills – What Needs Integrating? Haynes, Devereaux, and Guyatt, 2002

21. Evidence-based Healthcare ProcessesAdapted from J A Muir Gray (2001)

22. Individuals & OrganizationsAdapted from Gray, p.xxii

23. 2. Top DownTool kits/application kits/manuals/guidelines • Identify evidence-based, empirically supported practices • Develop tool/application kits/manuals/guidelines • Engage in broad scale effort to disseminate & facilitate local adoption by supporting training & working with stakeholders • Example • Implementing Evidence-Based Practice Project (Mueser, Torrey, Lynde, Singer & Drake, 2003) • Goal to promote widespread adoption of 6 EBP’s for mentally ill adults

24. 3. Top-down/Bottom-upCombining EBP with Community Stakeholder Consensus • Attempts to deal with: • gaps in empirically-based knowledge about effective interventions • unevenness of knowledge about implementation • need to involve community stakeholders in decision-making especially in the context of limited resources • need to adapt knowledge to local conditions • Example: • Texas Benefit Design Initiative (Cook, 2004) • review evidence re effectiveness & implementation needs • present evidence to community stakeholder consensus groups at state level – form recommendations • present evidence and recommendations to local consensus groups for implementation planning

25. Texas Benefit Design Initiative (Cook, 2004) • Goal: • Combine best evidence with community consensus in designing psychosocial rehabilitation services using public mental health services • Steps: • Experts reviewed evidence & presented findings at consensus conference which included over 200 citizens from around Texas • Consensus panel formulated service package principles & recommendations for service organization & implementation • Panels included stakeholder representatives from consumer & family organizations, service providers, advocates, state level administrators, researchers, & other interested parties • Second level consensus meetings were held in each of the pilot sites to design specific benefit package for local application

26. 4.1 Combine Staff Training & Organizational DevelopmentInteractive Staff Training • Interaction of educational & organizational approaches • Educational approach based on learning theory • designed to help individual practitioners acquire EBP principles & skills • Organizational approaches teach individual staff members how to work as a team to develop EB treatment programs that are user-friendly & meet consumer needs • Guidelines for disseminating & adopting evidence-based practice innovations: • Engage & prepare organization • Engage & form working alliance with treatment team • Develop a user friendly program based on identified program-development priorities & the innovation • Use established educational principles in training • Stepwise implementation of program • Plan for maintenance • (McCracken & Corrigan, 2004)

27. 4.2 Combine Staff Training & Organizational DevelopmentOutcomes & Objectives Orientation • Combines attention to: • practitioner training • trained to use critical-thinking process (Systematic Planned Practice) specifying: • client problem to be addressed • outcomes sought • intermediate objectives needed to attain these outcomes • self-evaluation skills • practitioners prepared to critically & selectively use evidence-based practices & guidelines made available to them by organization • organizational support • organizational barriers to EBP identified • organizational supports needed are identified • barriers & supports addressed in implementation strategy • Rosen, Proctor, Morrow-Howell, Auslander, & Staudt, 1993

28. 5. Professional Infrastructure as Agent

29. Interventions Required for these Four Intermediate Objectives • Interventions specific to practice infrastructure: • research • training • organizational culture • Based on theories & research about: • diffusion of innovations • knowledge utilization • quality improvement • Prochaska & Di Clemente’s stages of change model

30. Gira, Kessler and Poertner (2004) • “The literature from health care suggests that disseminating information alone is insufficient. Many interventions have been designed to improve practitioners’ adherence to EBP guidelines and are differentially effective. To date, no intervention has demonstrated powerful effects.” (p. 77-78) • They speculate that multiple strategies are needed rather than relying on any single intervention.

31. Selection of a Strategy for Use in Our Study • Review key findings from the literature regarding barriers to and strategies for implementation of EBP with agency teams • Together with the agency partners select a best-fit strategy or a modified strategy that emerges from the discussions

32. Selection of Partner Agencies • Three New York City agencies have elected to join the project & are collaborating with university researchers on the initial stages of the pilot design. • Two have programs delivered in a wide range of geographical and problem areas. They are among the largest voluntary social agencies in the City. The third agency is neighborhood based, small & informal. It too has a reputation of excellence, professionalism, and innovation. • All are known for excellence & the highest level of professionalism. • Two have considerable resources & the third has modest resources.

33. Initial Meetings with Agencies • Agency-university relationship -focus on partnership -respect for agency perspectives and practitioner wisdom -mixed reaction to unconventional research • Evidence based practice in the agencies -unfamiliar term -teams

34. Initial Meetings with Agencies • Problem formulation -large agencies with diverse programs -continual narrowing of focus -process vs. outcome interest • Team building -creative, innovative, or change oriented staff -midlevel administrator interest

35. Initial Meetings with Agencies • Resources -computers -time and scheduling • Agencies want trainings in EBP, but not to be told how to practice • Suspicion • Varying levels of expertise and education

36. Next Steps • Refining focus of questions/issue areas • Meeting with direct service staff • Building teams • Focus groups/trainings • Search for funding • Implementation • Evaluate/revise • Share findings

37. Concluding Note We have learned a great deal during this past year from the literature review, the interviews with EBP experts, and, from our agency partners. Evidence-based practice is a new innovation for social work. There are many challenges and unknowns, especially pertaining to implementation in complex social work programs serving populations that have many needs and problems. We look forward to exploring these challenges with our agency partners and those they serve in the year ahead.

38. References