Working Smarter: The implementation journey to improve student outcomes

Working Smarter: The implementation journey to improve student outcomes Michelle A. Duda, Ph.D., BCBA, Dean L. Fixsen, Ph.D. & Karen A. Blase Ph.D., Melissa Van Dyke, LCSW Frank Porter Graham Child Development Institute University of North Carolina – Chapel Hill

Implementation FluencyPart 2

Science to Service GAP SERVICE SCIENCE IMPLEMENTATION

Science to Service Gap • What is known is not what is adopted to help students, families, and communities • Implementation Gap • What is adopted is not used with fidelityand good outcomes for consumers. • What is used with fidelity is not sustained for a useful period of time. • What is sustained is not used on a scale sufficient to impact societal outcomes. Science to Service

Making use of Science Letting it happen Recipients are accountable Helping it happen Recipients are accountable Making it happen Implementation teams are accountable Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004

What is Implementation? • A specified set of purposeful activities at the practice, program, and system level designed to put into place a program or intervention of known dimensions with fidelity. What do we mean by implementation?

It’s all about changing the behavior of well intentioned people including practitioners, providers, community stakeholders, policy makers and funders! In Other Words…

Excellent evidence for what does not work Ineffective Methods • Implementation by mandatedoes not work • Implementation by “following themoney” does not work • Implementation without changingsupporting roles and functions does not work Paul Nutt (2002). Why Decisions Fail

Excellent experimental evidence for what does not work Ineffective Methods • Diffusion/disseminationof information by itself does not lead to successful implementation (research literature, mailings, promulgation of practice guidelines) • Training alone, no matter how well done, does not lead to successful implementation

IMPLEMENTATION Effective NOT Effective Implement Innovations Student Benefits Effective INTERVENTION Placebo NOT Effective PLACEBO: Something of no intrinsic remedial value that is used to appease or reassure another

What Works Effective intervention practices + Effective implementation practices = Good outcomes for consumers

Common Components of Fidelity Measures Adherence Dosage/Exposure Quality of Program Delivery

Fidelity Data Collection Methods Direct Observation Practitioner Self-report Consumer Self-report

Why are programs not implemented with fidelity? Lack of teacher training, Lack of required materials Use of some (but not all) of the required lessons and teaching strategies l Limited time _____________________________ Lack of funding Inadequate infrastructure Decentralized decision making Lack of administrative support

Why are programs adapted? Inadequate training/lack of understanding of program’s underlying theory Barriers (time, money, resources, accessing target population) Programs that are not user-friendly Lack of perceived efficacy, relevance, acceptance Brian Bumbarger, Prevention Research Center

Strategies for Maintaining Fidelity Creating awareness around the importance of maintaining fidelity Improving practitioner understanding of program’s underlying theory and rationale Developing sustainable infrastructure and processes for monitoring implementation and fidelity Monitoring fidelity continually

Evidence-Based Movement The “evidence-based movement” is an international experiment to make better use of research findings in typical service settings. The purpose is to produce greater benefits to students and society.

EBPs & Implementation Core intervention components Clearly described (who/what) Practical measure of fidelity Fully operationalized (do/say) Field tested (recursive revision) Contextualized (org./systems fit) Effective (worth the effort)

EBPs & Implementation Theusabilityof a program/practice has little to do with the quality or weight of the evidence regarding that program Evidence on intervention effectiveness for specific populations helps us choose what to implement Evidence on the effectiveness of the intervention does not help implement the program or practice successfully

EBPs & Implementation From an implementation perspective, what do we need to know about innovations such as evidence-based programs? Successful and sustainable implementation of evidence-based programs always requires organization and systems change.

Implementation Frameworks In order to ensure that an EBP maintains and sustains producing benefits to students over time, it is critical to ensure that the system is both multi-dimensional and fully integrated through : Implementation Stages (PART 3) Implementation Drivers (PART 4) Improvement Cycles (PART 5)

Michelle A. Duda, Ph.D. • 919-636-0843 • duda@mail.fpg.unc.edu For More Information Dean L. Fixsen, Ph.D. 919-966-3892 fixsen@mail.fpg.unc.edu • At the Frank Porter Graham Child Development Institute • University of North Carolina • Chapel Hill, NC • www.scalingup.org • http://nirn.fpg.unc.edu/ • http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/

For More Information Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/ To order the monograph go to: https://fmhi.pro-copy.com/

Working Smarter: The implementation journey to improve student outcomes