Ensuring RtI’s Successful Implementation

1. Ensuring RtI’s Successful Implementation Dean L. Fixsen, Karen A. Blase, Sandra F. Naoom, Melissa Van Dyke, Frances Wallace National Implementation Research NetworkLouis de la Parte Florida Mental Health Institute Virginia RtI Meeting 2007

2. A Functional System Policies Bureaucracy Agencies Practitioners

3. Evidence-Based Movement The “evidence-based movement” is an international experiment to make better use of research findings in typical service settings. The purpose is to produce greater benefits to students and society.

4. Education • 65 million kids • 6 million teachers and staff • 140,000 schools • 3,143 counties • 60 states & U.S. jurisdictions

5. Science to Service SERVICE SCIENCE IMPLEMENTATION GAP

6. Science to Service Science to Service Gap What is known is not what is adopted to help students, families, and communities • Implementation Gap • What is adopted is not used with fidelityand good outcomes for consumers. • What is used with fidelity is not sustained for a useful period of time. • What is sustained is not used on a scale sufficient to impact societal outcomes.

7. Making use of Science Letting it happen Recipients are accountable Helping it happen Recipients are accountable Making it happen Implementation teams are accountable Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004

8. Teaching–Family Replications Fixsen, Blase, Timbers, & Wolf (2001) 900 300 800 700 250 600 200 500 CumulativeHomes 150 400 300 100 CumulativeCouples 200 50 100 0 0 ≤1972 1974 1976 1978 1980 1982

9. Follow Through Programs Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press.

10. School Wide PBS 4.2% of all schools

11. Implementation Reviews Human service prevention and treatment programs (e.g. education, substance abuse, adult / children’s MH, justice, health) Advanced manufacturing technologies AMA clinical guidelines Engineering: bridge maintenance Hotel service management National franchise operations Cancer prevention & treatment

12. Ineffective Methods Excellent experimental evidence for what does not work • Diffusion/dissemination of information by itself does not lead to successful implementation (research literature, mailings, promulgation of practice guidelines) • Training alone, no matter how well done, does not lead to successful implementation

13. Ineffective Methods Excellent evidence for what does not work • Implementation by edict by itself does not work • Implementation by “following themoney” by itself does not work • Implementation without changing supporting roles and functions does not work Paul Nutt (2002). Why Decisions Fail

14. What Works Effective intervention practices + Effective implementation practices = Good outcomes for consumers

15. Implement Innovations IMPLEMENTATION Effective NOT Effective Student Benefits Effective INTERVENTION Placebo NOT Effective PLACEBO: Something of no intrinsic remedial value that is used to appease or reassure another

16. Implementation An effective intervention is one thing Implementation of an effective intervention is a very different thing

17. EBPs & Implementation From an implementation perspective, what do we need to know about innovations such as evidence-based programs?

18. EBPs & Implementation The usability of a program has little to do with the quality or weight of the evidence regarding that program • Evidence on intervention effectiveness for specific populations helps us choose what to implement • Evidence on the effectiveness of the intervention does not help implement the program or practice successfully

19. EBPs & Implementation Core intervention components Clearly described (who/what) Practical measure of fidelity Fully operationalized (do/say) Field tested (recursive revision) Contextualized (org./systems fit) Effective (worth the effort)

20. Implementation What do we need to know about successful implementation methods?

21. Stages of Implementation Implementation is not an event A mission-oriented process involving multiple decisions, actions, and corrections

22. Stages of Implementation Implementation occurs in stages: • Exploration • Installation • Initial Implementation • Full Implementation • Innovation • Sustainability 2 – 4 Years Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

23. Implementation Outcomes Intervention Outcomes 0% 100% Stages of Implementation Implementation occurs in stages: • Exploration • Installation • Initial Implementation • Full Implementation • Innovation • Sustainability Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

24. Implementation Drivers STAFF PERFORMANCE EVALUATION CONSULTATION & COACHING DECISION SUPPORT DATA SYSTEMS INTEGRATED & COMPENSATORY FACILITATIVE ADMINISTRATIVE SUPPORTS PRESERVICE TRAINING RECRUITMENT AND SELECTION SYSTEMS INTERVENTIONS

25. Integrated & Compensatory OUTCOMES (% of Participants who Demonstrate Knowledge, Demonstrate new Skills in a Training Setting, and Use new Skills in the Classroom) TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10% 5% 0% ..+Demonstration in Training 30% 20% 0% …+ Practice & Feedback in Training 60% 60% 5% …+ Coaching in Classroom 95% 95% 95% Joyce and Showers, 2002

26. Who does the work? • Implementation Teams • Develop effective, flexible, adaptable capacity to initiate and manage continual change • Requires new roles, functions, and skill sets that do not exist currently

27. Implementation Team • A group that knows the innovation very well (formal and practice knowledge) • A group that knows how to implement that innovation with fidelity and good effect • A group that accumulates data & experiential knowledge -- more effective and efficient over time (information economics, K. Arrow)

28. Implementation Team Policy members (change policy, barrier busters, facilitators) Practice members (do the innovation, test policies, feedback) Families and stakeholders Management members (roles and functions) Daily / Weekly / Monthly Meetings (urgent, unfiltered, goal focused)

29. Implementation Team Teacher • School • Management (leadership, policy) • Administration (HR, structure) • Supervision (nature, content) District State and Community Context Simultaneous, Multi-Level Interventions Implementation Team

30. Implementation Team Change the behavior of adult education professionals “Systems don’t change, people do” (J.W.) Change organizational structures, cultures, and climates Change the thinking of system directors and policy makers Successful and sustainable implementation of evidence-based programs always requires organization and systems change.

31. RtI

32. What is RtI?? Core intervention components Clearly described (who/what) Practical measure of fidelity Fully operationalized (do/say) Field tested (recursive revision) Contextualized (org./systems fit) Effective (worth the effort)

33. Response to Intervention: PhilosophyCore Principles (NASDSE, 2006) • We can effectively teach all children • We need to identify the curricular, instructional and environmental conditions for learning. • Intervene early • Solving small problems early is both more efficient and more successful. • Use a multi-tier model of service delivery • Needs-driven, resourced deployment systems to match instructional resources with student need. Dona Meinders, Silvia DeRuvo; WestEd, California Comprehensive Center

34. RtI Definition: Components • RtI is the practice of providing: • high-quality instruction and intervention • matched to student need, • monitoring progress frequently to • make decisions about change in instruction or goals and • applying child response data to important educational decisions. (NASDSE, 2005) Dona Meinders, Silvia DeRuvo; WestEd, California Comprehensive Center

35. RtI: Components (NASDSE, 2006) • Monitor student progress to inform instruction • Use assessment to collect information on how student is progressing • Use data to make decisions • Ongoing data collection systems in place and used to make informed decisions • Use assessment for three different purposes • Screening • Diagnostics • Progress monitoring Dona Meinders, Silvia DeRuvo; WestEd, California Comprehensive Center

36. Response to Intervention • Shifts accountability for outcomes from the children to the teachers and their supporting schools & education systems

37. Stages of Implementation Implementation occurs in stages: • Exploration • Installation • Initial Implementation • Full Implementation • Innovation • Sustainability 2 – 4 Years Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

38. RtI: Oregon Group • District level entry point; system change focus • Information, what it is, is not • Stakeholder buy-in, • Application/ selection process • Informed agreement, understand and defend the initiative The Tigard-Tualatin District; Carol Sadler et al

39. RtI: Oregon Group • District level system change • Give lots of rationales • Not a project, not patchwork • Focus on RtI functions • Establish a common vocabulary to ease communication • Build on what folks are doing already – help them get ready for change

40. RtI: Oregon Group • Demand far exceeds capacity • Select the willing • Be overly strict at the start • Work with others to help them become “RtI ready” • Develop capacity: Yr 3 sites now helping with newbies

41. Implementation Drivers STAFF PERFORMANCE EVALUATION CONSULTATION & COACHING DECISION SUPPORT DATA SYSTEMS INTEGRATED & COMPENSATORY FACILITATIVE ADMINISTRATIVE SUPPORTS PRESERVICE TRAINING RECRUITMENT AND SELECTION SYSTEMS INTERVENTIONS

42. RtI: Oregon Group • Guided development • Leadership involvement (will require more in the future) • Year of training with on-going coaching (will require more on-site visits in the future) • Include leaders in the training (well informed, able to explain and defend, willing to do what is required)

43. RtI: Oregon Group • Infrastructure development • DOs need information to guide the changes • Top down and bottom up approach • Never done, always changing, 3-ring binders and updated websites

44. Systems Change • Organizational changes (schools and districts) • System changes (state and federal)

45. RtI: Oregon Group • Infrastructure development • Designate funds to support implementation efforts right from the start (costs associated with making changes)

46. RtI: Oregon Group • Issues • Teacher education does not support RtI work (philosophy, values, skills) • Teacher certification may need to change

47. A Sobering Observation "All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get." R. Spencer Darling Business Expert

48. Systems Change New practices do not fare well in old organizational structures and systems Develop new position descriptions and job functions in state departments and districts focused on implementation (effective use) of policies and innovations

49. Academic Systems Behavioral Systems • Intensive, Individual Interventions • Individual Students • Assessment-based • High Intensity • Of longer duration • Targeted Group Interventions • Some students (at-risk) • High efficiency • Rapid response • Targeted Group Interventions • Some students (at-risk) • High efficiency • Rapid response • Universal Interventions • All students • Preventive, proactive • Universal Interventions • All settings, all students • Preventive, proactive 80-90% 80-90% Multi-tier Model • Intensive, Individual Interventions • Individual Students • Assessment-based • Intense, durable procedures 1-5% 1-5% 5-10% 5-10% Dona Meinders, Silvia DeRuvo; WestEd, California Comprehensive Center

50. Academic Systems Behavioral Systems • Intensive, Individual Interventions • Individual Students • Assessment-based • High Intensity • Of longer duration • Targeted Group Interventions • Some students (at-risk) • High efficiency • Rapid response • Targeted Group Interventions • Some students (at-risk) • High efficiency • Rapid response • Universal Interventions • All students • Preventive, proactive • Universal Interventions • All settings, all students • Preventive, proactive 80-90% 80-90% Multi-tier Model • Intensive, Individual Interventions • Individual Students • Assessment-based • Intense, durable procedures Attention, Effort, Precision 1-5% 1-5% 5-10% 5-10%