Advance Organizer

1. TOT Y3D3 Tier 3: Instruction/Intervention Design & IntegrityProgress MonitoringSchool Implementation Blueprints A collaborative project between the Florida Department of Education and the University of South Florida

2. Advance Organizer • Y3D2 Content Review • Skill Assessment Review • T3 Intervention Design • Intervention Integrity • T3 Progress Monitoring • Review of Randy - Comprehensive Intervention Plan Columns 4-5 • Step 4 Student Success Worksheet • Surveys - Personnel Satisfaction, Perceptions of Practices • School Blueprint • Implementation

3. Annual Growth/Catch-up Growth, Tier 3 Intervention Design YEAR 3, DAY 2 REVIEW

4. Review:Annual & Catch-up Growth Each year of instruction in core subject yields one year annual growth Students who are behind must achieve annual growth PLUS Primary driver of catch-up growth is increased instructional time

5. Review: Kenewick’s formula 13 percentile pts. = one year’s growth • Gr. 4 State standard: 50th percentile • Target 2nd grader: 12th percentile • The difference: 38 points • Point difference divided by 13: 2.9 yrs. Behind Example taken from Annual Growth…Catch-up Growth, Fielding et al., 2007

6. Review: Kenewick’s formula (cont.) Calculating instructional minutes: Daily minutes for annual 3rd gr. growth: 80 Daily minutes for annual 4th gr. growth: 80 Additional daily minutes to make three years of catch-up growth: 240 Total 3rd/4th grade daily minutes: 400

7. Review: Kenewick’s formula (cont.) Dividing the total minutes (400) between 3rd and 4th grade shows that the student needs 200 minutes of direct reading instruction in both 3rd and 4th grades to reach the 50th percentile by the end of fourth grade.

8. Review: Tier 3 Intervention Design Purpose is instructional match Adjusts what is taught and/or how it is taught Evidence-base for Tier 3 instruction relies on careful progress monitoring

9. Review: Tier 3 Intervention Design

10. Y3D2 Skill Assessment Review

11. Skill Assessment ReviewGuiding Questions • 1. Which components of developing a comprehensive intervention plan did your team complete successfully? For which components did the team have difficulty? What factors helped or hindered the team when developing the comprehensive intervention plan to address the student’s needs? • 2. How helpful was it to your team to specifically outline the intervention and support plans (i.e., who was responsible, what would be done, when it would occur, and where it would occur)? • 3. If your team was able to return to your school and implement the plan developed at the last training, to what extent did the level of specificity help ensure that the plan was implemented? • 4. If your team was able to return to your school and implement the plan developed at the last training, to what extent did the support plan you developed for the individuals responsible for implementing the intervention help ensure that the intervention was implemented as intended?

12. INTERVENTION INTEGRITY Tier 3 Considerations

13. Importance of integrity: • Rapid and widespread deployment of RtI has made urgent the need to attend to integrity • Integrity data are essential to making valid conclusions about outcomes • Issues related to integrity are central to the success of RtI (Sanetti and Kratochwill, 2009)

14. Best-practice/legal guidelines: • Evidence of exposure to appropriate instruction in general education setting • Adherence to Problem Solving steps • “Interventions shall be implemented as designed…and with a level of intensity that matches student’s needs.” (FL Gen. Ed. Intervention Procedures—6A-6.0331)

15. Best practice/legal guidelines (con’t): District to provide proof that: • Research-based instruction/intervention provided at each tier • RtI process consists of multiple tiers with defined decision points • Intervention team considers continuous progress monitoring data • Students provided appropriate instruction in gen. ed. setting delivered by qualified personnel. (Zirkel, 2008)

16. Parent involvement • Discussion with parent re: student RtI, supporting data, adjustments to interventions, and future action • Data-based/graphic documentation of repeated assessments at reasonable intervals provided to parents • Documentation of parental involvement and communication must be maintained (FL Gen. Ed. Intervention Procedures 6A-6.0331)

17. Another definition of Integrity: “Treatment integrity is the extent to which essential intervention components are delivered in a comprehensive and consistent manner, by an interventionist trained to deliver the intervention.” (Sanetti & Kratochwill, 2009)

18. Dimensions of integrity: • Content—what steps were delivered • Quantity—how much was delivered • Quality—how well the steps were delivered • Process—how the intervention was delivered (Sanetti & Kratochwill, 2009)

19. Effective interventions… • Interventionist adequately delivers critical elements of the intervention • Student is present and engaged when intervention delivered • Student masters skills and is able to use them • Use of skills is associated with improved outcomes for student (Schulte, Easton & Parker, 2009)

20. Integrity elements to consider: Delivery vs. Receipt Simple vs. Complex interventions Adherence vs. Adaptation Adaptation vs. Drift

21. Intervention “adaptation” Use of expertise to intentionally modify an intervention to meet the needs of a particular student, or group of students. When might low integrity result in increased outcomes for students?

22. Interventionist “drift” Unplanned, gradual altering of the implementation of an intervention by the interventionist. What might contribute to this?

23. What degree of implementation is required? Consider: • Integrity may not impact performance • Poor implementation may cloak effective intervention • Intervention components may not be equally important • Level of necessary integrity varies • Goal should be effectiveness

24. Allocating integrity assessment resources: Why do you need the data? Which treatment integrity methods are best matched to target areas? What decisions will be made using the data? What resources are available to collect that data?

25. Types of integrity assessment Direct observation Behavior rating scales Self-report Permanent products

27. Self-report Example (Lane & Beebe-Frankenberger, 2004)

28. Intervention Documentation Worksheet

30. Resources: • http://www.aea11.k12.ia.us/idm integrity assessment checklists— available for download • Lane, K. and Beebe-Frankenberger, M. (2004). School-Based Interventions: The Tools You Need to Succeed. Boston, MA: Allyn & Bacon/Pearson Education, Inc.

31. Practical considerations: Student RtI data Frequency of assessment Impact of observer Use of multiple methods Feasibility

32. Performance feedback: The only strategy, to date, proven to promote high levels of treatment integrity in education settings. Components: • Graphed integrity data (percentage of steps correctly implemented, e.g.) • Verbal recommendations re: intervention steps that need improvement

33. Tier 3 Integrity Group Activity: What integrity measures are currently in place at your school to assess Tier 3 interventions? What changes need to occur so that integrity of Tier 3 intervention is being adequately assessed and documented? How is parent involvement for students receiving Tier 3 interventions implemented and documented?

34. PROGRESS MONITORING Tier 3

35. Planning for Data Use “Research has found that up-front planning helped make data collection and use more efficient in many case study schools by clarifying what data were needed, aiding with integration of multiple data sources, and ensuring that data collection processes were on track (Keeney, 1998; Lachat, 2001).” -Kerr, et al., (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons learned from three urban districts. American Journal of Education, 112, 496-520.

36. Table Talk Data use at Tier 3: • Do staff understand procedures and timelines for assessments at Tier 3? • Do staff have the skills to participate in Tier 3 data analysis? • How are you coordinating data collected at Tier 3 with data collected at Tiers 1 and 2?

37. “Evidence-based” at Tier 3 Less More Measurement Frequency Measurement Precision Evidence Base Measurement Focus Group Size Depth of Problem Analysis Instructional Time Less More

38. Purpose of Progress Monitoring Evaluate student RtI Encourage accountability and fidelity of intervention implementation Inform accuracy of teamproblem-solving Inform instruction/intervention across tiers

39. Tier 3 Progress Monitoring • Build on existing Tier 2 concepts and structures • Increased frequency of PM? New PM? • Coordinate multiple assessments across tiers • Mastery measurement or General Outcome Measurement?

40. Planning Data Collection • Resource Map/Assessment Map • School Calendar and Daily Schedule • Existing infrastructure for Tiers 1 and 2 • Scheduling Data school-wide (and keeping it sacred): • Examples: • Tier 1: 1 week after assessment windows close • Tier 2: 1st Wednesday of every month; • Tier 2: Every 20 instructional days • Tier 3: Bi-weekly on Wednesdays • Identifying personnel to manage data

41. Mapping Your Assessments

42. PM Resources • http://www.studentprogress.org/chart/chart.asp • http://www.rtinetwork.org/Essential/Assessment • http://www.studentprogress.org/library/Webinars.asp#RTI • http://www.rtinetwork.org/Essential/Assessment/Progress/ar/LinkingMonitoring/1 • http://www.aea11.k12.ia.us/spedresources/ModuleFour.pdf (see Chapter 7, “Intervention Progress Monitoring”)   

43. PM Resources • http://www.rtitools.com/Progress_Monitoring/Tools/ • http://www.jimwrightonline.com/php/rti/rti_wire.php • http://www.rti4success.org/chart/progressMonitoring/progressmonitoringtoolschart.htm • http://www.rti4success.org/index.php?option=com_content&task=view&id=1172&Itemid=150 (CBM manuals/tools)

44. PLANNED DECISION MAKING Evaluating a Student’s RtI

45. Increasing Instructional Precision What to teach this student? How the student best learns new information? (How will we teach him/her?) What intensive conditions result in student improvement? (What conditions do not?) What can be learned from this case to prevent similar problems in the future?

46. Ongoing Decision Making Evidence based intervention linked to verified hypothesis is planned Evidence based intervention is implemented Student Outcomes (SO) Assessed Treatment Integrity (TI) Assessed Continue Intervention +SO +TI Implement strategies to promote treatment integrity Data-based Decisions -SO -TI Modify/change Intervention -SO +TI From Lisa Hagermoser Sanetti, 2008 NASP Convention

47. Tier 3 Decision Making Ongoing Decision Making Modify/change Intervention What do your data suggest? Did you ID the right problem? Are there any other validated hypothesis to consider? Any new hypotheses? Does the current intervention plan need tweeking or a complete redo? Is the dosage appropriate? Does the intervention match student need?

48. Questionable Good Poor Response to Intervention Expected Trajectory Performance Observed Trajectory Time

49. DecisionsWhat to do if RtI is: Positive Continue intervention with current goal Continue intervention with goal increased Fade intervention to determine if student has acquired functional independence.

50. DecisionsWhat to do if RtI is: Questionable Was intervention implemented as intended? If no - employ strategies to increase implementation integrity If yes - Increase intensity of current intervention for a short period of time and assess impact. If rate improves, continue. If rate does not improve, return to problem solving.