slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Advance Organizer PowerPoint Presentation
Download Presentation
Advance Organizer

Loading in 2 Seconds...

play fullscreen
1 / 63

Advance Organizer - PowerPoint PPT Presentation

  • Uploaded on

TOT Y3D3 Tier 3: Instruction/Intervention Design & Integrity Progress Monitoring School Implementation Blueprints. A collaborative project between the Florida Department of Education and the University of South Florida. Advance Organizer. Y3D2 Content Review Skill Assessment Review

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Advance Organizer' - dard

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

TOT Y3D3 Tier 3: Instruction/Intervention Design & IntegrityProgress MonitoringSchool Implementation Blueprints

A collaborative project between the Florida Department of Education and the University of South Florida

advance organizer
Advance Organizer
  • Y3D2 Content Review
  • Skill Assessment Review
  • T3 Intervention Design
    • Intervention Integrity
  • T3 Progress Monitoring
  • Review of Randy

- Comprehensive Intervention Plan Columns 4-5

    • Step 4 Student Success Worksheet
  • Surveys - Personnel Satisfaction, Perceptions of Practices
  • School Blueprint
    • Implementation
review annual catch up growth
Review:Annual & Catch-up Growth

Each year of instruction in core subject yields one year annual growth

Students who are behind must achieve annual growth PLUS

Primary driver of catch-up growth is increased instructional time

review kenewick s formula
Review: Kenewick’s formula

13 percentile pts. = one year’s growth

  • Gr. 4 State standard: 50th percentile
  • Target 2nd grader: 12th percentile
  • The difference: 38 points
  • Point difference

divided by 13: 2.9 yrs. Behind

Example taken from Annual Growth…Catch-up Growth, Fielding et al., 2007

review kenewick s formula cont
Review: Kenewick’s formula (cont.)

Calculating instructional minutes:

Daily minutes for annual 3rd gr. growth: 80

Daily minutes for annual 4th gr. growth: 80

Additional daily minutes to make

three years of catch-up growth: 240

Total 3rd/4th grade daily minutes: 400

review kenewick s formula cont1
Review: Kenewick’s formula (cont.)

Dividing the total minutes (400) between 3rd and 4th grade shows that the student needs 200 minutes of direct reading instruction in both 3rd and 4th grades to reach the 50th percentile by the end of fourth grade.

review tier 3 intervention design
Review: Tier 3 Intervention Design

Purpose is instructional match

Adjusts what is taught and/or how it is taught

Evidence-base for Tier 3 instruction relies on careful progress monitoring

skill assessment review guiding questions
Skill Assessment ReviewGuiding Questions
  • 1. Which components of developing a comprehensive intervention plan did your team complete successfully? For which components did the team have difficulty? What factors helped or hindered the team when developing the comprehensive intervention plan to address the student’s needs?
  • 2. How helpful was it to your team to specifically outline the intervention and support plans (i.e., who was responsible, what would be done, when it would occur, and where it would occur)?
  • 3. If your team was able to return to your school and implement the plan developed at the last training, to what extent did the level of specificity help ensure that the plan was implemented?
  • 4. If your team was able to return to your school and implement the plan developed at the last training, to what extent did the support plan you developed for the individuals responsible for implementing the intervention help ensure that the intervention was implemented as intended?
intervention integrity

Tier 3 Considerations

importance of integrity
Importance of integrity:
  • Rapid and widespread deployment of RtI has made urgent the need to attend to integrity
  • Integrity data are essential to making valid conclusions about outcomes
  • Issues related to integrity are central to the success of RtI

(Sanetti and Kratochwill, 2009)

best practice legal guidelines
Best-practice/legal guidelines:
  • Evidence of exposure to appropriate instruction in general education setting
  • Adherence to Problem Solving steps
  • “Interventions shall be implemented as designed…and with a level of intensity that matches student’s needs.”

(FL Gen. Ed. Intervention Procedures—6A-6.0331)

best practice legal guidelines con t
Best practice/legal guidelines (con’t):

District to provide proof that:

  • Research-based instruction/intervention provided at each tier
  • RtI process consists of multiple tiers with defined decision points
  • Intervention team considers continuous progress monitoring data
  • Students provided appropriate instruction in gen. ed. setting delivered by qualified personnel.

(Zirkel, 2008)

parent involvement
Parent involvement
  • Discussion with parent re: student RtI, supporting data, adjustments to interventions, and future action
  • Data-based/graphic documentation of repeated assessments at reasonable intervals provided to parents
  • Documentation of parental involvement and communication must be maintained

(FL Gen. Ed. Intervention Procedures 6A-6.0331)

another definition of integrity
Another definition of Integrity:

“Treatment integrity is the extent to which essential intervention components are delivered in a comprehensive and consistent manner, by an interventionist trained to deliver the intervention.”

(Sanetti & Kratochwill, 2009)

dimensions of integrity
Dimensions of integrity:
  • Content—what steps were delivered
  • Quantity—how much was delivered
  • Quality—how well the steps were delivered
  • Process—how the intervention was delivered

(Sanetti & Kratochwill, 2009)

effective interventions
Effective interventions…
  • Interventionist adequately delivers critical elements of the intervention
  • Student is present and engaged when intervention delivered
  • Student masters skills and is able to use them
  • Use of skills is associated with improved outcomes for student

(Schulte, Easton & Parker, 2009)

integrity elements to consider
Integrity elements to consider:

Delivery vs. Receipt

Simple vs. Complex interventions

Adherence vs. Adaptation

Adaptation vs. Drift

intervention adaptation
Intervention “adaptation”

Use of expertise to intentionally modify an intervention to meet the needs of a particular student, or group of students.

When might low integrity result in increased outcomes for students?

interventionist drift
Interventionist “drift”

Unplanned, gradual altering of the implementation of an intervention by the interventionist.

What might contribute to this?

what degree of implementation is required
What degree of implementation is required?


  • Integrity may not impact performance
  • Poor implementation may cloak effective intervention
  • Intervention components may not be equally important
  • Level of necessary integrity varies
  • Goal should be effectiveness
allocating integrity assessment resources
Allocating integrity assessment resources:

Why do you need the data?

Which treatment integrity methods are best matched to target areas?

What decisions will be made using the data?

What resources are available to collect that data?

types of integrity assessment
Types of integrity assessment

Direct observation

Behavior rating scales


Permanent products


Self-report Example

(Lane & Beebe-Frankenberger, 2004)


integrity assessment checklists— available for download

  • Lane, K. and Beebe-Frankenberger, M. (2004). School-Based Interventions: The Tools You Need to Succeed. Boston, MA: Allyn & Bacon/Pearson Education, Inc.
practical considerations
Practical considerations:

Student RtI data

Frequency of assessment

Impact of observer

Use of multiple methods


performance feedback
Performance feedback:

The only strategy, to date, proven to promote high levels of treatment integrity in education settings.


  • Graphed integrity data (percentage of steps correctly implemented, e.g.)
  • Verbal recommendations re: intervention steps that need improvement
tier 3 integrity group activity
Tier 3 Integrity Group Activity:

What integrity measures are currently in place at your school to assess Tier 3 interventions?

What changes need to occur so that integrity of Tier 3 intervention is being adequately assessed and documented?

How is parent involvement for students receiving Tier 3 interventions implemented and documented?

planning for data use
Planning for Data Use

“Research has found that up-front planning helped make data collection and use more efficient in many case study schools by clarifying what data were needed, aiding with integration of multiple data sources, and ensuring that data collection processes were on track (Keeney, 1998; Lachat, 2001).”

-Kerr, et al., (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons learned from three urban districts. American Journal of Education, 112, 496-520.

table talk
Table Talk

Data use at Tier 3:

  • Do staff understand procedures and timelines for assessments at Tier 3?
  • Do staff have the skills to participate in Tier 3 data analysis?
  • How are you coordinating data collected at Tier 3 with data collected at Tiers 1 and 2?
evidence based at tier 3
“Evidence-based” at Tier 3



Measurement Frequency

Measurement Precision

Evidence Base

Measurement Focus

Group Size

Depth of Problem Analysis

Instructional Time



purpose of progress monitoring
Purpose of Progress Monitoring

Evaluate student RtI

Encourage accountability and fidelity of intervention implementation

Inform accuracy of teamproblem-solving

Inform instruction/intervention across tiers

tier 3 progress monitoring
Tier 3 Progress Monitoring
  • Build on existing Tier 2 concepts and structures
  • Increased frequency of PM? New PM?
  • Coordinate multiple assessments across tiers
  • Mastery measurement or General Outcome Measurement?
planning data collection
Planning Data Collection
  • Resource Map/Assessment Map
  • School Calendar and Daily Schedule
  • Existing infrastructure for Tiers 1 and 2
  • Scheduling Data school-wide (and keeping it sacred):
    • Examples:
      • Tier 1: 1 week after assessment windows close
      • Tier 2: 1st Wednesday of every month;
      • Tier 2: Every 20 instructional days
      • Tier 3: Bi-weekly on Wednesdays
  • Identifying personnel to manage data
pm resources
PM Resources
  • (see Chapter 7, “Intervention Progress Monitoring”)   
pm resources1
PM Resources
  • (CBM manuals/tools)
planned decision making

Evaluating a Student’s RtI

increasing instructional precision
Increasing Instructional Precision

What to teach this student?

How the student best learns new information? (How will we teach him/her?)

What intensive conditions result in student improvement? (What conditions do not?)

What can be learned from this case to prevent similar problems in the future?

ongoing decision making
Ongoing Decision Making

Evidence based intervention linked to verified hypothesis is planned

Evidence based intervention is implemented

Student Outcomes (SO) Assessed

Treatment Integrity (TI) Assessed

Continue Intervention


Implement strategies to promote treatment integrity

Data-based Decisions


Modify/change Intervention


From Lisa Hagermoser Sanetti, 2008 NASP Convention


Tier 3 Decision Making

Ongoing Decision Making

Modify/change Intervention

What do your data suggest?

Did you ID the right problem?

Are there any

other validated

hypothesis to consider?

Any new hypotheses?

Does the current intervention plan need tweeking or a complete redo?

Is the dosage appropriate?

Does the intervention match student need?





Response to Intervention

Expected Trajectory


Observed Trajectory


decisions what to do if rti is
DecisionsWhat to do if RtI is:


Continue intervention with current goal

Continue intervention with goal increased

Fade intervention to determine if student has acquired functional independence.

decisions what to do if rti is1
DecisionsWhat to do if RtI is:


Was intervention implemented as intended?

If no - employ strategies to increase implementation integrity

If yes -

Increase intensity of current intervention for a short period of time and assess impact. If rate improves, continue. If rate does not improve, return to problem solving.

decisions what to do if rti is2
DecisionsWhat to do if RtI is:


Was intervention implemented as intended?

If no - employ strategies in increase implementation integrity

If yes -

Is intervention aligned with the verified hypothesis or are there other aligned interventions to consider? (Intervention Design)

Are there other hypotheses to consider? (Problem Analysis)

Was the problem identified correctly? (Problem Identification)

let s look at some examples
Let’s Look at Some Examples

Reading Fluency with DIBELS (CBM)



math cbm tier 3 data
Math CBM – Tier 3 Data

Courtesy of Monroe County School District

  • Case Study Review
  • Randy -
    • Comprehensive Intervention Plan: Documentation & Monitoring
    • Step 4 Student Success Worksheet
  • Your Student -
    • Comprehensive Intervention Plan: Documentation & Monitoring
    • Step 4 Student Success Worksheet
perceptions of practices survey
Perceptions of Practices Survey

Your project ID is:

  • Last 4 digits of SS#
  • Last 2 digits of year of birth
personnel satisfaction survey
Personnel Satisfaction Survey

Your project ID is:

  • Last 4 digits of SS#
  • Last 2 digits of year of birth