response to intervention using data to enhance outcomes for all students
Download
Skip this Video
Download Presentation
Response to Intervention: Using Data to Enhance Outcomes for all Students

Loading in 2 Seconds...

play fullscreen
1 / 148

Response to Intervention: Using Data to Enhance Outcomes for all Students - PowerPoint PPT Presentation


  • 179 Views
  • Uploaded on

Response to Intervention: Using Data to Enhance Outcomes for all Students. Amanda VanDerHeyden Education Research and Consulting, Inc. Objectives Today. Overview of RTI, RTI decision making, and expected outcomes Specific How-To for RTI

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Response to Intervention: Using Data to Enhance Outcomes for all Students' - elina


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
response to intervention using data to enhance outcomes for all students

Response to Intervention: Using Data to Enhance Outcomes for all Students

Amanda VanDerHeyden

Education Research and Consulting, Inc.

objectives today
Objectives Today
  • Overview of RTI, RTI decision making, and expected outcomes
  • Specific How-To for RTI
    • Interpreting Assessment Data to determine need for Tiers 1, 2, and 3 Interventions
    • Selection and implementation of Tier 2 Interventions
    • Selection and implementation of Tier 3 Interventions
  • Implementing intervention for sustenance and system change
what is rti
What is RTI?
  • A science of decision making and way of thinking about how educational resources can be allocated (or reallocated) to best help all children learn
  • Major premium on child outcomes
rti is not
RTI is Not
  • A program, a curriculum, an intervention, a particular model
data allow us to
Data allow us to
  • Provide faster, more effective services for ALL children
  • Work “smarter” not harder, better utilize the talents of the school psychologist and school-based assessment and intervention teams.
  • Make implementation SIMPLE and EASY for teachers (low cost, few errors)
  • Prevent diagnosis
early screening identifies children at risk of reading difficulty

5.2

2.5

Early Screening Identifies Children At Risk of Reading Difficulty

J

5

4

Low Risk on Early Screening

Reading grade level

3

2

At Risk on Early Screening

1

1 2 3 4

Grade level corresponding to age

This Slide from Reading First Experts

From Reading First

early intervention changes reading outcomes

With substantial instructional intervention

4.9

With research-based core but without extra instructional intervention

3.2

Intervention

Control

Early Intervention Changes Reading Outcomes

J

5.2

5

4

Low Risk on Early Screening

Reading grade level

3

2.5

2

At Risk on Early Screening

1

1 2 3 4

Grade level corresponding to age

This Slide from Reading First Experts

From Reading First

evolution
Evolution
  • “Wait to Fail”
    • Let’s provide services early!
  • Costly sp ed programs not improving learning
    • Let’s shift resources to provide services in less restrictive setting!
  • Increasing numbers of children struggling in general ed
    • Let’s provide help in general education!
  • Traditional measures are de-contextualized and the constructs are problematic
    • Let’s help children who struggle academically by measuring performance in response to certain intervention strategies and then deliver what works!
slide10

Rationale for System Change

  • Level and Rate of Performance
  • Return to General Education
  • Lack of Certified Teachers
  • No demonstrated instructional techniques that differentially benefit SLD
  • Drop-out
  • Disproportionate Representation by Ethnicity

200-300% increase in SLD

history of rti
History of RTI
  • Effective Instruction Lit, CBA/M Lit
  • Lab-Quality Intervention Programs
  • Progress Monitoring Data and Problem-Solving Models
  • Reading First data
consistent with
Consistent with
  • NCLB
  • Reauthorized IDEA
  • Recommendations of panel reports on: Minority students in special education, National Reading Panel, Science and Math Initiative
impetus
Impetus
  • Faster, more effective services for ALL children
  • Work “smarter” not harder, better utilize the talents of the school psychologist and school-based assessment and intervention teams.
  • Make implementation SIMPLE and EASY for teachers (low cost, few errors)
why rti
Why RTI?
  • Viable alternative to traditional diagnosis of high-incidence disabilities, particularly Learning Disability (LD)
  • Reauthorized IDEA guidelines for identifying LD state that:

a) A severe discrepancy between achievement and intellectual ability shall not be required

b) A response to intervention (RTI) may be considered

why rti15
Why RTI?
  • RTI can address the problem of disproportionate identification of children with LD by race and gender
  • The utility of curriculum-based measures (CBM) for:
    • Identifying children not likely to benefit from the general education curriculum without assistance,
    • Predicting important long-term outcomes
    • Tracking individual student growth and informing instructional programming changes has been established
considerations
Considerations
  • There has been some consensus concerning the need for change; however, there has not been consensus on how this change can best be achieved.
  • Whereas RTI has considerable promise as a tool within special and general education, it is a vulnerable construct if misapplied.
improved treatment validity
Improved Treatment Validity
  • Direct link to treatment or consequential validity
  • Efforts are focused to:
    • Properly articulate a concern
    • Develop targeted intervention to resolve the concern
    • Collect information to determine whether or not the concern has been adequately addressed or whether different solution efforts need to be implemented
  • This approach changes the goal of assessment from what some have described as “admiration” of the problem to problem-solving.
contextualized decision making
Contextualized Decision Making
  • RTI emphasizes the pre-referral conditions (child and environment) and this context becomes part of the decision-making equation.
  • Allows practitioners to quantify:
    • The state of instructional affairs in the child’s regular education environment
    • Potential learning given optimal instructional conditions
  • RTI may enable improvement in general education programming leading to children receiving assistance in a more efficient manner.
improved identification accuracy for ld
Improved Identification Accuracy for LD
  • Under RTI models intervention becomes a specified, operationalized variable, thus false positive identification errors should be reduced dramatically.
  • Removing the current reliance on teacher identification and requiring direct measures of child performance in context will enhance identification accuracy.
more effective intervention
More Effective Intervention
  • RTI is likely to facilitate less restrictive interventions and placements for children.
  • RTI allows school psychologists to bring their expertise to bear on assessment strategies at the classroom level and assist teachers to use data formatively to enhance their instructional programming.
decision making criteria
Decision-making Criteria
  • Must be operationalized and validated through research
  • The purpose of RTI will be critical to determining how implementation should proceed
new challenges for teams
New Challenges for Teams
  • Effective intervention delivery will depend on relevant intervention variables
  • To be effective the intervention must have been:
    • Properly identified
    • Implemented with integrity and with sufficient frequency, intensity, and duration
steep model

STEEP Model

Screening to Enhance Educational Progress

tier 1 math screening
Tier 1: Math Screening
  • Math Probe:
    • Group administered.
    • Materials: Worksheet consisting of a series of problems sampling the target skill(s) (e.g., sums to 5, double digit multiplication with regrouping).
    • Timing: 2 minutes
    • Information obtained: digits correct in two minutes.
math probe example
Math Probe Example
  • Total Digits: 38
  • Errors: 5
  • Digits Correct: 33
tier 1 writing screening
Tier 1: Writing Screening
  • Writing Probe:
    • Group administered.
    • Materials: story starter (e.g., If I had a million dollars…) printed at the top of a blank page.
    • Timing: 1 minute to think, 3 minutes to write.
    • Scoring: words written or correct word sequences in three minutes.
tier 1 reading screening
Tier 1: Reading Screening
  • Reading Probe:
    • Individually administered
    • Materials: A content-controlled reading passage.
    • Procedure: The student reads aloud as the teacher listens and records errors.
    • Timing: 1 minute
    • Information obtained: words read correctly in one minute.
cbm reading sample scoring
CBM Reading: Sample Scoring
  • TRW=63
  • Errors=6
  • CRW=58
tier 2 can t do won t do assessment
Tier 2: Can’t Do/Won’t Do Assessment
  • “Can’t Do/Won’t Do”
  • Individually-administered
  • Materials
    • Academic material that student performed poorly during class assessment.
    • Treasure chest: plastic box filled with tangible items.

3-7 minutes per child

response to intervention
Response to Intervention

Before Intervention

During Intervention

#Correct

Avg. for his Class

Each Dot is one Day of Intervention

Intervention Sessions

Intervention in Reading

slide40

Response to Intervention

Before Intervention

During Intervention

#Correct

Avg. for his Class

vehicle for system change system wide math problem
Vehicle for System Change:System-wide Math Problem

Instructional range

Frustrational range

Each bar is a student’s performance

class wide intervention
Class-wide Intervention

Teacher F Mult 0-12

120

100

80

Digits Correct Two Minutes

60

40

20

0

11/7/2003

10/24/2003

10/31/2003

11/14/2003

11/18/2003

Weeks

growth obtained
Growth Obtained

actual growth

aimline

effect on high stakes scores
Effect on High-Stakes Scores

VanDerHeyden, in prep

effect on high stakes scores51
Effect on High-Stakes Scores

VanDerHeyden, in prep

district wide implementation data
District-wide Implementation Data
  • Vail Unified School District
    • www.vail.k12.az.us
  • Three years, system-wide implementation of STEEP grades 1-8
system outcomes
System Outcomes
  • Referrals reduced greater than half
  • % who qualify from 50% stable baseline over three years to nearly 100%
  • SLD down from 6% of children in district in 2001-2002 (with baseline upward trend) to 3.5% in 2003-2004 school year
  • Corresponding gains on high-stakes tests (VanDerHeyden & Burns, 2005)
  • Intervention successful for about 95 to 98% of children screened

VanDerHeyden, Witt, & Gilbertson, 2007

cost reduction
Cost Reduction

VanDerHeyden, Witt, & Gilbertson, 2007

slide55

Findings

  • Number of Evaluations dramatically reduced– 70% at highest referral school
  • Diverse settings, psychologists of diverse backgrounds and no prior experience with CBM or functional academic assessment
  • Percentage qualify increased at 4 of 5 schools
  • Disproportionate representation of males positively affected
  • Number of children placed dramatically reduced

VanDerHeyden, Witt, & Gilbertson, 2007

slide56

Team Decision-Making Agreement

VanDerHeyden, Witt, & Gilbertson, 2007

team decision making
Team Decision-Making

VanDerHeyden, Witt, & Gilbertson, 2007

fall to spring reading growth
Fall to Spring Reading Growth

VanDerHeyden & Witt, 2005

what proportion of ethnicity represented before and after intervention in risk category
What Proportion of Ethnicity Represented Before and After Intervention in Risk Category?

VanDerHeyden & Witt, 2005

slide60

Criterion +

Criterion -

PVS +

PVS -

.76/.11=6.9: children with learning problem about 7 times more likely to have a failed RTI than children without a learning problem

.24/.89=.27: children without a learning problem are 3.7 times more likely to have a successful RTI than those children with a learning problem

Sens- .76

Spec- .89

PPP- .59

NPP- .95

+ LR- sens/(1-spec)

- LR- (1-sens)/spec

slide61

Criterion +

Criterion -

Probe +

Probe -

.94/.46=2.0: children with learning problem about 2 times more likely to perform below screening benchmark than children without a learning problem

.06/.54=.11: children without a learning problem are 9 times more likely to have a above criterion screening score than those children with a learning problem

Sens- .94

Spec- .54

PPP- .29

NPP- .98

identification accuracy
Identification Accuracy
  • High-achieving classrooms (<20%)
    • Procedures paired with RTI criterion were more accurate than other commonly used screening devices
  • Low-achieving classrooms (>50%)
    • Procedures paired with RTI criterion were more accurate than other commonly used screening devices

VanDerHeyden & Witt, 2005

questions
Questions
  • Is there a classwide problem?
  • Is there a gradewide problem?
  • What’s the most efficient way to deliver intervention?
screening tells you
Screening tells you
  • How is the core instruction working?
  • What problems might exist that could be addressed?
  • Most bang-for-the-buck activity
  • Next most high-yield activity is classwide intervention at Tier 2.
screening guidelines
Screening Guidelines
  • Efforts at Tier 1 pay off with fewer children needing individual intervention
  • 3 times per year, single probe
  • Use small team of trained coaches
  • Prepare all needed materials in a packet for each teacher
  • Score and return within 1 week on graph
  • Use data to generate aimlines, can be used to set benchmarks
slide66

Here is 3rd grade at Cottonwood. Each circle corresponds to a students score on a reading CBM probe in March and the AIMS reading score the same month. So the circle near the blue lines is a child who read 158 wc/min and scored 486 on the AIMS Reading. The diagonal line represents the “best fit” line or the line closest to all the circles. This shows there is a strong positive correlation between CBM and AIMS reading scores with this group.

how to set a benchmark
How to Set a Benchmark

This is words read correctly per minute. You move this line up and down to “catch” as many of those who will not pass as possible.

431 = “pass” for AIMS. This line does not move

setting 95 wc min as the pass standard for cbm
Setting 95 wc/min as the “pass” standard for CBM

These are children who were predicted to pass AIMS based on CBM and did pass. “Hits”

These are the children predicted to pass AIMS who actually failed. “False Negative Errors.” The worst kind of error.

These are the children predicted to fail AIMS who actually passed. “False Positive Errors.”

These are the children who were predicted to fail AIMS based on CBM who did fail. “Hits”

slide69
Moving the horizontal line up will “catch” two more cases who failed the AIMS, but will result in many more “false positive errors.”

Thus, 95 wc/min at 3rd grade at Cottonwood is a good standard that will tell you which children are likely to fail the AIMS reading section.

mastery model measurement cba
Mastery Model Measurement (CBA)

Letter naming fluency

Isolated sound fluency

Beginning sound fluency

Ending sound fluency

general outcome measurement cbm
General Outcome Measurement (CBM)

Beginning sound fluency

Letter naming fluency

Isolated sound fluency

Ending sound fluency

Words read correctly per minute

tracking year long growth
Tracking Year-Long Growth

mastery

aimline

instructional

slide74

Pass the AIMS

Digits Correct Two Minutes

1

12

Weeks

slide75

Academic Systems

Behavioral Systems

  • Intensive, Individual Interventions
  • Individual Students
  • Assessment-based
  • High Intensity
  • Of longer duration
  • Intensive, Individual Interventions
  • Individual Students
  • Assessment-based
  • Intense, durable procedures
  • Targeted Group Interventions
  • Some students (at-risk)
  • High efficiency
  • Rapid response
  • Targeted Group Interventions
  • Some students (at-risk)
  • High efficiency
  • Rapid response
  • Universal Interventions
  • All students
  • Preventive, proactive
  • Universal Interventions
  • All settings, all students
  • Preventive, proactive

Any

Curriculum

Area

1-5%

1-5%

5-10%

5-10%

Students

80-90%

80-90%

Dave Tilly, 2005

slide77

Academic Systems

Behavioral Systems

  • Intensive, Individual Interventions
  • Individual Students
  • Assessment-based
  • High Intensity
  • Of longer duration
  • Intensive, Individual Interventions
  • Individual Students
  • Assessment-based
  • Intense, durable procedures
  • Targeted Group Interventions
  • Some students (at-risk)
  • High efficiency
  • Rapid response
  • Targeted Group Interventions
  • Some students (at-risk)
  • High efficiency
  • Rapid response
  • Universal Interventions
  • All students
  • Preventive, proactive
  • Universal Interventions
  • All settings, all students
  • Preventive, proactive

Any

Curriculum

Area

1-5%

1-5%

5-10%

5-10%

Students

80-90%

80-90%

Dave Tilly, 2005

class wide intervention78
Class-wide Intervention
  • Use pair-peered practice (classwide peer tutoring, PALS)
  • Model, Guided Practice, Independent timed practice with delayed error correction, reward contingency
slide79

Instructional Hierarchy

Finally, problem-solving/ application practice should occur here with a mastery level skill– Core Instruction- Not Manipulated but could be

But fluency building should happen here with an instructional level skill– Intervention Focus was here

To gain the steepest growth, introduction of new skills should happen here– Core Instruction- Not manipulated

Generalization

Fluency

Acquisition

intervention plan 15 min per day
Intervention Plan- 15 Min per Day
  • Protocol-based classwide peer tutoring, randomized integrity checks by direct observation
  • Model, Guide Practice, Independent Timed Practice with delayed error correction
  • Group performance contingency
  • Teachers encouraged to
    • Scan papers for high error rates
    • Do 5-min re-teach for those with high-error rates
    • Provide applied practice using mastery-level computational skill
intervention plan
Intervention Plan
  • Class Median reaches mastery range for skill, next skill is introduced
  • Following promising results at one site in 2002-2003, lead to implementation district-wide grades 1-8 for all children by 2004-2005.
with teacher support
With teacher support
  • Consider time, resources, materials
  • Remove skill barriers with
    • classroom training for students
    • classroom coaching for teachers
  • Remove implementation barriers after use new steps
    • follow-up supportive meetings to problem solve.
    • frequent acknowledgment of a teacher’s efforts
slide91

Training Package

Tell

Rational

Step by step protocol

Show

Model

Do

Train students

Implement with guided practice

Implement independently with support

slide92

88% of interventions are not used

without support

Decisions do not always correspond to data (someone must check)

tier 2 class wide intervention104
Tier 2: Class-wide Intervention

Teacher F Mult 0-12

120

100

80

Digits Correct Two Minutes

60

40

20

0

11/7/2003

10/24/2003

10/31/2003

11/14/2003

11/18/2003

Weeks

pre post changes to performance detected by cbm
Pre-post changes to performance detected by CBM

Instructional range

Frustrational range

Each bar is a student’s performance

slide112
Computation Gains Generalized to High Stakes TestImprovements (Gains within Multiple Baselineshown as pre-post data)
identification accuracy114
Identification Accuracy

VanDerHeyden, et al., 2003

percent identified at each tier
Percent Identified at each Tier

VanDerHeyden, et al., 2003

slide116

Academic Systems

Behavioral Systems

  • Intensive, Individual Interventions
  • Individual Students
  • Assessment-based
  • High Intensity
  • Of longer duration
  • Intensive, Individual Interventions
  • Individual Students
  • Assessment-based
  • Intense, durable procedures
  • Targeted Group Interventions
  • Some students (at-risk)
  • High efficiency
  • Rapid response
  • Targeted Group Interventions
  • Some students (at-risk)
  • High efficiency
  • Rapid response
  • Universal Interventions
  • All students
  • Preventive, proactive
  • Universal Interventions
  • All settings, all students
  • Preventive, proactive

Any

Curriculum

Area

1-5%

1-5%

5-10%

5-10%

Students

80-90%

80-90%

Dave Tilly, 2005

tier 3
Tier 3
  • Assessment Data
    • Instructional level performance
    • Error analysis (high errors, low errors, pattern)
    • Effect of incentives, practice, easier task
    • Verify intervention effect
  • Same implementation support as Tier 2
  • Instructional-level materials; Criterion-level materials
tier 3120
Tier 3
  • Implement for 5-15 consecutive sessions with 100% integrity
  • Link to referral decision
  • Weekly graphs to teacher and weekly generalization probes outside of classroom, supply new materials
  • Troubleshoot implementation weekly
lessons learned
Lessons Learned
  • Most individual interventions for reading
  • Standard protocols with slight modifications are best
  • Most interventions are successful (should be successful)
  • Generalization must be attended to
  • Team will not follow data without support and training to do so
  • Coordinate intervention start times with principal and stagger start dates (10-15 at a time plus Tier 2’s).
  • Organize master schedule for data collection and Tier times
tier 3 intervention
Tier 3 Intervention
  • >5% of children screened (total population) IF solid Tier 1
  • Possibly as low as 2% IF solid Tier 1 and Tier 2
  • About 1-2% failed RTI; 10% of most at-risk
our recipe for intervention success
Our Recipe for Intervention Success

PREPARE

Identify and Use standard protocols for intervention

Develop all needed materials

Develop packets or put on a central web site

Determine graphing program

our recipe for intervention success133
Our Recipe for Intervention Success

TRAIN

Explain

Watch the teacher do it with the actual child before you leave

Call or meet teacher after first day to problem solve

our recipe for intervention success134
COLLECT DATA AND SUPPORT

Each week, graph intervention performance and do a generalization check with the child.

Graphed feedback to teachers with generalization checks for individual intervention once per week

Response-dependent performance feedback to sustain implementation accuracy

Monthly CBM to track growth and enhance existing Tier 1 Programs or advise new Tier 1

Data to principal weekly. Summarize effects and integrity of procedures.

Our Recipe for Intervention Success
our recipe for intervention success136
Our Recipe for Intervention Success

USE DATA TO MAKE DECISIONS

RTI successful if child performs criterion-level probe (from screening) in the instructional range. RTI unsuccessful if 15 consecutive intervention sessions and criterion probe is not in the instructional range.

Increase task difficulty for intervention if child scores at mastery on task during intervention sessions

infrastructure for implementation
Infrastructure for Implementation
  • Grade-level planning periods can be utilized
  • Special education “team” at school can be utilized
  • School Psych must be on-site 1 day/week
  • Developing master schedule for Tier 1, 2, and 3 intervention times is useful
  • Integrate efforts with evaluation referral team efforts (consider major reduction in meeting time and shift to intervention efforts!)
materials needed
Materials needed
  • Computer and software to organize data
  • Student data imported. Clerical person to enter data on-site for tier 1 screen only.
  • Color printer to print graphs + extra color cartridges
  • Probe materials, digital count-down timers
  • Intervention protocols, intervention materials (e.g., flashcard sets, reading materials)
  • Access to copier and some assistance with copying
  • Reinforcers for treasure chest (no more than $500 per school)
guidelines for implementers
Guidelines for Implementers

Use single trial scores for screening

Following screening, grade-wide graphs to principal

Return data to teachers within 48 hours with personal interpretation at grade-level team meeting

Include principal in critical meetings

Involve teachers at all stages

guidelines for implementers140
Guidelines for Implementers

Learn about curriculum and instruction.

Integrate RTI with ongoing school and system reform efforts

Thoughtfully merge to subtract duplicate activities and to enhance more comprehensive supplemental and core instructional support activities that may be in place

Use RTI data to evaluate the value of ALL instructional programs or resource allocation decisions. Quantify bang for the buck using student performance data.

lessons learned from vail usd
Lessons Learned from Vail USD

Infrastructure for education being a results-based enterprise

Accountability

Principal is the Instructional Leader of a school

Principal as change agent

School psychologist as change agent

Replace resources/substitute don’t add

Minimize meetings

Track outcomes that matter

slide142

Upset parent

Tantruming child

Bus 11 is late

Police on site for child abuse report

Principal

Visit from health department, bathroom out of paper towels again

Teacher out sick for rest of year

Meeting at district office/items due

slide143

Principal

FILTER-- How much time allocated to instruction? Children actively engaged? Standards introduced? Effective instruction occurring?

Upset parent

DATA on Learning

Check on health dept

Goal Setting

Check on police interview

Teacher Evaluation

Etc.

Allocation of Instructional Resources

great instructional leaders
Great Instructional Leaders

Have a filter

Allocate time and resources according to their filter

Use an AIMLINE

Have a framework for making data-driven decisions (know how to access the data they need to reach timely decisions)

Hold teachers, staff, students accountable

Research findings on effective schooling

great school psychologists
Great School Psychologists

Hand the principal the data the principal did not know to ask for but can’t live without

Follow the aimline and attend to implementation integrity

Understand the variables of effective instruction and engage in contextualized assessment that is technically valid for the purposes needed AND has treatment utility

Minimizes meeting time and “avoids the science of strange behavior…”

great districts
Great Districts

Minimize time away from school, but use time together to review school improvement implementation efforts and ongoing results

Have the will to proactively chart the course of a district

Provide adequate resources and space for principals to be effective instructional leaders and hold them accountable for results

Respect the role of parents and actively engage them

Have a framework for evaluating results (know how to access data for decision making)

Evaluate quality of all programs locally and make decisions about continued use based on DATA.

great teachers
Great Teachers
  • Use data to identify where more/different/less instruction is needed
  • Have as a goal to accelerate all learning of all children
  • Proactively address barriers to learning
  • Take responsibility for learning that occurs in the classroom
  • Are confident and ready to collaborate in the classroom
  • Appreciate childhood and children (a little humor, lots of patience, enthusiasm)
for more information
For More Information

amandavande@gmail.com

www.isteep.com

Thank you to the US Dept of Education for providing all film clips shown in this presentation

ad