Facilitating PS/ RtI Capacity: Tools , Skills, and Strategies for Practitioners - PowerPoint PPT Presentation

Facilitating ps rti capacity tools skills and strategies for practitioners
1 / 82

  • Uploaded on
  • Presentation posted in: General

Facilitating PS/ RtI Capacity: Tools , Skills, and Strategies for Practitioners. NASP 2013 Mini-Skills Presentation February 14 th , 2013 Amanda March Amber Brundage Clark Dorman Jose Castillo Kevin Stockslager University of South Florida. Purpose.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Facilitating PS/ RtI Capacity: Tools , Skills, and Strategies for Practitioners

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Facilitating ps rti capacity tools skills and strategies for practitioners

Facilitating PS/RtI Capacity: Tools, Skills, and Strategies for Practitioners

NASP 2013 Mini-Skills Presentation

February 14th, 2013

Amanda March

Amber Brundage

Clark Dorman

Jose Castillo

Kevin Stockslager

University of South Florida



  • To enhance practitioners’ understanding of empirically supported systems-change procedures, detailing systems-theory and principles identified as critical for success of PS/RtI initiatives.

Learning objectives

Learning Objectives

  • Discussion of critical components of PS-RtIimplementation &scale-up

  • Review of a comprehensive three-phase systems-change model

    • Consensus

    • Infrastructure

    • Implementation

  • Presentation, discussion, &practice of various tools to evaluate &progress monitor PS/RtI practices

  • Participants will leave with knowledge, skills, &tools to facilitate PS/RtI implementation &evaluation in their local settings

Advanced organizer

Advanced Organizer

  • PS/RtI& MTSS

  • MTSS and Program Evaluation in the Schools

  • Example Tools & Practice Activities

    • Beliefs on RtI Scale

    • Perceptions of RtI Skills Survey – Revised

    • Tier I and II Critical Components Checklist

  • Discussion

  • Resources & References



  • Who are we?

  • Who are you?

    • Clicker Activities

Ps rti mtss


Ps rti model of service delivery

PS/RtI Model of Service Delivery

  • RtI is the practice of providing high quality instruction matched to student needs and utilizing data to make educational decisions about students (Batsche et al., 2005)

  • Components of a PS/RtI Model

    • Integrated multi-tier model of service delivery

    • Problem-solving method

    • Integrated data collection and assessment system

Mtss defined

MTSS Defined

  • Evidence-basedmodel of educating students that uses data-based problem-solving to integrate academic and behavioral instruction and interventions;

  • Integrated instruction/interventions are delivered to students in varying intensities (multiple tiers) based on student need;

  • Need-driven decision making seeks to ensure the allocation of resources (district, school, classroom) are based on student need at the appropriate levels to accelerate performance of allstudents to achieve and/or exceed proficiency.



Florida’s MTSS




Changing systems

Changing Systems

Multi tiered system of support model in education

Multi-Tiered System of Support Model in Education

Systems Approach:

  • System - “The orderly combination of two or more individuals whose interaction is intended to produce a desired outcome.”(Curtis, Castillo, & Cohen, 2008)

  • Principles of Systems Change:

    • Shared Mission, Beliefs, and Values

    • Key Stakeholder Involvement

    • Effective, Committed Leadership

    • Systems Perspective – “Big” Picture

    • Structured Planning and Problem-Solving

      • Data-based decision making

      • Knowledge and skills to build capacity through professional development

Three phase change model

Three-Phase Change Model

Florida s change model

Florida’s Change Model




Three phase change model1

Three Phase Change Model


Belief is shared

Vision is agreed upon

Implementation requirements understood

Infrastructure Development


Training/Technical Assistance, Professional Development

Model (e.g., Standard Protocol)

Tier I and II intervention systems

E.g., K-3 Academic & Behavioral Support Plan

Data Management

Technology support

Decision-making criteria established


Mtss and program e valuation in the schools

MTSS and Program Evaluation in the Schools

Ps rti evaluation tool technical assistance manual revised

PS/RtI Evaluation Tool Technical Assistance Manual - Revised


Florida ps rti evaluation tools

Florida PS/RtI Evaluation Tools

  • Tools for Progress Monitoring PS/RtI

    • Self-Assessment of Problem-Solving Implementation (SAPSI)

  • Tools for Examining Consensus Development

    • Beliefs on RtI Scale

    • Perceptions of Practice Survey

  • Tools for Examining Infrastructure Development

    • Perceptions of RtI Skills Survey –Revised

    • Coaching Evaluation Survey – Revised

  • Tools for Examining Integrity of PS/RtI

    • Tier I and II Observation Checklist

    • Tier I and II Critical Components Checklist

    • Problem-Solving Team Meeting Checklists – Initial & Follow-Up

    • Tier III Critical Components Checklist

Today s featured tools

Today’s Featured Tools

  • Tools for Examining Consensus Development

    • Beliefs on RtI Scale

  • Tools for Examining Infrastructure Development

    • Perceptions of RtI Skills Survey –Revised

  • Tools for Examining Integrity of PS/RtI

    • Tier I and II Critical Components Checklist

Mtss evaluation issues

MTSS Evaluation Issues

  • The data you collect should be driven by the evaluation questionsyou want to answer

    • Are we implementing MTSS with fidelity?

    • Do we have the capacity to implement successfully?

    • Do staff buy into implementing MTSS?

Answering evaluation q uestions

Answering Evaluation Questions

  • Use data to inform evaluation questions

  • Use data to answer broad/specific questions

  • Align analysis and data display with evaluation questions

  • Consider available technology resources to facilitate analyses of data—online administration, automatic analysis, knowledge and skill of personnel

Beliefs on rti scale

Assessing Beliefs Integral to PS/RtI Practices

Beliefs on RtI Scale

Description and purpose

Beliefs of RtI Scale

Description and purpose

The beliefs of educators

The Beliefs of Educators

Research suggests:

  • Educators beliefs about student learning styles, styles of teaching, and instructional strategies impact their willingness to implement new practices (Fang, 1996; Sparks, 2002)

  • Educational reform occurs when educators understand the need for change and embrace this need as a moral imperative (Fullan, 2010; Sharratt & Fullan, 2009)

  • Beliefs of leaders communicated to stakeholders influence the climate for successful implementation of new practices (Sharratt & Fullan, 2009)

Description of survey

Description of Survey

  • Assesses educators beliefs regarding:

    • Student learning

    • The role of data in decision-making

    • Expectations for the effectiveness of instruction

  • 19 items; 5-point Likert scale

  • 1= Strongly Disagree …5= Strongly Agree

Purpose of instrument

Purpose of Instrument

Purpose of the Beliefs on RtI Scale is to measure and inform consensus development in two ways:

  • Assess impact of professional development on educator beliefs about PS/RtI

  • Identify commonly held beliefs that will likely facilitate or hinder implementation efforts

Administration procedures scoring

Beliefs on RtI Scale

Administration procedures & scoring

Administration procedures intended audience

Administration procedures-Intended Audience

  • Who should complete?

    • SBLT members

    • Instructional staff

  • Who should use results?

    • SBLTs

    • DBLTs

Directions for administration

Directions for Administration

  • Methods for administration/dissemination

    • Completed individually

    • Anonymity

    • Opportunity for questions

  • Role of school principal—explain the “why”

  • Role of RtI coach/coordinator/SBLT member

  • Frequency of use: resources, rationale, recommendations



Two techniques to analyze survey responses:

  • Mean rating for each item calculated to determine average perceived belief level

  • Frequency of each response option selected calculated for each item

Calculating item m ean

Calculating Item Mean

  • Overall assessment of reported beliefs of educators within a school/district

  • Can be done at domain (factor) and/or individual item level

    • Domain level: examine patterns in reported beliefs regarding

      • academic ability and performance of SWD

      • data-based decision-making

      • functions of core and supplemental instruction

    • Item level: identify specific beliefs staff report v. beliefs in need of exploration and support

Calculating frequency of response options

Calculating Frequency of Response Options

  • Provides information on range of belief levels

  • Can be used to determine what percentage of staff may require little, some, or high levels of support to implement PS/RtI

  • Informs professional development decisions

Interpretation and use of data

Beliefs on RtI Scale

Interpretation and use of data

Interpretation use of data cont

Interpretation & Use of Data (cont.)

  • Sharing data with stakeholders:

    • DBLTs, SBLTs, instructional staff

  • Use data to:

    • Develop/adjust consensus-building goals

    • Design training/coaching activities

    • Facilitate consensus-building discussions re: rationale for PS/RtI, patterns and changes in beliefs over time

Practice activity

Beliefs on RtI Scale

Practice Activity

Perceptions of rti skills survey

Assessing Perceptions of Skills Integral to PS/RtI Practices

Perceptions of RtI Skills Survey

Perceptions of skills

Perceptions of Skills

The likelihood of embracing new practices increases when:

  • Educators understand the need for the practice

  • Educators perceive they either have the skills to implement the practice or will be supported in developing required skills

Description and purpose1

Perceptions of RtI Skills Survey

Description and purpose

Perceptions of skills description and purpose

Perceptions of Skills—Description and Purpose

  • Theoretical Background:

    • Assess educators’ perceptions of skills they possess to implement PS/RtI

    • Understand perceptions of skills and how perceptions change as function of professional development to facilitate PS/RtI implementation

Description of survey1

Description of Survey

  • Assesses skills/amount of support needed for:

    • Applying PS/RtI practices to academic content

    • Applying PS/RtI practices to behavior content

    • Data manipulation and technology use

  • 50items; 5-point Likert scale

  • 1= I do not have the skill at all (NS)…5= I am highly skilled in this area and could teach others (VHS)

Purpose of instrument1

Purpose of Instrument

Purpose of the Perceptions of RtI Skills Survey:

  • Assess impact of professional development

  • Identify “comfort level” with PS/RtI practices to inform PD; allocate resources

Administration procedures scoring1

Perceptions of RtI Skills Survey

Administration procedures & scoring

Administration scoring

Administration & Scoring

  • Same procedures as Beliefs on RtI Scale

Interpretation and use of data1

Perceptions of RtI Skills Survey

Interpretation and use of data

Interpretation use of data

Interpretation & Use of Data

  • Three domains:

    • Perceptions of skills applied to academic content

    • Perceptions of skills applied to behavior content

    • Perceptions of data manipulation and technology use skills

  • Three methodologies:

    • Calculate mean at domain level

    • Calculate mean at item level

    • Frequency/percentage of who selected each response option

  • Identify specific skills/skills sets for PS/support

Practice activity1

Perceptions of RtI Skills Survey - Revised

Practice Activity

Implementation integrity

Implementation Integrity

What is integrity and why is it important

What is “Integrity”and why is it important?

  • Integrity is the degree to which something was done the way it was intended to be done.

  • When a process or procedure lacks “integrity,”few if any assumptions can be made about the outcome or impact of that process or procedure.

3 ways to assess implementation integrity

3 Ways to Assess Implementation Integrity


Most efficient

Least reliable

Permanent Product Reviews

Moderately efficient

Moderately reliable


Least efficient

Most reliable

Tiers i and ii critical components checklist

Tiers I and II Critical Components Checklist

Description and purpose2

Tiers I and II Critical Components Checklist

Description and Purpose

Theoretical background

Theoretical Background

  • Implementation of new practices is a gradual process that occurs in stages, not a one-time event (Fixen, Naoom, Blasé, & Wallace, 2005).

  • Since many educational reform initiatives fail due to lack of implementation (Sarason, 1990), it is critical to examine implementation integrity

  • Several methods for examining implementation integrity exist (Noell & Gansle, 2006)

    • Self-report

    • Permanent product reviews

    • Observations



  • Permanent product review

  • Measures the extent to which components of the PS/RtI process are evident in permanent products from data meetings addressing Tier I and/or Tier II content

  • 11 items organized around the 4-step problem-solving process

    • Problem identification

    • Problem analysis

    • Intervention development and implementation

    • Program evaluation/RtI

  • Response options: 0=Absent, 1=Partially present, 2=Present (N/A for some items)



  • To provide stakeholders with a practical methodology for evaluating the extent to which educators implement PS/RtI practices in data meetings addressing Tier I and /or II content

  • Permanent product reviews typically more reliable than self-report, but more resource-intensive

Administration procedures and scoring

Tiers I and II Critical Components Checklist

Administration Procedures and Scoring

Who should complete the checklist

Who should complete the checklist?

  • The person completing Tiers I and II CCC should have expertise in PS/RtI model and conducting permanent product reviews

    • Specifically, the 4 steps of the problem-solving process

  • PS/RtI Coaches, school psychologists, literacy specialists, etc.

Directions for administration1

Directions for Administration

  • Identify content areas/grade levels being targeted by the school

  • Identify when Tier I/II data meetings occur and who is involved in the meetings

  • Find out who to contact for permanent products that come from identified meetings and what products will likely be available

  • Gather any relevant documents for the period of time for which the checklists are being completed

  • Complete the checklists using the Tier I and II CCC Standard Scoring Rubric

  • Complete inter-rater procedures when applicable

Frequency of use

Frequency of Use

  • Consider resources available, including the time needed to:

    • Complete the instrument

    • Enter, analyze, graph, and disseminate data

    • Personnel available to support data collection

    • Additional data collection activities SBLT members and school staff participate in

  • General recommendations

    • Data collection aligned with school’s target content areas and grade levels

    • Aligned with the frequency of universal screening and progress monitoring data



  • Examples of two data analysis techniques

    • Calculate the mean rating for each item

    • Frequency distribution of each response option selected (i.e., Absent, Partially present, and Present)

  • Four domains

    • Problem Identification (Items 1-3)

    • Problem Analysis (Items 4-5)

    • Intervention Development and Implementation (Items 6a-7c)

    • Program Evaluation/RtI (Items 8-11)

Interpretation and use of the data

Tiers I and II Critical Components Checklist

Interpretation and Use of the Data

Examination of broad domains

Examination of Broad Domains

  • Start by examining broad domains to evaluate the extent to which permanent products indicate PS/RtI practices are being implemented

  • Examining the data graphically allows for educators to determine the extent to which the major steps of problem-solving are occurring

  • Examine implementation levels at each time point, as well as trends over time

Identification of specific needs

Identification of Specific Needs

  • Tiers I and II CCC can be used to identify which components of problem-solving are more vs. less evident

  • Consider what training educators have received and how long implementation efforts have been occurring

  • Stakeholders can use this data to identify components of the problem-solving process that require additional support to be implemented

    • Professional development

    • Policies and procedures

  • Important to consider all aspects of the school/district system that might contribute to implementation

Dissemination to stakeholders

Dissemination to Stakeholders

  • Important to disseminate implementation data to key school and district stakeholders as quickly and frequently as possible

  • Allow for stakeholders to discuss implementation levels, develop/alter implementation goals, and design strategies to increase implementation

Dissemination to stakeholders cont

Dissemination to Stakeholders (cont.)

  • Sample guiding questions

    • What are the patterns?

    • Are there indicators that show zero implementation? Why?

    • How have you progressed in implementing the PS model with fidelity?

Practice activity2

Tier I & II Critical Components Checklist – Domain I (Problem Identification)

Practice Activity

Tier i ii ccc practice activity materials

Tier I & II CCCPractice Activity Materials

  • Tier I & II CCC Scoring Rubric

    • Domain 1

    • Questions 1-3

Practice activity tier i ii ccc

Practice Activity: Tier I & II CCC

  • Domain 1

    • Problem Identification



  • What are you currently doing to examine these areas in your district or school?

    • What are the critical questions you ask?

    • What data sources do you have to answer them?

    • What questions do you already have that you cannot answer with available data?

    • How do you use the data you collect to inform decisions?

  • What areas need to be addressed as you return to your districts to plan? What are the priorities?

    • What critical questions do you need to start asking?

    • What data sources do you need?

    • How can you better use the data to inform decisions?

Additional resources

Additional Resources

Additional resources1

Additional Resources

  • MTSS Implementation Components: Ensuring Common Language & Understanding

  • http://www.floridarti.usf.edu/resources/format/pdf/mtss_q_and_a.pdf

Additional resources cont

Additional Resources, cont.

  • Implementing a Multi-Tiered System of Support for Behavior: Recommended Practices for School and District Leaders

    • http://flpbs.fmhi.usf.edu/pdfs/RtIB%20guide%20101811_final.pdf

  • Guiding Tools for Instructional Problem-Solving (GTIPS)

    • http://www.florida-rti.org/_docs/GTIPS.pdf

  • Florida’s PS/RtI Evaluation Tool Technical Assistance Manual – Revised (2012)

    • http://www.floridarti.usf.edu/resources/program_evaluation/ta_manual_revised2012/index.html

Floridarti usf edu


Flpbs fmhi usf edu


Facilitating ps rti capacity tools skills and strategies for practitioners

  • Amber Brundage

    • abrundage@mail.usf.edu

  • Jose Castillo

    • jmcastil@usf.edu

  • Clark Dorman

    • Dorman@usf.edu

  • Amanda March

    • amarch@usf.edu

  • Kevin Stockslager

    • kstocksl@usf.edu

  • Login