1 / 120

Craig A. Albers, Ph.D. Assistant Professor University of Wisconsin – Madison caalbers@wisc

RTI Training Series 2008 LEA Forum: Session 3 Monitoring Student Progress & Fidelity of Implementation May 8 th -9 th. Craig A. Albers, Ph.D. Assistant Professor University of Wisconsin – Madison caalbers@wisc.edu. Speaker: Craig A. Albers, Ph.D.

Download Presentation

Craig A. Albers, Ph.D. Assistant Professor University of Wisconsin – Madison caalbers@wisc

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. RTI Training Series 2008 LEA Forum: Session 3 Monitoring Student Progress &Fidelity of ImplementationMay 8th-9th Craig A. Albers, Ph.D. Assistant Professor University of Wisconsin – Madison caalbers@wisc.edu

  2. Speaker: Craig A. Albers, Ph.D. Dr. Albers, Assistant Professor at the University of Wisconsin in Madison, will provide the national perspective for Session 3 of this training series. He has conducted research in the areas of universal screening and progress monitoring, prevention and early intervention for students experiencing academic and/or behavioral difficulties, language proficiency and academic interventions for ELLs, and functional behavior assessment. He earned his doctorate in educational psychology with a specialization in school psychology at Arizona State University. Dr. Albers provides consultation and mentoring services to numerous schools, districts and state educational agencies regarding the implementation of RTI models. Dr. Albers has been identified as an Early Career Scholar by the Society for the Study of School Psychology, and recently guest-edited a Journal of School Psychology special issue relating to universal screening and progress monitoring within RTI models and the connection to improved educational and mental health outcomes. 2

  3. Session 3 Goals • Identify key components for universal screening and progress monitoring tools. • Examine academic and social-emotional functioning instruments and procedures that can be utilized as universal screening measures. • Develop specific criteria for decision-making to determine which students are in need of additional options. • Utilize frequent progress monitoring to determine effectiveness of interventions and fidelity of implementation. 3

  4. RTI Conceptual Models • RTI models for special education eligibility • RTI models for school-wide reform and improved outcomes for all student 4

  5. Response-to-Intervention • RTI is the practice of • providing evidence-based (scientific supported) instruction/intervention matched to student needs • using rate and level of performance as assessed over time to make important psychoeducational decisions 5

  6. Essential Components in RTI Models • Universal screening • Progress monitoring • Selection of universal (core), selected (supplemental) and targeted (intensive) options that are likely to be effective • Fidelity of implementation • Special education eligibility decisions (decision rules) Screening and progress monitoring are the foundation of data-based decision-making in RTI models! 6

  7. Connecticut RTI Model 7

  8. The R in RtI • Selecting At-Risk Students (“screening”) • Monitoring of At-Risk Students (“progress monitoring”) • Monitoring of Implementation Fidelity 8

  9. The R in RTIWhat Does Screening and Progress Monitoring Look Like Within Multi-Tiered Models? 9

  10. The I in RTI • The Focus is Primarily on Reading (but other domains are included) • Interventions are Multi-tiered • Problem Solving Model Used • Standard Treatment Protocol Used • Intervention-as-Test (Fuchs & Fuchs, 2006) • Intervention Integrity Important 10

  11. What is Universal Screening?How do we do it? 11

  12. The R in RTIEducational Decisions Linked to RtI • Screening for Resource Allocation • Screening for Further Evaluation • Assessment for Response-to-Intervention • Assessment for Instructional Decisions • Assessment for Program Eligibility (including special education) 12

  13. The R In RTI What is Universal Screening? • Conducted with everyone within a population (e.g., classroom, grade level, school, district, state, certain age, etc.); • Conducted to identify those at-risk of failure, emotional/behavioral difficulties, health issues, etc.; • Goal is to identify difficulties (1) before overt problems/symptoms are manifested, or (2) before the difficulties become significant and lead to impairment. 13

  14. The R In RTI What is Universal Screening? • Emphasis should be on early identification to prevent difficulties from further development or escalation • Two ways of looking at EARLY identification: • Early as in an “early” age (e.g., pre-kindergarten, etc.) • Early as in the “early” development stage of a difficulty • This should be the focus in a response-to-intervention model • Should be ONGOING • This is the connection to progress monitoring • Usually, only snapshots are taken of an individual that may not be an accurate portrayal of any significant issues 14

  15. The R In RTI What is Universal Screening? • A good screening program will also include appropriate intervention options at each level • Intent is to differentiate among: • Typically-developing children/adolescents • Those with elevated risk status • Those with life-course persistent problems 15

  16. The R in RTILegal Mandates and Recommendations for Universal Screening • NCLB • Focus on reading • IDEA 1997/2004 • 15% of funds can be used for early intervention (IDEA, 2004) • President’s Commission on Special Education • Recommends locally-driven universal screening of young children • NRC • Emphasizes early, universal screening for academic and behavioral issues • Best practices? Ethical responsibilities? 16

  17. A Comprehensive Data-Based Framework 17

  18. The R in RTIScreening-Related Variables (Glover & Albers, 2007) 18

  19. The R in RTIUniversal Screening Content Areas • Academic • Behavioral • Emotional • Mental Health • Social • Health 19

  20. The R in RTIUniversal Screening Intent • Prediction • Will difficulties arise in the future? How likely are future difficulties? • Identification • Is there evidence suggesting that difficulties currently exist? 20

  21. The R in RTIWho is Universal Screening Designed to Identify? Risk Factors Low Medium High Protective Factors High Medium Low 21

  22. The R in RTIUniversal Screening Frequency • Single time • Multiple times • Ongoing 22

  23. The R in RTIUniversal Screening Stages / Approaches • Single-gate • Multi-gate 23

  24. The R in RTIWhat are Multi-Gate Approaches? 24

  25. The R in RTIUniversal Screening Multi-Gate Approaches Gate 1 • Expected to over-identify difficulties (false positives) • Goal is to eliminate those who clearly are not having difficulties • Do not allow for definitive statements; at best may be a preliminary indication that something could be wrong • “Speculative screening” 25

  26. The R in RTIUniversal Screening Multi-Gate Approaches Gate 2 • Goal is to continue to remove students who clearly do not have significant difficulties from future screening activities Gate 3 • Goal is to clearly identify those with significant risk factors/lack of protective factors who are in need of intervention options 26

  27. The R in RTIUniversal Screening Informant(s) • Parent / caregiver • Teacher / educator • Self-report • Direct skills 27

  28. The R in RTIUniversal Screening Formats • Direct skills / measures • Rating scales • Checklists • Interviews • Nominations • Observations • Record reviews 28

  29. The R in RTIUniversal Screening Models • Focus on person • Focus on environmental variables • Focus on interaction between person variables and environmental variables 29

  30. The R in RTIUniversal Screening Outcome Measures Timing • Concurrent • Predictive (at some point in the future) 30

  31. The R in RTIDefinition of At-Risk • Who or what determines the “degree of at-risk”? • Related to: • Compatibility with service delivery availability (i.e., resource allocation) • Technical adequacy of instruments • Consequences of being identified 31

  32. The R in RTIConsiderations and Sample Questions for Evaluating Universal Screening Assessments (Glover & Albers, 2007) 32

  33. The R in RTIConsiderations and Sample Questions for Evaluating Universal Screening Assessments (Glover & Albers, 2007) 33

  34. The R in RTIConsiderations and Sample Questions for Evaluating Universal Screening Assessments(Glover & Albers, 2007) 34

  35. The R in RTIScreening Assessment Outcomes – Predictive Validity (a) (b) (c) (d) • Possible outcomes • A student may be screened and identified as having an early learning problem and be actually experiencing the early stages of a learning problem (a) • A student may be screened and identified as having an early learning problem but may not actually be experiencing the early stages of a learning problem (c) • A student may be screened and not identified as having an early learning problem but actually be experiencing the early stages of a learning problem (b) • A student may be screened and not identified as having an early learning problem and not be experiencing the early stages of a learning problem (d) Sensitivity - of those actually at risk, what proportion is correctly identified? a / (a + b) Specificity - of those actually not at risk, what proportion is correctly identified? d / (c + d) Positive predictive value - of those identified as at risk, what proportion is correctly identified? a / (a + c) Negative predictive value - of those identified as not at risk, what proportion is correctly identified? d / (b + d) Hit rate - what proportion of the total sample was correctly identified? 35

  36. Reading DIBELS CBM BACESS Other standardized achievement measures Mathematics CBM BACESS Other standardized achievement measures Or, create approach specific to your state – see next slides for example The R in RTIUniversal Screening Approaches: Academic Examples 36

  37. Creation of Math Progress Monitoring Process for WI 37

  38. 38

  39. 39

  40. Systematic Screening for Behavior Disorders (SSBD; Walker & Severson, 1990) School Social Behavior Scale (SSBS; Merrell, 1993) The Revised Behavior Problem Checklist (Quay & Peterson, 1987) Drummond’s Student Risk Screening Scale (SRSS; Drummond, 1993) Conner’s Rating Scales-Revised (CRS-R; Conner’s, 1990) Eyberg Child Behavior Inventory (ECBI; Eyberg & Ross, 1978) and Sutter-Eyberg Student Behavior Inventory (SESBI; Sutter & Eyberg, 1999) SSIS Multi-Tiered Assessment and Intervention Model (Elliott & Gresham, 2007) The R in RTIUniversal Screening Approaches: Social-Emotional / Behavioral Examples (Walker, Hope-Doolittle, Kratochwill, Severson, & Gresham, 2007) 40

  41. Ages & Stages Questionnaire (2002) Diagnostic Predictive Scales (2001) Strengths & Difficulties Questionnaire (1997) Conner’s Rating Scales (1997) Children’s Depression Inventory (1992) Multidimensional Anxiety Scale for Children (1997) Beck Depression Inventory-II (1996) Center for Epidemiologic Studies Depression Scale (1977) Columbia Depression Scale (2005) Personal Experience Screening Questionnaire (1991) The R in RTIUniversal Screening Approaches: Examples of Screening Instruments Relating to Mental Health (Levitt, Hunter, & Hoagwood, 2007) 41

  42. What is Progress Monitoring? 42

  43. What is Progress Monitoring? • A systematic process by which student performance data are frequently and repeatedly collected and analyzed • Used to assess student performance and evaluate the effectiveness of instruction. • Used to measure rate of improvement in relation to identified benchmarks • Frequent comparison of current to desired performance over a specified period of time 43

  44. What is Progress Monitoring? Continued. • Typically used to analyze a student response to a particular option (RTI) “(critical component of RTI models)” • Sensitive to small changes in student performance • You do not have to wait for 6 months to know if something is making a difference • Relatively quick and simple to carry out • Implemented with individual students or an entire class 44

  45. Progress Monitoring is not: • In-depth assessment of a content domain • An analysis of degree of curriculum alignment to standards • An analysis of student achievement in terms of lesson or unit content coverage • A diagnostic assessment of student strengths and weaknesses 45

  46. Assessment for Intervention Identifying, Verifying, and Aiding…. …in the selection of an appropriate intervention. Assessment of Intervention Determine whether intervention is having sufficient impact so that the student can reach defined goals Is the intervention being implemented as intended? Progress Monitoring: Necessary Distinctions Between Assessment for Intervention and Assessment of Intervention Note – see: Albers, C. A., Elliott, S. N., Kettler, R. J., & Roach, A. T. (2005). Evaluating intervention outcomes. In R. Brown-Chidsey (Ed.), Problem-solving based assessment for educational intervention (pp. 329-351). New York: Guilford Publications for more descriptions of assessment for – and assessment of intervention in a problem-solving model. 46

  47. Universal options Selected options Targeted options RTI Models – Progress Monitoring Intervention Intensity Progress Monitoring Frequency Low Quarterly, etc. More frequently High 47

  48. A Progress Monitoring System is Designed to Identify... • Rate of all students’ performance • Effectiveness of instructional options • Students who may need additional supports 48

  49. On-Going Progress Monitoring Facilitates… • Efficient selection and use of effective educational options • Appropriate resource allocation • Focus on explicit outcome indicators • Objective data-based decisions about options • Clear expectations for students • Continuous feedback 49

  50. What are the Minimum Requirements for Progress Monitoring? • Technical adequacy (i.e., reliability & validity) • Feasibility of administration (quick/brief) • Sensitive to student growth • Sufficient number of alternate forms • Useful for instructional planning From: www.studentprogress.org 50

More Related