Improving Results Using State and Local Special Education Data

1. Bureau of Indian EducationSpecial Education AcademyUsing State and Local Data to Improve Results Sandy Schmitz, Ph.,D DAC Tampa, FLSeptember 12 - 15, 2011

2. DAC’s Goal Form partnerships in states that join state and local agencies in the use of data to drive improved results

3. Premises Data Use involves: • Working through a Collaborative Team approach. • Engaging Team in a Continuous Improvement Process. • Relating the Data to specific Problem/Issue. Using Data is an Iterative Process!

4. DAC Concepts for Data Use 1. Identify relevant data Consist of three phases w/ several steps: • Phase 1: Preparation • Identify relevant data • Phase 2: Inquiry • Conduct data analysis • Determine Root Cause • Phase 3: Action • Plan for improvement • Implement Plan • Evaluate progress Action Preparation 6. Evaluate Progress 2. Conduct data analysis to generate hypothesis 5. Implement Plan Data Analytics 3. Test Hypothesis to determine root cause 4. Plan for Improvement Inquiry

5. Data Quality Standards • Data collected, submitted, analyzed, and reported must be: • Timely • Accurate • Reliable • Consistent • Objective • Valid • Complete • Credible • Secure • Useful • Interpretable • Relevant • Transparent • Accessible

6. MUST INVOLVE A TEAM • Administrator • Data Person • Special Education • General Education • Parents • Others as needed

7. How to identify relevant data? Module 1: Identify Relevant Data

8. The key to identifying relevant datais to ensure that you clearly define or select a specific problem or issue.

9. Problem Description The problem should be a clear concise statement of the issue(s) that need to be addressed by a problem solving team.

10. Problem Description The described problem should answer: • Who has the problem? This should explain who needs the solution and who will decide the problem/issue has been resolved 2. Whatis the problem? This should explain why the team is needed 3. Where did the problem occur? 4. Whendid the problem occur? This should provide the context and timeframe of the problem/issue

11. EXAMPLE: ISBE Problem For more than three years, as documented in the APR and in public reports, the State of Illinois has reported percentages of students with disabilities included in regular classrooms 80% or more of the day at a lower rate than the national average.

12. Educational Environments: 2006 Percent Served Inside Regular Class >80% of the School Day(Highest and Lowest ranked Entities) 71% (3) 78% (1) 65% (9) 49% (46) 69% (5) 66% (8) 50% (43) 67% (6) 50% (44) 41% (48) 70% (4) 50% (41) 49% (47) 10% (50) D.C. 50% (45) 67% (7) 50% (43) 74% (2) 21% (49) BIE 65% (10) (Reporting States, DC & BIE Schools Average = 54% ) States Not Ranked- Denominator <10 Highest Ranked for Percent Served in Setting Source: Dec 1, 2006 count. IDEA Data provided by OSEP Table 5.8 See www.ideadata.org– State Ranks-Part B Lowest Ranked for Percent Served in Setting X= suppressed data

13. Educational Environments: 2007 Percent Served Inside Regular Class >80% of the School Day(Highest and Lowest ranked Entities) 51% (46) 77% (2) 51% (45) 69% (5) 67% (8) 71% (3) 69% (7) 70% (4) 52% (42) 42% (48) 48% (47) 67% (9) 52% (44) 69% (6) 52% (41) D.C. 17% (50) 52% (43) 79% (1) 18% (49) 64% (10) 64% (10) B.I.E (Reporting States, DC & BIE Schools Average= 57% ) States Not Ranked- Denominator <10 Highest Ranked for Percent Served in Setting Source: Dec 1, 2007 count. IDEA Data provided by OSEP Table 5.2. See www.ideadata.org– State Ranks-Part B Lowest Ranked for Percent Served in Setting X= suppressed data

14. Educational Environments: 2008 Percent Served Inside Regular Class >80% of the School Day(Highest and Lowest ranked Entities) 43% (48) 50% (45) 77% (2) 52% (42) 70% (5) 71% (4) 70% (6) 72% (3) 44% (47) 49% (46) 68% (9) 52% (41) 68% (10) 70% (7) 52% (44) D.C. 18% (49) 52% (43) 81% (1) 15% (50) B.I.E 69% (8) (Reporting States, DC & BIE Schools Average= 57% ) States Not Ranked- Denominator <10 Highest Ranked for Percent Served in Setting Source: Dec 1, 2007 count. IDEA Data provided by OSEP Table 5.2. See www.ideadata.org– State Ranks-Part B Lowest Ranked for Percent Served in Setting X= suppressed data

15. Educational Environments: 2009 Percent Served Inside Regular Class >80% of the School Day(Highest and Lowest ranked Entities) 49% (47) 72% (5) 50% (46) 76% (2) 51% (44) 70% (9) 73% (3) 70% (8) 72% (4) 45% (48) 51% (45) 54% (41) 70% (10) 71% (7) 51% (43) D.C. 36% (49) 53% (42) 82% (1) 17% (50) B.I.E 71% (6) (Reporting States, DC & BIE Schools Average= 57% ) States Not Ranked- Denominator <10 Highest Ranked for Percent Served in Setting Source: Dec 1, 2007 count. IDEA Data provided by OSEP Table 5.2. See www.ideadata.org– State Ranks-Part B Lowest Ranked for Percent Served in Setting X= suppressed data

16. Illinois Sw/D Indicators 5a, 5b, 5c

17. Problem Description • Begin with an initial State Agency problem statement. • Local agencies revise the statement as necessary when additional data are collected and analyzed throughout the data analytics process

18. Given your district problem statement…. Identify which data needed to gather the evidence necessary to answer “why” the problem or issue exist.

19. Identify Relevant Data (cont’d) • Relevant information may include: • District, building, or school - level data • Disaggregated student population data • Data on the status of highly qualified personnel disaggregated by building/school

20. Module 2 Step 2: Conduct Data Analysis to Generate Hypothesis

21. What is data analysis and how is it done? Data Analysis

22. Analysis is based upon what the problem is… The question “WHY” comes into play

23. Analysis involves organizing and understanding data based on criteria you develop; it is useful when you want to find some trend or pattern. Source: Purdue Online Writing Lab

24. Drill down involves accessing information by starting with a general category and moving through the hierarchy of field to file to record; it is the act of focusing in to get to the root cause. Source: Adapted from Webopedia

25. What is a hypothesis and how do you generate one? Hypothesis

26. A hypothesis is defined as “……. a starting-point for further investigation from known facts”.  (The Concise Oxford Dictionary, 1990)

27. Example of a Hypothesis Sw/Ds who have greater access to the general curriculum as measured by time spent with typical peers will perform better on tests of reading proficiency than those Sw/Ds who have less access to the general.

28. Another example of a Hypothesis IF-Then If reading proficiency is related to access to the general education curriculum, then increasing SW/Ds access to the general education curriculum will improve their performance on the statewide reading assessment.

29. What is an analysis plan and why develop one? Data Analysis Plan

30. The analysis plan provides an outline of additional data that need to be analyzed to test the hypothesis and determine root cause; it helps with preparing a clear and concise presentation of the results of your analysis activities

31. Data Analysis Report - Components The analysis should include: Types of data to be examined (e.g., demographic data; data about programs, process and outcomes; etc.) • Student records • Interviews (e.g., position and roles of building and district personnel) • Observations

32. Let’s get you started on Module 3 Step 3: Test Hypothesis to Determine Root Cause

33. “ In order to shake a hypothesis, it is sometimes not necessary to do anything more than to push it as far as it will go. Denis Diderot

34. What is it and how do you do it? Data Triangulation

35. Data triangulation is a process of examining multiple sources of data to form a conclusion or generalization. Source: Wikipedia

36. Why determine the root cause? Root cause analysis

37. Helps Dissolve the Problem • Eliminates Patching • Conserves Resources • Facilitates Discussion • Provides Rationale for Strategy Selection Determining the Root Cause:

38. Module 4 Plan for Improvement

39. When data indicate a problem or issue, districts and schools should develop or revise an improvement plan that outlines the course of action it will take to improve results. Adapted from ESEA, Title I, Sec. 1116 (b)(3)

40. What is an improvement plan and why develop one? Improvement Plan

41. Basic Components of anImprovement Plan • Goals • Activities • Timelines • Person(s) Responsible • Resources • Evidence of Change

42. Setting goals is critical to determine how much progress (as documented in an improvement plan) is acceptable and what amount of progress is not; it establishes internal accountability, high expectations and a trajectory by which to evaluate progress. Boudette, City, and Murnane - Datawise

43. First Consider Evidence of Change • Indicate that the changes in the system have yielded “significantly” improved results for students with disabilities in the problem area • Must be demonstrated through gains in student results data • Not about “effort” but about “impact”

44. Setting SMART Goals Specific Measurable Attainable Realistic Timely Source: goal-setting-guide.com

45. Organize Data Sources to Assess Progress Short-term data – information that can be collected daily or weekly Medium-term data – information gathered systematically at the building-, grade-, or district-level at periodic intervals during the year Long-term data – information gathered annually (e.g., students’ performance on statewide tests)

46. Module 5 Step 5: Implement Plan

47. Remember! • Implement with Fidelity • Team work • Timely • Continuously assess progress • Keep your eye on the ball (Evidence of Change)

48. Module 6 Step 6: Evaluate Progress

49. District MET goals and showed evidence of change. Action: Create evaluation strategies to ensure sustainability OR • District DID NOT meet goals and show evidence of change. Action: Re-evaluate process Two potential pathways

50. Pathway 1: Sustainability Does the team have: • a plan to ensure sustainability over time? • strategies to keep the work fresh/ongoing? • routine checks to review data to ensure sustainability? • strategies to ‘raise the bar’?