1 / 59

Top-to-Bottom Ranking & Priority/Focus/Reward Designations

Top-to-Bottom Ranking & Priority/Focus/Reward Designations. Understanding the. Why the TTB Ranking?. Move from metric as designation only (“stick”) to leveraging the metric as a diagnostic tool for schools

Download Presentation

Top-to-Bottom Ranking & Priority/Focus/Reward Designations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Top-to-Bottom Ranking &Priority/Focus/RewardDesignations Understanding the

  2. Why the TTB Ranking? • Move from metric as designation only (“stick”) to leveraging the metric as a diagnostic tool for schools • Resist urge for “more data” until we understand the metrics available; avoids “dying in data” • Focus of this presentation: • Overview of calculations • How to interpret results (for schools and districts)

  3. Breaking it Down

  4. The Top-to-Bottom List • Statewide ranking of ALL schools that meet ranking criteria • Bottom 5% = Priority (PLA) Schools • 10% schools with largest achievement gaps = Focus Schools • Also used for Reward School status: • Use top 5% from overall ranking = Reward Schools • Use top 5% improving schools = Reward Schools • Use Beating the Odds Schools = Reward Schools

  5. Top to Bottom(TTB) Ranking • Three main components by subject: • Achievement • Improvement in achievement over time • Achievement gap between the top scoring 30% of students versus the bottom scoring 30% of students • Each component tells schools something about their overall performance and can be used for diagnostics

  6. TTB Ranking • Graduation rates are included in the statewide Top-to-Bottom Ranking. • Schools with a graduation rate have it included in the following two ways: • Graduation Rate • Improvement in graduation rate over time

  7. Who Receives a Ranking? • Schools with 30+ full academic year (FAY) students over the last two years in at least two state-tested content areas; school must be OPEN at time of list generation • Application  Some schools do not receive a ranking if they: • Have too few FAY students • Only have one year of data

  8. Tested Grades & Subjects • Reading and Mathematics: Grades 3-8 and 11 • In grades 3-8, testing every year allows us to figure out student performance level change (our current “growth” metric) in reading and math • Students can either significantly improve, improve, maintain, decline or significantly decline • Writing: Grades 4 & 7 • Science: Grades 5 & 8 • Social Studies: Grades 6 & 9

  9. What About Reconfigured Schools? • A school must change by four or more grades in order to get a new code • Example: A K-2 building becoming a K-6 building. • New codes NOT granted when a school is reopened as a charter, for example • If not, the school retains the old code and continues to have data “point” at it from all students for whom that code is their feeder school • There is no “phase reset” like there was in AYP • If school population changed by 51%, could request a phase reset—still got AYP calculations, but sanctions delayed • Under Priority/Focus interventions, would simply have a customized intervention.

  10. How Is the Top to Bottom Ranking Calculated • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

  11. How do we get Standardized Scale Scores for Each Student? • Step #1: Take each student’s score on the test they took and compare that score to the statewide average for students who took that same test in the same grade and year • This creates a student-level z-score for each student in each content area • Compare • MEAP to MEAP • MEAP-Access to MEAP-Access • MME to MME • MI-Access • Participation to Participation • Supported Independence to Supported Independence • Functional Independence to Functional Independence

  12. What do we do with those Standardized Scores? • Step #2: Once each student has a z-score for each content area (based on the test they took), we take all of the students in a each school, and rank order the students within the school. • Z-scores will have come from different tests, and compare students to statewide average for that grade, test, and subject • But they can now be combined for the school • Step #3: Add up all z-scores and take the average. This is now the average standardized student scale score. • Step #4: Define the top and bottom 30% subgroups, based on that rank ordering.

  13. Average Z-score (average standardized student scale score): 0.28 (sum all z-scores, divide by 15)

  14. Top 30% Bottom 30%

  15. How Is the Top to Bottom Ranking Calculated? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

  16. What is Important to Show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Step #1: Achievement How well did the school do in that subject? Positive number = better than average Near zero = average Negative number = worse than average Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

  17. What is Important to Show Schools? Step #2: Improvement Is the school improving in that subject? Positive number = greater rate of improvement than average Near zero = average improvement Negative = slower rate of improvement than average; can also mean they are declining • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

  18. What is Important to Show Schools? Raw value is also meaningful: Positive number: More students improving than declining Negative number: More students declining than improving • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

  19. What is Important to Show Schools? Step #3: Achievement Gap Is the gap in that subject between top 30% and bottom 30%: (positive number) = smaller gap than average (negative number) = larger gap than average (near zero) = average gap • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

  20. Once you have Looked at each Component, Discuss: • What’s the overall pattern? • Low achievement? • Declining achievement? • Large gaps? • Where are the actionable areas? • Which subjects need the most attention? • Is everyone doing poorly (small gap, low achievement) or are some students doing well and others falling behind (decent achievement, but large gap)

  21. What is Important to Show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Reward Schools (for improvement) Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Focus Schools Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

  22. Weighted Performance Level Change (PLC) • A weighted composite of individual student performance level change is used to calculate improvement in grades 3-8 reading and mathematics • Rewards large improvements more heavily, rewards maintenance of proficiency if a student was already proficient

  23. How is the Top-to Bottom Ranking Calculated? • For science, social studies, writing, and grade 11 all tested subjects Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Four-Year Achievement Trend Slope School Performance Achievement Trend Z-Score School Content Area Index Content Index Z-score 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

  24. How is the Top-to-Bottom Ranking Calculated? • For graduation rate Two-Year Average Graduation Rate School Graduation Rate Z-Score 2/3 Four-Year Graduation Rate Trend Slope School Graduation Rate Trend Z-Score School Graduation Rate Index Grad Index Z-score 1/3

  25. How is the Top-to-Bottom Ranking Calculated? • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate)

  26. How is the Top-to-Bottom Ranking Calculated? • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate)

  27. How is the Top-to-Bottom Ranking Calculated? • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate) Slope = 2.3%

  28. How is the Top-to-Bottom Ranking Calculated? • Calc an overall rank for a school with a grad rate School Mathematics Std Index 18% School Reading Std Index 18% Overall Standardized School Index Overall School Percentile Rank School Science Std Index 18% School Social Studies Std Index 18% School Writing Std Index 18% School Graduation Rate Std Index 10%

  29. How is the Top-to-Bottom Ranking Calculated? • Calculating an overall ranking for a school without a graduation rate School Mathematics Std Index 20% Look at each subject index. Help schools understand which subjects are strong/weak for them. Positive number: better than average Negative number: below average Near zero: near average School Reading Std Index 20% Overall School Standardized Index Overall School Percentile Rank School Science Std Index 20% School Social Studies Std Index 20% School Writing Std Index 20%

  30. How is the Top-to-Bottom Ranking Calculated? • Calculating an overall ranking for a school without a graduation rate and without a writing score School Mathematics Index 25% School Reading Index 25% Overall School Standardized Index Overall School Percentile Rank School Science Index 25% School Social Studies Index 25%

  31. Which years of data are in the ranking?

  32. For elementary and middle schools • Michigan tests in the fall. • These fall tests reflect the learning of students in the previous school year. SY 2008-2009 SY 2009-2010 SY 2010-2011 Fall 2010 Testing Fall 2008 Testing Fall 2009 Testing Fall 2011 Testing

  33. For High Schools • Michigan tests in the spring • The spring test (MME and MI-Access) measures what students have learned from grades 9, 10 and grade 11 prior to the MME testing.

  34. What do the 2012 Rankings reflect? • For elementary/middle schools: • Performance on the MEAP and MI-Access tests in fall 2011 (which represents learning from school year 2010-2011) and before • For high schools: • Performance on the MME and MI-Access tests in spring 2012 (which represents learning from school year 2011-2012 prior to testing) and before.

  35. Resources Available • Complete list of all schools and their ranking • Individual school look-up to see your school’s results • Overview presentation with voice over • FAQ • Business rules by which the rankings were calculated • Complete data file and validation file You can access these resources at www.mi.gov/ttb You can also request individual assistance by calling the Evaluation, Research and Accountability unit at 517-373-1342, or emailing mde-accountability@michigan.gov

  36. Priority Schools Identifying

  37. Schools in the bottom 5% of the Top to Bottom ranking. • MDE will ensure that the number of schools identified as Priority schools is equal to at least 5% of the state’s Title I schools as Priority schools. • In addition to the bottom 5%, MDE will also add any school with a graduation rate of less than 60% for three consecutive years, and any Tier I or Tier II school using SIG funds to implement a turnaround model. What are priority schools?

  38. Based on the Top-to-Bottom (TTB) ranking methodology which includes data from • Achievement results • Improvement results • Achievement gap results • All tested subjects are included in the ranking where possible for a school. • A ranking is determined for all open schools with 2 years of data for • 30 or more students • two or more tested subjects How are they determined?

  39. For a school to exit priority school status, it has to receive a Green, Lime, Yellow or Orange on the Accountability Scorecard at the close of its third year in the priority school intervention. • A school must either meet aggressive proficiency targets (toward 85% of students proficient by 2022), or must have demonstrated significant improvement. • The proficiency and/or improvement gains must be demonstrated all nine traditional ESEA subgroups as well as in the new “bottom 30%” subgroup. How does a school EXIT Priority School status?

  40. This means that a priority school who achieves a Green, Lime, Yellow or Orange on the Accountability Scorecard and exits priority status has: • Met all interim measurements of progress for priority schools (approved plan, leading and lagging indicators). • Met proficiency and/or improvement targets on average as a school. • Increased the proficiency rate of all traditional subgroups • Increased the proficiency rate of their very lowest performing students. How does a school EXIT Priority School status?

  41. Focus Schools Identifying

  42. What is a Focus School? • Schools with the largest achievement gaps in scale score between the top 30% of students and bottom 30% of students within a school • Focus schools may have high average performance overall, but have a significantly large gap, suggesting struggles addressing low achieving students

  43. What is the purpose of Focus School designation? • Identifying Focus Schools is a critical component to closing achievement gaps within schools and statewide • Focus schools highlight where changes in teaching and learning practices need to be undertaken to respond to the learning needs of low-achieving students. • These changes are difficult, and both accountability and support need to be differentiated.

  44. How was our school identified as a Focus School? • Top-to-Bottom list includes a component that examines the gap in achievement scores between top 30% and bottom 30% of students within a school • Gaps are standardized between all students using a common assessment within a school, and then averaged for the school • Gaps are calculated for all subject areas and combined

  45. How were Focus schools identified (continued)? • Average gap is then standardized and ranked among all schools • 10% with largest gaps are identified as Focus Schools

  46. Common concerns about this metric Z-scores compare schools • Are Focus schools only high-achieving schools? • Are Focus schools only high socioeconomic status schools? • Is the bottom 30% subgroup in Focus schools actually high performing? • Are schools more likely to be Focus schools if they have [fill in the blank group] kids?

  47. Not just high achieving...

  48. Not just high (or low) socioeconomic...

  49. Bottom 30% are not high achieving ... Across all subject areas and grade levels, the bottom 30% subgroup consistently had average achievement z-score below zero, and most of them are between -0.5 and -1.5. (This example: E/MS Reading) Top 30% Bottom 30%

More Related