2011 Top to Bottom Ranking

1. Understanding How the Ranking is Calculated 2011 Top to Bottom Ranking

2. The statewide top-to-bottom ranking takes into account both student achievement on state tests and graduation rates. Student achievement on state tests is included in the statewide top to bottom ranking in the following three ways: Achievement at the elementary, middle, and high school levels Improvement in achievement over time The largest achievement gap between two subgroups calculated based on the top scoring 30% of students versus the bottom scoring 30% of students Top to Bottom (TTB) Ranking Rationale for the ranking and some background information that will put it into perspective. So, one critical piece to understand is that in the national and statewide discussion about school performance, the focus has shifted from identifying schools who fail to meet a set of criteria to identifying the schools that are persistently low performing, in order to target resources and interventions more specifically to those schools most in need of assistance.� To help identify these schools, the Michigan Department of Education, or MDE, developed a statewide Top to Bottom ranking.� All k-12 public schools are ranked based on student proficiency, student improvement, and achievement gaps between students within a school.� A school with a high ranking is one that has a high level of proficiency, is improving over time, and is ensuring that all students are learning and achieving at a high level.� This ranking helps to identify those low performing schools that would benefit from additional assistance. It also provides important information to all schools regarding their performance and how it compares to other schools similar to themselves. � So let�s get into the components and the calculations behind the ranking. Rationale for the ranking and some background information that will put it into perspective. So, one critical piece to understand is that in the national and statewide discussion about school performance, the focus has shifted from identifying schools who fail to meet a set of criteria to identifying the schools that are persistently low performing, in order to target resources and interventions more specifically to those schools most in need of assistance.� To help identify these schools, the Michigan Department of Education, or MDE, developed a statewide Top to Bottom ranking.� All k-12 public schools are ranked based on student proficiency, student improvement, and achievement gaps between students within a school.� A school with a high ranking is one that has a high level of proficiency, is improving over time, and is ensuring that all students are learning and achieving at a high level.� This ranking helps to identify those low performing schools that would benefit from additional assistance. It also provides important information to all schools regarding their performance and how it compares to other schools similar to themselves. � So let�s get into the components and the calculations behind the ranking.

3. In addition to the achievement components, student graduation is included in the statewide top-to-bottom ranking for schools with a graduation rate in the two following ways: Graduation Rate Improvement in graduation rate over time TTB Ranking

4. Schools with 30 or more full academic year (FAY) students tested over the last two years in at least two state-tested content areas: Mathematics Reading Science Social Studies Writing Who receives a ranking?

6. Quick Reference for Z-Scores What is a Z-Score?

7. Why do We Use Z Scores? Z-scores are a standardized measure that helps you compare individual student (or school) data to the state average data (average scores across populations). Z-scores allow us to �level the playing field� across grade levels and subjects Each Z-score corresponds to a value in a normal distribution. A Z-Score will describe how much a value deviates from the mean.

8. Z-Score �Cheat Sheet� Student z-score = (Student Scale Score) � (Statewide average of scale scores) Standard Deviation of Scale Score School z-score= (School Value) � (Statewide average of that value) Standard deviation of that value Z Score Summary PowerPoint and Business Rules- http://www.michigan.gov/mde/0,4615,7-140-37818_56562---,00.html

9. Z-Score �Cheat Sheet� Z-scores are centered around zero Positive numbers mean the student or school is above the state average Negative numbers mean the student or school is below the state average

10. Z-Score Examples Your school has a z-score of 1.5. You are better than the state average.

11. Z-Score Examples Your school has a z-score of .2. You are better than the state average, but not by a lot.

12. Z-Score Examples Your school has a z-score of -2.0. You are very far below state average.

13. For grade 3-8 reading and mathematics How Is the Top to Bottom Ranking Calculated The brackets show how the index is created for reading and mathematics in grades 3-8. � As shown at the top left, a standardized scale score is created for each student taking a test. A standardized scale score comes from calculating the student z-score of each student against all students statewide who take the same test in the same grade level in the same year. This assures that each student is compared only to students taking the same type of test in the same year and grade level. The average standardized scale score for each school is then calculated across the two most recent years. � Following the arrow to the right, a school achievement z-score is calculated by comparing the school�s two-year average standardized scale score to all other schools in the state on that content area. That quantity is then multiplied by � to contribute to the overall school index in the content area. � As shown in the middle box on the left side of the brackets, a two-year average performance level change index is calculated using scores displayed in the chart on the next slide. For each the school�, the performance level change scores are summed across students and an average is taken to create the two-year average performance level change index. The index for each school is then compared to the rest of the schools in the state to create a school-level performance level change z-score. That z-score is then multiplied by � to contribute to the overall school index in the content area. � Finally, as shown on the bottom left of Figure 1, a two-year average bottom 30% minus top 30% z-score gap is created by obtaining the average z-scores of the bottom 30% of z-scores in the school and subtracting from that the average of the top 30% of z-scores in the school. This gives a negative number which when compared to all schools in the state assures that schools with the highest achievement gap receive the lowest z-scores as intended. The school z-score for achievement gap is then multiplied by 1/4 to contribute to the overall school index in the content area. � Combining these elements together creates a school content area index. This is then translated into a final z-score, in order to compare that school�s content area index to other elementary/middle schools or other high schools. The brackets show how the index is created for reading and mathematics in grades 3-8. � As shown at the top left, a standardized scale score is created for each student taking a test. A standardized scale score comes from calculating the student z-score of each student against all students statewide who take the same test in the same grade level in the same year. This assures that each student is compared only to students taking the same type of test in the same year and grade level. The average standardized scale score for each school is then calculated across the two most recent years. � Following the arrow to the right, a school achievement z-score is calculated by comparing the school�s two-year average standardized scale score to all other schools in the state on that content area. That quantity is then multiplied by � to contribute to the overall school index in the content area. � As shown in the middle box on the left side of the brackets, a two-year average performance level change index is calculated using scores displayed in the chart on the next slide. For each the school�, the performance level change scores are summed across students and an average is taken to create the two-year average performance level change index. The index for each school is then compared to the rest of the schools in the state to create a school-level performance level change z-score. That z-score is then multiplied by � to contribute to the overall school index in the content area. � Finally, as shown on the bottom left of Figure 1, a two-year average bottom 30% minus top 30% z-score gap is created by obtaining the average z-scores of the bottom 30% of z-scores in the school and subtracting from that the average of the top 30% of z-scores in the school. This gives a negative number which when compared to all schools in the state assures that schools with the highest achievement gap receive the lowest z-scores as intended. The school z-score for achievement gap is then multiplied by 1/4 to contribute to the overall school index in the content area. � Combining these elements together creates a school content area index. This is then translated into a final z-score, in order to compare that school�s content area index to other elementary/middle schools or other high schools.

14. A weighted composite of individual student performance level change is used to calculate improvement in grades 3-8 reading and mathematics Rewards large improvements more heavily, rewards maintenance of proficiency if a student was already proficient Weighted Performance Level Change The weighted performance level change metric is designed to heavily reward significant improvements, reward improvements, reward maintenance of performance level for students who were already proficient, and disincentivize all declines and significant declines. A ceiling clause is also implemented here such that any student who declines in performance level but remains in the top performance level can be considered to have maintained his or her performance level.The weighted performance level change metric is designed to heavily reward significant improvements, reward improvements, reward maintenance of performance level for students who were already proficient, and disincentivize all declines and significant declines. A ceiling clause is also implemented here such that any student who declines in performance level but remains in the top performance level can be considered to have maintained his or her performance level.

15. For science, social studies, writing, and grade 11 all tested subjects How Is the Top to Bottom Ranking Calculated This slide shows how the index is created for science, social studies, writing in all grades; and for reading and mathematics in grade 11. The only difference between this slide and the previous slide is that rather than performance level change, a four-year achievement trend slope is calculated by regressing two-year average z-scores on school year. This improvement slope is then compared to the improvement slopes for all other schools to derive a school performance achievement trend z-score, which is then multiplied by � to contribute to the overall school index in the content area. � Combining these elements together creates a school content area index. This is then translated into a final z-score, in order to compare that school�s content area index to other elementary/middle schools or other high schools. This slide shows how the index is created for science, social studies, writing in all grades; and for reading and mathematics in grade 11. The only difference between this slide and the previous slide is that rather than performance level change, a four-year achievement trend slope is calculated by regressing two-year average z-scores on school year. This improvement slope is then compared to the improvement slopes for all other schools to derive a school performance achievement trend z-score, which is then multiplied by � to contribute to the overall school index in the content area. � Combining these elements together creates a school content area index. This is then translated into a final z-score, in order to compare that school�s content area index to other elementary/middle schools or other high schools.

16. For graduation rate How Is the Top to Bottom Ranking Calculated This slide shows how the index for graduation rate is created for schools that have a graduation rate. � Starting at the top left of the schematic, the two-year average graduation rate is calculated, and compared to all other schools� graduation rates to create a school graduation rate z-score. That z-score is multiplied by 2/3 to contribute to the school graduation rate index. Moving to the bottom left of the schematic, a four year graduation rate trend slope (or annual improvement rate) is calculated by regressing graduation rate on year. That slope is then transformed into a z-score by comparing the school�s slope to the slopes of all other schools. That graduation rate trend z-score is then multiplied by 1/3 to contribute to the school graduation rate index. This slide shows how the index for graduation rate is created for schools that have a graduation rate. � Starting at the top left of the schematic, the two-year average graduation rate is calculated, and compared to all other schools� graduation rates to create a school graduation rate z-score. That z-score is multiplied by 2/3 to contribute to the school graduation rate index. Moving to the bottom left of the schematic, a four year graduation rate trend slope (or annual improvement rate) is calculated by regressing graduation rate on year. That slope is then transformed into a z-score by comparing the school�s slope to the slopes of all other schools. That graduation rate trend z-score is then multiplied by 1/3 to contribute to the school graduation rate index.

17. Calculating a four-year slope (e.g., graduation rate) How Is the Top to Bottom Ranking Calculated The following three slides demonstrate how a four year slope is calculated. First, you plot the school�s values from each year. In this slide, graduation rate is highlighted. The following three slides demonstrate how a four year slope is calculated. First, you plot the school�s values from each year. In this slide, graduation rate is highlighted.

18. Calculating a four-year slope (e.g., graduation rate) How Is the Top to Bottom Ranking Calculated Next, you plot a linear regression line through the plots. Next, you plot a linear regression line through the plots.

19. Calculating a four-year slope (e.g., graduation rate) How Is the Top to Bottom Ranking Calculated The final step is to calculate the slope of that line. This gives you the school�s annual improvement rate. In the example shown, the school has a slope of 2.3%, which means they have improved an average of 2.3% each year of the four previous years.The final step is to calculate the slope of that line. This gives you the school�s annual improvement rate. In the example shown, the school has a slope of 2.3%, which means they have improved an average of 2.3% each year of the four previous years.

20. Calculating an overall ranking for a school with a graduation rate How Is the Top to Bottom Ranking Calculated All of the standardized school content area indices and the standardized graduation rate index, where applicable, are then combined to create an overall school index as shown in the slide above. � To create an overall standardized school index, the school standardized graduation rate index is multiplied by 10% to contribute to the overall index. The remaining 90% is equally divided among the content areas for which the school has an index. In the case presented above, for example, the remaining 90% is divided five ways to account for the five content areas in which the school has an index. All of the standardized school content area indices and the standardized graduation rate index, where applicable, are then combined to create an overall school index as shown in the slide above. � To create an overall standardized school index, the school standardized graduation rate index is multiplied by 10% to contribute to the overall index. The remaining 90% is equally divided among the content areas for which the school has an index. In the case presented above, for example, the remaining 90% is divided five ways to account for the five content areas in which the school has an index.

21. Calculating an overall ranking for a school without a graduation rate How Is the Top to Bottom Ranking Calculated For schools without a graduation rate, 100% of the overall school index is divided equally among the content areas for which the school has an index. For example, in the case of a school that has assessment results in five content areas, each would be weighted 20% For schools without a graduation rate, 100% of the overall school index is divided equally among the content areas for which the school has an index. For example, in the case of a school that has assessment results in five content areas, each would be weighted 20%

22. Calculating an overall ranking for a school without a graduation rate and without a writing score How Is the Top to Bottom Ranking Calculated In the case of a school with results from four content areas, each would be weighted 25%.In the case of a school with results from four content areas, each would be weighted 25%.

23. Which years of data are in the ranking?

24. Michigan tests in the fall. These fall tests reflect the learning of students in the previous school year. For elementary and middle schools

25. Michigan tests in the spring The spring test (MME and MI-Access) measures what students have learned from grades 9, 10 and grade 11 prior to the MME testing. For High Schools

26. For elementary/middle schools: Performance on the MEAP and MI-Access tests in fall 2010 (which represents learning from school year 2009-2010) and before For high schools: Performance on the MME and MI-Access tests in spring 2011 (which represents learning from school year 2010-2011 prior to testing) and before. What do the 2011 Rankings reflect?

27. For elementary/middle schools: Fall 2010 MEAP & MI-Access For high schools Spring 2011 MME & MI-Access When is performance in 2010-2011 measured?

28. For elementary/middle schools: Fall 2011 MEAP, MI-Access & MEAP-Access For high schools Spring 2012 MME & MI-Access When is performance in 2011-2012 measured?

29. TTB vs PLA How is this different From the PLA list?

30. This list represents a ranking of all schools in the state of Michigan, using our preferred methodology developed in collaboration with many stakeholders. MDE also published, according to state statute, a list of Persistently Lowest Achieving schools. This is the PLA list. The PLA list of schools was generated by a set of federally-approved and required rules that differ from our Top to Bottom ranking. TTB versus PLA

31. Why are the lists different?

32. Resources to Understand my Ranking

33. Complete list of all schools and their ranking Individual school look-up to see your school�s results Overview presentation with voice over FAQ Business rules by which the rankings were calculated Complete data file and validation file You can access these resources at: http://www.michigan.gov/baa Click on �Michigan Schools Top to Bottom Ranking� You can also request individual assistance by calling the Evaluation, Research and Accountability unit at 517-373-1342, or emailing mde-accountability@michigan.gov Resources Available