Dropout Summit Policy Issues - H

1. Dropout SummitPolicy Issues - H Dr. Marco Muñoz, Evaluation Specialist Jefferson County Public Schools Lillian Pace, Legislative Director Congressman John Yarmuth Office Kathy Zandona, Director of Education Greater Louisville, Inc.

2. Policy Issues - H H-24. Federal Accountability should be based on student improvement (Growth Model) instead of Current Status (AYP) measures. H-25. Supplemental Educational Services (SES) should be open to School Districts to provide and should be accountable for the same accountability standards held for schools. H-26. All state tests should have concurrent validity with the National Assessment of Educational Progress (NAEP) HIGH SCHOOL DROPOUT SUMMIT

3. Issue H-24 Federal Accountability should be based on student improvement (Growth Model) instead of Current Status (AYP) measures. HIGH SCHOOL DROPOUT SUMMIT

4. Current Status Model • Such models set threshold levels of performance--% of students scoring at a proficient /above level on state assessments of reading and math: • As a group and as sub-groups (LEP, ECE, Race, SES) • Set each year from 2002 to 2014 so that they lead to 100% proficient or above in 2014 • Safe Harbor provision—% of students who are NOT at proficiency level, but declines by at least 10% HIGH SCHOOL DROPOUT SUMMIT

5. Our Policy Proposal: Growth-Based Accountability Models • Measure student achievement over time • Greater flexibility in tracking individual students’ annual progress • Following the examples of: • 2005-06 : Tennessee and North Carolina (2) • 2006-07: Alaska, Arizona, Arkansas, Delaware, Florida, Iowa, and Ohio (7) • 2007-08 Proposals: District of Columbia, Michigan, Minnesota, Missouri, New Mexico, and Pennsylvania (6) HIGH SCHOOL DROPOUT SUMMIT

6. Elements of the Policy Proposal • The primary focus shifts to the change/growth in achievement from the previous year • Kentucky should evaluate success of their schools and districts in meeting student achievement goals by measuring students’ progress from year to year. • The modern concept of “value-added” growth model—accountable for the value they “add” to students’ starting points! HIGH SCHOOL DROPOUT SUMMIT

7. Support from Research • Inferences about school effectiveness from differences in student test performance at a fixed point in time are “scientifically indefensible” (Linn, 2006; Raudenbush, 2004) • Current status on achievement tests is “contaminated with factors other that school performance, in particular the average level of achievement prior to entering first grade – average effects of student family and community characteristics on student growth from first grade through the grade in which the student is tested (Myers, 2000). HIGH SCHOOL DROPOUT SUMMIT

8. Issue H-25 Supplemental Educational Services (SES) should be open to School Districts to provide and should be accountable for the same accountability standards held for schools. HIGH SCHOOL DROPOUT SUMMIT

9. Setting the Stage • At the National level, given the current policy debate associated with the reauthorization of NCLB, it is important to make data-informed policy-making. • At the District level, it is important to assess the extent to which SES programs can demonstrate positive effects on reading and mathematics using appropriate measurement HIGH SCHOOL DROPOUT SUMMIT

10. NCLB SES Overall Impact • Reading: • Program effect [F=1.69, p=.19] indicating non-significant differences between SES and comparison students. • Mathematics: • Program effect was not significant [F=1.52, p=.22] between SES and comparison students. HIGH SCHOOL DROPOUT SUMMIT

11. Overall lessons learned • One-on-one tutoring WITH certified personnel AND knowledge of research-based instructional practices • Providers with “proven” educational programs • Alignment to state/district curriculum (CCA 4.1) • Alignment to statewide assessment (KCCT) • On-going, two-way communication between classroom teachers and service providers • Link between intervention and everyday classroom practices (Herman, 2006). • Implementation fidelity checks—walk the talk! HIGH SCHOOL DROPOUT SUMMIT

12. Looking ahead JCPS’s Support Branding Characteristics: • System perspective with feedback-loops • Training/monitoring for impact on student learning • High levels of communication Explore using JCPS as a Service Provider: • Results might be different for students who select JCPS as a provider instead of outside organization (Los Angeles Unified) HIGH SCHOOL DROPOUT SUMMIT

13. Issue H-26 All state tests should have concurrent validity with the National Assessment of Educational Progress HIGH SCHOOL DROPOUT SUMMIT

14. Concurrent Validity • Validity refers to the accuracy of the interpretations of test scores • Concurrent validity is a type of validity evidence--based on relations to other tests • KCCT (state) and NAEP (national) tests • Converge evidence of the validity involves correlating a test with existing tests that measure similar constructs (e.g., SAT vs. ACT) HIGH SCHOOL DROPOUT SUMMIT

15. NAEP and the Variety of State Assessments • NAEP and state assessments measure different content—there is a lack of curriculum alignment. • Gaps measured in terms of percent proficient or above on state assessments could be quite misleading due to the wide variation in the stringency of state definitions of the proficient performance standard (Hall & Kennedy, 2006; Peterson & Hess, 2006). HIGH SCHOOL DROPOUT SUMMIT

16. NAEP vs. State Assessments • 33 states that had 2005 grade 8 mathematics assessments • Percentages proficient or above on the 2005 grade 8 NAEP math assessment for 33 states • The 25 point (from 13% to 25%) range of percentages in NAEP is clearly much smaller than the 71 point range (from 16% to 87%) for the state assessment results (Linn, 2005) HIGH SCHOOL DROPOUT SUMMIT

17. HIGH SCHOOL DROPOUT SUMMIT

18. HIGH SCHOOL DROPOUT SUMMIT

19. Another Example Comparing State vs. NAEP Assessment Results • The graph shows that 4th grade students in Massachusetts who score at or above proficient on their state math assessment would also score at or above NAEP's proficient achievement level, while students in Tennessee may score proficient on their state assessment but score below basic on NAEP. HIGH SCHOOL DROPOUT SUMMIT

20. HIGH SCHOOL DROPOUT SUMMIT

21. Contact Information Dr. Marco Muñoz, Evaluation Specialist Jefferson County Public Schools marco.munoz@jefferson.kyschools.us Lillian Pace, Legislative Director Congressman John Yarmuth Office lillian.pace@mail.house.gov Kathy Zandona, Director of Education Greater Louisville, Inc. kzandona@greaterlouisville.com HIGH SCHOOL DROPOUT SUMMIT