Performance Funding Ideas in Education

1. Performance Funding Ideas in Education September 13, 2011 meeting Paul Teske Dean, SPA

2. Why? What is the purpose of thinking about performance funding? • Quasi-business model • Incentivize high performance and reward it (and punish failure?) • Easy to toss out the idea, harder to implement • Failure – businesses disappear, districts and schools probably can not go away • Do we expect student achievement and other measures to improve, and how quickly? Paul Teske, Dean SPA

3. What is the purpose of thinking about performance funding? (2) • What is the logic model? • Districts work harder? Smarter? • Hire different people? • Or, is it part of building an overall culture of performance and accountability? • And, compared to what? – current system rewards districts for kids in seats on October 1 Paul Teske, Dean SPA

4. Who is doing this in education? • Teacher performance pay (district level) • Many failed experiments in 1980s • Round 2.0 – Denver Procomp, Eagle, JeffCo, TIF grants, many others • Issues for teacher pay rewards • individual v. group • bonus v. base pay • Based upon single measure v. multiple

5. Who is doing this in education? (2) • State incentives to districts (or schools) • A few recent experiments or proposals • Montana • Michigan Governor has proposed • Irony – many states doing it with an eye to save money, during budgetary shortfalls, when it should probably be done in times of budgetary prosperity

6. Who is doing this in education? (3) • State Higher Education • Up to 43 states claim to have some version of performance rewards for higher ed institutions • CO is one of those – performance contracts with CCHE • Research studies show no real outcomes changes from these plans, most of which are minimal • Tennessee has done a version of this for 20 years, rewards for graduation, retention, etc • Research studies show little outcome changes there too • Only about 5-6% of state money was related to performance – in a new plan 100% will be tied to performance targets • Indiana

7. Performance funding for inputs, outputs and/or outcomes • Have been funding inputs, student count • based upon Oct 1 count • Outputs are what the system does • For example, teacher induction, professional development, teacher evaluations, programs • Outcomes are the differences made by the system – student achievement most importantly – could also be grad rates, workforce and college readiness

8. Performance funding for inputs, outputs and/or outcomes (2) • Could be a mix • Essentially all money today is for inputs • Could get more serious about WSF adjustments • Not clear we want to fund outputs • Outcomes – student achievement, graduation

9. Who receives the performance funding money ? • Districts, schools, teachers, families, or students? In what form? • What/who are we incentivizing? • Do we include charters, magnets, innovation schools if districts or schools are getting the extra funds? • Do districts decide how to push bonus money down to schools? • What would funding students mean? Could give each child a “pot” of money for their P12 career and let the family decide how to spend it • Lots of problems - use it up, etc.

10. What are they rewarded for? • Do the incentivized parties believe that they can influence the outcomes? • What combination of outcome measures makes sense ? • Too few puts too much reliance on measures that may not be reliable, valid, full of integrity • Too many may obscure performance accountability and increase complexity (e.g., ProComp has a lot) • Sweet spot in-between ? • Perhaps weighted index, like the CDE performance framework (which is somewhat arbitrary)

11. Confidence in the reliability, validity, and integrity of the measures? • Student test score growth is the one measure most agreed upon – but poorly measured right now – annual CSAPs. • Theoretically, hard to accurately ascribe “student value added” to teachers, schools – partly due to small numbers • Somewhat more valid at district level • Higher stakes for measures, more cheating

12. How big is the performance funding? • If it is too small, key actors will probably ignore it • HR studies of incentive pay suggest need for 10% or more in play to be meaningful to employees • If it is too big, stakes will be high • Possible gaming/cheating on measures • winners/losers will be extreme dollar differences • “Goldilocks” – just right, in between

13. Possible Outcome Example • What if we pay CO districts for increasing their number of graduates (compared to prior years or a predicted rate)? • Assume PPOR now $8,000 • Say $1,000 bonus per every additional graduate (12%) • About 20,000 senior dropouts per year in CO • If all graduated, this would reward districts with $20 million in new performance funding (less than ProComp mill levy ) • But, districts would presumably have to spend non-trivial resources getting these students to graduate

14. Possible Output Example • Use new state money to fund implementation of good district-level teacher pay for performance plans, tied to evaluation systems (SB 191) • Competitive application process from districts to CDE • Like a state innovation fund • Could be established for other outputs that state values • In essence, not rewarding demonstrated student performance by districts, but trying to incent them to change their way of doing business • How do you make sure districts deliver a good program?

15. Input Example: (How) does this relate to factors already in the SFA? • Do we give districts a lot more for the harder to teach students? (adjustments to base) • Does rewarding outputs or outcomes also take account of these elements? (e.g., do we reward districts more for graduating an at-risk student?) • Does it just add on?

16. What if high SES districts/schools are mostly the winners? • Colorado performance framework rewards growth more than status (arbitrary percentage, but based upon legislative guidance). • While status is highly correlated with higher SES of students, early evidence suggests that growth is too. • Should we get concerned if the schools/districts that are “winners” are mostly high SES? (e.g., is this a tool for everyone, or more for lower income districts?)

17. What to do with losers? • In a symmetric system, there will be losers. In the business marketplace they disappear (theoretically, at least, unless “too big to fail”). • We may not want districts or schools to disappear • What are the costs if schools, perhaps districts, continue to perform poorly, get their funding cut, and spiral down even more? (but this can and does happen today, anyway, in other ways) • What are the “turnaround” scenarios – are they the same or different than today?

18. Or, do you just fund “upside”? • Colorado’s own internal Race to the Top ? • No losers, if you create a new pot of money for those districts and/or schools doing better over time? • The politics of this are much better (ProComp opt-in) • The equality notions of this might be much better • Or, could be a opt-in system, of risk/reward? • Doesn’t penalize bad or flat performance, except that you don’t get extra resources

19. What do we expect winners to actually do with the extra money? • Just keep doing a better job? • Put resources into better teaching? • Reward the key actors with more money – superintendent, principals, teachers, other employees ? • Expand their market somehow? • Makes more sense in higher ed – get more students • K12 - take over nearby loser districts? • Create competitive schools to attract more students from outside the district? What else?

20. Growth v. status rewards: penalize those currently doing well? • If you start rewarding from today’s baseline, in a sense you penalize districts presently doing a better job graduating kids or achieving high • Another idea would be to figure out an “expected, predicted” rate of dropouts, based upon FRL eligibility and perhaps other student factors, from a model across the state – and reward districts for above predicted performance, not compared to their own performance last year

21. How to prevent gaming/cheating of system with high stakes? • Evidence shows that, with high stakes tests, there will be gaming and cheating. It can be reduced through good implementation of tests, but that costs money. • How much “teaching to a test” is too much, if the test captures what we think is real learning? • If we focus on graduation rates, does graduation get “watered down” (e.g., North HS credit recovery program)

22. How to insure separate, continuing pot of money ? • Otherwise, sure failure • From teacher pay to other types of performance funding, if the money is zero-sum, and if the timing happens when resources are already being cut (like today), we may not get the hoped-for results, and many unintended consequences are likely. • If money is put aside, but then disappears in a year or two, because of budget problems, the key actors no longer trust the process, and it likely won’t work going forward.

23. How long to reward district/school – one year, more? • Are these like annual bonuses, or are they built into base funding? • Bonuses reward performance from last year, which is timely. Base funding allows a longer term perspective. • How do you address the fact that, to produce a graduate might require interventions in sixth grade, that aren’t rewarded until 6 years later?

24. Summary (1) • More questions than answers • So far, little evidence of success from other jurisdictions • Lots of 2.0 versions in higher ed and maybe K12 • Do you want only “upside” rewards? • Who, how, how long do you provide rewards?

25. Summary (2) • What do you incentivize? • What will winners do with the new money? • Is it less about incentives and more about creating a climate of performance expectations?