Overview of the ProComp Evaluation

1. Overview of the ProComp Evaluation What to talk about when you can’t talk results. Robert Reichardt September 14, 2011

2. Overview • DPS background • Improving human capital • ProComp background • Evaluation questions • Data sources • Others have found • Questions?

3. DPS Background • 2nd largest district in the state • 72,000 in 2005, 78,000 in 2010 • Largest urban district: 72% FRL, 80% minority in 2010 • 178 schools in 2010 • 4,500 teachers • Multiple Reform initiatives • ProComp in 2006 • Portfolio district in (about) 2008 • 32 existing charter plus 16 new in 2010 • Alternative Routes into teaching, expanded in 2008

4. Improved Student Outcomes Improving Human Capital Exit In-effective Teachers Retain Effective Teachers Recruit Effective Teachers Teacher Human Capital Improve Effectiveness Through Professional Learning

5. ProComp • Negotiated between DPS and the Union (DCTA) • Four year Pay-for-Performance Pilot • Applies to all DCTA members: essentially all school professionals except administrators • Funded by a taxpayer approved $25 million fund • Implemented in 2006-07 • Optional for existing teachers • Mandatory for all hired after Jan 1, 2006 • Changed in 2009, moving incentives from senior to new teachers

6. Alternative Compensation • Traditional Compensation System: • Increases in pay for experience and education • Easy to administer • Transparent and ‘fair’ • Alternative Compensation • Incentive to work harder • Incentive to work smarter • Incentives to change the workforce composition • A different definition of ‘fair’ • Expectancy Theory and Incentives, to work people must • Know about the incentive • Know how to get the incentive • Be motivated by the incentive

7. ProComp Structure * Base-building ** Base building for the first 14 years of experience *** Base building if a teacher accomplishes 2 SGOs, not base-building if only one SGO is accomplished.

8. ProComp Evaluation • Two evaluations • Internal: conducted by Ed Wiley and colleagues at CU Boulder • External: conducted by Proctor & Reichardt at CU Denver and Goldhaber at UW • External evaluation focused on: • Has ProComp affected student performance? • Has ProComp affected recruitment and retention? • Has a sustainable system been implemented?

9. Main Data Sources • For achievement analysis: • Linked student teacher records within DPS from 2005-6 through 2009-10 (and earlier in some cases) • For recruitment and retention: • Statewide linked teacher records 2004-05 through 2009-10 • Survey of new graduates of traditional teacher preparation programs • For implementation • Teacher and principal surveys in 2009 and 2010 • Interviews in 15 schools in 2009 • Interviews with district administrators and stakeholders

10. What Have Others Found • Evaluation of the Pilot found • Student Growth Objectives are associated with improved student performance • Wiley et al: • Over the period of ProComp implementation teacher effectiveness has increased • Over the period of ProComp retention has increased • Those in ProComp have different attitudes about ProComp then those who did not join

11. Important questions to consider • What does it take to implement these types of changes? • Systems infrastructure • Sustainability • What environmental supports were there for this innovation? • Earlier reforms • Time & money • Local control • How do professionals get better at their work? • Individually or together? • What implications does this have for resource allocation? • How should systems be structured to support this innovation?

12. Questions? Results will be available next month. Thank You Robert Reichardt Robert.reichardt@ucdenver.edu