Local, national and international benchmarking for improvement

1. Programme for International Student Assessment Local, national and international benchmarking for improvement First results from the OECD test for schools based on PISA School Board Meeting Fairfax, VA 4 April 2013 Alejandro Gomez Palma Policy Analyst / PISA Instrument Development Directorate for Education and Skills

2. In the dark all schools and school systems look the same… But with a little light….

3. In the dark all schools and school sytems look the same… But with a little light…. …important differences become apparent….

4. A worldofchange in the global talentpoolApproximatedbypercentageofpersonswithhighschoolorequivalentqualficationsin theagegroups 55-64, 45-55, 35-44 und 25-34 years % 1 13 1 27 1. Excluding ISCED 3C short programmes 2. Year of reference 2004 3. Including some ISCED 3C short programmes 3. Year of reference 2003.

5. PISA countries in 2003 2000 2001 2006 2009 1998 Coverage of world economy 83% 77% 81% 85% 86% 87%

6. Mathematics in PISA The real world The mathematical World Making the problem amenable to mathematical treatment A mathematical model A model of reality Understanding, structuring and simplifying the situation Using relevant mathematical tools to solve the problem A real situation Validating the results Mathematical results Real results Interpreting the mathematical results

7. High reading performance Average performanceof 15-year-olds in reading – extrapolate and apply Woodson High School (W.T.), serves middle-class students Langley High School, serves upper-class neighbourhoods Performance distribution in US 18% do not reach baseline Level 2 (16% when excluding immigrants) (Finland 6%, Canada 9%) • Economic cost: 72 trillion $ 10% are top performers (Shanghai 20%) North Star Academy College Preparatory High School, serves students in disadvantaged neighbourhoods (social context like in Chile, Portugal or Shanghai) Northeast Suburban schools Midwest Not just about poor schools in poor neighbourhoods West Urban schools South … 17 countries perform below this line Low reading performance

8. High reading performance Average performanceof 15-year-olds in reading – extrapolate and apply Woodson High School (W.T.), serves middle-class students Langley High School, serves upper-class neighbourhoods Reading Pilot Results PISA 2009 Results Schools in theUnited States Schools in theShanghai-China Schools in Mexico Performance distribution in US 18% do not reach baseline Level 2 (16% when excluding immigrants) (Finland 6%, Canada 9%) • Economic cost: 72 trillion $ 10% are top performers (Shanghai 20%) North Star Academy College Preparatory High School, serves students in disadvantaged neighbourhoods (social context like in Chile, Portugal or Shanghai) 10% above Northeast Suburban schools 25% above Midwest 10% above 50% above/below 25% above West 25% below Urban schools 50% above/below Performance on PISA scale 10% above South 10% below 25% below 25% above 10% below 50% above/below 25% below 10% below … 17 countries perform below this line Low reading performance

9. High reading performance Average performanceof 15-year-olds in reading – extrapolate and apply Woodson High School (W.T.), serves middle-class students Langley High School, serves upper-class neighbourhoods Performance distribution in US 18% do not reach baseline Level 2 (16% when excluding immigrants) (Finland 6%, Canada 9%) • Economic cost: 72 trillion $ 10% are top performers (Shanghai 20%) Northeast Suburban schools Midwest West Urban schools South … 17 countries perform below this line Low reading performance

10. High reading performance Average performanceof 15-year-olds in science – extrapolate and apply High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low reading performance

11. High reading performance 2009 Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low reading performance

12. High reading performance 2009 Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low reading performance

13. High performing systems often prioritize the quality of teachers over the size of classesContribution of various factors to upper secondary teacher compensation costsper student as a percentage of GDP per capita (2004) Percentage points

14. High reading performance 2009 Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low reading performance

15. High reading performance 2000 Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low reading performance

16. High reading performance 2000 - 2009 Durchschnittliche Schülerleistungen im Bereich Mathematik High average performance Large socio-economic disparities High average performance High social equity Strong socio-economic impact on student performance Socially equitable distribution of learning opportunities Low average performance Large socio-economic disparities Low average performance High social equity Low reading performance

17. Changes in performance by type of task Increase percentage correct Japan Japan OECD OECD

19. Learning from the World’s top performers

20. A commitment to education and the belief that competencies can be learned and therefore all children can achieve • Universal educational standards and personalisation • … as opposed to a belief that students have different destinations to be met with different expectations, and selection/stratification as the approach to heterogeneity • Clear articulation who is responsible for ensuring student success and to whom Lessons from PISA on successful education systems

21. Clear ambitious goals that are shared across the system and aligned with high stakes gateways and instructional systems • Well established delivery chain through which curricular goals translate into instructional systems, instructional practices and student learning (intended, implemented and achieved) • High level of metacognitive content of instruction Lessons from PISA on successful education systems

22. Capacity at the point of delivery • Attracting, developing and retaining high quality teachers and school leaders and a work organisation in which they can use their potential • Instructional leadership and human resource management in schools and districts • Keeping teaching an attractive profession • System-wide career development Lessons from PISA on successful education systems Sahl, SIN

23. Incentives, accountability, knowledge management • Aligned incentive structures • Forstudents • How gateways affect the strength, direction, clarity and nature of the incentives operating on students at each stage of their education • Degree to which students have incentives to take tough courses and study hard • Opportunity costs for staying in school and performing well • For teachers • Make innovations in pedagogy and/or organisation • Improve their own performance and the performance of their colleagues • Pursue professional development opportunities that lead to stronger pedagogical practices • A balance between vertical and lateral accountability • Effective instruments to manage and share knowledge and spread innovation – communication within the system and with stakeholders around it • A capable administration with authority and legitimacy to act Lessons from PISA on successful education systems

24. School autonomy, accountability and student performanceImpact of school autonomy on performance in systems with and without accountability arrangements PISA score in reading • Posting achievement data publicly

25. Local responsibility and system-level prescription Trend in OECD countries Finland and other improvers today Every school an effective school System-level prescription ‘Tayloristic’ work organisation Schools today The industrial model, detailed prescription of what schools do Schools tomorrow? Building capacity Schools leading reform Teachers as ‘knowledge workers’

26. Lessons from PISA on successful education systems • Investing resources where they can make most of a difference • Alignment of resources with key challenges (e.g. attracting the most talented teachers to the most challenging classrooms) • Effective spending choices that prioritise high quality teachers over smaller classes CHN

27. A learning system • An outward orientation to keep the system learning, technology, international benchmarks as the ‘eyes’ and ‘ears’ of the system • Recognising challenges and potential future threats to current success, learning from them, designing responses and implementing these Lessons from PISA on successful education systems SIN

28. Coherence of policies and practices • Alignment of policies across all aspects of the system • Coherence of policies over sustained periods of time • Consistency of implementation • Fidelity of implementation (without excessive control) Lessons from PISA on successful education systems CAN

29. A tool for greater resolution for schools and local educatorsPISA for SchoolsIn the U.S.:OECD Test for Schools(based on PISA)

30. OECD Test for Schools (based on PISA)Uses of the assessment tool The test isnot… • A mandated standardised test • Intended to influence – in of itself – everyday teaching practices • An accountability tool or a tool for “rankings” or “league tables”

31. OECD Test for Schools (based on PISA)Overview of results from the pilot Schools in the U.S. that participated in PISA 2009 Reading Performance on PISA scale Pilot Schools in the U.S. Note: Size of bubbles is proportional to the number of students enrolled at the school. Socioeconomic status (ESCS)

32. OECD Test for Schools (based on PISA)Overview of results from the pilot Schools in the U.S. that participated in PISA 2009 Mathematics Performance on PISA scale U.S. average PISA 2009 Pilot Schools in the U.S. U.S. average PISA 2009 Note: Size of bubbles is proportional to the number of students enrolled at the school. Socioeconomic status (ESCS)

33. OECD Test for Schools (based on PISA)Overview of results from the pilot Schools with students from similar socioeconomic backgrounds Schools in the U.S. that participated in PISA 2009 Science U.S. average PISA 2009 Performance on PISA scale Pilot Schools in the U.S. U.S. average PISA 2009 Note: Size of bubbles is proportional to the number of students enrolled at the school. Socioeconomic status (ESCS)

34. OECD Test for Schools (based on PISA)Overview of results from the pilot Reading Pilot Results PISA 2009 Results Schools in theUnited States Schools in theShanghai-China Schools in Mexico 10% above 25% above 10% above 50% above/below 25% above 25% below 50% above/below Performance on PISA scale 10% above 10% below 25% below 25% above 10% below 50% above/below 25% below 10% below

35. OECD Test for Schools (based on PISA)Overview of results from the pilot Reading What does the same mean mean? 543 and 543 Level 1 and be`lowLevel 2 Level 3 Level 4 Level 5 Level 6

36. OECD Test for Schools (based on PISA)Overview of results from the pilot Content of reports I. Introduction: Understanding your school’s results What students in your school know and can do in Reading, Mathematics and Science Student engagement and the learning environment at your school (teacher-student relations, disciplinary climate, student confidence and attitudes towards mathematics and science) Your School Compared with similar schools in Your Country Your School’s Results in an international context How your school compares internationally OECD Test for Schools Pilot Trial 2012 School name School District State United States

37. OECD Test for Schools (based on PISA)What next? Opportunities for development… • Receiving feedback… • District-level information based on school-level results • Reporting on country-specific variables • Examples of policies, practices and insights based on feedback from pilot participants (and results from PISA 2012) • Peer-to-peer exchanges… examples of practices and network-specific follow-up

38. OECD Test for Schools (based on PISA)What now? Next Steps Availability of the assessment in the United States • Starting in September 2013 (for testing) • Interested schools and districts can begin planning now To sign up: http://www.oecd.org/pisa/pisa-basedtestforschools/ OECDTestforSchoolsSignUp@oecd.org • OECD will accredit a service provider

39. Thank you ! Find out more about PISA at… OECD www.pisa.oecd.org All national and international publications The complete micro-level database Email: Alejandro.GomezPalma@OECD.org … and remember: Without data, you are just another person with an opinion

40. OECD Test for Schools (based on PISA)Overview of results from the pilot • 105Schools in the U.S.: 48 school districts in 22 states • 1 in 3 of the pilot schools are Title I • 6 magnets, 6 charters and 1 private school • 3 of 4 schools with 20% or more of students from diverse ethnic backgrounds • Several schools from the same districts and networks (EdLeader21, Fairfax,…)