Dr Harold Hislop Chief Inspector  Príomh-Chigire Department of Education and Skills, Ireland

1. Applying an Evaluation and Assessment Framework: An Irish perspectiveBetter Assessment and Evaluation to Improve Teaching and LearningEU Presidency Conference  Dublin  20 March 2013 Dr Harold Hislop Chief Inspector Príomh-Chigire Department of Education and Skills, Ireland INSPECTORATE FEABHAS NA FOGHLAMA A CHUR CHUN CINN

2. Purpose of the paper • OECD: Education systems ought to view arrangements for evaluation and assessment in their countries as a structured and planned system • How could you apply the framework in Ireland? • What questions and implications would this raise for Ireland and others? • What is the significance of the OECD framework?

3. Overarching themes • Governance • Design and procedures: the components that one might consider using • Capacity: the ability of the system and the institutions and individuals to operate evaluation and assessment arrangements • Use of results

4. GOVERNANCE

5. GOVERNANCE • Need for clear structures and defined responsibilities for evaluation and assessment OECD • Ireland would appear to have at least the main elements of governance in place, though some challenges remain….

6. Significant legislation is in place • Education Act, 1998 • Education Welfare Act, 2000 • Teaching Council Act, 2001 • Education for Persons with Special Educational Needs Act, 2004

7. Minister responsible for… • evaluation To monitor and assess the quality, economy, efficiency and effectiveness of the education system provided in the State ……and publish, in such manner as the Minister considers appropriate, information relating to such monitoring and assessment (Education Act, section 7(2)(b))

8. Minister responsible for …. • evaluation • curricular policy • making regulations concerning appointment and qualifications of teachers • determining policy framework in which the Teaching Council operates • making regulations concerning the inspection of schools

9. Statutory bodies are in place • National Council for Curriculum & Assessment • State Examinations Commission • Inspectorate • Educational Research Centre • (non-statutory) • Teaching Council

10. Legislation contains checks and balances • Powers are vested in Minister BUT • Minister can only exercise his powers following consultation with the key stakeholders in the education system AND • Statutory bodies must also consult with stakeholders

11. Consensus in policy making Advantages • Stakeholder “buy-in” • Teachers and teacher unions • School managers • School leaders and their national organisations • National organisations of parents and students • Business and community groups and interests • Supports teacher professionalism

12. Consensus in policy making Advantages • Stakeholder “buy-in” • Supports teacher professionalism Disadvantages • Responsiveness • Sometimes radical change is needed

13. So, reflecting on governance… • Structures of governance exist in Ireland for an evaluation and assessment framework, BUT • Have we articulated an overall policy about evaluation and assessment in the system?

14. AN OVERALL POLICY ABOUT ASSESSMENT AND EVALUATION?

15. National Literacy and Numeracy Strategy, 2011 • the closest we have to a national statement regarding how aspects of curriculum, assessment and evaluation knit together • but focussed on literacy and numeracy • linkages need to be more explicit

16. A reflection on the OECD framework… • We could find it beneficial to examine deliberately how we want the essential components of an evaluation and assessment framework to develop in a coherent way that will support the sort of student learning to which we aspire in the school system of the 21st century

17. DESIGN AND PROCEDURES THE COMPONENTS OF A FRAMEWORK

18. DESIGN AND PROCEDURES • Student assessment • School evaluation • Inspections and School Self-Evaluation • Teacher appraisal & Appraisal of school leaders • Education system evaluation

19. Student assessment: primary • Early adopter of standardised tests for national sampling in 1970s– Educational Research Centre • Standardised tests not used in consistent way in schools until recently • Strong reaction against testing and league tables in 1990s • Teacher education: not enough attention paid to assessment skills and competences

20. Student assessment: secondary level • Dominance of State examinations • Externally set and marked examinations at end of lower secondary level (15-16 years) and end of upper secondary level (17-18 years) • Less use of assessment for learning • Teacher education: not enough attention paid to assessment skills and competences

21. Student assessment: developments • Curricula with learning outcomes & examples of students’ work • Assessment for Learning • Standardised tests for literacy and numeracy • School based assessment & certification at lower secondary

22. A number of policy directions highlighted in the OECD Framework for Assessment and Evaluation • Ensure that parents are engaged • Support assessment for learning approaches • Build assessment capacity in system • A balance between formative/summative assessment • A broader range of assessment methodologies • Need to invest in and use ICT systems • A balanced use of student assessment • keep existing policies and arrangements under review

23. School evaluation: Inspection of schools • Expanded and developed since 1998 • A well-embedded feature of primary and second-level schools • Particular focus on teaching and learning • Advice to teachers and school leaders • Improvement agenda as well as accountability • Inspection reports published since 2006 • Proportionate inspection in schools

24. School inspections: some features • Range of inspection models, including • Some notified inspections – examine leadership, management, teaching and learning, capacity to improve • Unannounced inspections – focussed on teaching and learning; used to scan system and for risk planning • Follow-up inspections – both generally and in schools with severe weaknesses • Confidential questionnaire data from parents and students

25. Parents are very positive about many aspects of the work of schools • “The school is well run” • Agree/strongly agree 90.2% • Disagree/strongly disagree 4.5% • Don’t know 5.4% 8,680 parental responses in second-level schools, provisional 2012 data, subject to confirmation

26. Parents are very positive about teaching and learning in the school • “Teaching is good in the school” • Agree/Strongly agree 86.5% • Disagree/Strongly disagree 7.5% • Don’t know 6.0% • “My child is doing well in school” • Agree/Strongly agree 93.0% • Disagree/Strongly disagree 4.6% • Don’t know 2.3% 8,680 responses; provisional data from 2012, subject to confirmation

27. Students are positive about many aspects of their experience in school • “There is a good atmosphere in the school” • Agree/strongly agree 83.2% • Disagree/strongly disagree 5.5% • Don’t know 11.3% 12,893 responses, provisional 2012 data, subject to confirmation

28. Students are generally positive about many aspects of their experience in school • “The school deals well with bullying” • Agree/strongly agree 69.6% • Disagree/strongly disagree 10.7% • Don’t know 19.8% 12,893 responses, provisional 2012 data, subject to confirmation

29. School evaluation: School self-evaluation • Less well-developed • Guidelines/resources

30. A number of policy directions highlighted in the OECD Framework for Assessment and Evaluation • Clarity about role and purpose of school evaluation • Accountability or Improvement • Public confidence and availability of data • A framework of standards • Capacity of school leaders and teachers • Investment in data capture and usage

31. Teacher appraisal &Appraisal of school leaders • Formal systems: least well developed aspect of system • Teaching Council: “fitness to teach” enquiries, power to de-register teacher • Appraisal in probationary period and for under-performing teachers • School leadership is considered in inspections but • No regular school-based appraisal for teachers or principal

32. A policy directions highlighted in the OECD Framework for Assessment and Evaluation • Given the experience of many countries documented by the OECD, it’s not unreasonable to ask “Why has the Irish school system apparently chosen not to use some form of teacher appraisal as a component in its evaluation and assessment arrangements?”

33. But what should teacher appraisal be about? • Weakness in terminology and construct • “Teacher appraisal” has implicit emphasis on accountability • We want reflective practitioners as well as appraisal for evaluation • Improvement as well as accountability

34. We should attempt to have… • Reflective practice • Collaborative, peer review • Collaborative whole-staff conversations about practice and how to improve • Observation and constructive feedback from peers and principal • Collegial Professional Reflection & Appraisal

35. System evaluation • National monitoring surveys of literacy and numeracy (Educational Research Centre) • Specialised evaluations and research (ERC and others) • initiatives and policies; disadvantaged groups • Inspectorate reports • International surveys • TIMSS and PIRLS, PISA, PIAAC, etc.

36. IMPLICATIONS

37. IMPLICATIONS: Concerns that may be expressed • Over-use of student assessment • “Weighing the pig will not make it any heavier” • Effect of measurement, performance and accountability • Attempting to measure teaching and learning can alter teacher and student behaviour • Dynamic around international surveys • Indiscriminate policy borrowing

38. IMPLICATIONS • An honest, valuable and significant attempt to avoid ill-informed policy making • Getting the right mix of tools and approaches • Move away from narrow focus on student assessment – towards a balanced framework • Takes into account country context and capacity of each educational system • Interesting challenges ahead!!

39. Go raibh maith agaibh! Thank you! Dr Harold Hislop Chief Inspector Department of Education and Skills, Dublin 1, Ireland harold_hislop@education.gov.ie Full paper available at www.education.gov.ie