Applying an Evaluation and Assessment Framework: An Irish perspectiveBetter Assessment and Evaluation to Improve Teaching and LearningEU Presidency Conference Dublin 20 March 2013 Dr Harold Hislop Chief Inspector Príomh-Chigire Department of Education and Skills, Ireland INSPECTORATE FEABHAS NA FOGHLAMA A CHUR CHUN CINN
Purpose of the paper • OECD: Education systems ought to view arrangements for evaluation and assessment in their countries as a structured and planned system • How could you apply the framework in Ireland? • What questions and implications would this raise for Ireland and others? • What is the significance of the OECD framework?
Overarching themes • Governance • Design and procedures: the components that one might consider using • Capacity: the ability of the system and the institutions and individuals to operate evaluation and assessment arrangements • Use of results
GOVERNANCE • Need for clear structures and defined responsibilities for evaluation and assessment OECD • Ireland would appear to have at least the main elements of governance in place, though some challenges remain….
Significant legislation is in place • Education Act, 1998 • Education Welfare Act, 2000 • Teaching Council Act, 2001 • Education for Persons with Special Educational Needs Act, 2004
Minister responsible for… • evaluation To monitor and assess the quality, economy, efficiency and effectiveness of the education system provided in the State ……and publish, in such manner as the Minister considers appropriate, information relating to such monitoring and assessment (Education Act, section 7(2)(b))
Minister responsible for …. • evaluation • curricular policy • making regulations concerning appointment and qualifications of teachers • determining policy framework in which the Teaching Council operates • making regulations concerning the inspection of schools
Statutory bodies are in place • National Council for Curriculum & Assessment • State Examinations Commission • Inspectorate • Educational Research Centre • (non-statutory) • Teaching Council
Legislation contains checks and balances • Powers are vested in Minister BUT • Minister can only exercise his powers following consultation with the key stakeholders in the education system AND • Statutory bodies must also consult with stakeholders
Consensus in policy making Advantages • Stakeholder “buy-in” • Teachers and teacher unions • School managers • School leaders and their national organisations • National organisations of parents and students • Business and community groups and interests • Supports teacher professionalism
Consensus in policy making Advantages • Stakeholder “buy-in” • Supports teacher professionalism Disadvantages • Responsiveness • Sometimes radical change is needed
So, reflecting on governance… • Structures of governance exist in Ireland for an evaluation and assessment framework, BUT • Have we articulated an overall policy about evaluation and assessment in the system?
National Literacy and Numeracy Strategy, 2011 • the closest we have to a national statement regarding how aspects of curriculum, assessment and evaluation knit together • but focussed on literacy and numeracy • linkages need to be more explicit
A reflection on the OECD framework… • We could find it beneficial to examine deliberately how we want the essential components of an evaluation and assessment framework to develop in a coherent way that will support the sort of student learning to which we aspire in the school system of the 21st century
DESIGN AND PROCEDURES THE COMPONENTS OF A FRAMEWORK
DESIGN AND PROCEDURES • Student assessment • School evaluation • Inspections and School Self-Evaluation • Teacher appraisal & Appraisal of school leaders • Education system evaluation
Student assessment: primary • Early adopter of standardised tests for national sampling in 1970s– Educational Research Centre • Standardised tests not used in consistent way in schools until recently • Strong reaction against testing and league tables in 1990s • Teacher education: not enough attention paid to assessment skills and competences
Student assessment: secondary level • Dominance of State examinations • Externally set and marked examinations at end of lower secondary level (15-16 years) and end of upper secondary level (17-18 years) • Less use of assessment for learning • Teacher education: not enough attention paid to assessment skills and competences
Student assessment: developments • Curricula with learning outcomes & examples of students’ work • Assessment for Learning • Standardised tests for literacy and numeracy • School based assessment & certification at lower secondary
A number of policy directions highlighted in the OECD Framework for Assessment and Evaluation • Ensure that parents are engaged • Support assessment for learning approaches • Build assessment capacity in system • A balance between formative/summative assessment • A broader range of assessment methodologies • Need to invest in and use ICT systems • A balanced use of student assessment • keep existing policies and arrangements under review
School evaluation: Inspection of schools • Expanded and developed since 1998 • A well-embedded feature of primary and second-level schools • Particular focus on teaching and learning • Advice to teachers and school leaders • Improvement agenda as well as accountability • Inspection reports published since 2006 • Proportionate inspection in schools
School inspections: some features • Range of inspection models, including • Some notified inspections – examine leadership, management, teaching and learning, capacity to improve • Unannounced inspections – focussed on teaching and learning; used to scan system and for risk planning • Follow-up inspections – both generally and in schools with severe weaknesses • Confidential questionnaire data from parents and students
Parents are very positive about many aspects of the work of schools • “The school is well run” • Agree/strongly agree 90.2% • Disagree/strongly disagree 4.5% • Don’t know 5.4% 8,680 parental responses in second-level schools, provisional 2012 data, subject to confirmation
Parents are very positive about teaching and learning in the school • “Teaching is good in the school” • Agree/Strongly agree 86.5% • Disagree/Strongly disagree 7.5% • Don’t know 6.0% • “My child is doing well in school” • Agree/Strongly agree 93.0% • Disagree/Strongly disagree 4.6% • Don’t know 2.3% 8,680 responses; provisional data from 2012, subject to confirmation
Students are positive about many aspects of their experience in school • “There is a good atmosphere in the school” • Agree/strongly agree 83.2% • Disagree/strongly disagree 5.5% • Don’t know 11.3% 12,893 responses, provisional 2012 data, subject to confirmation
Students are generally positive about many aspects of their experience in school • “The school deals well with bullying” • Agree/strongly agree 69.6% • Disagree/strongly disagree 10.7% • Don’t know 19.8% 12,893 responses, provisional 2012 data, subject to confirmation
School evaluation: School self-evaluation • Less well-developed • Guidelines/resources
A number of policy directions highlighted in the OECD Framework for Assessment and Evaluation • Clarity about role and purpose of school evaluation • Accountability or Improvement • Public confidence and availability of data • A framework of standards • Capacity of school leaders and teachers • Investment in data capture and usage
Teacher appraisal &Appraisal of school leaders • Formal systems: least well developed aspect of system • Teaching Council: “fitness to teach” enquiries, power to de-register teacher • Appraisal in probationary period and for under-performing teachers • School leadership is considered in inspections but • No regular school-based appraisal for teachers or principal
A policy directions highlighted in the OECD Framework for Assessment and Evaluation • Given the experience of many countries documented by the OECD, it’s not unreasonable to ask “Why has the Irish school system apparently chosen not to use some form of teacher appraisal as a component in its evaluation and assessment arrangements?”
But what should teacher appraisal be about? • Weakness in terminology and construct • “Teacher appraisal” has implicit emphasis on accountability • We want reflective practitioners as well as appraisal for evaluation • Improvement as well as accountability
We should attempt to have… • Reflective practice • Collaborative, peer review • Collaborative whole-staff conversations about practice and how to improve • Observation and constructive feedback from peers and principal • Collegial Professional Reflection & Appraisal
System evaluation • National monitoring surveys of literacy and numeracy (Educational Research Centre) • Specialised evaluations and research (ERC and others) • initiatives and policies; disadvantaged groups • Inspectorate reports • International surveys • TIMSS and PIRLS, PISA, PIAAC, etc.
IMPLICATIONS: Concerns that may be expressed • Over-use of student assessment • “Weighing the pig will not make it any heavier” • Effect of measurement, performance and accountability • Attempting to measure teaching and learning can alter teacher and student behaviour • Dynamic around international surveys • Indiscriminate policy borrowing
IMPLICATIONS • An honest, valuable and significant attempt to avoid ill-informed policy making • Getting the right mix of tools and approaches • Move away from narrow focus on student assessment – towards a balanced framework • Takes into account country context and capacity of each educational system • Interesting challenges ahead!!
Go raibh maith agaibh! Thank you! Dr Harold Hislop Chief Inspector Department of Education and Skills, Dublin 1, Ireland email@example.com Full paper available at www.education.gov.ie