NATIONAL EDUCATION EVALUATION AND DEVELOPMENT UNIT

1. NATIONAL EDUCATION EVALUATION AND DEVELOPMENT UNIT A PRESENTATION TO Portfolio Committee of Basic Education 15 November 2011 Cape Town NEEDU

2. SCHOOLS VISITED Between February and September, NEEDU team (five members) visited schools in five provinces: Mpumalanga, Eastern Cape, KwaZulu-Natal, Northern Cape, and Western Cape A total of 74 schools were visited, 42 primary and 32 high schools No systematic selection of schools. Data mainly anecdotal—a variety of evaluation and observation instruments were used Purpose of school visits vary from province to province: To provide some indication as to why high schools and their feeder primary schools were underperforming; To identify good practices from good performing primary schools; To evaluate the extent to which schools implement the national initiatives introduced by the Minister to improve learning outcomes, incl. ANA, learner workbooks, and the Foundations for Learning Programme 2

3. PURPOSE OF SCHOOL VISITS Purpose of school visits vary from province to province: To conduct a pre-evaluation survey of school landscape in various provinces To provide some indication as to why high schools and their feeder primary schools were underperforming To identify good practices from good performing primary schools To evaluate the extent to which schools implement the national initiatives introduced by the Minister to improve learning outcomes, incl. ANA, learner workbooks, and the Foundations for Learning Programme 3

4. APPROACH TO SCHOOL EVALUATIONS Schools were selected based on their performance in ANA and Grade 12 national exams Data collection processes included: Discussion with the principals, school management teams, and selected teachers School records, incl. teacher and learner attendance registers, logbooks, asset registers, learner assessment records, school improvement plans, etc. Review of teachers’ work, incl. their work schedules, daily lesson preparations, classroom and personal timetables, assessment plans and tasks, and other records Review learners’ work, incl. class work, homework, assessment books, and learner workbooks provided by the Department Selecting learners randomly and asking them to read (especially in primary schools) and doing mental mathematics Entering classrooms and observing teachers teaching—focusing on language and mathematics (refer to the pack for the instrument that was used) 4

5. POST SCHOOL EVALUATION • Provide feedback to teachers after classroom observations • Provide feedback schools and district officials on the findings and recommendations • It is expected that the latter will provide the necessary support to schools • A full report will be submitted to the Minister for her consideration

6. GUIDING PRINCIPLES • Informed by the Sector Plan (Action Plan to 2014 Towards the realisation of Schooling 2025) and the Minister’s Delivery Agreement, NEEDU is guided by the following principles when evaluating schools: • The quality of teaching and learning is central in improving low levels of learner performance • School evaluation initiatives must lead to improved quality of teaching and learning; • School improvement will be enhanced if there is an improvement in the quality of support provided to schools by the relevant structures • There must be visible improvement in the functionality of schools; • Empowered school principals are able to provide necessary leadership, particularly where it matters the most, i.e., curriculum delivery; and • There must be sustainable learner performance improvement annually, as reflected in the set targets in the Sector Plan

7. FOCUS AREAS FOR EVALUATION • While NEEDU looked at other contextual issues (refer to the pack) that have a bearing on curriculum delivery, it mainly focused key essential drivers of quality to improve learner outcomes: • Monitoring student learning • Instructional leadership of the principal; • Curriculum quality; • Opportunity to learn (OTL); and • Professional development and collaboration • The latest research published in 2011 (Carnoy et al; Shepherd; Spaul; Taylor; and Van der Berg) consistently identify the above as essential drivers of quality

8. FINDINGS: Monitoring student learning • Output 2, sub-output 1 in the Minister’s delivery agreement talks about establishing a world class system of standardised national assessments • NEEDU examined how student learning was monitored or assessed in schools and how results from such assessments were used (refer to the pack for details). NEEDU observed that: • In all schools, teachers were in full compliance with regard to keeping the prescribed number of formal learner performance or assessment records. • The main challenges in schools included over utilisation of a single form of assessment, quality and use of assessment results, whether school-based of external, e.g. ANA:

9. FINDINGS: Monitoring student learning • Specifically, NEEDU found that: • In most schools, teachers focus on formal written assessments. Other equally important forms of assessments are hardly used (doing the minimum) • Most school-based assessment tasks, particularly in primary schools, are of low quality and are not comparable with external assessments. They test low level skills. High order skills such as problem solving, independent thinking are not tested • School-based assessments do not adequately cover the curriculum • Only in few schools did NEEDU find that assessment results are analysed to identify problematic areas in the curriculum • Except in isolated cases, no evidence could be found that either school-based assessments or external assessments were used to inform planning, teaching strategies and school-based or district-based interventions

10. FINDINGS: Principals as instructional leaders • Output 4, sub-output 1 in the Minister’s delivery agreement refers to the strengthening of school management and promotion of functional schools. • NEEDU observed instructional leadership practices of principals (refer to the pack for the practices) and found that the majority of principals have abdicated their curriculum management responsibilities, i.e. • Principals spend most of their time on other tasks but curriculum matters • Most principals do not have systems and procedures to monitor curriculum delivery, incl. controlling classroom practices and monitoring teachers’ work

11. FINDINGS: Principals as instructional leaders • NEEDU also found that: • Little evidence to suggest that principals initiate and facilitate staff development (based on teacher professional development needs established through IQMS) to improve learning outcomes • Only a handful of primary school principals know performance levels in language and mathematics in their schools • In very few cases did primary school principals use evidence from ANA to identify appropriate school intervention programs, to develop a School Improvement Plan (as required by WSE policy) or to develop an Academic Performance Improvement Pan (as required in terms of the Education Laws Amendment Act)

12. FINDINGS: Curriculum Quality • Output 1, sub-outputs 1 and 2 in the Minister’s delivery agreement refer to the improvement of teacher capacity and practices, and the increase of access to high quality materials. • NEEDU observed teacher practices in and outside class (refer to the pack for the practices) and observed the following: • While most teachers kept files with lesson plans, NEEDU observed that some teachers did not have clear daily lesson plans • In almost all schools visited, work schedules and lesson planning were not informed by findings in ANA • Most primary school teachers have very low expectations for their learners--resulting in them focusing on inappropriate level of cognitive demand, e.g., spending more time on single words and not extended writing, and in mathematics completely ignoring word sums

13. FINDINGS: Curriculum Quality • NEEDU also found that: • Except in very few cases, most teachers did not seem to have a variety of strategies to teach reading. The five key components of teaching reading ( phonemic awareness, word recognition [sight words and phonics], comprehension, vocabulary and fluency) were completely ignored in many schools • While there were pockets of good practice, in many classrooms visited there was no differentiated teaching to cater for learners with different learning styles and learners with barriers to learning, e.g., learners with foetal alcohol syndrome mostly in the Northern Cape • Most schools do not have sufficient reading materials incl. graded readers. Learner workbooks supplied by the Department were not used appropriately in most schools. • Learners (in some cases, teachers) have a challenge using English as a language of teaching and learning

14. FINDINGS: Opportunity to learn • Output 4, sub-output 1 in the Minister’s delivery agreement acknowledges that there has been insufficient attention paid to whether teachers complete the year’s learning programme within the year. • When evaluating whether learners are provided with adequate opportunities to learn, NEEDU observed three elements: the effective use of time, curriculum coverage and homework (refer to the pack for details). Following were NEEDU observations: • Evidence in many schools indicates that there is a problem with school timetables. The prescribed time for teaching reading (incl. time for shared reading, guided reading, independent reading or reading for pleasure) and mathematics is not observed in a good number of schools. Extreme cases included learners being taught three hours of mathematics a week instead of a minimum of six.

15. FINDINGS: Opportunity to learn • Other findings: • When observing lessons in class, NEEDU focused on the amount of time spent specifically on instruction and found that instructional time was further lost on non-instructional matters, such as maintaining order, etc. • There was a problem with curriculum coverage and pacing. While work schedule clearly indicated coverage of the curriculum, there was usually no congruence between work schedules, daily lesson plans (where they existed) and exercises in learner workbooks or homework books. • In many schools, learners were not given sufficient written work in language or mathematics in class or as homework. Where written work was provided, NEEDU found no evidence to indicate that enough time and energy were put on addressing problematic areas in the curriculum as revealed by ANA • Many schools did not have a homework policy. Thus, many schools did not give homework; where homework was given it was done haphazardly with no clear purpose

16. FINDINGS: Professional development • Output 1, sub-output 1 in the Minister’s delivery agreement acknowledges that there is a need for the development of new training packages for schools, largely through distance education and e-Education. • In terms of professional development and collaboration, NEEDU observed that: • Following the identification of individual teacher development needs through the IQMS processes, these needs are seldom addressed • Almost all school improvement plans do not indicate how professional development needs of teachers will be addressed

17. FINAL REPORT ON 100 SCHOOLS A more controlled scientific study, involving an stratified random sample of 100 schools, will be conducted between January and March 2012 The purpose of this study is to: Establish the causes of poor performance in language and mathematics schools, Examine the extent to which the national initiatives introduced by the Minister (incl. ANA, learner workbooks, CAPS) are making a difference Test NEEDU’s school evaluation instruments The study with focus on in-school factors, particularly the quality of teaching and learning Procurement processes are being followed to appoint a service provider that will provide technical assistance to NEEDU to conduct this study

18. MILESTONES: • NEEDU has developed instruments that will be used to evaluate the quality of teaching and learning in a sample 100 schools across the provinces • A service provider has been appointed to quality-assure instruments developed by NEEDU before they are used to ensure their credibility • NEEDU interviewed over 100 applicants for 21 NEEDU evaluators' posts across the nine provinces. It is anticipated that all 21 evaluators will assume duties in January 2012 and will be involved in collecting data in 100 schools. • The appointed service provider will train NEEDU evaluators in January 2012 • Data collection in 100 schools will commence in February/March 2012 • The final report will be available in April 2012