From Education to Learning: Using evidence to catalyze change

1. ASER : Measure | Analyze | Communicate | Change • From Education to Learning: Using evidence to catalyze change • In English, ASER stands for Annual Status of Education Report. • In Hindi, ASER means impact.

2. Overview Evolution: How was ASER born? Design: What are the key elements of ASER? Findings: What does ASER find? Impact: What difference have we made?

3. Genesis: Pratham experience • WHAT WERE THE CHALLENGES: • Do we know our children: Large families / diverse and multi-grade classrooms • Parents “send” children to school and are concerned about “inputs”. Parents often over-estimate what children can do (JPAL study) • Teachers“teach” the course for the grade level and often over-estimate what children can do (SchoolTELLS) • Schools in India are not structured to identify or to help those who “fall behind” • Learning was “assumed” by all sides. Hence the need to highlight the “learning crisis” in India Since 1996, Pratham has been working with children who were “left out” (out of school) or “left behind” (in-school but lagging behind) While it was relatively easy to bring children to school, bringing children to their appropriate age-grade level of learning was much harder

4. Policy environment in Education UNIVERSAL ENROLLMENT UNIVERSAL LEARNING • BUT MDGs make no reference to children’s learning goals • MDG goals refer to universal enrolment • BUT our national (SSA) goals are broad - “education of satisfactory quality relevant for life” ; “learning enhancement” • India’s national educational goals (SSA) refer to universal enrolment, retention and reduction of social and gender gaps • BUT government measurements of learning levels come out every few years and without any discussion • In India, enrollment figures from school/village level to state & national level are collected frequently, published and discussed Challenge was bring learning to centre of stage at micro & policy level. How? Address and alleviate progress & equity

5. Can children read? What could we do at ground level? Simple tool useful for instruction and assessment. Helpful in engaging parents and teachers about what to do. Grade 1 level text Grade 2 level text Simple words Letters

6. Mobilizing communities around children’s learning Big meeting in village to discuss hamlet & village education report card. Attended by parents, villagers, village committee members, headmaster and teachers : 100-150 people attend especially if the hamlet level discussions have been held well Discussion in small groups in each hamlet : Start action. Track measurable changes in children’s learning & in school functioning Agree on next steps… On what can be done in village Local people help in making the hamlet report card. Report card focuses on whether child goes to school & on whether child can read simple text and do basic arithmetic “Demo” class run in the village for 4-5 days to show how children can learn to read quickly

7. Moving from evidence to action Enable analysis based on evidence 2 • UNDERSTAND EVIDENCE • Digest information as it is being generated close to the ground • Move from anecdote or personal experience to aggregate picture 1 ENGAGE AND EVALUATE • Generate curiosity • Encourage participation • Engage via simple activities • Use basic tools & methods • Generate your own data De-mystify assessment Demand accountability, seek collaborations 4 • COMPARE WITH PROVISIONS • Local demands for information about plans & provisions • Push for institutional mechanisms for linking govt and citizens 3 Link assessment to action • THINK ABOUT SOLUTIONS • What can we do as citizens? • Local strategies, common goals, collective involvement

8. Architecture of ASER: Nuts & Bolts • 4. Where to measure? • All rural districts: • India has ~600 district • 30 villages randomly selected ion each district (PPS sampling) • 20 HHs randomly selected in each village (In school / Out of school) • In each village, one govt. school is also visited 1. What to measure? • Basic reading and arithmetic tasks to all children. The highest in reading is Grade 2 level & in Math is Grade 4 level • 2. Who to measure? • All children in age group 5 to 16 • 3. When to measure? • Once every year to review progress and produce comparable data • 5. Who measures? • Local organizations and institutions in each district. • NGOs, colleges, universities, youth & women’s groups Sample size is 600,000 - 700,000 children each year

9. ASER Report Card: India (Rural) 2011 About half of all children will complete primary school without learning to read fluently. 48% children can read at Std 2 level 24% can read at Std 1 level Huge implications for instruction & equity. Urgent corrective action needed. 13% can read letters & less 14% can read words Reading level Std 5 in India

10. Not much change over time India is in a “big stuck”. Learning levels are not improving. What are we doing year on year to move children from the current level towards the “desired” level of basic reading? % Children in Grade 3 to 5 who can read Grade 2 level text “Desired” level Actual current level Source : ASER 2006-2011

11. Right to Education in India • Since 2010 ASER has been tracking Right to Education indicators in order to measure govt. outlays and results on ground. • Teachers: Pupil-Teacher ratio • Infrastructure: Building, classroom, computer rooms, number of classrooms, toilets etc. • Outlays and expenditure

12. Dissemination & Impact: What we do with the findings • National release of the ASER report each year in January • Followed by extensive dissemination exercises where state / district-wise data is taken back to the regions by ASER team • Discussions held with district level partners & surveyors who collected the data to understand what it means • Village level meetings (gram sabhas) organized with “village report cards” • People are beginning to understand the difference between “schooling” and “learning” • Partnering with state / district administration (SSA) in order to conduct learning assessments and inform policy at ground level • State / district & even villagers (parents, teachers) are encouraged to use ASER tools to conduct their own assessments

13. Impact of ASER in 7 years : 2005 to 2001 WITHIN INDIA • Has brought the issue of children’s learning outcomes to the forefront of policy discussions • Quoted and used by government and other sources at Central, state and district level • Leading to more such measurements of learning • Govt’s annual planning exercise at district level for elementary education has a line item allowing money to be allocated for “learning enhancement”. ABROAD • ASER-like initiatives in many other countries – ASER in Pakistan, UWEZO in Kenya, Tanzania, Uganda and BEKONGO in Mali • Giving “voice” on global platforms for universal learning goals and importance of basic reading / math • Other agencies are using ASER tools (JPAL, Room to Read, Effective intervention etc.)

14. Using ASER-like approach to improve quality of education • BENEFITS… • Simple to do, to analyze and to understand • No delay between data collection and results • Inexpensive to conduct in terms of both money and manpower • Provides trends over time and comparisons of key variables across units • LIMITATIONS… • Large sample but “thin” slice of data • Tells us what but not why • Captures key inputs and outcomes but not processes • Focuses on children and what they know, not on schools and what they do. ASER type rapid assessment is a good place to start in countries and contexts where outcomes orientation, evidence based analysis and planning are not used widely.

15. Challenges ahead There are big challenges at the macro level and at the micro level within India and internationally: Govt. is still focused on inputs and access rather than outcomes. No culture or capacity for measurement or evidence. Governments need to move their mindset from policy based in ideology to focusing on practice according to “what works”. Teachers to need to have access to easy and do-able tools to measure the learning levels of their children and to straightforward strategies of improving learning outcomes in real time. Many parents have not been to school themselves, yet they understand the importance of schooling. Now, the big challenge is that illiterate parents need to participate in the journey for universal learning. Biggest resource and biggest challenge is how to make the educated people in the community and country united together to work to ensure that every child is in school and learning well.

16. PurnimaRamanujan purnima.ramanujan@asercentre.org ASER Centre B4/54 Safdarjang Enclave Near Kamal Cinema New Delhi 110 029. India. www.asercentre.org contact@asercentre.org