PISA 2009 Evaluating systems to improve education

1. Programme for International Student Assessment PISA 2009Evaluating systems to improve education Sophie Vayssettes Indicators and Analysis Division, EDU, OECD

2. PISA 2009 in brief • Half a million students in 74 countries/economies covering 87% of world economy • 2-hour test of skills that go beyond reproducing what is taught in schools • Information on student background, attitudes and their schools + data from parents, principals and system leaders • 19 countries participated in digital reading assessment

3. OECD’s computer-based assessments • PISA 2009 digitalreadingassessment • 19 countries • PISA 2012 • 43 countries • PISA 2015 • Almost all countries • PIAAC - international survey of adult skills • interviewing adults aged 16-65 years in their homes – 5 000 in each participating country; • literacy and numeracy skills and ability to solve problems in technology-rich environments; • results will be publishedin 2013

4. Why PISA 2009 included a digital reading assessment • “Digital reading” means reading on screen, mostly via the Internet • Digital reading  reading on line  is an increasingly important form of reading in the 21st century • Digital reading has different features to print reading and makes new demands on readers

5. Why PISA 2009 included a digital reading assessment • “Digital reading” means reading on screen, mostly via the Internet • By March 2011, more than 2 billion people were using the Internet worldwide • Digital reading is an increasingly important form of reading in the 21st century • Digital reading has different features to print reading and makes new demands on readers

6. Why PISA 2009 included a digital reading assessment • “Digital reading” means reading on screen, mostly via the Internet • Digital reading  reading on line  is an increasingly important form of reading in the 21st century • taxpayers fill in online forms; • students search the web for information; • jobseekers look up ads on employment websites; • consumers order goods in online stores; • people build and maintain social communities on line. • Digital reading has different features to print reading and makes new demands on readers .

7. Digital reading has different features to print reading and makes new demands on readers • There are different kinds of texts • There are many mixed texts, including short pieces, diagrams, graphics, fragments • There are many interactive texts, where readers can change, add to, or create their own texts • The reader has more responsibility, in selecting and constructing what they read • There is often no guide to the order in which a text should be read – The reader can take many pathways • There is an almost infinite number of texts available • The extent of a text is abstract and unknown – the reader cannot see the whole text • The reader has more responsibility, in evaluating the text • Traditional filters, like publishers and booksellers, are absent The reader has to ask, “Is this text useful for me?” “Do I trust this text?”

8. Digital reading features and demands – How were these features implemented in PISA 2009? • There are different kinds of texts • The tasks are based on a range of typical digital text types and formats • The reader has more responsibility, in selecting and constructing what they read • Students often needed to navigate to find the relevant part of the text • Digital reading texts included many different pages • The reader has more responsibility, in evaluating the text • Some tasks asked students to evaluate the text • An example task

9. Example of a PISA digital reading task (1)

10. Example of a PISA digital reading task (2)

11. Example of a PISA digital reading task (1)

12. Example of a PISA digital reading task (3)

13. Example of a PISA digital reading task (4)

14. Example of a PISA digital reading task (5)

15. Example of a PISA digital reading task (6)

16. Example of a PISA digital reading task (7)

17. Example of a PISA digital reading task (8)

18. Example of a PISA digital reading task (9)

19. Identifying key elements of digital reading • text processing locating, interpreting and evaluating words on the screen • navigatingfrom one screen to another, to find relevant information

20. Performance in digital reading

21. How proficient are students in digital reading? Percentage of students

22. Differencesbetween boys and girls comparedwithprintreading Gender difference in print reading performance (girls- boys) Poland Girls have a larger advantage in print reading Norway Sweden New Zealand Iceland Austria France Japan Ireland Hungary OECD Average Macao - China Australia Korea Hong Kong -China Belgium Denmark Spain Chile Girls have a larger advantage in digital reading Colombia Gender difference in digital reading performance (girls-boys)

23. Navigation • Navigation is considered to be part of the cognitive process of digital reading • Tracking and analysing the sequences of pages students visit provide insights into effective navigation behaviours in digital reading

24. Number of relevant pages visited and digital reading performance The index number of relevant pages visited describes how many of the pages judged to be relevant to a task were accessed while the student worked on that task

25. ICT questionnaire • Students’ access to and use of computers and Internet • Their general attitudes towards and self-confidence in using computers • 29 OECD countries and 16 partner countries and economies • Not designed to assess the quality of ICT use at school and the integration of ICT in pedagogy • Focus on students’ use of ICT to access, manage and present information

26. Changes in percentage of students who reported having a computer at home between PISA 2000 and 2009

27. Changes in computers-per-student ratio between PISA 2000 and 2009

28. Digital divide by country However, some countries lag behind on digital use Percentage of students who use a computer at school Percentage of students who use a computer at home

29. Digital divide by socio-economic background within countries Within countries, there is a digital divide by socio-economic background of students Disadvantaged students have less access to a computer at school Disadvantaged students have less access to a computer at home Disadvantaged students have more access to a computer at school

30. ICT use at home for leisure and digital reading performance

31. ICT use at home for schoolwork and digital reading performance

32. ICT use at school and digital reading performance

33. After accounting for performance in print reading Digital reading performance Leisure at home Schoolwork at home At school

34. Computer use and navigation (students with above-average reading scores) Index of the number of relevant pages visited

35. Some conclusions • Identifying effective strategies to teach digital reading skills is an important policy objective • Ability to critically evaluate the quality and credibility of texts, integrate information from multiple texts and – crucially – navigate effectively • ICT use at home for leisure is – up to a point – positively related to performance, navigation skills and self-confidence in completing high-level ICT tasks • Parents and teachers need to encourage students to use computers so that they can improve their navigation skills but also provide guidance on balancing time spent using computers with time for other activities • Digital reading can be a lever to reduce the gender gap • The gender gap in digital reading is much smaller than in print reading, and relates to differences in navigation skills between boys and girls • Reading more and reading with enjoyment promotes better reading, and better reading fosters stronger engagement .

36. Some conclusions • Access to computers has improved significantly, both at home and at school, but important gaps remain between countries and social groups • Strategies that promote wider access to ICT at school can help minimise impact of social background on digital competency gaps, with possible consequences for future employment opportunities • Schools can do more to integrate ICT into learning • Opportunities for students to solve problems using ICT • Reading methods that improve students’ ability to distinguish between relevant and irrelevant material, and to structure, prioritise, distil and summarise texts • ICT can… • Enable students to obtain more regular feedback on their learning processes • Make students more active participants in learning processes in classrooms and tailor these processes to individual students’ needs • Provide students with up-to-date access to the world’s current research and thinking .

37. THANK YOU Find out more about PISA at… OECD www.pisa.oecd.org All national and international publications The complete micro-level database Email: Sophie.Vayssettes@OECD.org