The Use of ICT for the Assessment of Key Competences

1. Irish Presidency Conference Better Assessment and Evaluation to Improve Teaching and Learning 19-20 March 2013 The Use of ICT for the Assessment of Key Competences Christine Redecker Yves Punie

2. IPTS- Who we are IPTS: Part of Joint Research Centre of the EC: 7 Research Institutes across Europe Mission: “to provide customer-driven support to the EU policy-making process by developing science-based responses to policy challenges that have both a socio-economicas well as a scientific/technologicaldimension”

3. IPTS Studies on ICT for Learning Future Learning Spaces in 2020 eLearning in the EU10 MS Review ICT Impact - Digital Competence - Active Ageing & Learning Learning 2.0 in formal E&T Learning 2.0 in informal settings Creativity & Innovation in schools in EU The Future of Learning Tell-Net Teacher Networking 2006 2007 2008 2009 2010 2011 2012 2013 ICT for the Assessment of Key Competences Digital Competence DC Follow up Mainstreaming Creative Classrooms 1to1 Learning initiatives in Europe Mapping Technologies for Learning Open Education in Europe

4. Generation Re-invention Transformative strategy: use technology to change test formats Where we stand Efficient testing Personalised learning 1990 1995 2000 2005 2010 2015 2020 2025 Generation 1 Generation 2 Generation 3 Generation 4 Adaptive Testing Automated administration and/or scoring of conventional tests Data Mining & Analysis Continuous integrated assessment Personalised feedback and tutoring Behavioural tracking Learning Analytics Multiple Choice Short Answer Automated Feedback Bunderson, Inouye & Olsen (1989). The four generations of computerized educational measurement. In R. L. Linn (Ed.), Educational measurement (Third ed., pp. 367-407). New York: Macmillan Bennett, R. E. (2010). Technology for Large-Scale Assessment. In P. Peterson, E. Baker & B. McGaw (Eds.), International Encyclopedia of Education (3rd ed., Vol. 8, pp. 48-55). ford: Elsevier. Replication of complex real life situations within a test Intelligent Tutors Games Virtual Worlds Online collaboration Virtual laboratories Peer assessment Simulations Learning 2.0 Collaborative multimedia learning environments Computer-Based Assessment (CBA) Technology-enhanced learning

5. Current use of ICT in assessment in schools There is scope for improvement Central recommendations on the use of ICT in pupil assessment in compulsory education in primary and general secondary education (ISCED 1, 2, 3), 2009/10 Source: Eurydice (2011). Key Data on Learning and Innovation through ICT at School in Europe 2011, Education, Audiovisual and Culture Executive Agency.

6. Examples 1st Generation CBA: Problem-Solving http://www.worldclassarena.org/v5/flash/09_year_old/backtofront.swf World Class Arena Sample Test: Problem Solving (9 years)

7. Examples Generation Re-Invention: Problem-Solving http://www.worldclassarena.org/v5/flash/13_year_old/pyramids.swf Problem Solving 13 years

8. Examples 1st Generation CBA: Situational Judgement EPSO sample situational judgement test questions; http://europa.eu/epso/discover/prepa_test/sample_test/index_en.htm

9. Examples Generation Re-Invention: e-tray exercise EPSO sample assessment centre e-tray exercise http://europa.eu/epso/discover/prepa_test/sample_test/index_en.htm

10. Generation Re-invention Transformative strategy: use technology to change test formats Future eAssessment Strategies Efficient testing Personalised learning 1990 1995 2000 2005 2010 2015 2020 2025 Generation 1 Generation 2 Generation 3 Data Mining & Analysis Adaptive Testing Automated administration and/or scoring of conventional tests Continuous integrated assessment Behavioural tracking Learning Analytics Multiple Choice Generation 4 Personalised feedback and tutoring Short Answer Automated Feedback Bunderson, Inouye & Olsen (1989). The four generations of computerized educational measurement. In R. L. Linn (Ed.), Educational measurement (Third ed., pp. 367-407). New York: Macmillan Bennett, R. E. (2010). Technology for Large-Scale Assessment. In P. Peterson, E. Baker & B. McGaw (Eds.), International Encyclopedia of Education (3rd ed., Vol. 8, pp. 48-55). ford: Elsevier. Intelligent Tutors Games Virtual Worlds Replication of complex real life situations within a test Online collaboration Virtual laboratories Peer assessment Simulations Learning 2.0 Collaborative multimedia learning environments Computer-Based Assessment (CBA) Technology-enhanced learning

11. Examples Towards Generation 3: Datamining Signals http://www.itap.purdue.edu/tlt/signals/signals_final/index.htm Similar: GPS at Northern Arizona University http://resourceconnect.nau.edu/grade-performance-status.asp

12. Examples Towards Generation 3: Behaviour Tracking Example: SNAPP SNAPP is a software tool that allows users to visualize the network of interactions resulting from discussion forum posts and replies. Teachers can rapidly identify patterns of user behaviour. SNAPP can extract all user interactions from various commercial and open source learning management systems, such as BlackBoard and Moodle.

13. Examples Towards Gen 4: Intelligent Tutoring Systems GnuTutor (http://gnututor.com/) OpenSource version of AutoTutor https://www.youtube.com/watch?v=k6HDEEnYuIA&feature=player_embedded AutoTutor https://www.youtube.com/watch?v=aPcoZPjL2G8

14. Technological trends Efficient testing Personalised learning 1990 1995 2000 2005 2010 2015 2020 2025 Generation 1 Generation 2 Generation 3 Adaptive Testing Automated administration and/or scoring of conventional tests Continuous integrated assessment • Multimedia collaborative learning environments support key competence development. • Peer and portfolio assessment evolve naturally. • Data mining and analysis will make integrated assessment and tutoring possible in the future. • mainstream use for high stakes testing • further development of innovative test formats • increased ease of use at school level • trend towards automated scoring • → cultural change: • increased acceptance of CBA Generation 4 Personalised feedback and tutoring Bunderson, Inouye & Olsen (1989). The four generations of computerized educational measurement. In R. L. Linn (Ed.), Educational measurement (Third ed., pp. 367-407). New York: Macmillan Bennett, R. E. (2010). Technology for Large-Scale Assessment. In P. Peterson, E. Baker & B. McGaw (Eds.), International Encyclopedia of Education (3rd ed., Vol. 8, pp. 48-55). ford: Elsevier. We are in a transitional period: Moving from the “Testing Paradigm” to the (future) “Tutoring Paradigm” in which explicit testing could become obsolete Generation Re-invention Learning 2.0 Collaborative multimedia learning environments Replication of complex real life situations within a test Computer-Based Assessment (CBA) Technology-enhanced learning

15. Assessing Key Competences Key Questions • Which ICT tools, formats, environments, devices etc. are most suitable for the assessment of which Key Competences? • How can ICT be used to support competence-based assessment strategies that adequately capture the skills and attitudes dimensions of all eight Key Competences and their transversal nature?

16. Focus Areas Intelligent Tutors etc. Peer Assessment Self Assessment Virtual worlds games Simulations ePortfolios Quizzes / simple games CBA

17. PART 1 INSPIRATION

18. Mother Tongue Literacy Matching spoken and written language http://www.familylearning.org.uk/sight_word_games.html

19. Foreign language learning http://www.nickjr.de/games/107-swipers-buchstabier-buch

20. Mathematical games Mathematical games using simulation for secondary school education Source: http://hotmath.com/games.html.

21. Peer Assessment for Collaborative Work Example: SparkPlus http://spark.uts.edu.au/

22. http://www.youtube.com/watch?v=Tp8v2UBbmN4 Peer Assessment using moodle www.moodle.org Moodle is a free web application that educators can use to create effective online learning sites. Moodle Statistics Registered sites: 77,213 Countries: 229 Courses: 7,116,466 Users: 66,349,750 Teachers: 1,292,731 Enrolments: 56,306,274 Forum posts: 118,413,272 Resources: 63,891,942 Quiz questions: 147,057,434

23. ePortfolios ePortfolios http://sites.google.com/site/eportfolioapps/overview/model-k-12

24. ePortfolios Level 1 - ePortfolio as Storage Collection: Creating the Digital Archive (regularly – weekly/monthly)A Focus on Contents & Digital Conversion Level 2 - ePortfolio as Workspace Collection + Reflection: Immediate Reflection on Learning & Artifacts in Collection (regularly) A Focus on Process& Documentation of Learning http://sites.google.com/site/eportfolioapps/overview/levels Level 3 - ePortfolio as Showcase Selection/Reflection/Direction (each semester? End of year?)A Focus on Product & Documentation of Achievement → A (formative/summative) assessment approach on the basis of artefacts → Focus on production and reflection

25. A game approach to arts ePortfolios meetVirtual reality Pheon is a multimedia scavenger hunt that tests players’ ingenuity through exploring the collection of the Smithsonian American Art Museum, making objects, and texting through the galleries http://pheon.org; http://pheon.org/intro.html; http://americanart.si.edu/multimedia/games/pheon.cfm.

26. Science: Molecular Workbench http://mw2.concord.org/public/part2/diffat/page3.cml

27. Simulations for mathematics Sample task from US future New State Tests(2014/15) http://www.k12center.org/rsc/pdf/a-sea-change-in-assessment-letter-size.pdf

28. Cognitive Tutor (Carnegie) http://www.youtube.com/watch?v=RR4HPrGlUz8&feature=related

29. Science: Quest Atlantis Example: Quest Alantis Quest Atlantis (QA) is an international learning and teaching project that uses a 3D multi-user environment to immerse children, ages 9-16, in educational tasks. QA combines strategies used in the commercial gaming environment with lessons from educational research on learning and motivation.

30. Citizenship Global Conflicts http://www.globalconflicts.eu/ Example: Child labour in Bangladesh: http://www.youtube.com/watch?v=mf9Ju86IVYw Educational game, designed to help teach concepts in citizenship, geography, and media. Developed by Serious Games International, it has detailed lesson plans and assignments for students.

31. Examples PART 2 CRITICAL INSPECTION

32. CBA forDigital Competence Digital Competence → Focus on knowledge Internet and Communication Core Certification (IC3) exam sample question http://www.certiport.com/Portal/common/htmllibrary/Sales-Video-Demos/demo_ic3.html

33. CBA forDigital Competence Digital Competence: IC3 → Focus on operational skills Internet and Communication Core Certification (IC3) exam sample question http://www.certiport.com/Portal/common/htmllibrary/Sales-Video-Demos/demo_ic3.html

34. Entrepreneurship Questionnaire Entrepreneurship Questionnaire ? → Focus on personality/disposition as a static fact https://www.careeringahead.com.au/survey/Entrepreneurship/Generic/questions.asp

35. Conclusions ICT offers powerful, engaging assessment formats, in particular: Self / Peer Assessment Educational software Quizzes / simple games ePortfolios CBA Simulations Virtual worlds/ games Basic literacy (foreign/mother tongue) Creative writing (foreign/mother tongue) Learning to Learn Scientific competence Mathematical competence Digital competence Social and civic competences Basic mathematical competence Cultural awareness and expression Caveat: CBA lends itself to simplified, knowledge-based assessment → Pedagogy comes first! Careful selection of question formats ICT Assessment approaches are underdeveloped for: Learning to Learn Sense of initiative and entrepreneurship Social and civic competences

36. The Paradigm Shift Era of Computer-Based Assessment (CBA) Era of Technology-enhanced learning Focus on: Efficient testing Focus on: Personalised learning 1990 1995 2000 2005 2010 2015 2020 2025 Technology CBA (Gen.1) Adaptive Tests (Gen.2) Automatic Scoring Gen. Re- Invention Learning 2.0 Immersive Learning Learning Analytics Intelligent Tutors Pedagogy Efficient Testing Validity of Scores Quick Feedback Authentic Contexts ? ? ? ? → Primacy of Pedagogy: Pedagogical guidelines for intergrated assessment

37. Thank you very much for your attention! Christine Redecker Yves Punie Christine.Redecker@ec.europa.eu Yves.Punie@ec.europa.eu http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=5719

38. further information ……http://is.jrc.ec.europa.eu Newsletters: http://is.jrc.es/pages/newsletter.html Publications: http://is.jrc.ec.europa.eu/pages/Publications.html Contact: jrc-ipts-is-NEWS@ec.europa.eu