Assessment and Learning: A Vision

1. Assessment and Learning: A Vision Valerie Shute, FSU Webinar (Oct. 1, 2010)

2. Shute, V. J., Levy, R., Baker, R., Zapata, D., & Beck, J. (2009). Assessment and learning in intelligent educational systems: A peek into the future. In S. D. Craig, & D. Dicheva (Eds.), Proceedings of the Artificial Intelligence and Education (AIED '09) Workshop on Intelligent Educational Games (pp. 99 – 109). Brighton, UK. http://myweb.fsu.edu/vshute/pdf/peek.pdf Assessment Learning Valerie Shute, FSU Webinar (Oct. 1, 2010)

3. Education, Writ Large Education is a social process. Education is growth. Education is not a preparation for life; education is life itself. –Dewey

4. Why Education Needs to Change The world has changed a lot in the past 100 years. Education has not. Classroom photo, 1910. Classroom photo, 2010.

5. Why Assessment Needs to Change

6. Learning & Assessment Definitions ASSESSMENT: Collecting, analyzing, and interpreting info about student understanding and/or performance re: educational goals. Various procedures used for evaluating student work & learning. LEARNING: Lifelong process of accessing, interpreting, and evaluating information & experiences into knowledge, skills, behaviors, values, identities, and dispositions.

7. Current Assessment Practices Student Learning Teach. Stop. Test. Repeat.

8. Assessment Student Learning

9. Collaborative Learning

10. Fuse Learning & Assessment

11. Themes

12. Seamless & Ubiquitous Assessment Comprehensive Learner Models Assessment Info for Decision Making Theme 1 Theme 2 Theme 3 Aligned along 3 dimensions: vertically, horizontally, and temporally.

13. Setting Up A Metaphor for Theme 1 On June 26, 1974, the first product with a bar code was scanned at a check-out counter. It was a 10-pack of Wrigley's Juicy Fruit chewing gum. Today, the pack of gum is on display at the Smithsonian Institution's National Museum of American History. 1974

14. Theme 1 Metaphor (via Pellegrino, Chudowsky, & Glaser, 2001) Business • Before 1974, businesses had to close down once a year to take inventory. • Barcodes enabled access to a continuous stream of informationto monitor inventory. • Now businesses continue without interruption, supporting real-time inventory management.

15. Theme 1 Metaphor (via Pellegrino, Chudowsky, & Glaser, 2001) School • Schools shouldn’t have to interrupt normal instruction to give external tests. • Instead, assessment should be continual & invisible, supporting real-time, JIT learning. • This is the first of 3 themes envisioned, 20 years hence (seamless & ubiquitous assessment).

16. Stealth Assessment • Advances in measurement let us administer evidence-based assessments to: • Extract ongoing information from a learner • Make accurate inferences of competencies • React in immediate and helpful ways. • Accomplished via automated scoring and machine-based reasoning techniques. • Seamlessly woven into the fabric of the learning environment.

17. How Does It Work? • Stealth assessment can be used to improve learning: • Gathering evidence for accurate estimates of competencies • Info on competencies used by teachers, the system/game, and/or students • Current estimates of competency levels integrated into the game and displayed as progress indicators. • This elevates valued competencies to the same level as health and weapons!

18. Theme 1: Summary “Seamless and Ubiquitous Assessment” Idea: A continuous process fusing assessment and learning. Seamless: Removal of false boundaries between learning and assessment. Ubiquitous: Constant nature and need to feedback back results of assessment into learning.

19. Theme 2: Comprehensive Models Current State Goal State

20. Theme 2: Comprehensive Models • Research needed to identify, model, and assess21st century competencies as well as other attributes, like affective variables, values, dispositions, etc. • These will comprise rich evolving profiles representing a synopsis of what’s known, what can be done, what’s believed, preferred/valued, and felt.

22. ECD focuses on the evidence of competencies. It provides a way to reason about assessment design, and a way to reason about learner performance. As an assessment framework, it can support both Theme 1 (seamless/ubiquitous assessment) and Theme 2 (comprehensive models).

23. Assessment Design Competency Model What do you want to say about the person? Evidence Model What observations would provide best evidence for what you want to say? Task/Action Model Model What kinds of tasks let you make the necessary observations?

24. Design and Diagnosis For more information about ECD, please see ECD for Dummies.

25. Example Conceptual Model

26. Example Computational Model

27. Summary of ECD ECD should be used as the framework for new assessments because it can: • Yield valid assessments for different purposes • Provide accurate estimates of complex competencies and dynamic performances • Aggregate information from various sources • Afford transparency to stakeholders via evidentiary reasoning to support claims.

28. Wrapping It Up • New competencies need to be identified, modeled, assessed and supported. • Assessments need to be valid and reliable to meet our educational needs (e.g., ECD and stealth assessment). • Immersive games are fun, and enable learning within complex and relevant environments.

29. Themes over time Wow! Theme 1 Theme 2

30. Major Funding Targets • Fusing assessment and learning • Understanding full complement of characteristics involved with learning • Rendering assessments useful to all parties

31. Thank you! Questions? Email: vshute@fsu.edu Website: http://myweb.fsu.edu/vshute

32. Collaboration Critical Thinking Systems Thinking Curiosity Creativity Grit Empathy Go Back

34. Barriers (or Opportunities!)