Teague Orblych Michelle Dunaway

1. Formative Assessment: Transforming Education in the Library LOEX Fort Worth, Texas May 6, 2011 Teague Orblych Michelle Dunaway

2. Road Map • Introduction to clickers • Formative assessment overview • Background of project • Theoretical framework • Outcomes • Questions iStockPhoto.com: emptyclouds

3. Salted peanuts Almonds? Pecans? Cashews? Walnuts? Allergic to nuts? This is nuts!

4. Salted peanuts: Extroverted, sensitive, charming; enjoy parties and karaoke. Almonds: Scrupulous, conscientious, intensely motivated, moral perfectionists. Pecans: Devoted, loyal, true friends; generous, dependable, team players. Cashews: Empathic, calm, always knows just what to do. Walnuts: Aggressive, achievement oriented, competitive; natural leaders. Fisher Nut Study (Hirsch, 2010) For more information, visit: http://www.fishernuts.com/fisher/home.asp

5. “We start with the self-evident proposition that teaching and learning are interactive.” Black & Wiliam, 1998

6. I. Formative Assessment

7. Formative Assessment • A process wherein teachers make adjustments to instruction that is in progress, based on assessment-elicited evidence of students’ learning. Freedigitalphotos.net: Renjith Krishnan

8. Why formative assessment? Formative assessment is a useful way of addressing students’ widely varying information skills. iStockPhoto.com: subjug

9. II. Project development

10. The beginning Engaged faculty in the process of designing a way to: 1) instruct based on students’ existing skills 2) engage students in library instruction sessions. iStockPhoto.com: vernonwiley

11. Pre-assessment • This was done in order to develop a roadmap for each instruction session. • Also, this engaged students in the research process, giving them the opportunity to confront their own misconceptions about their research skills. • Note that this is not formative assessment; formative assessment takes place during instruction.

12. In-class assessment This grew from our efforts to engage students in the instruction session. How can we involve students in the process of creating instruction sessions that are responsive to their individual learning needs? Freedigitalphotos.net: Salvatore Vuono

13. Audience response system

14. I’ve used them a lot I’ve used them a little I’ve never used them I’ve thought about using them I’ve never heard of them What do you know about the clickers?

15. The Breakthrough: in-class assessment = formative assessment Freedigitalphotos.net: nuchylee

16. III. Theoretical framework • Deleo, Eichenholtz, & Sosin (2009): Bridging the information literacy gap with clickers, differentiated instruction • Oakleaf (2009): ILIAC, assessing and adjusting instruction over time • Black & Wiliam (1998): Formative assessment in k-12 Freedigitalphotos.net: digitalart

17. IV. Current methodology

18. The new deal Formative assessment allows instructors to make adjustments to teaching activities, based on students’ existing skills, while instruction is in progress.

19. The assessment process • Pre-assessment: assessment for learning • In-class assessment with clickers: formative assessment • Post-assessment: assessment for learning

20. Learning objective 1 Evaluating information and sources critically • Pre-assessment question: what features do you look for to identify high quality sources? • Session assessment question: what group of characteristics of an information source make that source high quality? • Post assessment question: what features do you look for to identify high quality sources?

21. Learning objective 1:Pre-assessment  in-class assessment (LING 482) Pre-assessment In-class assessment In your own words, what features do you look for to identify high-quality sources? What group of characteristics indicates a high quality source? 66% Peer-reviewed, expert author, researched, credible 6% Popular, from a known and trusted publication 0% Biased, popular, known to a few, unverifiable 29% Dated, incredible, scholarly, from a known and trusted publication 35% answered incorrectly!

22. Learning objective 2 Selecting methods and sources • Pre-assessment question: Describe, in detail, the specific steps you would follow to find a high quality source on this topic. • Session assessment question: Which research tool would be best to use to begin your search for a high quality source? • Post assessment question: Describe, in detail, the specific steps you would follow to find a high quality source on this topic.

23. Learning objective 2:Pre-assessment  in-class assessment(LING 482) Pre-assessment in-class assessment Describe, in detail, the specific steps you would follow to find a high quality source. 25% indicated that they would use the library 6% indicated that they would use the Internet 69% identified a specific library resource What research tool would you use to begin your search for a high quality source? 20% indicated they would use the library web site 6% indicated they would use the library catalog 28% indicated they would use a research database 0% indicated they would use the Internet 20% responded with less specificity than necessary.

24. Learning objective 3 Citing sources correctly • Pre-assessment exercise question: complete the process you described in question 2 to identify a high quality source on your topic, and give the citation of the best source you've identified. • Session assessment question: which of the following is a correctly cited source and could potentially indicate a high quality source? • Post assessment question: complete the process you described in question 2 to identify a high quality source on your topic, and give the citation of the best source you've identified.

25. Learning objective 3:Pre-assessment  in-class assessment (LING 482) Pre-assessment in-class assessment Provide the citation of the best source that you’ve identified. Which of the following is an example of a source that is cited correctly, and is likely a high quality source? 32% answered correctly 68% answered incorrectly 19 % provided a citation to a quality source, but did not correctly cite the source 50% provided a citation to a source that was not a quality source, or did not include a citation at all 31 % provided accurate citations to quality sources Based on the in-class assessment, we determined that we needed to address this topic during instruction.

26. Summary • In-class assessment represents the formative assessment part of this process. • Pre-assessment exercise provided a roadmap for each instruction session. • Post-assessment exercise provided a measure of the impact of formative assessment on the effectiveness of the instruction sessions.

27. V. Outcomes

28. Outcomes: Effectiveness Marked improvement: • Statistical increase in inclusion of peer review as a quality • Went from 52% in terms of using a specific library resource to 91% • Went from 30% in terms of locating and citing usable materials to 74%

29. The sessions were more fun! Outcomes: Engagement morgueFile.com: MissyRedBoots

30. Take aways • Formative assessment is the process of adjusting the content of library instruction sessions based on in-class assessment of students’ information skills. • Formative assessment can be applied to library instruction sessions by incorporating in-class assessments that elicit evidence of students’ mastery of information concepts, then adjusting instruction activities accordingly. • Audience response system technologies can be used in library instruction sessions to gather formative assessment data. iStockPhoto.com: STILLFX

31. Future research What’s next? iStockPhoto.com: Sharply_done

32. Questions? Contact us for more information: Teague Orblych mtorblyc@umd.umich.edu Michelle Dunaway mkd22@pitt.edu More information about this project can be found at: http://libguides.umd.umich.edu/LOEX2011FormativeAssessment