It's A ll in What You Ask: Techniques for Enhancing Reflection and Learning in an Online Course

1. It's All in What You Ask: Techniques for Enhancing Reflection and Learning in an Online Course Karen Diaz Nancy O’Hanlon LOEX Conference, April 2010

2. Agenda • Describe several techniques for encouraging metacognition and improving student learning. • Discuss how we used these techniques in an online course to enhance understanding of a difficult concept. • Demonstrate methods that you could apply in your own teaching.

3. Clicker Test: Are you really in the right room today? • Yes • Probably • Not sure • No, but too embarrassed to leave now. Answer Now

4. Speed kills . . . brain cells! "Deep log studies show that, from undergraduates to professors, people exhibit a strong tendency towards shallow, horizontal, 'flicking' behaviour in digital libraries. Society is dumbing down." Information Behaviour of the Researcher of the Future, 2008. http://www.ucl.ac.uk/infostudies/research/ciber/downloads/ggexecutive.pdf

5. Fostering Habits of Reflection • We teach students to skim and make snap judgments about quality. • Reflection requires more time and conscious effort.

6. Why? Improved Outcomes • Learning is enhanced when students engage in metacognitive activities such as self-assessment and monitoring (Lin, 2001). • Self-monitoring is particularly important in Web-based environments. • Tools or features prompting students to reflect on learning were effective in improving outcomes (Means et al, 2009).

7. How? Prompts • Prompts (questions) are an important tool for encouraging reflection. • Prompts that ask students to assess what they understand and where they should direct additional effort were effective in research studies on information skills (Stadtler, 2007; Wopereis, 2008).

8. How? Process Modeling • Lin (1999) notes that “reflective thinking involves actively monitoring, evaluating, and modifying one’s thinking and comparing it to both expert models and peers.” • Pace & Middendorf (2004) suggest that instructors can help students overcome learning obstacles (bottlenecks)by reflecting in depth on the steps an expert would take to accomplish a particular task and then modeling it for students. 

9. Techniques: Summary

10. Reflection 1 Think about your own teaching experiences. Is there a concept or skill that your students find particularly difficult to grasp? How could you provide opportunities for students to assess their own progress?

11. Our Approach: Action Research

12. New Course Content: Movie Recognizing Bias: • Providesa definition of bias. • Offers a rationale for learning the skill of assessing bias. • Introduces a cognitive strategy, represented as a mnemonic, MAPit. 

13. Modeling the Process

14. Deconstructing the Message

15. Recognizing Bias Movie http://liblearn.osu.edu/movies/bias.htm

16. New Course Content: Quiz • Recall: 4 multiple choice questions that assess recall of content from the movie. • Application: 2 multiple choice questions with links to websites, to assess students’ ability to apply the MAPit strategy. • Reflection: 2 open ended questions (metacognitive prompts) intended to encourage reflection by students.

17. Bias Quiz Knowledge • Question 1 (no change from Autumn 2008 to Winter 2009) • The letters MAP in the MAPit strategy for recognizing bias stand for: • Medium, Audience, Purpose • Message, Author, Purpose • Message, Audience, Practicality

18. Bias Quiz Knowledge • Question 2 (no change from Autumn 2008 to Winter 2009) • Worthwhile factors to consider in analyzing a message for bias are whether it is: (Select ALL correct options) • Fact or opinion • Sources cited or not cited • Right or wrong • Fair or selective

19. Bias Quiz Comprehension • Question 3 (no change from Autumn 2008 to Winter 2009) • Why does an author's credibility matter when looking for bias? • 1) Authors with deep knowledge of a field, and who have withstood scrutiny of reviewers, are less likely to be biased. • 2) Authors who have been reviewed a lot are likely to say what they think the reviewers want to hear, so may be more biased. • 3) Authors who are new to a field bring an important fresh perspective that is less likely to be biased.

20. Bias Quiz Comprehension • Question 4 (no change from Autumn 2008 to Winter 2009) • What factors can help you determine if the purpose of a publication is partisan? (Select ALL that apply) • Where it gets its funding • How many members it has • It's own mission statement • Who is on its board of directors

21. Bias Quiz Application • Question 5 (no change from Autumn 2008 to Winter 2009) • Using the strategies described in the movie, connect to the following site and analyze it for bias. Drug Policy Alliance • Which statement is most true about this site? • 1) It is unbiased • 2) It  is biased • 3) It is commercial

22. Bias Quiz Application • Question 6 (no change from Autumn 2008 to Winter 2009) • Using the strategies described in the movie, connect to the following site and analyze it for bias. Who's Watching What You Eat? • Which statement is most true about this site? • 1) It is unbiased • 2) It  is biased • 3) It is commercial

23. Bias Quiz Reflection • Question 7 (Autumn 2008) • How well do you understand what is needed and how prepared do you feel to recognize bias in other information you encounter?  • (To receive credit, your answer must include at least one complete and clear sentence.)

24. Bias Quiz Reflection • Question 7 (Winter 2009) • How well do you understand what is needed and how prepared do you feel to recognize bias in other information you encounter?  • 1) I understand the concepts related to, and feel confident that I will be able to recognize bias in all other information sources. • 2) I understand the concepts needed to recognize bias, but think it may still be difficult to recognize bias in some information sources. • 3) Not all the concepts related to recognizing bias are easy for me, and am not confident in my ability to recognize bias in all information sources. • 4) I do not feel prepared at all to recognize bias in information sources.

25. Bias Quiz Reflection • Question 8 (Autumn 2008) • What do you think you should do to improve your own abilities for recognizing bias?  • (To receive credit, your answer must include at least one complete and clear sentence.)

26. Bias Quiz Reflection • Question 8 (Winter 2009) • What do you think you should do to improve your own abilities for recognizing bias?  • 1) I need to remember to apply the MAPit strategy in the future. • 2) There are one or two elements in the MAPit strategy that I know are challenging for me, and need to pay attention to those when I try to detect bias. • 3) I need to pay more attention to elements from information sources (such as author credentials, or "about us" links). • 4) I should spend time practicing going to web sites and applying the MAPit strategy to get good at it.

27. Bias Quiz - Reflection

28. Capstone Application • Question 8 (Autumn 2008 and Winter 2009) • What is the PRIMARY purpose of this site? (Pick one option below and type it into the answer box): • Advocacy (sway opinion or get support for a cause) • Reference/Information (provide balanced view of a topic) • Commercial (sell a product; offer product support)

29. Capstone Application • Question 11 (Autumn 2008 and Winter 2009) • Evaluation of Web Sites--Content Bias/Balance • An advocacy site may attempt to sway the reader's view one way or another, while a reference/information site will provide a balanced, objective look at the topic. Examine some of the pages on the site before answering the following questions. • Also, remember the MAPit strategy for recognizing bias that wasintroduced in the movie you viewed during week three.  Be sure to consider these factors when you make a decision. • Choose the best description of the site's content from the options listed below.. • Biased (slanted) • Balanced (objective)

30. Capstone Analysis • Question 24 (Autumn 2008 and Winter 2009) • Write a paragraph covering all of the points listed below. Use complete sentences. Your answer will be judged on accuracy, clarity of expression, and completeness. • For the site that you found with your search (Questions 22, 23), indicate: • Who is the author? Provide some background information about the author. • What is the purpose of this site? Find a statement of purpose on the site and quote from it. • When was this site last updated? Indicate where you found this information. • Does the site's content appear balanced (objective) or biased (slanted)? Provide support for your answer.

31. Bias Quiz - Performance

32. Capstone - Performance

33. Capstone – Qualitative look at Performance • Case Studies • Terminology • Reflective • MAPit (instructed)

34. Applied to another course • Provide students with an article to read • How We Failed the Net Generation by William Badke • Quiz them over it

35. Article Quiz Comprehension • Question 1 • This article critiques the current state of information literacy and offers suggestions to improve information literacy education.  Who is the intended audience of this article? • a) Those teaching information literacy at the University level. • b) University students trying to understand information literacy. • c) Those teaching information literacy at the K-12 level.

36. Article Quiz Knowledge • Question 2 • Why might a library database present a better alternative to Google? • a) Library materials are reviewed and regulated. In contrast, no one checks the reliability of Web pages. • b) Libraries are better funded than most Web pages. Therefore, they are more accurate. • c) The author states that Google is the best way to search for library materials.

37. Article Quiz Knowledge • Question 3 • Where does the author think students learn most of their Internet search skills? • a) At home. • b) At their hometown libraries. • c) In their K-12 education.

38. Article Quiz Knowledge • Question 4 • Why does the author think it is important to teach stronger Web search skills? • a) In the author's opinion, the Web is really only good for finding music or blogs. • b) Information provided by search engines is limited and often lacking precision when compared to library databases. • c) The author is a librarian and wants to promote libraries.

39. Article Quiz Reflection Question 5 After reading this article, pick which of the following assumptions the author makes that you MOST agree with:

40. Your Turn to Reflect • I understand what reflective questions look like and will be able to apply them to my assignments. • I need to know more about the use of reflection and metacognition before trying to use it. • I will probably never build reflective questioning into my assignments. • I feel something different than any of these alternatives.

41. References • Lin, X., Hmelo, C., Kinzer, C.K., & Secules, T.J. (1999). Designing technology to support reflection. Educational Technology Research and Development47 (3), 43-62. •  Lin, X. (2001).  Designing metacognitive activities. Educational Technology Research and Development49, 23-40. • Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Washington, D.C.: U.S. Dept. of Education, Office of Planning, Evaluation, and Policy Development. Retrieved from: http://www.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf • Pace, D. & Middendorf, J. (Eds.) (2004). Decoding the disciplines: Helping students learn disciplinary ways of thinking. New Directions for Teaching and Learning, 98. San Francisco: Jossey Bass. • Stadtler, M. & Bromme, R. (2008).  Effects of the metacognitive computer-tool met.a.ware on the web search of laypersons.  Computers in Human Behavior24 (3), 716-37. • Wopereis, I., Brand-Gruwel, S., & Vermetten, Y. (2008).  The effect of embedded instruction on solving information problems. Computers in Human Behavior24 (3), 738-52.