Engaging students outside of class as a tool for enhanced learning

1. Engaging students outside of class as a tool for enhanced learning Dan Bernstein University of Kansas UBC Centre for Teaching and Academic Growth 22 November 2006 djb@ku.edu

2. Unhappy with answers to questions • What is the major point of quantum mechanics that makes prediction impossible? What level of prediction is affected? What level is not? • What effect do quantum mechanics have on scientific objectivity as part of our process of knowing? • Describe the Heisenberg uncertainty principle (briefly). What relevance do you think it has for contemporary research and clinical practice in psychology? • Does quantum theory render Newtonian mechanics useless to humans in their interactions with the physical environment? Why or why not?

3. Simple problem led me onto a path of continuous inquiry that has lasted a decade

4. Started with Experimental Studies • In search of the best way to generate high level student answers to those questions • Compared multi-media presentation with lecture, reading, and irrelevant information • Measured rote and complex understanding with multiple choice, fill-in, short essay • Advantages of random assignment and controlled conditions • Goal ultimately a practical one

5. Basic Procedures • Commercial and home-made multi-media • Used video and live lecture with same content as multi-media • Created text samples with same content • Assessment of levels of understanding with mixed methods of responding • Individual instruction with unlimited time • Participants from introductory psychology

6. Learning from Lecture

7. Learning from Lecture v. Reading

8. Lecture, Reading, and Computer

9. Percentage Correct by Instructional Mode and Level of Understanding

10. Percentage Correct by Instructional Mode and Question Type

11. Mean Instructional Time by Instructional Mode

12. Upgraded the lecture component • Live lecture to portion of introductory class • Topic is reactive measurement and quantum mechanics • Matched to locally authored multi-media and text -- also irrelevant text control • Motivation sustained by possibility of test • Assessment layered by type of understanding and format of question

13. Distribution of total test performance resulting from four methods of instruction

14. Number of Correct Responses Across Levels of Understanding

15. Time on task (in minutes)

16. What do you make of these data? • How would it help me teach? • Any ideas for next innovation? • Critical reactions to the evidence?

17. My take home messages • Students learning by computer or reading showed greater and deeper understanding • Multi-media has potential but very high cost to get started • Reading consistently took the least time • Reading is a good way to learn if you have a way to assure that it is done

18. Next problem arose when I presented my data • Challenged by Tom Hatch, from Gardner’s Project Zero group • Described “Deep Understanding” • Use ideas in a context not taught • Ideas retained without memorization • I presumed that abstract was more general • Agreed to test the idea in my class

19. Tracking understanding over time • Learning and motivation • Sophomore level course • 20 to 45 students • Taught 9 times in 7 years • Professional grade reading • Hybrid online and F2F • Largely discussion in class • All essay examinations • Conceptual writing required

20. Wanted Richer Essay Exams • Why is magazine training so critical to the process of shaping a new response class? Sometimes the sound of the operation of the food magazine is called a bridging stimulus; what might this metaphor mean? • What is the evolutionary mechanism for the cultural survival of religious taboos against slaughtering and eating cattle? You should identify the conditions that make this evolution most likely. • Explain the notion of selection by consequences as a description of the change in distribution of behavior that takes place during shaping. How is this process different from selection that takes place under naturally occurring conditions?

21. Reasonably Conceptual Items • Describe the results of [Razran’s] second experiment on semantically mediated generalization (numbers) done with a 13-year old boy. Be sure to identify the respondent behavior, the training stimuli, and the test stimuli. Why is this example semantically rather than physically mediated generalization? • What do chicken scratches, raccoon rubbings and pig rooting have in common? What stimuli have pre-existing relations these bits of behavior? What does the “misbehavior of organisms” have to do with the generality of principles of learning?

22. Transform into Problems • Old question: • What were the reinforcing consequences in the Welsh, Bernstein & Luthans (restaurant) study? How were the consequences identified? • Problem-based Assessment : • Suppose you were asked to implement a motivational program in a business with 25 employees engaged in the following activities: production planning, inventory delivery, direct production, packaging, and marketing. Based on your understanding of the restaurant study, how would you improve the quality of the employees work by using access to activities as a motivator? Describe the costs and benefits of the program and make a recommendation about implementation.

23. Old question: • The behavioral context into which punishment is added must be considered in evaluating the likely effect of a given stimulus. What conditions are essential in this analysis? • New question: • Suppose it is your task to decrease the frequency of college students engaging in binge drinking or games involving high levels of intoxication. You have been asked to set up a punishment program to eliminate this program before someone is hurt. What context for binge drinking would you identify first? What punishing consequence would you use? What characteristics would you include in your punishing system to maximize effectiveness? Please give an example of an additional element you would need to include to make the plan maximally effective.

24. Points for Question 1 • 4pts - reinforcement for drinking • 5 pts - ID specific severe punishing consequence • 4 pts - Practical plan for measurement of abusive drinking • 12 pts - Immediate consequence (4 pts); severity (more than change action) (4 pts); consistent system (4pts) • 5 pts - reinforcement plan for alternative actions that are preferred, if possible using original class of reinforcers

25. Old question: • Describe the results of the experiment on semantically mediated generalization done with a 13-year-old Soviet boy. Be sure to identify the respondent behavior, the training stimuli, and the test stimuli. Why is this example semantically rather than physically mediated generalization? • New question: • Generate your own example of classical conditioning using the neutral stimuli “idea” and “rock” along with the eliciting relation between a very loud noise and increased heart rate. Your example should include all of the following components: a description of a conditioning procedure that would produce differing reactions to the two stimuli, a description of the procedure that tests for the direct effects of conditioning, a description of a procedure that would test for physically mediated generalization, a description of a procedure that would test for semantically mediated generalization, and the likely results of the three test procedures.

26. Points for Question 2 • 4 pts - specific conditioning -- say “idea” (2) and then puff of air in eye (2) • 7 pts - Direct test: Present “idea” (3) w/o air (2) and see blink (2) follows • 7 pts - Physically mediated generalization: Present sound similar to “idea” (3) w/o air (2) and see weak or little blink (2) • 7 pts - Semantically mediated generalization: Present sound or picture or word related to “idea” (3) w/o air (2) and see strong blink (2) more than physically mediated

27. Use existing work:Keep an archive of evidence

28. Distribution on 1st Attempt

29. Change of Questions

30. What do you make of the data? • Is this a surprise? • What examples do you have in your own work? • What does it take to “problematize” a concept for assessment? • Keep going in this direction or turn back to abstraction?

31. I relied on my experimental results • The key is getting students to read • If only they were prepared, we would all be better off • Post hoc argument from fluency -- free up mental resources for problem solving [away from details]

32. Organizing units by topics

33. Online questions on reading

34. Feedback until mastery of topics

35. Students consider reading

36. Complete roster services

37. Click on name for full record

38. See items, make comments

39. Aggregate Performance 1st Try

40. Addition of EDU Out of Class

41. What do you make of these data?Would you keep using it? I did.

42. Next idea came from Grant Wiggins • Welding teacher story • Generate internalized criteria • Like composition studies and editing • Used prior student work with permission • No feedback from me

43. Bb - consider others’ work

44. Students share reactions w/o me

45. Exam - Eliminate Problem Actions

46. Exam Question onCultural Anthropology

47. Take home complex essay

48. Add Generation to Recognition • Posted another variation on the question • Each student was to write an answer • Also expected to comment on one answer • Collected a sample of answers • They were focus of class discussion • Students comment on completeness • Offer suggestions to refine answers

49. Learning With Full Techno BoostReport: http://www.unl.edu/peerrev/examples/bernstein/index.html

50. Comparison with last three offerings (percent of students)