1 / 33

End of lecture: The future of evidence-based teaching

End of lecture: The future of evidence-based teaching. Sub-subtitle: PLEASE understand how important you are. Why are we still lecturing? . I don’t believe that active learning can work in a large lecture. (UW professor, 8/12).

obert
Download Presentation

End of lecture: The future of evidence-based teaching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. End of lecture: The future of evidence-based teaching Sub-subtitle: PLEASE understand how important you are

  2. Why are we still lecturing?

  3. I don’t believe that active learning can work in a large lecture. (UW professor, 8/12) • I just know that students .... (UW professor, 3/09) • Although it did not occur to us .... to collect data, we consistently observed … (Barzilai 2000) • … we feel that our junior-senior cell biology course ... works extraordinarily well …” (Lodish et al. 2005) • We think that our objective of teaching the students to think was well-accomplished. (Miller & Cheetham 1990) • We strongly believe that they lead to deeper understanding.... (Rosenthal 1995)

  4. I don’t believethat active learning can work in a large lecture. (UW professor, 8/12) • I just know that students .... (UW professor, 3/09) • Although it did notoccur to us .... to collect data, we consistently observed … (Barzilai 2000) • … we feel that our junior-senior cell biology course ... works extraordinarily well …” (Lodish et al. 2005) • We think that our objective of teaching the students to think was well-accomplished. (Miller & Cheetham 1990) • We strongly believe that they lead to deeper understanding.... (Rosenthal 1995)

  5. Today’s question: Is the first generation of research on undergrad STEM education over? aka: Does active learning really work? Started this project on: 2 January 2008 “Ended” this project on: Started this project on: “Ended” this project on: 12 May 2014

  6. A meta-analysis: Five criteria for admission Contrast any active learning intervention with traditional lecturing (same class and institution); “cooperative group activities in class,” worksheets/tutorials, clickers, PBL, studios … 2. occurred in a regularly scheduled course for undergrads; 3. limited to changes in the conduct of class sessions (or recitation/discussion); 4. involved a course in Astronomy, Bio, Chem, CompSci, Engineering, Geo, Math, Physics, Psych, Stats; 5. included data on some aspect of academic performance—exam/concept inventory scores or failure rates (DFW).

  7. Searching Hand-search (read titles/abstracts) every issue in 55 STEM education journals from 6/1/1998 to 1/1/2010; 2. query seven online databases using 16 terms; 3. mine 42 bibliographies and qualitative or quantitative reviews; 4. “snowballing”—check citation lists of all pubs in study.

  8. Coding 642 papers: SF reads 5 criteria? no yes 398 two coders (SF + MPW, MKS, MM, DO, HJ) 244 “easy rejects” • confirm 5 criteria? no reject • identical assessment, if exam data? yes • students? • instructor? • meta-analyzable data? (exam scores; DFW) no reject Missing data search (91 papers, 19 successful) Data analysis: 225 studies (SE)

  9. Results: Failure rate data Biology Chemistry Computer science Engineering Geology Math Physics Overall Overall odds ratio = 1.94 • Average failure rate 21.8% vs. 33.8% = a 55% increase • Risk ratio = 1.5; students in lecture are 1.5x more likely to fail STEM discipline • Biomed RCTs stopped for benefit: mean relative risk of 0.53 (0.22-0.66) and/or p < 0.001. • In our sample: 3,516 fewer students would fail; ~$3.5M in saved tuition.

  10. Other results: • No difference in failure rates in small, medium, vs. large classes • No difference in failure rates for intro v upper-division courses

  11. Are results due to publication bias? (file drawer effect) Standard error

  12. Results: Exam performance data Overall effect size = 0.47 • In intro STEM, 6% increase in exam scores; 0.3 increase in average grade. • Students in 50th percentile under lecturing would improve to 68th percentile. … other meta-analyses … K-12

  13. What is the effect size for concept inventories vs. instructor-written exams?

  14. Do effect sizes vary with class size?

  15. Other results: • No difference in effect sizes for majors v non-majors courses • No difference in effect sizes for intro v upper-division courses • Of the original studies with statistical tests, 94 reported significant gains under active learning while 41 did not (70%)

  16. Does variation in methodological rigor—control over student equivalence—impact effect sizes?

  17. Does variation in methodological rigor—control over instructor equivalence—impact effect sizes?

  18. Are results due to publication bias? (file drawer effect)

  19. Statistical tests confirm asymmetry in the plot for exam scores (no asymmetry in the plot for failure rates). • BUT, • calculating fail-safe numbers (# studies of 0 effect to make overall effect size trivial); • Duval & Tweedie’s trim-and-fill (substitute missing studies and re-calculate effect size); • and analysis of extreme values (drop and re-calculate) • …. all suggest no substantive impact

  20. TO SUMMARIZE: Two fundamental results • Students in lecture sections are 1.5 times more likely to fail, compared to students in sections that include active learning; • Compared to students in lecture sections, students in active learning sections have exam scores that are almost half a standard deviation higher—enough to raise grades by half a letter. Note: students who leave STEM bachelor’s or associate’s degree programs have GPA’s 0.5 and 0.4 lower than persisters. Seymour & Hewitt, other studies on STEM retention: higher passing rates, higher grades, and increased engagement in courses all play a positive role.

  21. Fallout I: follow-up pubs in PNAS

  22. Fallout II: Professional press—“the industry” takes note

  23. Fallout III: Popular press—a broader conversation Can’t beat Bowie retirement You are here BUT! We top the Pope baptizing aliens & Sean Connery’s soccer career

  24. Fallout III: Popular press—a broader conversation

  25. What now? Is it really the End of Lecture? Note: In the K-12 world, effect sizes of 0.20 are considered grounds for policy interest. Note: Work here and elsewhere has shown that active learning has disproportionate benefits for students from disadvantaged backgrounds—it closes the achievement gap. Michael Young But is change possible in the university setting?

  26. 1. The Rider is the evidence that change is good—the knowledge. 2. The Elephant is the emotional element—the ganas. 3. The Path is the tools and resources that make change possible—the how-to.

  27. 1. The Rider is the data. Now we need … 2. The Elephant is a commitment to evidence-based teaching, and a system that rewards it. 3. The Path is reading, listening asking, copying … and a willingness to experiment—to start small and fail at first.

  28. My perspective: Policy-makers have been trying to advocate change from the top-down for 30 years. The system (tenure, IDCR, departmental culture) is extremely conservative. It has not changed, and it will not change. • If students (and parents) demanded excellence—meaning, evidence-based teaching—change would happen in a HURRY.

  29. Thanks to: You, for producing the evidence that will make our faculty better teachers and our students better learners. And the molecules that made this study possible:

More Related