Performance Metrics

1. Performance Metrics Improving Undergraduate Science Education at UBC Canadian Council of Deans of Science Conference Simon Peacock June 2008

2. UBC’s Carl WiemanScience Education Initiative CWSEI is an evidence-based approach to improving undergraduate science education (as opposed to approaches to teaching based on tradition or anecdote) “Science education is a science in itself, and a scientific approach can be applied to optimize its delivery andimpact.” Classroom studies Brain research Cognitive psychology http://www.cswei.ubc.ca

3. Research on Learning Learning Results from Traditionally Taught Courses Retention of information from lecture: 10% after 15 minutes lecture: Gain in conceptual understanding: 25% Beliefs about physics and problem solving: significant drop Courses Using Research-Based Pedagogy Retention of information from lecture: More than 90 % after 2 days Gain in conceptual understanding: 50-70% Beliefs about physics and problem solving: small improvement

4. Evidence-based Scientific Approach to Education • Establish what students should learn - measurable learning goals • Scientifically measure what they are actually learning • e.g., Force Concepts Inventory used in Physics • Implementation of teaching methods aimed at maximizing achievement • e.g., peer instruction, personal response systems, interactive computer simulations • Scholarly dissemination and adoption • Sustainable and continued optimization

5. Success requires: • Organizational change (an unnatural act) • Change at the course and departmental program level • Faculty changing the way they teach • Personality and self-awareness challenges • Cultural challenges (dept course vs. my course) • Substantial resources to effect transition

6. CWSEI Investments • Funding is based on Department proposals • Most departments have opted to use Science Teaching and Learning Fellows (STLFs) • Developing clear measures of student learning (assessment of educational outcomes) • Creating and testing new educational methods and materials • Developing faculty expertise in effective education

7. Performance Metrics • How many of our first-year courses specify (and distribute to students) clear learning outcomes? • Longer term goal - specify learning outcomes for all courses and programs • Student retention rates and graduation rates (cohort studies) • Reasons for students leaving (e.g., academically ineligible) • Careful analysis of inflows and outflows • Student evaluations • Review teachers with low “effectiveness” scores and require improvement plan • Focus groups and on-line evals provides rich feedback

8. Performance Metrics • Scrutinize courses with anomalous failure rates and/or anomalous grade distribution. Possible factors contributing to low success rates include: • Insufficient resources • Students are insufficiently prepared (prerequisites?) • Course is poorly taught (instructor, course activities) • Course expectations are too high* • Means of assessing performance are flawed*

9. In Development • Benchmarking of course (and lab) requirements in specific programs with comparable programs at peer universities • Number of students with substantive out-of-classroom educational experiences: e.g., - Co-op programs (18%) - Research experiences, Honours theses - Exchange program, Service learning, … • Number of courses taught by contract faculty • Number of faculty who do not teach undergraduates in a given year