1 / 23

Evaluating Student Learning Experiences

Evaluating Student Learning Experiences. Michael Prosser Higher Education Academy. 1. Academy Mission. The Academy’s mission is to help institutions, discipline groups and all staff to provide the best possible learning experience for their students. Presentation based upon:

levi-mendez
Download Presentation

Evaluating Student Learning Experiences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluating Student Learning Experiences Michael ProsserHigher Education Academy 1

  2. Academy Mission The Academy’s mission is to help institutions, discipline groups and all staff to provide the best possible learning experience for their students.

  3. Presentation based upon: • Over 20 years research into the student learning experience in higher education in the United Kingdom, Australia, Sweden and Hong Kong • Experience in Australia with the Course Experience Questionnaire • Interpretations of the results of evaluations of student learning experiences are not value or theory free: • Interpretations in terms of student satisfaction • Interpretations in terms of student learning experiences

  4. Overview of the student learning perspective Figure 1: Model of Student Learning

  5. STUDENT APPROACHES TO LEARNING • Surface Approach • Intention to reproduce • - rote memorise information needed for assessment • - failure to distinguish principles from examples • - treat tasks as external impositions • - focus on discrete elements without integration • Deep Approach • Intention to understand • meaningfully memorise information for later use • - relate new ideas to previous knowledge • - relate concepts to everyday experiences • relate evidence to conclusions  • Approaches vary between tasks and modules

  6. STUDENTS’ EPERIENCES OF THE LEARNING CONTEXT • Early research by Entwistle and Ramsden (1983) using both interviews and questionnaires identified a number of student experiences relating to the way they approached their studies • Student experiences of: • Quality of teaching – including quality of feedback • (NSS: Teaching, Assessment and Feedback)) • Clearness of goals of course and standards of assessment • (NSS: Assessment and feedback) • Workload so high that it was not possible to understand everything • Assessment measuring reproduction and not understanding • were found to relate to how they approach their studies and to learning outcomes (exam results and other indicators)

  7. RELATIONSHIP BETWEEN COURSE EXPERIENCES AND APPROACHES Study of over 8000 students in first year subjects around Australia Amongst the data collected were students responses to: contextualised Ramsden's Course Experience Questionnaire and a contextualised Biggs Study Process Questionnaire 1994-1996: Australian Research Council ; Academic Departments and the Quality of Teaching and Learning; Paul Ramsden, Griffith University, Elaine Martin, RMIT, Michael Prosser, La Trobe University, Keith Trigwell, UTS

  8. Approaches to Study Surface Approach 32. Although I generally remember facts and details, I find it difficult to fit them together into an overall picture 35. The best way for me to understand what technical terms mean is to remember the textbook definitions Deep Approach 28. I try to relate ideas in this subject to those in other subjects, wherever possible 34. In trying to understand new ideas, I often try to relate them to real life situations to which they might apply. Biggs Study Process Questionnaire

  9. Student Experiences Of Learning Environment Good Teaching 15. The staff made a real effort to understand difficulties students might be having with their work. Clear Goals and Standards 1. It was always easy to know the standard of work expected 6. I usually had a clear idea of where I was going and what was expected of me in this subject. Appropriate Workload 25. The sheer volume of work in this subject meant that it couldn't all be thoroughly comprehended (-). Appropriate Assessment 8. To do well in this subject, all you really need is a good memory (-). Ramsden’s Course Experience Questionnaire

  10. Factor Analysis of Experiences of T & L Context and Approach to Study _________________________________________________________________ Scale Factors ____________________________________ 12 _________________________________________________________________ Experiences of Context Good teaching .80 Clear Goals and Standards .67 Appropriate Workload -.69 Appropriate Assessment -.65 Approach to Study Surface Approach .81 Deep Approach .73 _________________________________________________________________ Principal Components, Variamax Rotation, n=8837

  11. In each subject: • 1.   A deep approach is associated with experiences that the teaching is good and the goals and standards are clear (NSS: Teaching, Assessment and Feedback) • 2.   A surface approach is associated with experiences that the workload is too high and assessment tests reproduction • That is, variation in students’ experiences of the learning environment within subjects is associated with the approaches to study within subjects – within subject variation in perception is not measurement error. • 1994-1996: Australian Research Council ; Academic Departments and the Quality of Teaching and Learning; Paul Ramsden, Griffith University, Elaine Martin, RMIT, Michael Prosser, La Trobe University, Keith Trigwell, UTS

  12. Comments on NSS • Substantial amount of research linking results on surveys such as the NSS with student approaches to study and learning outcomes • Student experiences are a function of both their prior experiences and understandings and the course design and teaching • Do not necessarily improve student satisfaction by focusing on satisfaction - the individual items or scales – need to better understand why they responded the way they • Spread of results – proportions responding in certain ways – better than mean of responses • Changes is scores over time of surveys such as the NSS represent broad cultural change – departmental & institutional experiences • Can only expect small effect sizes in changes in scores over time - .2 of a SD – if over 3 - 5 years changes of the order of .1 to .2 points (3.5 to 3.6 or 3.7). • Follow up with more qualitative studies – interviews, open-ended written statements, focus groups to better understand experiences

  13. Overview of the student learning perspective Figure 1: Model of Student Learning

  14. Recent Review of Research and Evaluation on Student Learning Experience in • E-learning • Rhona Sharpe, Greg Benfield, Ellen Lessner and Eta DeCicco,: • Much research on design of learning experiences – teacher, course or program focused • Much research on student observable behaviours • Little or no research on the student learning experiences of those designs (intentions, perceptions, beliefs, outcomes etc) • Little research on the student experience of the integration of online with face-to-face – how e-learning relates to their whole learning experience and to their lives and jobs

  15. Learning Through Blended Discussion • Peter Goodyear, Rob Ellis, Mike Prosser • (acknowledge Australian Research Council Support) • Work in Progress • Student experience of learning through online and face-to face discussion • Using a phenomenographic approach to analysing student learning experiences • Key aspects of differences in the way something is experienced – not rich, thick descriptions • Second-order description and analysis – not describing what it is but how it is experienced. • Categories of description

  16. Investigate associations amongst: • how the students approached their discussion within blended learning context, • how this variation was related to the quality of what they thought they were learning through discussion, and • their performance in the course as a whole • In-depth, semi structured interviews; short open-ended questionnaire, assessment results • Interviews: 30 minutes, transcribed, main data for construction of categories and outcome space • Short open-ended questionnaire – same questions and leading questions for interview

  17. Questions: • What did you learn through discussion in your course? This includes all of the discussions that you were involved in in the course • How did you approach engaging in face-to-face discussion in your course? What sorts of things did you do to engage in the discussions? Why did you use these strategies to engage • How did you approach engaging in online discussion in your course? What sorts of things did you do to engage in the discussions? Why did you use these strategies to engage • Sample: • 51 second your university undergraduate students studying social work in a large, research intensive metropolitan Australian University – 19 completed semi-structured interviews

  18. Categories of Description of Conceptions of Learning through Discussions

  19. Categories of Description of Approaches to Learning through Online Discussion

  20. Chi-square=24.2, phi=.69, p<.001 Statistically significant and large effect size No cases of Fragmented conception of learning with Deep approach to discussion on-pile Student understanding of why they are engaging in discussion fundamental to adopting deep approaches to discussion online.

  21. Relationship between Conceptions, Approaches and Performance

  22. Focus on the student experience rather than satisfaction – better understand their experiences • Comprehensive and aligned set of student evaluation instrument designed and interpreted in terms of student experiences • Conceptual model of teaching and learning based upon the student experience underlying all developments • Follow up with more in depth qualitative studies • Focus on the student experience

More Related