Learning Set liaison: Felicity Rose, Pharmacy Learning Set members: John Harris, Biosciences

1. Evaluating the Learning Outcomes of the Undergraduate Project: Perceptions vs. Reality. Learning Set liaison: Felicity Rose, Pharmacy Learning Set members: John Harris, Biosciences Sarah McMullen, Biosciences Rong Qu, Computer Science & IT Angus Davison, Biology Martin Gering, Biology Richard Roberts, Biomedical Sciences Jim Maas, Biosciences In Memory of our Learning Set Advisor, Dr Martin Willis

2. Background • The final year undergraduate research project (UGRP) is: • An opportunity to carry out an independent, intensive piece of original work. • The project may take different forms (Cowie, 2005, Hollingsworth, Mahon & Thomas, 2004), but the main criteria are the same: • Independent Working. • Problem Solving. • Critical Analysis (literature, data). • Apply research approaches and methods. • Communication Skills. • QAA encourages inclusion of UGRP & is required by certain professional bodies (e.g. RPSGB).

3. Previous Work • Ryder & Leach, 1999 (University of Leeds) • Research environment affects motivation of students. • Murphy & Valenzuela, 2002 (Pharmacy USA), Ali & Seville, 2005; Sandhu & Seville, 2005 (Aston University) • UGRP was a valuable learning tool and should be kept as part of the course. • Orsmond et al, 2004 (Staffordshire University) • Pre-project perceptions differed from experience. • Students’ perception of the project’s learning outcomes were rarely in line with their supervisors’ perceptions. • Originality of our project: • Few studies in this area. • Application of background ideas to courses at Nottingham that are revising the UGRP.

4. Background • The UG research project makes up between 10 and 30% of the final degree mark: • Biology: 25% • Biomedical Sciences (BMedSci): 30% • Biosciences: 23% • Computing and IT: 20% • Pharmacy: 13% • Forms of undergraduate research projects: • Laboratory or field based research projects. • Bioinformatics projects. • IT. • Survey/questionnaire. • Literature-based projects. • Clinical Audit.

5. Background • Laboratory-based projects: • Original research carried out in lecturer’s lab space using departmental equipment & funding. • Literature-based projects: • Systematic analysis of research literature. • In comparison to literature-based projects, lab-based projects: • Require greater supervision (indirect or direct). • Place a strain on research equipment and lab space. • In general, cost more. • With increasing numbers of students we may have to: • Offer more literature based projects. • Offer more group projects.

6. PGCHE Project Aims • Do literature-based projects yield training outcomes equivalent to those of laboratory-based projects? • What are the learning objectives of the UGRP? • Do staff and students feel that the lab-based and literature-based projects we offer yield expected learning outcomes? • Would staff and students welcome more group projects? • Are staff and students satisfied with the way we currently assess the UGRP? • What is the impact of the research project on future career choices?

7. Methods • To answer these questions: • Consulted the module handbooks to find out whether learning objectives were defined. • Gave an anonymous questionnaire (5 point Likert scheme) to students and staff at the end of the UGRP in 5 different Schools: • - Biology. • - Biomedical Sciences. • - Biosciences. • - Pharmacy. • - Computer Science and IT. • Data input by MEADS Ltd and data analyzed by group. • Use of questionnaire enabled large sample set to be surveyed anonymously, and consistently across Schools.

8. Methods

9. Methods

10. Results • Total response rate: Students 456 (59%), Staff 150 (67%). • Among the students • Laboratory Projects: 72%; Literature Projects: 27%, unknown: 1% • Literature based projects: • Biology 22% • Biomedical Sciences 29% • Biosciences 37% • Pharmacy 12% • Computer Science and IT 33% • Individual Projects: 74%, Group Projects: 24%, unknown: 2% • Group Projects: • Biology 10% • Biomedical Sciences 35% • Biosciences 0% • Pharmacy 41% • Computer Science and IT 30%

11. Learning Objectives • We consulted the module handbooks: • For example, in the School of Biology & Biomedical Sciences: • Learning objectives are not defined explicitly. • In the School of Pharmacy: • Read, consolidate and contextualize information from scientific journals or reference material. • Contribute to the design and execution of data gathering-and-analysis, based in laboratories or elsewhere. • Develop strong problem-solving skills. • Carry out critical analysis and evaluation of experimental data. • Develop high quality generic communication and time management skills. • Competently describe the work in a poster format. • Present the research work in the form of an abstract for a scientific meeting. • Use poster presentation format to maximize the visual impact of presented information. • Present the research work in the form of a concise scientific dissertation.

12. Learning Objectives Are learning objectives clear even though they may not be defined explicitly?

13. Students Staff Perceived Objectives of the UG Project Communication skills Laboratory techniques Putting theory of lectures into practice Writing Problem solving Working independently Literature searching, data analysis and interpretation Major objective (%)

14. Learning Objectives The purpose of the UGRP is to learn / teach laboratory skills.

15. Group v Individual UGRPs Do students prefer to work as part of a group or individually?   

16. Group v Individual UGRPs I would prefer working as part of a group rather than individually.

17. Group v Individual UGRPs I would prefer working as part of a group rather than individually. BMedSci students 45 %

18. Assessment / Successful outcomes of the UGRP Are literature based projects of value and should they be assessed in the same way?  

19. Value of Literature-based UGRPs Wholly literature based projects have limited value.

20. Open Comments “…literature projects should not exist for science based degrees, nothing is achieved by just reading literature.” [Student – Biosciences]

21. Assessment of Literature-based UGRPs Literature and laboratory based projects should be assessed in the same way.

22. Learning outcomes Do staff and students feel that the learning outcomes are being met?

23. Students Staff Learning outcomes Communication skills Experimental design Laboratory techniques Data analysis Literature review Writing skills Working independently Agree (%)

24. The UGRP’s influence on future career Do staff and students feel that the UGRP influences career choice?

25. Students Staff The UGRP’s influence on future career Is a good introduction to working in research Influences students to consider a career in research Discourages students from a career in research Influences students’ career choice Has no relevance to future career Is a useful experience regardless of future career Agree (%)

26. Open Comments “Discouraging students from a career in research can be beneficial if they realise that they are not well suited.” [Staff – Biosciences] “This [the discouragement] can be a good thing…better to realise this in the context of the UGRP rather than during a PhD.” [Staff – Biosciences]

27. The UGRP’s influence on future career There were differences between vocational (Pharmacy) v non-vocational (Biosciences) degrees.

28. Open Comments “…the UGRP is entirely pointless…it is completely irrelevant to the degree and the Pharmacy job.” [Student - MPharm]

29. Conclusions • Learning objectives seem to be clear to staff and students even in Schools that do not explicitly state them. • Most, but not all, of the learning objectives are achieved in literature and laboratory-based projects. • Students and staff feel that literature and laboratory-based projects should not be assessed in the same way. • In general, students and staff would not welcome more group projects. • The project is valued in its current format by staff and students independent of future career choices.

30. Limitations • Methodology adopted ensured large sample size and consistency across Schools but… • Some questions may have influenced responses. • Questionnaire may have been difficult for staff to answer if they had supervised both literature and laboratory based UGRP. • Would have been useful to have surveyed students before the start of the UGRP to see if responses were influenced by the experience. • Students may experience questionnaire fatigue (with the use of SEMs and SETs).

31. Recommendations • All Schools must ensure that learning objectives are written in the module profile. • Differences in skills acquired between literature and laboratory-based projects should be highlighted to students. • The assessment criteria must be appropriate for the differential outcomes of the two types of project. • If implementing group project work, Schools need to investigate methods to ensure that they are effective. • The research projects are valued by staff and students and should therefore continue to be an important part of the curriculum with a high weighting of marks.

32. Open Comments “The work I had to do in my UGRP has improved my overall ability to work in all areas” [Student – Biosciences]. “The UGRP should be standardised…there is no point in giving us invaluable (useless?) skills and torturing us…………..we are paying for our degree and surely as a consumer we want value for money” [Student - MPharm].

33. Acknowledgements • Martin Willis, Learning Set Advisor. • Ken Levine, Survey Unit. • Staff and students for completing the questionnaires. • MEADS Ltd. • University of Nottingham Learning and Teaching Development Fund.

34. References • Ali J.A. and Seville P.C. (2005) Usefulness of final year MPharm projects: supervisors’ opinions. J. Pharm. Pharmacol. 2005; 57 (Sept sppl.), S46. • Cowie R. (2005) Practice within UK institutions. Presentation at LTSN Staff Development Event ‘Making the most of final year projects’ Durham, 8/2/05. Report available on line at ftp://www.bioscience.heacademy.ac.uk/events/dur05/cowie.pdf • Hollingsworth M., Mahon M. and Thomas L. (2004) Web projects for Life Science students. Bioscience Education E-journal 4 paper 5. Available on line at http://www.bioscience.heacademy.ac.uk/journal/vol4/beej-4-5.htm. • Mills P. (2003) Group Project Work with Undergraduate Veterinary Science Students. Assessment & Evaluation in Higher Education, Vol. 28, No. 5. • Orsmond P., Merry S. and Reiling K. (2004) Undergraduate project work: can directed tutor support enhance skills development? Assessment & Evaluation in Higher Education Vol. 29, No. 5. • Sandhu D. and Seville P.C. (2005) Usefulness of final year MPharm projects: students’ opinions. J. Pharm. Pharmacol. 2005; 57 (September supplement), S45-S46.