Engaging Students in Quantitative Research Methods

1. An Evaluation of Assessment for Learning (AfL) Strategies on an Undergraduate Social Research Methods ModuleDr Ciaran Acton and Dr Bernadette McCreightSchool of Sociology, University of Ulster Engaging Students in Quantitative Research Methods

2. Outline of the HEA TRM Project • Aims of the Project • To obtain student feedback on AfL strategies employed on a quantitative methods module • To use this feedback to develop the assessment strategy and enhance the student learning experience • To work towards a model of best practice   • Research Design • Stage 1 - Questionnaire on students’ perceptions and experiences of research methods • Stage 2 – Focus group interviews

3. Developing Research Skills • Research-Related Modules • Year 1 – Introduction to Qualitative Research Methods • Year 2 – Quantitative Methods; Research Methodology • Year 3 – Sociological Placement; Dissertation • Explicit links to other modules • Year 1 –Sociology of Health • Year 2 – Information Society; Modern Ireland • Year 3 – Sociology of Education

4. Widening participation • UU has a large proportion of students historically under-represented in HE • 45.8% of full-time degree entrants in UU in 2011/12 from NS-SEC Group 4-7 (Benchmark = 34; UK average = 31.2) • Impacts on certain disciplines more than others and creates challenges for the teaching of particular parts of the sociology curriculum • Quantitative Research Methods Module • 80 to 100 students • Diverse group (socially and academically) • 67% of students’ parents did not attend HE • 46% Grammar; 42% Secondary; 12% Other

5. Background to the Module • Aims of the module • To give students the opportunity to develop the research skills necessary to devise, administer and analyse the results of a social survey • To equip students with the ability to interpret and critically assess quantitative research findings published within the field of sociology • Content and Delivery • Twelve 2-hour lectures • Twelve 1-hour seminars • Twelve 90 minute computer-based workshops

6. Assessment and Feedback • Summative Assessment • Research Report (2,500 words) • End of module exam (3 hours) • Formative Assessment • Tutor and peer feedback • 1. Seminar Activities (group work) • (a) Questionnaire Design • Choose a topic; design 6 questions; pilot the questions • (b) ‘Reading’ Quantitative Data • Interpreting quantitative data in published research (e.g. Ark Research Updates; Sociology articles)

7. ARK Research Updates

8. Tutor and Peer Feedback (cont) • 2. Computer-based Workshops • Informal ‘no stakes’ assessment on SPSS • Student learning in the workshops is assessed on two occasions (descriptive statistics; inferential statistics) • Collaborative learning • 3. Interactive Lectures • The use of personal response system (PRS) technology • (a) Assessing students’ understanding of the lecture material • Allows the lecturer to identify problems at an early stage • Anonymity makes it easier for students to participate • (b) Promoting active learning • Encourages reflection on statistical problems • Facilitates collaborative learning

9. Which of the following requires an accurate sampling frame of the total population? • Quota sample • Cluster Sample • Stratified sample • Convenience sample

10. Correlation Matrix (SPSS Output)

11. 17 of 113 Discuss the correlation matrix with the person next to you and consider the following question:How would you describe the relationship between age and attitudes to teenage pregnancy? • Sig., +ve & strong • Sig., -ve & strong • Sig., +ve & weak • Sig., -ve & weak • Not significant

12. Preliminary results from Survey & Focus Groups • Questionnaire adapted from Williams (2007) • Student perceptions and experiences of quantitative methods • Disseminated to 2nd and 3rd year students (Sociology & Sociology with Criminology) • Key themes: • Attitudes to quantitative methods • Students’ experience on the module • Assessment and feedback • Two Focus Groups with 2nd and 3rd year students • Addressed questionnaire themes and other issues (barriers to learning; developing student confidence; applying research skills)

13. Fears and anxieties • Questionnaire Results • All have GCSE maths (or equivalent) but 28% had ‘a bad experience’ of maths at school • 34% said ‘the idea of learning statistics makes me feel anxious’ • Focus Groups • ‘I was petrified. It scared me a lot. When I heard computers I was really frightened. I am more confident now that I could actually sit down and use SPSS.’ • ‘When I used to see tables I never knew how they got there. I didn’t know sociologists had to do this sort of work.’ • ‘Whenever I heard it was going to involve numbers I thought I was going to fail it.’

14. Assessment and Feedback 1. Seminar Activities • Questionnaire Results • 86% ‘agreed’ that giving feedback on other students’ survey questions ‘helped me to develop my own understanding of questionnaire design’ • 73% felt ‘confident’ or ‘very confident’ in their ability to ‘read journal articles which discuss the findings of quantitative research’ • However 35% of students ‘preferred to work on my own rather than within a group environment’

15. Assessment and Feedback 1. Seminar Activities (Questionnaire Design) • Focus Groups • ‘Designing the questions was really difficult.’ • ‘You have to get your hands dirty. We thought [this] would be so easy but it wasn’t. How wrong were we?’ • ‘We thought our questions were brilliant, but the pilot proved they weren’t. If we had to do the questionnaire ourselves it would be really scary.’ • Questionnaire design was difficult if other people did not turn up.’

16. Assessment and Feedback 1. Seminar Activities (‘Reading’ Quantitative Data) • Focus Groups • ‘I can now criticise other research papers.’ • ‘Quantitative statistics are in a lot of articles I would read for most modules.’ • ‘Especially those journal articles. You are more confident to have a flick through it instead of turning the page.’ • ‘There was lots of statistics in the criminology fear of crime module. It gives us an advantage because we can now look at the statistics and are familiar with them.’

17. Assessment and Feedback 2. Computer-based Workshop Assessments • Questionnaire Results • 87% ‘agreed’ that the workshop assessments ‘enabled me to assess my ability to analyse data using SPSS’ • 91% ‘agreed’ that the workshop assessments were ‘good preparation for the research project’ • However, 53% felt that ‘marks should be awarded for the workshop assessments’

18. Assessment and Feedback 2. Computer-based Workshop Assessments • Focus Groups • ‘When it came to carrying out the research project you could do it because you had had the practice beforehand. The workshops allowed you to try it out.’ • ‘The computer test on the life and times survey was helpful.’ • ‘It give you a chance to play with it and work out what you had to do.’

19. Assessment and Feedback 3. Interactive Lectures (PRS) • Questionnaire Results • 67% ‘agreed’ that PRS ‘encouraged interaction with other students during the lecture’ (31% ‘undecided’) • 70% ‘agreed’ that PRS provided them with the opportunity ‘to gauge my understanding of the lecture material (26% ‘undecided) • 59% ‘enjoyed’ using PRS (35% undecided)

20. Assessment and Feedback 3. Interactive Lectures (PRS) • Focus Groups • ‘The questions were good, instead of just a 2 hour lecture.’ • ‘It gave you the opportunity to know what to do. If you didn’t know the answer you knew you were behind and you had to push yourself a bit more.’ • ‘It gave you time to let the information absorb in, rather than just a 2 hour lecture where you are being fed information. You can test yourself.’ • ‘It’s good the way it is anonymous, you know, if you get the answer wrong it’s not like you’re ashamed.’

21. Concluding Points • Giving students the freedom to make mistakes • Embedding research skills throughout the degree • Developing students’ understanding of assessment and feedback • Building confidence • ‘It was the one module I did best on, yet it was the module I thought I was going to fail.’