Dr Nadia Wager Principal Lecturer in Criminological Psychology and

Dr Nadia Wager Principal Lecturer in Criminological Psychology and Portfolio Leader in Psychology and Sociology Emotional, Attitudinal and Behavioural Responses to Assignment Feedback

Gibbs (2005) - motivated to raise standards in Higher Education ??? The we need to begin conceptualising assessment as a learning modality as opposed to a measurement tool. Ramsden (1991) the helpfulness of feedback is the single factor that distinguishes between students’ perceptions of their favourite and least favourite modules. Why the interest in feedback??

Hattie (1987) – meta-analysis of 87 studies of student achievement – feedback is the most potent factor in enhancing performance BUT often not used by students – Wager (2004) – 29% of the students surveyed admitted to only looking at the grade and ignoring the feedback

Strategic learning style Feedback lacks clarity and direction Leaves some students confused and feeling helpless Emotional inhibition Good grade – motivated to maintain a positive mood Low grade unable to tolerate further assaults in self-esteem Why the reluctance to engage with the feedback?

Language avoid ‘final’ vocabulary, e.g. good, right or poor - does not permit change or repetition Use formative rather than summative language, e.g. ‘do, don’t and how’ rather than ‘did, didn’t and why’ - gives direction for improvement Focus of Feedback comment on the essay and not the author Boud – power differences between lecturer and student Good practice according to the literature

Two Parts: Survey of student experiences of good and bad feedback Experimental study examining the emotional, attitudinal and anticipated behavioural responses to feedback So… my new study

Opportunistic sample of 47 psychology students from all three year groups Asked to reflect on their last piece of assignment feedback Questions on survey were open ended Participants

26 (55.3%) reported a positive emotional response 4.9% a negative response 10.6% relative indifference 19% inappropriate or missing data Q(1): How did the feedback make you feel?

motivating and empowering experiences confirmation experience of conflicting emotions Positive emotions: Three themes

42.3% of the positive experiences Made me feel positive for future assignments as it gave me something to work towards and improve upon. Really pleased that I received positive feedback – chuffed and felt like I’d got more confidence to do it again. It helped me understand better how to improve my grade and made me feel more confident. It made me feel great and excited. Motivating and empowering experiences

27% of the positive experiences Reassured Relieved Very good. I knew exactly what my lecturer was looking for. Proud of the mark considering I had put the effort in Happy that it passed Confirmation

Good, but I was also annoyed (with myself) because some of the feedback related to things I should have done, but I just forgot to do. I was disappointed with the mark, but the feedback was good because it told me how to improve and where I went wrong I felt happy with the positive comments and low with the negative ones The duality of emotions

3(43%) referred to a sense of disappointment, 2 (28.6%) related a sense of unjust accusations, 1 (14.3%) describes a diminution in self-worth 1 (14.3%) describes a loss in confidence. Negative Emotions

Down and like my work wasn’t good enough, although I had worked hard. Also some wasn’t accurate so the teacher hadn’t read my work properly. Upset and frustrated, I was accused of sticking too close to the text when I feel that was not the case. Worthless - Very disappointed with the grade and the manner in which the lecturer addressed my work made me feel worthless. ‘Unjust accusations’

A bit confused, but overall okay. It made me feel okay as it was very detailed but straight to the point. Okay, the content was positive and well written, but I would have liked to have known how to improve further. Feeling okay…but confused

All 47 reported having experienced good feedback 7 themes emerged from the responses: positive encouragement & highlighting areas for improvement 28.6% offering guidance on how to improve - 25.7% writing with a future rather than a past orientation -20% clarity of both expectations expressed by lecturers and the wording of the feedback opportunity to discuss the feedback with the lecturer. confirmation of success What was ‘good’ about the good feedback?

It gave direction for the future, i.e. what was wrong and how to change it. Telling me my mistakes and how to improve on them for my next coursework. The importance of clear directive guidance

Well explained, but I still feel the need to get together with someone to work out how to improve. Advice for the future and where it went wrong, however, I am unclear where to go with it next. But other’s highlight how we can point out what but they really do need the how….

15% of the sample claimed that they had never received bad feedback!!!! 3 themes emerged; the lack of direction for improvement -23% the lack of detail – 23% the perception of being insulted – 12.8% Less Favourable Experiences of Feedback

The feedback said that some points were wrong, but it didn’t give an explanation for how to improve. Where I felt I had put a great deal of effort into the work but was only told that I didn’t answer the question, with no indication of where I went wrong. Lack of direction for improvement

It only contained criticism so I didn’t benefit from it. The lecturer was rude to me. To the point of giving me no positive feedback for my work only, negative. Feedback that insinuates that because the level of work was good the student must have copied the text. It was bad because it didn’t motivate me to work harder. Also because it put down my work. It was a bit humiliating, not enough detail for improvement. Perceived Insults

3 super-ordinate themes Empowerment – 58.7% reassurance – 8.7% confidence building – 15.2% How did the good feedback help you?

It motivated me to improve myself. Enabled me to see what I needed to do on future assignments. It means that I know what to look for in my work that could be improved. It encouraged me to do more independent research for my exams. Better understanding. It woke me up and made me want to work much harder. Empowerment and taking responsibility for own improvement

It gave me confidence, as I wasn’t sure how well my writing skills were due to the original feedback I had previously received. It made me feel that I wasn’t as bad as I had originally thought. It improved my confidence because I can actually do the assignments. It boosted my self-esteem and made me what to work harder to keep up the standard. Confidence Building

It meant that I am able to keep up with what I am doing right. It pointed out that what I am doing is right. It motivated me to continue with my style of writing and essay structure. Reassurance

22% did not believe that their experience of bad feed back had any negative impact on them 5.5% felt it had made them work harder 72.3% felt it had hindered their progress in some way – majority referred to the lack of guidance given and the consequent sense of helplessness Bad feedback – did it hinder you???

If I’d been given ideas on how to improve I could have understood where I had gone wrong. I couldn’t see where mistakes were made Yes, because I only know where I got it right and where I went wrong, but not why and how to improve. I don’t know what made the good areas good. Lack of guidance

It made me feel really crap and I didn’t want to continue. It affected me quite badly. Yes it reduced my confidence entirely. It made me reluctant to start my dissertation as I felt I could not write cohesively in an academic manner. This slowed the process of writing my dissertation and confidence that I could do it right. I was left feeling very negative about the tutor, my course and the experience. Others reported more extreme and enduring effects …

I wasn’t confident or comfortable handing in work to this lecturer again. Felt low for days. Felt slightly de-motivated when it came to writing the next assignment. De-motivated me to work hard for this subject. It made me feel that I wasn’t capable of achieving.

Avoid providing feedback to student through Blackboard with our ID numbers which everyone can see. It has been good to see lecturers address feedback to the whole class if most of the class encountered the same problem with the assignment. Provide an example of a good paper so that we can compare and understand how to achieve this level. Be polite. Students’ Advice to Lecturers

An experimental study of student emotions, attitudes and anticipated behavioural responses on receiving fictitious feedback. 2 independent variables were manipulated in the study To whom the feedback is directed - 2 conditions addressed to the author and written in the past tense written to the assignment and written in the future tense. Whether or not guidance as to ‘how to improve’ statements were provided. Thus four different variants of the stimulus material. Age was entered as a covariate The experimental study…

All sources used were appropriate for this assignment however, remember to use original journal articles and some of the key texts highlighted on the reading list. Many of the key arguments have been identified, although inclusion of one or two more may have benefited this assignment. Clear referencing in the text, appropriate use of the Harvard system, just remember to be particular about the use of punctuation. Example of feedback written addressing the assignment with a future orientation

All sources used were appropriate for this assignment however, remember to use original journal articles and some of the key texts highlighted on the reading list. The range can be broadened by looking at both theoretical and empirical material relating to the topic. There are some chapters or journals which are specifically theoretical (e.g. …….) whilst other journals tend to focus on empirical studies (e.g. …..) – try to use a selection of both. The electronic database collections can be particularly useful. Many of the key arguments have been identified, although inclusion of one or two more may have benefited this assignment. Remember that reading around the topic when planning the essay should begin by being quite broad and often a focus on ‘review’ papers/ chapters can be quite helpful at the start. This will help the identification of all of the central arguments and important search terms. Clear referencing in the text, appropriate use of the Harvard system, just remember to be particular about the use of punctuation. The full stop goes after the reference in brackets at the end of the sentence rather than before the bracketed reference [e.g. Cats are always clever (Smith, 1984).] This means that we know that the reference relates to the preceding sentence. Now with ‘how to improve’ suggestions

You have selected appropriate sources, but the range is a little narrow. You have missed a few of the key arguments. Your referencing in the text was largely okay, however, your punctuation in relation to referencing was incorrect. Same comments directed at the author and in the past tense

You have selected appropriate sources, but the range is a little narrow. The range can be broadened by looking at both theoretical and empirical material relating to the topic. There are some chapters or journals which are specifically theoretical (e.g. …….) whilst other journals tend to focus on empirical studies (e.g……..) – try to use a selection of both. Make sure you attempt to use the electronic database collections. However, you have missed a few of the key arguments. Remember that reading around the topic when planning the essay should begin by being quiet broad. Often focusing on ‘review’ papers/ chapters can be quite helpful at the start of reading for the assignment. This will help you to identify all of the central arguments and important additional search terms. Your referencing in the text is largely okay, however, your punctuation in relation to referencing was incorrect. The full stop goes after the reference in brackets at the end of the sentence rather than before. Now with ‘how to improve’ suggestions

Please indicate what emotions were evoked when you read the feedback whilst imagining that it related to your own assignment? Pleased Not at all _/_/_/_/_/_/_/_/_/_ Extremely Angry Not at all _/_/_/_/_/_/_/_/_/_ Extremely Relieved Not at all _/_/_/_/_/_/_/_/_/_ Extremely Upset Not at all _/_/_/_/_/_/_/_/_/_ Extremely Proud Not at all _/_/_/_/_/_/_/_/_/_ Extremely Shamed Not at all _/_/_/_/_/_/_/_/_/_ Extremely Content Not at all _/_/_/_/_/_/_/_/_/_ Extremely Embarrassed Not at all _/_/_/_/_/_/_/_/_/_ Extremely Deflated Not at all _/_/_/_/_/_/_/_/_/_ Extremely Motivated Not at all _/_/_/_/_/_/_/_/_/_ Extremely Happy Not at all _/_/_/_/_/_/_/_/_/_ Extremely Depressed Not at all _/_/_/_/_/_/_/_/_/_ Extremely Inspired Not at all _/_/_/_/_/_/_/_/_/_ Extremely Irritated Not at all _/_/_/_/_/_/_/_/_/_ Extremely 2 scales: Positive and Negative Emotions Assessing Emotions

On reading the feedback, how likely you are to think the following, if this feedback related to your assignment? a) The comments were fair Not at all _/_/_/_/_/_/_/_/_/_ Extremely b) The comments were helpful Not at all _/_/_/_/_/_/_/_/_/_ Extremely c) The comments were insulting Not at all _/_/_/_/_/_/_/_/_/_ Extremely d) The comments were accusatory Not at all _/_/_/_/_/_/_/_/_/_ Extremely e) The comments were encouraging Not at all _/_/_/_/_/_/_/_/_/_ Extremely f) The comments were meaningless Not at all _/_/_/_/_/_/_/_/_/_ Extremely Two scales Favourable perceptions Unfavourable perceptions Meaningless removed from the analysis Assessing Attitudes towards the feedback

Having received this feedback, how likely do you think you would do each of the following in relation to the assignment and the feedback received? Approach the lecturer and ask for more guidance Not at All _ / _ / _/ _ / _ / _ / _ / Very Likely Think of the lecturer as incompetent Ignore the feedback Check over the assignment to see whether the comments were justified Ask someone else to read through your assignment to see if the comments are justified Decide on a course of action to rectify one or two of the weaknesses Decide to start working earlier on future assignments. Seek further guidance from the Academic Skills Support Unit File the feedback sheet safely so that you can refer to it when preparing for your next assignment Throw the feedback sheet away Discuss the feedback with a friend to help workout how to improve Anticipated behavioural responses

Four factors/ subscales De-motivated inaction Seeking clarification Empowered action Disbelief in the feedback Behavioural Responses (Cont.)

A 2 X 2 MANCOVA was performed controlling for the age of the participant Preliminary assumptions testing was conducted to check for normality, linearity, univariate and multivariate outliers, homogeneity of variance-co-variance matrices and multicollinearity. The following two violations were noted. Box’s Test of Equality of Covariance Matrices produced a p = .001, thus suggesting that this assumption had just been violated. Consequently, Pillai’s Trace statistic was used to determine significance (Tabachnick & Fidell, 1996) and the Bonferroni adjustment for multiple analyses was applied. Thus a significance level of p = .006 is required on the between test of between-subjects effects. However, the Levene’s test of Equality of Error variances, was also violated on one of the dependent variables (holding an unfavourable perception of the feedback) and thus a more conservative alpha level, (p =.003) was utilised in determining significance in relation to this variable. Analysis

Both of the independent variables produced results that attained a level of significance or borderline significance. Where, the Pillai’s Trace for the presence or absence of ‘how to improve’ suggestions revealed (F8,31 = 2.258, p = .050, h2 = .368) Whether the feedback was addressed to the assignment or the author revealed (F8,31 = 2.084, p = .068, h2 = .350). Results

Only two of the dependent variables attained a borderline level of significance. Firstly, the inclusion of ‘how to improve’ suggestions was associated with an almost statistically significant higher score on favourable attitudes towards the feedback (mean = 17.14, sd = 7.42, F8,31 = 8.057, p = .007, h2 = .175), in comparison to when these suggestions are omitted (mean = 13.38, sd = 9.70). And almost statistically significant in terms of the anticipation of engaging in de-motivated inaction. Thus, those who did not receive the ‘how to improve’ suggestions were more inclined (mean = 8.35, sd = 4.52, F8,31 = 7.761, p = .008, h2 = .170) to report de-motivated inaction in comparison to student who were presented with such suggestions (mean = 5.50, sd = 3.03). Main effect of ‘how to improve’ suggestions

Addressing the feedback to the author and writing it in the past tense was associated with: significantly less positive affect (mean = 29.0, sd = 16.19, F8,31 = 9.146, p = .004, h2 = .194) and higher levels of negative affect (mean = 36.07, sd = 14.28, F8,31 = 7.959, p = .008, h2 = .173) in comparison to when the feedback is written to the assignment and in the future tense (positive affect mean = 40.93, sd = 15.15; negative affect mean = 23.07, sd = 11.68). No significant interaction effect was observed for the two independent variables. Main effects of ‘to whom the feedback is addressed’ (i.e. author or assignment)

The ‘how to improve’ suggestions appear to impact upon attitudes and intended behaviour and thus we can imagine would be directly related to whether or the students attempt to use the information. It is proposed that the how to improve statements may actually enhance self-efficacy. Addressing the feedback directly to the author, particularly in the context were either the mark is poor or there are significant areas for improvement can have a detrimental impact on the student – which may erode self-esteem and confidence. Conclusions

Small group activity Thank you… Constructing ‘how to improve’ suggestions

Dr Nadia Wager Principal Lecturer in Criminological Psychology and