Feedback on learning
Download
1 / 59

Feedback on learning In the hands of the student David Nicol - PowerPoint PPT Presentation


  • 140 Views
  • Uploaded on

Feedback on learning In the hands of the student David Nicol Emeritus Professor of Higher Education University of Strathclyde, Scotland Visiting Professor, University of Ulster Adjunct Professor, University of Swinburne, Australia

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Feedback on learning In the hands of the student David Nicol ' - esben


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Feedback on learning

In the hands of the student

David Nicol

Emeritus Professor of Higher Education

University of Strathclyde, Scotland

Visiting Professor, University of Ulster

Adjunct Professor, University of Swinburne, Australia

Expert Consultant to JISC: Assessment and Feedback Programme

Talking about Teaching, York St John University, 25 Jan 2013


Focus of presentation
Focus of presentation

Theoretical perspective – cognitive dimensions

Teacher feedback and peer feedback

Written assignments – open-ended and complex tasks

Practice – illustrative cases – peer review

UK JISC-funded PEER project, see website www.reap.ac.uk/Peer.aspx

Nicol (2012) Resituating feedback from the reactive to the proactive


Purpose of feedback
Purpose of feedback

Feedback should develop the students’ capacity to make evaluative judgements about their own and others work (Boud and Associates, 2010: Cowan, 2010; Sadler, 2010)

Feedback should serve the function of progressively enabling students to better monitor, evaluate and regulate their own learning, independently of the teacher (Nicol and Macfarlane-Dick, 2006: Nicol, 2009)


Teacher feedback in higher education
Teacher feedback in higher education

Students are less satisfied with teacher feedback than with any other aspect of their course (National Student Survey UK)

feedback is not timely

feedback is not detailed

feedback did not clarify things I did not understand


Some institutional remedies
Some Institutional Remedies

Faster turnaround times for assignments

Electronic feedback, including audio feedback

Feedback calendars to clarify timings

Improved feedback rubrics/criteria

More attention to providing corrective advice

Feedback on exams

Awareness raising campaigns to highlight student role in feedback

Etc.


Cognitive processes in use of teacher feedback
Cognitive processes in use of teacher feedback

Teacher activity

Deliver feedback

Student activities

Decode feedback messages, internalise, construct meaning from them

Evaluate and compare this external feedback against own work

Identify discrepancies, misconceptions, alternative views, gaps in work

Revise and reconstruct knowledge based on the inner feedback that these processes generate

Transfer this revised understanding to new contexts and tasks


Cognitive processes
Cognitive processes

For feedback to be support learning, and develop evaluative judgement, two cognitive processes must be elicited:

Evaluation – students must reflect on and evaluate their own work in relation to a feedback input – identify gaps, misconceptions, errors, structural deficiencies, other perspectives, weaknesses etc.

Knowledge building – students must use the feedback they generate from these evaluative processes to repair misunderstandings and construct a better understanding. This entails knowledge building.


Terminology
Terminology

I use the following term borrowed from Roscoe and Chi (2008) to represent both processes of evaluation and knowledge building:

reflective knowledge building (RKB)

The question addressed is not just how do we improve use of teacher feedback but rather how do we harness and strengthen inner feedback processes?


Practical strategies to encourage rkb
Practical strategies to encourage RKB

Students evaluate/respond to comments – e.g. put them in own words and say what will do about them

When submit next assignment say how used earlier feedback (University of London)

Sequence assignments so that feedback is actually used – drafts and redrafts

Multi-stage projects and overlapping tasks

Patchwork text – different texts ‘stitched’ together (Winter, 2003).

For 3,4 and 5 suggestive feedback rather than corrective advice.

Technology: Recording comments and responses so that both teachers and students can easily revisit earlier feedback


Specific example steve draper glasgow univ
Specific example: Steve Draper (Glasgow Univ)

Proforma (prompt sheet) for making responses to feedback – the responses are discussed in a tutorial.

Re-phrase (each of) my comments on your essay in your own words: what do they mean? what did they apply to? what future actions do they imply?

If you had to re-edit this essay then how would you apply my feedback to do this, if at all?

How will you apply my feedback to writing your next essay?

Is the feedback I wrote at all useful to you personally, as far as you can tell now?


Guiding principle of good teacher feedback
Guiding principle of good teacher feedback

Guiding principle in educational policy documents

Teacher feedback should be planned in ways that ensure that students are provided with explicit opportunities to respond to, evaluate and act on it.

Might be the single most powerful way of addressing the NSS issue


Limitations with this model
Limitations with this model

Can promote learning of scripted responses – students dependent on teacher (Orsmond and Merry, 2011)

Assumes that ‘others are required to identify and provide the information students need to learn and that learning is driven by how others go about this’ (Boud and Malloy, 2012)

Good feedback is adaptive (Vygotsky, 1978)

Still a transmission model even though the feedback loop is closed – students act on feedback

Practicality – workload


Feedback in professional and workplace settings
Feedback in professional and workplace settings

In the professions, feedback never comes from a single source: task is usually to evaluate, weigh up and reconcile and respond to different and sometimes contradictory feedback perspectives.

Professionals are not just ‘consumers’ of feedback but also ‘producers’

Nicol (2013) Rethinking feedback practices in higher education: a peer review perspective


Peer review
Peer review

Definition of peer review

Peer review is an arrangement whereby students make evaluative judgements about the work of peers and provide a written feedback commentary.

In peer review, students produce as well as receive feedback.


The focus
The focus

Not talking about scenarios involving

..….informal feedback in collaborative tasks

.....students evaluating each other’s contribution to group working

.....students grading/marking each other’s work, although some rating might be part of peer design


Peer feedback receipt augmenting teacher feedback
Peer feedback receipt: augmenting teacher feedback

Increases quantity and variety of feedback

No extra workload on teacher when software used (e.g. PeerMark)

Often perceived as more understandable as peers ‘on the same wavelength’ (Topping, 2003: Hounsell, 1987)


Cognitive perspective receiving reviews
Cognitive perspective: receiving reviews

After receiving peer feedback students have opportunities to update their assignments. Hence activates reflective and knowledge re-construction

Simulates professional scenarios – evaluating and reconciling different feedback perspectives – calls for acts of judgement

Cho and MacArthur (2010) showed that students made more complex revisions to their work when feedback received from multiple peers when compared against a single peer or a single teacher.


However
However...

Not enough attention has been focused on the potential of peer feedback not just as a way of increasing the quantity and quality of the feedback students receive, butalso as a way of giving students practice in making evaluative judgements and constructing feedback

See Nicol (2010) Developing students’ ability to construct feedback. Available at:

http://www.enhancementthemes.ac.uk/resources/publications/graduates-for-the-21st-century


Peer project funded by jisc uk
PEER project funded by JISC UK

One aim was to:

Separate out the different feedback components involved in peer review – receipt versus production

Prior research had either reported only receipt of peer feedback

Or on combined effects of producing and receiving

Much research on peer review was confounded by an emphasis on students marking each other’s work

see http://www.reap.ac.uk/peer.aspx


Cognitive benefits of feedback construction
Cognitive benefits of feedback construction

High-level cognitive activity: students cannot easily be passive

Students actively exercise assessment criteria from many perspectives

Writing commentaries causes students to evaluate and rehearse their own knowledge

Evaluate different approaches to same work and learn that quality can be produced in different ways

Shifts responsibility to student – puts them in role of assessor exercising critical judgement


Cognitive benefits of feedback construction1
Cognitive benefits of feedback construction

Learn to evaluate their own work – as exactly the same skills are involved

Develops capacity to make evaluative judgements - a fundamental requirement for life beyond university. Also, this capacity underpins all graduate attribute development (Nicol, 2010)

Nicol, D (2011) Developing students’ ability to construct feedback, Published by QAA for Higher Education, UK

http://www.enhancementthemes.ac.uk/resources/publications/graduates-for-the-21st-century


Example 1 engineering design
Example 1: Engineering Design

Peer Project case study

  • DM 100 Design 1: first-year class

  • Dr Avril Thomson, Course Leader, Design Manufacturing and Engineering Management (DMEM), University of Strathclyde

    [email protected]

  • Caroline Breslin, Learning Technology Adviser, University of Srathclyde

    [email protected]

    Funded by JISC: see www.reap.ac.uk/PEER.aspx


Engineering design 1
Engineering Design 1

  • 82 first-year students

  • Design a product – ‘theme eating and resting in the city’

  • Research in groups (in city, in library etc.)

  • Individually produce a Product Design Specification (PDS) – detailed requirements for and constraints on design (rationale, performance,standards, manufacturing etc)

  • Given a PDS exemplar from another domain to show what’s required (stainless steel hot water cylinder)

  • Online learning environment: Moodle and PeerMark part of Turnitin suite



Dm 100 design 1
DM 100: Design 1

  • Peer review task

  • Individually, each student peer-reviewed and provided feedback on the draft PDS of two other students

  • Criteria: (i) completeness (ii) convincingness of rationale (iii) specificity of values (performance) (iv) one main suggestion for improvements with reasons

  • Students used experience, giving and receiving feedback to update own PDS

  • Peer review not assessed directly but 10% marks for professionalism which included participation in peer review.


Evaluation
Evaluation

Online survey completed by 64 students

Focus group interviews

Peer review comments recorded online

Course work marks compared to previous years


Results 1
Results 1

  • Which aspects of the peer review did you learn from?

  • Giving feedback 10.9%

  • Receiving feedback 26.6%

  • Giving and receiving feedback 54.7%

  • Neither giving or receiving 7.8%


Results 2
Results 2

Did you modify your initial submission as a result of the peer review activity?


Results student comments
Results: student comments

  • If yes, please give specific examples of modifications (n=41)

  • [Comments are from different students]

  • I added a couple of paragraphs and improved existing paragraphs, this added two full A4 pages to my work

  • I provided more specific numeric values and expanded my rationale after seeing someone else’s PDS and after receiving feedback

  • I added a legal and patents section

  • Improved the rationale, included more facts

  • I made some of my numeric points more specific to my final design concept.


Results learning from receiving feedback
Results: learning from RECEIVING feedback

  • Please give examples of what you learned from RECEIVING peer reviews from other students (n=54)

  • Specific content mentioned: Depth of analysis needed, more numerical data and figures, stronger rationale, how to structure it better etc.

  • Receiving peer reviews gave me insight into what others thought of my work and gave me a direction to improve (reader response)

  • Where the PDS was confusing to understand (reader response)

  • Parts that I had previously missed were brought to eye such as market competition (noticing)

  • The person who peer reviewed my PDS gave me positive feedback which helped me a lot (motivational)

  • Not much, they...[the peer reviews]...weren’t very good (no value)


Results learning from providing feedback
Results: learning from PROVIDING feedback

  • Please give examples of what you learned from PROVIDING peer reviews of other’s work (n=47)

  • How to look at work critically that isn’t your own [critical judgement]

  • Thinking from a critical point of view [critical judgement]

  • I was given a greater understanding of the level of the work the course may be demanding [attention to expectations/criteria]

  • Allowed me to see from an assessor’s perspective [expectations/criteria]

  • When giving advice to people on theirs, it gave me greater perception when reviewing my own work by listening to my own advice for example [transfer]

  • I had a chance to see other peoples work and aspects of their work that I felt were lacking in my work, this helped me to improve my work [transfer]


Results how you carried out peer review
Results: How you carried out peer review

  • Could you make any comments about how you carried out the peer review? How did you evaluate the quality of the work to provide a response to the peer review questions? (n=37)

  • I compared it to mine and ...and said how I would improve it

  • Partly by comparing my work to theirs

  • I tried to think about what I wrote and whether this PDS was better or worse


Focus groups
Focus groups

How did you go about reviewing?

‘I read it through and compared it with what I had done to see if they had put something I had not done and then I added it in if they hadn’t. The four questions...[provided by the teacher]...were useful as they provided a framework for the review. If we hadn’t had the questions it would have been difficult. I did the reviews separately and then answered one then the other. The first was a better standard than the other – so I used the ideas from the better one to comment on the weaker one. I also read the guidelines in class when I did the peer review. There were ideas from the good one that I hadn’t even thought of in mine’


Focus groups1
Focus groups

What do you think is best for learning – giving or receiving feedback?

‘For me it would probably be to give feedback because I think seeing what other people have done is more helpful than getting other people’s comments on what you have already done. By looking at other people’s work you can see for yourself what you have forgotten or not even thought about. When people give feedback on yours they generally just talk about what is there. They don’t say, well I did this on mine and you could put that in yours.’


Focus groups2
Focus groups

What do you think is best for learning – giving or receiving feedback?

I think when you are reviewing...[the work of peers]...it’s more a self-learning process, you’re teaching yourself; well, I can see somebody’s done that and that’s a strength, and I should maybe try and incorporate that somehow into my work. Whereas getting...[teacher]... feedback you’re kind of getting told what to do; you’re getting told this is the way you should be doing it, and this is the right way to do it. You’re not really thinking for yourself.... I think...[reviewing]... would help you not need so much of teacher feedback, if there was more of this. Whereas, I think if you’re not being able to do...[reviewing]... then you will always be needing more...[teacher feedback]...


Results
Results

Would you choose to participate in a peer review exercise in the future?

  • Yes 76.6%

  • No 3.1%

  • Maybe 18.8%

  • Don’t know 1.6%


Theoretical considerations peer review
Theoretical considerations : peer review

Elicits multiple acts of evaluative judgement

Evaluate peer’s work against own

Evaluate one peer’s work against another (and own)

Evaluate work against given criteria to produce response

Students both create and apply evaluative criteria

Create criteria as they compare work with own (holistic)

Apply explicit criteria (analytic) to instances of practice

New perspective on good feedback practice

Students construct feedback ‘meanings’ for themselves while they produce it for others

More control over feedback and less need for teacher feedback


Some tentative conclusions
Some tentative conclusions

Giving and receiving feedback are qualitatively different

Feedback receipt helps bring deficiencies in own work to students’ attention, can be motivational and gives a experience of different audiences (readers)

Feedback production a more direct means of developing evaluative judgement – critical thinking, using criteria and standards, being more objective about own work

Fortunately receiving and giving usually occur in the same domain of assignment production – hence double duty – as both processes help develop important professional skills.


Enhancing evaluative judgement through peer review
Enhancing evaluative judgement through peer review

Maximise the number of reviews within practical limits

Integrate peer and self reviews – would enhance transfer

Embed opportunities for dialogue in reviewing process

Broaden the scope of the review criteria –beyond ‘critiquing’

Make reviewing a regular activity

Change focus of teacher feedback to commenting on quality of peer reviews.


Design decisions
Design decisions

Target task – draft, factual or open-ended (design, computer programme, essay, report etc)

Unit for task: individual, pair, group

Unit for review: individual, pair, group work

Matching reviewers: random, ability, topic

Number of reviews (counteracting poor reviews)

Privacy: anonymous or public outputs

Peer review rubric – not-given: guidelines: fixed format

Review focus: generic v disciplinary, holistic v analytic

Using peer feedback: drafts, self-review, new task

Requesting and responding to feedback

Grading: no marks, marks for participation, for reviews, marks for self-review after peer review


Principles of effective peer review
Principles of effective peer review

Ensure an atmosphere of mutual trust and respect

Engage students in active use of criteria and standards

Involve students in constructing commentaries in relation to peer judgements, not just awarding marks

Give practice in both analytic (componential) and holistic (configurational) judgements about quality

Facilitate dialogue around the object and quality of the review

Integrate self-review activities within peer review designs

Ensure that peer review is a regular activity and not a one-off event

www.reap.ac.uk/PEER.aspx


References
References

Boud, D. and Associates (2010) Assessment 2020: Seven propositions for assessment reform in higher education. Sydney: Australian Learning and Teaching Council (ALTC) Available from www.assessmentfutures.com

Cowan, J. (2010) Developing the ability for making evaluative judgements, Teaching in Higher Education, 15(3), 323-334.

Nicol, D. and Macfarlane-Dick, D. (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218

Nicol, D (2009) Assessment for learner self-regulation: enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher Education, 34(3), 335-352.

Nicol, D (2010) From monologue to dialogue: improving written feedback in mass higher education, Assessment and Evaluation in Higher Education, 35:5, 501-517

Nicol, D (2010) The foundation for graduate attributes: developing self-regulation through self and peer assessment, QAA Scotland, Enhancement Themes. Available at: http://www.enhancementthemes.ac.uk/resources/publications/graduates-for-the-21st-century

Nicol, D (2011) Developing students’ ability to construct feedback, QAA Scotland, Enhancement Themes. Available at http://www.enhancementthemes.ac.uk/resources/publications/graduates-for-the-21st-century

Nicol, D (in press), Resituating feedback from the reactive to the proactive. In D. Boud and L. Malloy (Eds) Effective Feedback in Higher and Professional Education: understanding it and doing it well, Routledge UK

Roscoe, R. & Chi, M. (2008) Tutor learning: the role of explaining and responding to questions, Instructional Science, 36, 321-350.

Sadler, D.R (2010) Beyond feedback: Developing student capability in complex appraisal, Assessment and Evaluation in Higher Education, 35:5, 535-550


Examination of module and course approval documents
Examination of module and course approval documents

Rare to find a learning outcome of the following types:

At the end of this module:

…you will be able to evaluate critically the quality and/or impact of your own work

…you will be able to evaluate critically the quality and impact of the work produced by others

see

http://www.enhancementthemes.ac.uk/documents/G21C/Assessment_150910.pdf



Example 2 biochemistry labs
Example 2: Biochemistry labs

Peer Project case study

  • Biochemistry: third-year class with 30 science students (BM310)

  • Professor Peter Halling, University of Strathclyde

  • [email protected]

  • Dr Sue Barnes, Learning Technology Adviser, University of Strathclyde

    [email protected]


Bm310 biochemistry labs
BM310: Biochemistry labs

  • Students do 3 labs in groups of 4 – produce shared results on spreadsheet template

  • Labs (i) Enzyme Kinetics (EK) (ii) Adenylate Kinase Equilibrium (AKE) and (iii) Fat Analysis (FA)

  • Individually produce their own interpretation

  • Lab report format comprises a set of 18-21 questions to answer

  • Question types – analysis of data, judge significance of results, do calculations, suggest an explanation (simple and complex, where other explanations possible)

  • Online learning environment: PeerMark part of Turnitin suite


Aims of biochemistry pilot
Aims of Biochemistry pilot

  • To ensure that students actively engage with, process and use teacher feedback

  • To develop students own skills in making evaluative judgements and in producing feedback

  • Study possible because teacher had been producing feedback for many years on labs – saw this feedback data as a potential reusable resource


Bm310 biochemistry labs1
BM310: Biochemistry labs

  • Assessment and feedback – a summary

  • Wrote three lab eports

  • Got feedback and mark from academic on one (FAT)

  • Got feedback from fellow students on other two (EK and AKE)

  • Gave feedback on two reports written by fellow students

  • Was given a mark for the quality of feedback on one of these two reports


Scaffolded peer review task
Scaffolded peer review task

  • Three conditions of support for peer review task:

  • Menu of Feedback Comments (MFC)

  • e.g. ‘You are confusing the meaning of retention times and areas – retention times tell you nothing about whether a reaction is proceeding. It’s only the areas that tell you whether the concentrations are still changing’

  • Menu of Feedback Questions (MFQ)

  • e.g. ‘Have they understood the link with increased pH of the borate promoting full ionization of nitrophenol to the yellow nitrophenolate anion?’

  • No menu (NM) – must produce comments un-aided.


Evaluation1
Evaluation

Analysis of quality of feedback comments provided to peers under the 3 different scaffolding conditions: this involved coding the students feedback provision

Survey asking students about their experiences

[Only selected data reported here]


Findings
Findings

Analysis of students’ feedback on work of peers

Initial results show that the MFQ (feedback questions) led to better quality feedback comments than the MFC (feedback comments)

No Menu condition not significantly different from MFQ or MFC conditions.

However, further research needed as effects might be due to difficulty of questions and have not tested pairings of students for reviews.


Findings1
Findings

Survey responses: students’ reports

All reported that reviewing engaged them in comparing peer’s work with own work and with the menu template (MFQ or MFC). Most made a physical comparison.

Reported that MFQ made them think more than MFC and were more helpful for their understanding

Were less happy about the NM condition


Findings mfc versus mfq
Findings: MFC versus MFQ

The teacher’s feedback comments help me understand the basic point to the assignments, however the questions allowed me to understand the assignment better as I had to research the answers myself and this allowed me to gain a better understanding of the topic

I prefer the questions provided from the AK equilibrium because it made me think more about the right answers and it allowed me to write my own comments using the questions as guidance. Having feedback comments from the Enzyme kinetics made it too easy to just copy and paste previous feedback comments in and I did not understand what some of the feedback comments referred to.


Findings survey no menu nm condition
Findings survey: No Menu (NM) condition

NM condition had mixed results as some students claimed that without any external input difficult to go beyond the ideas they had for their own work.

Without teacher support we only know what we answered – our perspective

Without the teacher feedback we lose an important area of the question we may not have known was even a requirement to answer

Highlights value of having an external reference point when providing feedback


Interpretation
Interpretation

Students engage with teacher feedback while also developing their evaluative skills to question others’ work.

Teacher’s feedback on the peer review helps develop students’ own feedback skills.

Implementation in a science discipline where feedback might differ - about accuracy as well as quality of judgement

Addresses issue that use of teacher feedback leads to ‘scripted’ responses (Boud, 2012).

Enables feedback to become a reusable resource.



Problems with feedback as telling
Problems with feedback as ‘telling’

Research shows that students learn through ‘reflective knowledge building’ not through information receipt (Phillips, 1995)

No evidence that delivery of more teacher-feedback, on its own, improves learning [despite Black and Wiliam, 1998)]

Feedback must be adapted to students’ prior knowledge (Vygotsky, 1978: Orsmond & Merry, 2010) and to current level of understanding.

The feedback paradox!

Does not directly develop students’ capacity to evaluative and judge the quality of their own work and that of others.


ad