Using e-learning for self-assessment in advanced productive skills: from essay-writing to liaison interpreting

1. Crossing Frontiers: Languages and the International Dimension CILT/LLAS - Cardiff 6-7/07/06 Using e-learning for self-assessment in advanced productive skills: from essay-writing to liaison interpreting María Fernández-Toro University of Newcastle upon Tyne maria.f.toro@ncl.ac.uk

2. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Overview • Rationale • Peer-assessment in essay-writing • Self-assessment in liaison interpreting • Learning through self-assessment

3. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Rationale A number of outstanding problems experienced by students and teachers in HE: • Increasing numbers of students, all needing teacher feedback ® increasing marking load for the tutors • Final year UG students returning from their year abroad still have many form-related problems despite their improved fluency. • Students do not know how to improve their accuracy in productive tasks in the absence of individual feedback from a teacher.

4. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Possible strategies • Grouping students to reduce the number of assignments to be marked by the tutor • Giving students alternative resources from which to obtain (some of) the feedback they need • Training learners in effective peer- and self-assessment techniques • Setting up an economical and efficient framework for the use of tutor feedback to individual students

5. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Peer-asessment in essay-writing • Students: 72 Modern Language finalists on a Spanish module component in advanced writing skills, taught in 4 classes of 18 students by 3 staff. • Task: Writing a 300 word mini-essay based on a complex argumentative article from a Spanish newspaper. Format based on one of the questions presented in the final examination paper.

6. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Procedure • In class, each team assesses the other teams’ work, using a checklist of criteria based on the examiners’ marking form. • Each summary is then discussed with the whole class, covering the criteria provided in the checklist. • Each team is given a print out of their team’s work with further annotations from the tutor. • Individuals who wish to follow this up may submit a revised vrsion of the summary for tutor feedback. • Students write their summaries in teams of 4 (5 summaries per class). • Team-written summaries are sent to the tutor through the VLE (Blackboard) • Tutor uploads them onto the relevant Group folder after deleting the students’ names and correcting language mistakes. • Students read all summaries from their class and bring print-outs to the next session.

7. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Evaluation Advantages Disadvantages • Students learn by talking to each other (cf. social constructivism) • Feedback on problem areas is less personal ® can be safely shared with the whole class • Students learn to rely on sources other than a tutor for feedback • Fewer assignments to mark for the tutor! • More able students feel that they could do better & faster by themselves • Difficult to prevent students from identifying each other’s contributions in a class of 20 • As the examination essay is written individually, the formative validity of group writing is arguable • Students want individual tutor feedback regardless of any other modalities available

8. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Uses of Blackboard in the group-writing task • Submission of group-written essays (anonymous submissions would be possible in theory) • Fast distribution of work from all teams before the peer-assessment session • Written feedback to individual teams using the Revisions feature in MS Word • All the materials related to the task (directions, checklists, essays from other groups/classes) remain available on Blackboard for future reference

9. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment The liaison intepreting course Students: 60 final-year UGs taking a compulsory module component in Spanish Liaison Interpreting at Newcastle University Aims of the Interpreting component: • To develop language competence (accuracy, fluency and appropriacy) by applying the skills acquired so far to a new type of integrative task • To introduce the skills involved in liaison interpreting (emphasis on communication strategies, but covering also memory strategies, note-taking, researching unfamiliar topics at short notice, etc.)

10. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Live practice in groups Spanish speaker English speaker Practice with recorded dialogues Interpreter Live practice with two tutors Student Student Student Student Student Spanish speaker English speaker Student Student Student Spanish speaker English speaker Interpreter Modes of delivery

11. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Blackboard resources Uses of Blackboard in the interpreting course Student’s tasks • Introductory scenarios + tips for preparation • Download scenario and directions. Prepare scenario. • Download dialogue and play back using Media Player. Stop after each turn and interpret as appropriate • Source dialogues (MP3 audio files) • Recording software (Sanako) licensed for download from VLE • Download Sanako software and record own output on a new audio file. Save new file onto student’s own disc space, USB pen, etc. • Template and directions for self-transcript exercise (Word docs) • Download template and type in self-transcript while playing back own output from new file (eg using Media Player)

12. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Turn Original conversation[Transcript provided on ESA template] Self-transcript[Written by student] Corrections Justification & comments S4 Preguntale cuándo se reúnen y qué tengo que hacer si me interesa. Mrs Alonso would like to know when do you meet and what would she have to do in order to come along. -what does she have to do if she is interested.. More exact E4 We meet every Thursday. If you want to come along next Thursday with me, you’d be welcome. Dice que se encuentran cada jueves y si os gustaria ir con el Sr Francis, lo podria hacer. -Se reunen -Si le gustaria ir el jueves que viene con el Sr Francis.. Wrong person Example of self-transcript and error self-analysis (ESA)

13. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Error self-analysis (ESA) What is ESA? Error analysis performed by learners themselves on the basis of their own output in a learning task involving productive skills. • Output may be either written or spoken • Errors may be identified in two ways: • highlighted by an instructor • spontaneously ‘noticed’ by learners themselves • Learners correct their errors and justify the corrections made • ESAs are then evaluated in terms of skill area and specificity (see next) How does it work?

14. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment • Lexis “I didn’t know the word for ‘village hall’.” • Grammar “It should be ‘existen’ as the subject is plural.” • Content “Missed out detail of where the reunion takes place.” • Structure & style “The corrected way is more organised”;“Too much repetition of ‘no’ and ‘si’” • Pragmatics “I should have ‘usted’, not ‘tu’ as it was a fairly formal environment”; “As a non-English speaker she may not know ‘pub’.” • Oral features “Hesitation before ‘al lugar’” ; “…not to waffle”; “Mispronunciation of ‘gorilla’” • Strategy “Didn’t get across everything to do with building sets, perhaps because I was too busy trying to think of vocabulary!” Area: To which of the following areas was your comment related?

15. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Specificity: How specific was your analysis of the problem that you were addressing? • Category A broad category to which the problem/comment relates is explicitly identified. “Grammatical mistakes”; “More suitable vocab, flows a little bit better.” “I was not sure how to translate ‘building the set’” ; “‘el próximo pueblo’ is wrong” • Problem A specific problem is identified in the comment • L2 revision A specific correction is proposed in the target language “Hacer el decorado”; “‘El pueblo al lado’ for ‘next village’” “ ‘vestidos’ means ‘dresses’, not ‘costumes’ ”; “The subjunctive is needed after ‘es posible que’ ” • Rule The solution/analysis is formulated in terms of a “rule” of some kind “Mainly problems with vocabulary here, which good research beforehand may have eliminated.” • Strategy A strategy for improvement is proposed

16. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Possible applications of ESA • As follow-up to any writing or speaking task • For self-assessment of peer-assessment • For diagnostic purposes • linguistic competence • metacognitive awareness • attitudes and affective responses to the task • As a formally assessed task (following appropriate training) • As an individual-specific strategy for developing vocabulary and grammar

17. Rationale Peer-assessment in essay-writing Self-assessment in liaison interpreting Learning through self-assessment Integrating ESA in the language curriculum • Year 1: Introducing ESA for writing tasks (starting with peer assessment of team-written compositions) • Year 2: Introducing ESA for speaking tasks (e.g. self-assessment of previously recorded and self-transcribed oral tasks) • Yr. abroad: Written submissions via VLE, formally assessed, consising of: • The firstdraft of a composition • Subsequent corrections (after live feedback from native speaker or course tutor via the VLE) • ESA based on the corrections proposed • Final year: Formative assessment of liaison interpreting using ESA. Students using ESA independently in their writing tasks.

18. References • Anderson, J.R. 1985. Cognitive Psychology and its Implications. New York: Freeman. • Doughty, C. & J. Williams, eds. 1998. Focus on Form in Classroom Second Language Acquisition. Edited by M. H. Long and J. C. Richards, Cambridge Applied Linguistics. Cambridge: Cambridge University Press. • James, C. 1998. Errors in Language Learning and Use. Exploring Error Analysis. Edited by C. Candlin, Applied Linguistics and Language Study. London & New York: Longman. • Lesser, M.J. 2004. Learner proficiency and focus on form during collaborative dialogue. Language Teaching Research 8 (1):55-81. • Long, M. 1991. Focus on Form: A Design Feature in Language Teaching Methodology. In Foreign Language Research in in Cross-Cultural Perspective, edited by K. de Bot, R. Ginsberg and C. Kramsch. Amsterdam: John Benjamin. • Wenden, A. 1998. Metacognitive knowledge and language learning. Applied Linguistics 19 (4):515-537. • Wenden, A. 1999. An introduction to metacognitive knowledge and beliefs in language learning: Beyond the basics. System 27:435-441.