Everything is Formative!

1. Everything is Formative! Creating a Culture of Peer and Self Assessment in Workplace Maths Mark Sivills

2. Aims for today – to explain: • The context - Don College, Level 3 Maths and Level 2 Maths • Where we were in 2012 • How we embedded Peer and Self-Assessment in Maths Level 3 (2013-2015) • Trialling a Peer and Self-Assessment cycle in Workplace Maths (Level 2) in 2014 • The results • The changes we have made in 2015 as a result

3. Don College • Don College is a 16-18 Year 11/12 College in Devonport • 71 teachers, approx (66 FTE) • 954 students in 2015 • 497 on a University pathway • 457 on Vocational pathway • With some doing both type of course

4. Don College • Serves city and rural population of about 40,000 • Devonport and North-West Coast face significant social and economic challenges and disadvantages • We are one of the poorest regions in one of the poorest states in the nation • Reflected in our student population • Despite this Don College consistently performs above state averages with a median ATAR of 80+ for last 5 years

5. Maths at Don in 2015 • 4 Level 2 Maths Courses • Essential Skills Numeracy (67) (AC ‘E’) • Workplace Maths (84) (AC ‘D’) • General Maths Foundation (189) (AC ‘C’) • Maths Methods Foundation (16) (AC ‘B’) • 2 Level 3 Maths Courses • General Maths (192) (AC ‘A/B’) • Maths Methods (56) (AC ‘A’ / MTM2 ‘A/B’) • 2 Level 4 Maths Courses • Maths Specialised (10) • UTAS High Achiever’s Programme (1)

6. Workplace Maths: where we were in 2012 Need for numeracy component of TCE (Tasmanian Certificate of Education) Ostensibly for those on a Vocational Pathway Anyone with Year 10 AC ‘C’ or higher encouraged into General Maths Foundation (or higher) Reality is it’s for those who have ‘failed’ maths at high school (Year 10 AC ‘D’)

7. Workplace Maths: where we were in 2012 Cohort generally not enthusiastic about maths – only experience to date is of ‘failure’ (‘D’) Completion rate 77% Pass rate 60% Raising retention and pass rates in Workplace Maths became a focus for 2013 Initial plan going into 2013 was for half of Workplace maths classes to be combined in order to have team-teaching with an extra maths specialist (timetable restrictions prevented all classes being team-taught)

8. Context of Maths Department in 2013-2014 Teacher Learning Community in Maths and Science established in Jan 2013 Based on work of Dylan Wiliam Purpose to enable teachers to support each other in embedding formative assessment “The teacher’s job is not to transmit knowledge, nor to facilitate learning. It is to engineer effective learning environments for the students. The key feature of effective learning environments are that they create student engagement and allow teachers, learners and their peers to ensure that the learning is proceeding in the intended direction. The only way we can do this is through assessment. That is why assessment is, indeed, the bridge between teaching and learning.” Dylan Wiliam, Embedded Formative Assessment, 2011

9. Formative Assessment in 2013 • Over the course of a year we experimented with different ways of making assignments more formative including using: Pre-Flight Checklists, Comment Only Marking, Double-Deadlines, “Recipe for Future Action” • Several studies cited by Dylan Wiliam in Embedded Formative Assessment (2011) influenced the design of our method: • Elawar and Corno (1985) – comments vs grades • Butler (1987) – comments vs grades • Day and Cordon (1993) – scaffolding and opportunity to fix vs solutions and further questions • Boulard, Simard, DeMelo (1990) – importance of 15 minutes immediately following feedback • Kluger and DeNisi (1996) – meta-analysis – ‘recipe for future action’

10. Assignment Procedure Deadline 1 – attempted Self Assess for Completion Criteria Checklist – by Peer Feedback and time to improve Deadline 2 – completed Peer and/or Self Assessment of Content Action Setting

11. ‘Everything Is Formative’ Cycle – developed over 2013-2014 implemented department wide in 2015

12. WILF Cover Sheets which build in 2 deadlines and provide rubric for self-assessment Trying to find the balance between accuracy and clarity in terms of the detail of the rubric Keeping it simple and using student friendly language

13. Checklist for Peer Assessment

14. Post-Test Self-Assessment

15. Post-Test Action Setting

16. Student Example - Post-Test Action Setting

17. Reports from General 3 and Methods 3 • Teachers in Level 3 courses reported: • Standards of assignment work increased dramatically • Quality of dialogue and discussion increased • Teachers felt their feedback was really being used • End of unit test results increased in some classes (not uniformly) • Students analysed tests in far greater detail and made improvements afterwards • Students in other subjects started to request that their teachers also make use of double-deadlines

18. Meanwhile in Workplace Maths • 2013 – team-teaching was reported as partially successful from point of view of teachers • Results increased: • 2012: 60% Pass with 77% completion • 2013: 75% Pass with 80% completion • Plans for 2014 • Repeat team-teaching • Additional Maths Specialist (2 maths specialist, 1 non-maths specialist) • Introduce some of the formative techniques being used in Level 3 into Level 2

19. Concerns expressed about bringing peer and self-assessment to Workplace Maths Behavioural issues in Workplace maths Concerns about handing over control to students Concerns about whether students would engage with activities Concerns about whether it would be of any benefit to lower-level students Concerns about new ideas and practices

20. What happened in 2014? Team-teaching enabled introduction of some formative practices in two groups such as using mini-whiteboards Some self-assessment introduced at the end of the unit In the middle of the year, decided to try and implement a full cycle of formative assessment based on the Level 3 work

21. Trialling the formative assessment cycle in Workplace Maths Class 3 - control Deadline ___ ___ Self-marked against solutions ___ ___ Revision Relay Test ___ Grade only Opportunity to improve given only to those who failed

22. Trialling the formative assessment cycle in Workplace Maths Trialling the formative assessment cycle in Workplace Maths Class 2 – partial cycle Class 3 - control Deadline ___ ___ Self-marked against solutions ___ ___ Revision Relay Test ___ Grade only Opportunity to improve given only to those who failed • Deadline 1 – RAG Self Assessment • Peer Checking of Assignment using Checklist • ___ • Self-marked against solutions • ___ • ___ • Revision Relay • Test • Test Checklist • Provided with explicit advice on what to improve • Opportunity to improve given to all

23. Trialling the formative assessment cycle in Workplace Maths Class 2 – partial cycle Class 1 – full cycle Class 3 - control Deadline ___ ___ Self-marked against solutions ___ ___ Revision Relay Test ___ Grade only Opportunity to improve given only to those who failed • Deadline 1 – RAG Self Assessment • Peer Checking of Assignment using Checklist • ___ • Self-marked against solutions • ___ • ___ • Revision Relay • Test • Test Checklist • Provided with explicit advice on what to improve • Opportunity to improve given to all • Deadline 1 – RAG Self Assessment • Peer Checking of Assignment using Checklist • Deadline 2 – RAG Self Assessment • Self-marked against solutions • Actions – Designing Questions on Weak areas • Testing and Mark designed questions • Revision Relay • Test • Test Checklist • Provided with explicit advice on what to improve • Opportunity to improve given to all

24. TheResults • Class 3 almost flat but dips in Unit 4 – implies Unit 4 more challenging? • Class 2 and 3 show significant improvement in Unit 3 • All dip in Unit 4 • Not starting at the same point • Composition of class • Behavioural issues • ‘Harsher marker’? • As several differences between classes, need to compare performance against each class’ own performance in previous unit

25. Average grade improvement of 0.5 when using full cycle What is the effect size when comparing using the cycle against not using it within the three groups? Is this significant?

26. Effect Sizes Effect Sizes calculated as the difference in means of Unit 3 results compared to Unit 2 divided by mean SD of the two sets of results Would expect effect size close to zero for Control group if Unit 3 is similar level of difficulty to Unit 2 Effect Size of 0.4 is internationally replicated standard value of a year’s worth of average improvement 0.38 (0.31 above ‘not used’ group) in 6 weeks is significant 0.73 (0.66 above ‘not used’ group) in 6 weeks is highly significant

27. Trialling the formative assessment cycle in Workplace Maths Class 2 – partial cycle Class 1 – full cycle Class 3 - control Deadline ___ ___ Self-marked against solutions ___ ___ Revision Relay Test ___ Grade only Opportunity to improve given only to those who failed • Deadline 1 – RAG Self Assessment • Peer Checking of Assignment using Checklist • ___ • Self-marked against solutions • ___ • ___ • Revision Relay • Test • Test Checklist • Provided with explicit advice on what to improve • Opportunity to improve given to all • Deadline 1 – RAG Self Assessment • Peer Checking of Assignment using Checklist • Deadline 2 – RAG Self Assessment • Self-marked against solutions • Actions – Designing Questions on Weak areas • Testing and Mark designed questions • Revision Relay • Test • Test Checklist • Provided with explicit advice on what to improve • Opportunity to improve given to all 0.73 0.07 0.38

28. Designing Questions Activity Short video on one of the Post-Test Actions activities that we designed – Methods Foundation

29. 2014 results in Workplace Maths • Successfully challenged the notion that lower level students could not engage in meaningful peer and self assessment • Unit 3 results increased dramatically when formative assessment used • Effect size of 0.73 in full cycle group • 2012: 60% pass 77% completion • 2013: 75% pass 80% completion • 2014: 78% pass 94% completion

30. 2015 and beyond Formative Assessment has been built into the fabric of the course The ‘Everything is Formative Cycle’ is now being used in all units in Workplace Maths this year The year plan / scope and sequence has been designed around these processes In addition individual lessons and series of lessons are being designed around more formative processes

31. Workplace Maths Year Plan with: Deadline 1, 2, Actions built into the timeline

32. Workplace Maths Assignments with WILF sheets for Self-Assessment

33. Workplace Maths Checklists for Peer Assessment

34. Self-Assessment Post Test Activity in Workplace Maths Students colour code their responses to each question They go through solutions to the test and analyse their responses Action Setting They choose three areas to work on They use the solutions to fix the three questions They then find or design three new questions based on the originals They then solve their new questions They can be used to test each other in similar way as previous activity

35. Revisit Aims • The context - Don College, Level 3 Maths and Level 2 Maths • Where we were in 2012 • How we embedded Peer and Self Assessment in Maths Level 3 (2013-2015) • Trialling a Peer and Self Assessment cycle in Workplace Maths (Level 2) in 2014 • The results • The changes we have made in 2015 as a result

36. Thank you for listeningCopies of all materials (except student videos) are available for sharing Mark Sivills, AST, Don College, Tasmania email: mark.sivills@education.tas.gov.au twitter: @doncollegemark youtube: bit.ly/doncollegemark