Quality of assessment practices

1. Quality of assessment practices Rob Stowell Learning Australia

2. Background In late 2008 the NQC commissioned Learning Australia to undertake a scoping study to: review what constitutes and contributes to quality assessment and how perceptions or understandings of this may have changed in the last five years from the point of view of key stakeholders, including industry. identify the key issues impacting on the integrity and quality of assessment outcomes. analyse the breadth of existing materials available to support the VET workforce in determining quality assessment outcomes. determine what the critical components of quality assessment are.

3. Structure of report Two stage project that built on Investigations into industry expectations of vocational education and training assessment - approved by NQC in mid 2008. Stage 1 report comprises two volumes: Volume one describes the outcomes of the scoping study on the quality of assessment practices. Volume two provides a listing of assessment support materials Stage 2 report provides: review of assessment materials identified in stage 1. a strategy for addressing the gaps, and establishes a plan for the dissemination and implementation of any new, updated or existing materials for supporting quality assessment.

4. Quality assessment Clear and well understood benchmarks for assessment. Principles of validity, reliability, fairness and flexibility. Assessment decisions based on evidence which comply with the rules of evidence. Quality assured - the AQTF2007 Essential Standards for Registration. An integral part of learning and training process and the outcomes of assessment are actually used to inform the development of programs, modify training delivery to meet the needs of learners and improve learning. In quality assessment practices, evidence and feedback are used to identify where learners are in their learning, what they need to do and how best to achieve this. Quality judgements.

5. Key recommendations in scoping study Clear benchmarks for assessment Status and capability of assessors Improve assessment quality assurance by supporting the effective implementation of the current AQTF standards High quality, industry endorsed model assessment support materials Build assessment expertise within the VET sector Improve capability of assessors to make quality assessment judgements Ensure that assessment is an integral part of the learning and training process.

6. Changing emphasis in assessment

7. Assessment and learning Assessment may be seen as a summative process which has little or no connection to the processes of training and teaching. Assessment evidence used for certification purposes and may not be used to identify learners� achievements, diagnose their strengths and weaknesses and inform the design of learning plans that assist learners to achieve competency. Summative or �assessment of learning� is important but there is increasing interest in �assessment for learning� or and its role in promoting learner achievement.

8. Assessment for learning Assessment for learning focuses on the gap between where learners are in their learning, and where they need to be � the desired goal ie: competent performance. It is based on the idea that learners will improve most if they understand: the competency to be achieved, where they are in relation to the required level of performance how they can achieve the required level of performance.

9. Refining formative assessment Paul Black and Dylan Wiliam, Inside the Black Box - 1998 Refined the term �formative assessment� by emphasising that assessment is only formative when: it is an integral part of the learning and teaching process; and assessment evidence is actually used to modify teaching to meet the needs of learners and improve learning.

10. The Black Box: findings

11. Finally, Black and Wiliam concluded that no other single improvement initiative improved performance levels to the extent that formative assessment did. Black and Wiliam�s review of at least 20 studies on classroom assessment showed that significant, and often substantial, quantifiable learning gains were achieved in the classrooms conducting formative assessment experiments. There is a long known effect of drift as learners get older, especially among those learners at the bottom end of the performance range. Evidence shows a tendency for learners at the bottom end of the range to �catch up� due to the increased performance assessment for learning stimulates. This slide shows the range of improvement indicated in the research. It reflects learners of all learning abilities. The improvement ranges from 15 to 30 percent, with differential effects in post-primary of around 2 full grades at GCSE after 2-3 years. These figures have since been underpinned by subsequent research in UK schools like the Kings-Medway-Oxfordshire formative assessment project (1999 onwards) and The Gillingham Partnership�s formative assessment project (2000-2002).Finally, Black and Wiliam concluded that no other single improvement initiative improved performance levels to the extent that formative assessment did. Black and Wiliam�s review of at least 20 studies on classroom assessment showed that significant, and often substantial, quantifiable learning gains were achieved in the classrooms conducting formative assessment experiments. There is a long known effect of drift as learners get older, especially among those learners at the bottom end of the performance range. Evidence shows a tendency for learners at the bottom end of the range to �catch up� due to the increased performance assessment for learning stimulates. This slide shows the range of improvement indicated in the research. It reflects learners of all learning abilities. The improvement ranges from 15 to 30 percent, with differential effects in post-primary of around 2 full grades at GCSE after 2-3 years. These figures have since been underpinned by subsequent research in UK schools like the Kings-Medway-Oxfordshire formative assessment project (1999 onwards) and The Gillingham Partnership�s formative assessment project (2000-2002).

12. Research on assessment for learning in VET Lack of a research base Policy and practice in formative assessment driven by concerns about learner motivation, engagement and inclusion. Connection between formative assessment and learner performance not adequately explored in VET. Some recent work Davies and Ecclestone 2008 Boud UTS � www. assessment futures.com.

13. Assessment for learning principles Assessment for learning is part of effective planning for VET delivery focuses on how learners� learn is central to training delivery involves high quality interactions between VET trainers and learners is a key professional skill of Vet practitioners takes account of the importance of learner motivation assumes that learners understand the criteria on which they are assessed assumes that learners receive constructive guidance about how to improve develops learners� capacity for self assessment so that they become reflective and self managing recognises the full range of achievements of all learners

14. Five key strategies �

15. �and one big idea. Use evidence about learning to adapt instruction to meet learner needs.

16. Skills of VET practitioners Provide learners with constructive guidance about how to improve. This may involve: Ensuring that learners know what competent performance is; Pinpointing learners� strengths and advising on how to develop them; Being clear and constructive about any weaknesses and how they might be addressed; and Providing opportunities for learners to improve upon their work. Develop learners' capacity for self-assessment so that become reflective and self-managing learners. Obtain information about learner progress and use it in planning at individual, group and program levels. Design and implement learning activities that promote high quality interactions with learners that feature thoughtful questions, careful listening and reflective responses. Plan assessment, observe learning, analyse and interpret evidence of learning, give feedback to learners and support learners in self-assessment. Negotiate and manage flexible assessment options including self assessment, peer assessment and collaborative assessment. Ensure that learners understand the criteria on which they are assessed so that they know what it is they are trying to achieve.

17. Activity � Rate VET Practitioner skills

18. Implementing assessment for learning VET practitioners �know� most of this already So the problem is not a lack of knowledge The problem is how to do assessment for learning Telling VET practitioners what to do doesn�t work - there is a need for action research to develop practitioners skills to implement assessment for learning Experience alone is not enough - if it were, then the most experienced teachers would be the best teachers - we know that�s not true (Hanushek, 2005) People need to reflect on their experiences in systematic ways that build their accessible knowledge base, learn from mistakes, etc. (Bransford, Brown & Cocking, 1999)

19. Further information Finally, here�s a list of publications and websites where you can find more information about Assessment for Learning practice in classrooms. (Pass out Handout 1) Details on these same publications appear on this handout.Finally, here�s a list of publications and websites where you can find more information about Assessment for Learning practice in classrooms. (Pass out Handout 1) Details on these same publications appear on this handout.