Implementing measures of success: training resource

1. Implementing measures of success: training resource This presentation is designed to assist your organisation in its staff development and training regarding the measures of success. It contains speaker notes which can be viewed or printed in notes page view It can be downloaded and delivered as it stands, or can be customised to meet your needs It is intended for use in presenting the measures to: Training Managers and Quality Managers in work-based learning providers The presentation, including notes, can be downloaded from the LSC website: www.lsc.gov.uk/ffe

2. Section 1Measures of Success: Introduction

3. What are the measures of success? • They are a suite of measures developed to measure the success of learners and the performance of providers more accurately and fairly thus helping to demonstrate the sector’s existing strengths and to identify areas for improvement • There are currently seven measures being developed for the learning and skills sector. These are: • Value added (VA) and distance travelled (DT) • Qualification success rates • Recognising and recording progress and achievement (RARPA) • Learner satisfaction • Learner destination • Responsiveness to employers • Staff measures

4. How are the Measures of Success relevant to me? • The measures: • are the key tools for making judgements about how well your learners are performing • can help you monitor the progress of different groups of learners • enable you to highlight strengths and weaknesses in performance relating to learners’ progress and achievement • enable trainers to assist individual learners to maximise progress and achievement • contribute to effective quality management • integrate key information you use for quality management with that used by the inspectorates, LSC and other agencies

5. Which of the measures apply to my organisation? The measures which are currently applicable to WBL are: Qualification success rates DT RARPA These and the other measures are being developed as part of the LSC’s Framework for Excellence. The application of all of the measures to WBL is part of this.

6. Scope of the measures of success • Qualification success rates • All learners on courses leading to external qualification/certification • DT • Learners on courses leading to other approved qualifications at Levels, 1, 2 & 3, including non-graded • RARPA • Learners on courses that do not lead to external qualification/ certification

7. Section 2Relationship of the measures to the LSC’s Framework for Excellence

8. Framework for Excellence • The Framework for Excellence is a new, all-inclusive and radical approach to managing performance across the learning and skills sector including WBL providers. • Further development of the measures is being taken forward through the Framework for Excellence • All of the measures of success will be included in the evidence for the seven key performance indicators that form the basis of the Framework for Excellence.

9. Overall rating Finance Responsiveness Effectiveness Quality of outcomes Financial health Delivery against plan Quality of provision Financial control Responsiveness to employers Responsiveness to learners Framework for Excellence: Key Performance Indicators Measures data

10. Section 3How you can use measures of success to manage quality in your own organisation?

11. The measures: • will be used consistently by external agencies • provide information for use in: • The self-assessment process • Development plans • Assessment visits • LSC funding and planning decisions • Inspection • are indicators of your organisation’s quality, efficiency and effectiveness • are the basis for your organisation’s quality cycle Uses of measures of success

12. Fundamental questions to ask when using the measures • Do the learners for whom you have responsibility: • have qualification success rates which are better/equivalent/worse than the national standards? • perform better / as expected / worse in terms of the DT from entry to final achievement? • have adequate support to progress and achieve to their full potential? • Are you and your colleagues taking action to remedy identified weaknesses and to build on success?

13. The use of measures of success for inspection • The Adult Learning inspectorate (ALI) has been piloting and gradually introducing the measures • ALI will be encouraging WBL providers to ensure that: • You and your colleagues have accessed measures data and information • You have been involved in using the information to support your learners and to help evaluate the quality of your organisation’s provision for them • You are involved in action taken or planned as a result of reflecting on the data and information

14. Section 4Qualification success rates

15. What are Qualification Success Rates? • The number of qualifications achieved as a proportion of the total number started • They provide you with broad measures of learners’ success in achieving qualification goals • Apply to all learners on LSC-funded courses • Measure used in work-based learning for some time • Changes introduced in 2006 to harmonise success rate calculations across whole Learning and Skills sector

16. Changes to qualification success rates in WBL • Two measures for WBL: overall success rate and timely success rate • Calculations now to be based onplanned end date of programmes, rather than actual as before • Taken together, they can help you and your colleagues to identify and deal with instances of slow progress of learners

17. How should I make use of qualification success rates data? • To assess the overall achievement performance of your learners, how it compares with the national pattern of achievement for similar learners and what the trends are • To compare your learners’ achievements in the given year with those which are ‘timely’ – i.e. achieved on or within one calendar month of the planned end date • To monitor how your learners perform for progress and overall achievement against the national picture, and tailor your quality management strategies appropriately

18. Section 5Distance travelled

19. Distance travelled: Distance travelled • shows the progress of individual learners based on their prior attainment • measures how your learners perform when compared to learners with similar starting points in other providers. • covers NVQs at levels 2 and 3, as part of Advanced Apprenticeships and Apprenticeships, and as part of WBL programmes as well as full Advanced Apprenticeships and full Apprenticeships. • show areas of strengths and weaknesses in a provider’s overall performance • shows details of the performance of learners in individual subject/qualification areas

20. Distance travelled output • The LSC has developed software called the Learner Achievement Tracker (LAT) which produces three types of DT output • Summary charts • National comparison graphs • Chances charts

21. The LAT Tutorial A tutorial on the Learner Achievement Tracker showing how to access DT and other data relevant to your organisation is available online at http://ffe.lsc.gov.uk/staff-development-resources/

22. Section 6Recognising and Recording Progress and Achievement

23. What is RARPA? • RARPA aims to help you assure the quality of provision while focusing on learner achievement. • Originally developed to help record and recognise learners’ progress and achievement in provision which does not lead to nationally recognised qualifications or external accreditation. • Bulk of this provision in adult education but also currently relevant to WBL in respect of Entry to Employment provision (E2E) • Work now under way to extend RARPA to all learners – including those on WBL courses leading to recognised qualifications

24. The RARPA approach • Based on five clear standards that help ensure your learners enjoy a high quality experience and maximise their success. • The five standards are inter-related • They relate to the key questions in the Common Inspection Framework (CIF), as used by the Adult Learning inspectorate (ALI). • They follow a logical sequence inline with recognised educational practice. • They currently apply to learners on courses that do not lead to external qualification / certification

25. Aims of the 5 RARPA standards • Encourage learners to take responsibility for their own learning and achievements by helping them to set their goals and monitor their own progress towards them • Allow trainers and managers to ensure learning is taking place and to recognise the related achievement • Allow you to provide full answers to questions about your learners’ achievements asked by • Should not produce additional bureaucracy – if your organisation has effective procedures for ensuring learners’ progress and success, you will be able to accommodate RARPA within your current arrangements

26. Who does RARPA benefit? • Learners: It makes learning more effective by encouraging your learners to engage more actively by measuring their progress and achievements • Trainers: By following the stages in the process, trainers can ensure that learning is taking place and that the learner is making progress • Managers: By looking at evidence that the process is being followed, you can establish whether teaching, learning and assessment are effective • Inspectors: The five stages are cross-referenced with the CIF, so evidence of implementing RARPA can also provide evidence of effective practice for inspectors

27. Meeting RARPA standards • Meeting RARPA standards means that you should: • ensure that staff are supported and trained to implement the RARPA process • help staff to devise fit for purpose approaches to RARPA within your organisation’s overall frameworks • minimise bureaucratic burden on trainers and learners • monitor and moderate RARPA processes

28. Section 7Further Information

29. Further information on the Framework for Excellence • The Framework for Excellence website: www.lsc.gov.uk/ffe(from mid-November 2006) • The LSC website’s Publications area: www.lsc.gov.uk/Publications

30. Implementing measures of success: The Handbook • The Handbook contains an introduction to the Framework for Excellence and comprehensive details of: • What the measures of success are • Explanations of the methodologies used • Timetables for piloting and implementation • Case studies

31. Other sources of information • The Provider Gateway: LSC’s web-based internet portal which enables providers to access information which supports self-assessment and quality improvement, available at: https://gateway.lsc.gov.uk/providergateway • The QIA’s Excellence Gateway: resources, materials, support and examples of effective practice, available at: http://excellence.qia.org.uk/vision

32. End of presentation © LSC September 2006 Published by the Learning and Skills Council. Extracts from this publication may be reproduced for non-commercial educational or training purposes on condition that the source is acknowledged and the findings are not misrepresented. This publication is available in an electronic form on the Learning Skills Council web site: www.lsc.gov.uk. Publication reference: LSC-P-NAT- xxxxxx. If you require this publication in an alternative format or language please contact the LSC Help Desk: 0870 900 6800