Helen Scott LSN Consultant

1. Using your distance travelled and qualification success rate measures to evidence ‘effectiveness, efficiency and capacity to improve’ Helen Scott LSN Consultant

2. Aim • To enable work-based learning providers to consider the use of distance travelled measures and qualification success rates to support improvements in the quality of their provision and their organisation’s overall efficiency, effectiveness and capacity to improve

3. Objectives • To interpret the “distance-travelled” and “timely and overall success rate” charts produced in the Learner Achievement Tracker in relation to the New Measures of Success • Recognise how these values can be used to identify good practice to be shared and poor practice to be improved • To develop strategies to support improvements in the quality of provision and the overall effectiveness of WBL.

4. Context • NMoS part of the Success for All strategy (DfES 2002) • Agenda for Change, Planning for Success • Developed following consultation in 2003/04 with LSC, DfES, Ofsted and ALI • Measures developed from 2004/05

5. Development of the NMoS • Fair, valid and fit for purpose measures for whole sector, including employers • Consistent use by official bodies • Putting teaching and learning at the heart of what we do • Greater decision-making opportunities for providers • Avoid additional bureaucracy

6. Qualification success rates (QSR) VA and DT measures Recognising and recording progress and achievement (RARPA) in non-accredited learning Learner satisfaction Provider responsiveness to employers Learner destinations Value for money Staff measures Suite of measures

8. Interaction between the measures • “There are clear links between all the New Measures of Success – indeed they are intended to be seen as a ‘basket’ of measures which, taken together, provide a contextualised picture of the performance of a provider. These case studies emphasise the interaction between the three measures described in detail in this release of the Quality Improvement Pack”

9. The Quality Improvement Pack • Provides general information on the measures and more detail on: • Value Added (VA) and Distance Travelled (DT)  • Qualification Success Rates and  • Recognising and Recording Progress and Achievement (RARPA) • Purpose to: • Raise awareness and prepare providers for piloting / implementing the measures

10. The Learner Achievement Tracker (LAT) • Produces reports for use by managers, inspectors and LSC • Accessible via the Provider Gateway • Uses most robust statistical methods available • Builds on proprietary systems currently available • Software and data free of charge to all LSC-funded providers

11. The principles behind distance travelled and qualification success rates…..

12. Qualification Success Rate (QSR) There will now be 2 success rates: • overall success rate- the proportion of learners who achieve the qualification/framework • timely success rate – the proportion of learners who achieve by their expected end date (or within one calendar month of it)

13. QSR (cont.) • there will be a six-week ‘period of grace’ and learners who drop out or transfer during this period will not be included in the calculation for success rates • learners who transfer on to another programme within the same provider are still excluded from the success rate calculations. However, transfers to another provider will now be counted as withdrawals • Key Skills, Technical Certificates, NVQs and Frameworks will be reported separately - to be progressively rolled out as detailed data to support this analysis becomes available

14. Distance Travelled • Distance Travelled applies to other qualifications at Levels 1-3 such as GNVQ and NVQ. • The measures are intended for: • The providers - to help them assess and improve their performance • the LSC - to help contribute to planning, self- assessment and three year development plans • the inspectorates - to inform their initial ideas on learners’ achievements and then to support their judgements about learners’ progress at institutional, area of learning, qualification and subject levels • the DfES and other stakeholders - to help policy-making and monitoring

15. Prior Attainment example • (6) GCSEs grade C = 6 x 40 = 240 points • (4) Intermediate GNVQ merit = 184 points • Sub-total: 424 points • (10) Is the ‘number of qualifications.’ • Average prior attainment is therefore: • 424 divided by 10 = 42.4 (average points)

16. Overall effectiveness • How effective and efficient are the provision and related services in meeting the full range of learners’ needs and why? • What steps need to be taken to improve the provision further?

17. Capacity to Improve • ‘The provider has demonstrated that it is in a good position to maintain • the high quality of provision.’ • ‘The provider has demonstrated that it is in a good position to make • improvements.’ • ‘The provider has demonstrated that it has sufficient capacity to make • improvements.’ • ‘The provider has demonstrated that it is in a poor position to make • improvements.’

18. Inspection • The new inspection arrangements mean that Ofsted and ALI will be placing more emphasis on performance data both to plan inspections and to assess quality. • Measures will be embedded in existing and new processes such as CIF, FfA and LSC Business Cycle

19. Inspection (cont.) • Measures will be used for pre-inspection hypotheses and evidencing how providers intend to use the new measures to inform their self assessment/evaluation processes • Ofsted and ALI will use DT for 16-19 learners as a supplement to provider reports and will evaluate these and other new measures for application to inspection

20. Links between NMoS and capacity to improve The self-assessment report (SAR) will provide inspectors with crucial evidence to evaluate the college’s capacity to improve. • The provider’s development plan and internal policies provide a well-researched and realistic framework for improvement • The extent to which WBL provider plans are consistent with local LSC objectives • The provider has appropriate teaching and support staff and other resources to carry through its plans for improvement • Leaders and managers have a good record of bringing about improvement through setting and meeting challenging targets • The commitment of staff at all levels to secure real improvements

21. QSR - Implications for inspection? • From April 2006, QSRs will be main source of data used during inspection • Work Based Learning – • Full introduction of new QSR’s from April 06. • National averages available. • No other data collected. • Providers will be expected to ‘own’ the data and explain differences with existing measures. • QA analysis and target setting should start taking account of the new QSR’s. • Note. Inspectors will expect managers to speak the ‘new’ language!

22. DT - Implications for inspection? • Work Based Learning – • Similarly for WBL the DT graphs available from the LAT, based on success rates will become a common feature of inspection. • Inspectors will increasingly expect providers to understand and interpret what the data might indicate at all levels. • Note. Inspectors will expect managers to speak the ‘new’ language!