Welcome To The HN & Vocational Senior External Verifier Event August 29th 2013

1. WelcomeTo The HN & Vocational Senior External Verifier Event August 29th 2013

2. Agenda

3. QA Past, Present and Future….

4. And then……………

5. Your Role ‘A Senior External Verifier provides leadership and support to a team of external verifiers through the processes in accordance with SQA policy and instruction.’

6. Alastair MortonOperations PlanningManager (HN and Vocational Qualifications)

7. Topics Review of Verification Activity 2012 – 13 College Review Future of Reporting

8. Comments: 75 Holds reported (1st Aug 2012 – 31st July 2013) 54% of Allocated Events were Successful 38% of Events were updated as Not Running vs 36% Session 2011 - 12 Includes visits conducted under new approach Profile excludes China and International Centres

9. How SQA plans to tackle this Colleges • Provided Verification Plans to all other Scotland’s Colleges • Process worked well with 10 partner colleges over last 2 sessions • Colleges highlighting Not Running – internally select for accreditation purposes – SQA will set to Not Running • First time this process has been considered ETPs • Reviewed entry patterns for all approved SVQs over last 2 years • Allowed SQA to identify areas of candidate activity • Also allowed SQA to target individual Work Assessed units • Will select -> de-select as per College process Longer Term • IQA project – significant expenditure in designing complete system

10. Profile of Verification Activity

11. Comment: 150 more Successful events vs session 2011-12 (2112 vs 1962) Includes visits conducted under new approach Profile excludes China and International Centres.

12. Comparing 2012-13 vs 2011-12:

13. Impact of Verification Profile • Resource impact on Centres • Resource impact on Appointees • Missing assessment windows – (e.g. Semester 1) • Availability of candidate evidence at Centres may drive part of this profile • Moving away from single allocation period -> quarterly allocations • React to changing developments at centre / across subject area / industry • Allocations will be issued throughout the session -> ongoing verification • Initial concentration on ETPs – assessment often throughout year

14. Scotland’s College Update

15. Scotland’s Colleges Update • Verification Planning • All Scotland’s Colleges – met / discussed finalised Verification plan • Real focus on Entry information • Discussed verification areas which have over / not verified • Plans are agreed in partnership • Swing from HN qualification to NCs • College Mergers • Number of Colleges in process of merging • Sensitive to the transitional period, particularly around availability of candidate evidence across sites • Majority of merging centres have yet to finalise single SEED number and will be operating under original numbers for next 6 – 12 months

16. Challenges of Changing Quality Assurance Approaches in Merging Colleges Scenario 2 Scenario 1 Scenario 3 Scenario 4 New New New New Old Old New New Old New New New Old New Old

17. College Feedback Verifiers are not liaising with the SQA Coordinator Some Feedback sessions could be more comprehensive Sampling needs to be more widespread Visit Plans are not being provided ahead of visit. Verifiers using a College email address and not a private one…. Some Verifiers are not using standard email templates for contacting / planning Visit plans are issued less than 2 weeks before visit More Comments need to be provided against Criteria

18. HN and VQ Approach to Quality Assurance Walkthrough Preparing Planning Verification Carrying Out Send Visit Plan Docs Request Information Prepare Accept EV+ Select Sample Prepare for Visit Contact Agree Agenda Conduct Sampling General Points Completing Report & Follow-Up Communicating Decision Judge sufficiency Plan Feedback Complete Report Follow-up Flowchart Explain Decision Reporting Flowchart

19. Planning Accepting Centre Allocation on EV+ • Do’s • Accept allocation within 14 days • If you decline a visit, please provide a reason • Once you have registered, please remember keep your password safe. SQA will not provide you with a password. Verification allocations will only be provided via EV+ system Accept You will receive notification of allocation from EV+ via email. Decline • If you decline the allocated visit, please provide a reason • If you do not accept your allocation within 14 days, an automatic reminder email will be sent by EV+ • SQA may re-allocate to another Verifier if no response is received within this timescale • Please highlight any conflict of interest to QA Logistics Team • Don’ts

20. Communicating Decision • Do’s • Consider confidentiality and location of feedback session • Give balanced feedback • Recognise centre’s efforts where sufficient evidence provided. • Take time out between making the decision and giving feedback Plan to deliver balanced feedback • Communicate verification decisions on criteria compliance and overall risk rating. • Highlight positive aspects of visit, including good practice. • Think clearly – what really is Good Practice ? • No specific guidance on length of time required for feedback. • Don’ts • Focus solely on Action Points

21. Criteria Reporting – The Way ahead

22. Evolution of Verification Outcome Reporting Pre 2010 SEV Report Count of Holds EV8A Reports Activity 2010-11 Count of Holds SEV Report EV8A Reports Activity Senior Review 2012-Now New Verification Reports Detailed Analysis SEV Report Senior Review Activity

23. Detailed Analysis

24. The Way Forward… Activity Database (QAMS) ONLINE WEB FORMATTED REPORT Benefits from this session: • Real time information available – easy to extract and disseminate • Understand “pockets” of non-compliance quickly • Criteria compliance reporting supported by narrative • Provide information on more frequent basis • Share with Centres / SQA teams and Appointees to highlight, understand and fix • Explore use of Social Media

25. Example Database Activity Information via Profiles Centres Appointees SQA teams Implement and Communicate Fix Awareness and analysis through number of views / reports Highlight and understand Non Compliance pattern Contains profile of criteria (R / A / G) and Good Practice

26. David Pirnie Lead Verifier – HN & VQ

27. Results of Survey Questionnaire June 2013 Centres • Website is easy to follow - 93% • Guidance informs what to expect from a Qualification Verification • Visit – 75% • Qualification Verification forms and templates are clear – 64% Qualification Verifiers Clear flowcharts and procedures - 100% Understand the type of evidence required – 74% Guidance is effective - 81% Qualification Verification forms and templates are clear – 10%

28. Veronica KirkQuality Enhancement Manager

29. QEM feedback • Anecdotal feedback from centres on the new systems and criteria.

30. Tea Break

31. Juliette McGinley Quality Assurance Logistics Manager

32. Overview • Introduction and team update • EV+ functionality • Quality Assurance Management System

33. Quality Assurance Logistics Team

34. EV+ documents and templates • Review of documents • Template index and restructure of documents • Quality Framework documents archived

35. Allocations via EV+

36. EV+ Summary • Continue to accept/decline allocations • Log estimate and actual visit dates • Availability of visit plan PDF • Some reminder facilities will remain

37. Quality Assurance Management System (QAMS)

38. Benefits of the Quality Assurance Management System (QAMS) • Online Reporting for Systems and Qualification Verification • Workflow for verification activity • Information Management System • Intelligence to inform future selections for Systems and Qualification Verification • Reporting output to centres

39. Benefits of the Quality Assurance Management System (QAMS) • SQA • Information Management System • Reduction in manual monitoring, logging and tracking • Intelligence building for each centre • Centre • Improved report layout with non-compliances at glance • Improved Service Level Agreements • Evidence review only contains non-compliant criteria • Qualification and System Verifiers • Ease of use, web based system • Built in Guidance and on screen errors • Only need to view active, outstanding workload • Evidence review only contains non-compliant • criteria

40. Quality Assurance Management System (QAMS) . • Online Reporting – do not use OLD excel report forms • Web link to QAMS included in guidance and EV+ • Automated reminders email addresses • - EV+ noreply@sqa.org.uk • - QAMS asv@sqa.org.uk

41. Welcome Back

42. David Pirnie Lead Verifier – HN & VQ

43. Supporting Qualification Verification Update of SQA’s Quality Assurance Guidance New system for dealing with Qualification Verification queries Introducing the Standardisation Record

44. Quality Assurance Guidance has been updated The purpose of SQA’s Quality Assurance Guidance is to: • support Qualification Verifiers • promote consistency and transparency in Qualification Verification • provide a source of reference for verifiers and centres

45. Visit Planning 1. Accept allocations on EV+ 2. Make initial contact with each centre and ascertain optimum time to visit (if visit is feasible) 3. *Request information from centre 4. Select sample, formulate and send Visit Plan *Where centres have agreed to provide evidence for all selected units, they should still let you know who the assessors/verifiers are and list the assessment sites being used

46. Criteria wording changes 2.8 There must be evidence of initial and on-going reviews of assessment environment(s), assessment procedures, equipment, learning and assessment materials 4.1 The centre’s assessment and verification procedures must be documented and implemented to meet qualification and SQA requirements(the “review” requirement removed – now in 2.8) 3.2 Candidate’s development needs and prior achievements (where appropriate) must be matched against the requirements of the award (“These must be related to agreed personal action/training plans” has been removed and placed in examples of evidence

47. QCF Assessor/Verifier Requirements General requirements apply: All Assessors/Internal Verifiers In general terms, SQA requires all Assessors/Internal Verifiers to be competent in the subject/occupational area to a level appropriate to the qualification and competent in assessment/internal verification of the type involved in the qualification. QCF and Customised Awards Assessors/Internal Verifiers do not need to show evidence of having achieved nationally recognised assessor/verifier qualifications e.g. A1/V1/9DI as per SVQs.

48. Context section has been added Criterion 2.8: There must be evidence of initial and on-going reviews of assessment environment(s), assessment procedures, equipment, learning and assessment materials. Rationale for criterion inclusion It is each centre’s responsibility to ensure that the assessment environment, assessment procedures, equipment, learning and assessment materials are up to date and provided in sufficient quantity to meet ……………………….. Context “Evidence of planned or/and past reviews can be Considered in judging whether sufficient evidence exists” .

49. Context section has been added Context - other examples: 3. 2 “Collective and/or individual candidate development needs can be considered when reviewing evidence” 4.1 “Comparing expected practice (documented procedures) with actual practice (the implementation of those procedures) can help provide appropriate evidence” 4.2 “Assessment methods/instruments and decisions should be sampled “

50. Other changes to Quality Assurance Guidance Interviewing candidates is important to the verification process (all qualifications) andshould take place whenever possible Where the qualification(s) sampled include one or more SVQ Units, the relevant Sector’s Assessment Strategy applies to those units New Guide to Assessment about to be published