Sqa s approach to quality assurance of assessment
This presentation is the property of its rightful owner.
Sponsored Links
1 / 95

SQA’s Approach to Quality Assurance of Assessment PowerPoint PPT Presentation


  • 133 Views
  • Uploaded on
  • Presentation posted in: General

SQA’s Approach to Quality Assurance of Assessment. Matthew McCullagh Quality Manager Welcome 欢迎. Why are we here?. To introduce you to SQA’s New Approach to Quality Assurance To help you understand the key changes to Quality Assurance process To help you become familiar with:

Download Presentation

SQA’s Approach to Quality Assurance of Assessment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Sqa s approach to quality assurance of assessment

SQA’s Approach to Quality Assurance of Assessment

Matthew McCullagh

Quality Manager

Welcome

欢迎


Why are we here

Why are we here?

  • To introduce you to SQA’s New Approach to Quality Assurance

  • To help you understand the key changes to Quality Assurance process

  • To help you become familiar with:

    • The new Quality Assurance Criteria

    • The new Confidence Statements

    • How visits are planned, conducted and reported


Thursday 6 th september 2012

Thursday 6th September 2012

9.00 Coffee and Registration

9.30 Introduction, New Approach to QA Overview, Aims and Objectives of Training Days

10.10 Preparation for Qualification Verification Activity, Attitudes and Behaviours, KPMs

11.00 Break

11.15Preparation for Qualification Verification Activity,

Attitudes and Behaviours

12.00 Planning for Qualification Verification Activity – the theory

12.30Lunch

1.30Conducting the Qualification Verification Visit – the theory

2.45Break

3.00Introducing the Quality Assurance Criteria

3.20Quality Assurance Criteria Workshop

4.30Close


Friday 7 th september 2012

Friday 7th September 2012

9.00 Coffee and Registration

9.30Feedback: The Quality Assurance Criteria

11.15Break

11.30Decision Making

12.30Lunch

1.30 Communicating Decisions

2.00 Theory of Completing Excel Reports/Action Plans

2.30 Break

2.45Writing effective Verifier Reports

3.45Case Study Assignment

4.00Summary and Q & A

4.30Close


Sqa s approach to qa

SQA’s Approach to QA

  • SQA is moving to introducing a new approach to Quality Assurance of qualifications delivered in China

  • It is based on SQA’s

    Principles of Quality Assurance:

    • Open, fair and transparent

    • Aimed at risk reduction

    • Proportionate

    • Intelligence led

    • A shared responsibility

    • Constantly reviewed and improved


What is the new approach to quality assurance

What is the new approach to Quality Assurance

SQA now operates a transparent, intelligence-led, risk based model for quality assurance of approved centres and potential centres delivering our HND qualifications in China.


Timeline of implementation of new approach in china

Timeline of Implementation of New Approach in China

May

2012

September 2012

September 2011

March

2012

September

2012-2013


New quality criteria

New Quality Criteria

  • Five Categories:

    • Management of a centre

    • Resources

    • Candidate Support

    • Assessment & Verification

    • Records/Data Management

  • All signposted to four QA processes

  • All pre-rated as High/Medium/Low impact

  • All supported by possible sources of evidence


The new approach to quality assurance the four processes

The New Approach to Quality Assurance: The Four Processes

  • Systems Approval

  • Qualification Approval

  • Systems Verification

  • Qualification Verification


High medium low impact criteria

High/Medium/Low Impact Criteria

  • All criteria are important

  • However, some are more important than others

  • Those that are most important are rated as High Impact criteria

  • Those that are less important are rated as Low Impact criteria

  • In between these two are medium impact criteria.

  • Failure to meet a High Impact criteria will have a greater impact on the outcome of the visit


Sufficiency of evidence

Sufficiency of evidence

  • No Systems Verifier or Qualification Verifier can change the impact rating of a criteria.

  • For each criterion you verify, you must decide whether the centre has presented you with:

    • Sufficient evidence to meet the criterion

    • Some, but insufficient evidence to meet the criterion

    • No evidence to meet the criterion


Good practice and guidance

Good Practice and Guidance

  • As part of the verification visit, you should identify good practice that is evident in the centre.

  • You should also give advice and guidance to help centre staff improve the delivery of our qualifications

  • This is a mandatory part of your role as a Qualification Verifier


Sufficiency of evidence1

Sufficiency of evidence

  • We will support you and help you to understand what sort of evidence centres can provide to meet each criterion

  • Once you have made your decision regarding the sufficiency of evidence, SQA staff will then calculate our level of confidence

  • This will depend on the impact level of the criteria and the sufficiency of the evidence presented


Introduction of confidence statements

Introduction of Confidence Statements


How confidence statements are calculated

How Confidence Statements are calculated

  • Confidence statements are given for the outcome of each Systems Verification and Qualification Verification visit for each of the categories of criteria verified

  • These are based on the sufficiency of the evidence provided to the you

  • A summary statement of confidence is also given, based on the confidence statements for each category


What if non compliances are identified

What if non-compliances are identified?

  • Where there is No evidence or insufficient evidence to meet a criterion, you must agree with centre staff, during the visit:

    • the action the centre needs to take in order to become compliant.

    • The evidence they need to produce

    • Where to send this evidence

    • The date by which the action must be taken


What happens then

What happens then?

  • SQA will ask you to consider the evidence the centre sends us and ask you to consider whether it is Sufficient or insufficient.

  • Depending on the sufficiency of the evidence, the Confidence Statement will be re-calculated.


Sanctions

Sanctions

  • If the outcome of a verification visit results in a Confidence Statement of Reasonable, Minimal or No Confidence, SQA staff may decide to place a sanction on a centre until our confidence increases to Broad or High.

  • Our confidence may decrease if a centre fails to meet action points agreed during a verification visit.

  • Sanctions will only be used where necessary.


Sanctions1

Sanctions


Benefits of new approach for centres

Benefits of new approach for centres

  • Open and transparent to all

  • Clear guidance available to verifiers

  • Proportionate response where issues are identified

  • Focused on Good Practice and improvement as well as compliance


Questions

Questions


Sqa s approach to quality assurance of assessment1

SQA’s Approach to Quality Assurance of Assessment

David Pirnie

Lead Verifier

Welcome

欢迎


Qualification verification

Qualification Verification

The Qualification Verifier role activities

Prepare for verification

Plan verification

Carry out verification

Communicate verification decision

Complete verification report


Preparing for qualification verification

Preparing for Qualification Verification

The Qualification Verifier role activities

Prepare for verification

Plan verification

Carry out verification

Communicate verification decision

Complete verification report


Being prepared

Being prepared

“The best preparation for good work

tomorrow is to do good work today”.

Elbert Hubbard


Preparing for qualification verification1

Preparing for Qualification Verification

Before carrying out any activities, verifiers are

required to have a full understanding of their role

and responsibilities. This involves acquiring

knowledge and keeping it up to date.


Preparing for qualification verification2

Preparing for Qualification Verification

Required knowledge – three key areas:

Knowledge of your subject area

Centre roles and centre responsibilities

Qualification Verification


Preparing for qualification verification3

Preparing for Qualification Verification

Knowledge of your subject area:

Have a common interpretation of the Unit Standards including the Evidence Requirements for each unit you have been assigned to verify

Know how to access Arrangements Documents, Instruments of Assessment and Marking Schemes to gain a full understanding of content


Preparing for qualification verification4

Preparing for Qualification Verification

Knowledge of centres roles and responsibilities:

Centre Co-ordinators

Assessors

Internal verifiers


Preparing for qualification verification5

Preparing for Qualification Verification

Knowledge of the Qualification Verifier role:

Role Profile

Sources of support

Verification policies and procedures

Values


Qiips quality integrity innovation partnership service

QIIPS:Quality, Integrity, Innovation, Partnership & Service

Sounds like:

“QUIPS” – a dictionary definition:

“A clever remark often prompted by the occasion”.


Sqa values

SQA Values

Quality

Integrity

Innovation

Partnership

Service


Sqa values1

SQA Values

Quality

As custodians of the standards we verify, we

continually strive to maintain consistent quality in

their assessment.


Sqa values2

SQA Values

Integrity

We use open and honest communication with internal and external customers (centres), promoting transparency, and building trust with others.

We take a standardised approach to verification as laid down by SQA and agreed by consensus within our Qualification Verification team, even if our personal opinions may at times conflict.


Sqa values3

SQA Values

Innovation

We view unfamiliar approaches to assessment and verification with an open mind, whilst ensuring quality is maintained.

We believe that creativity should be seen not as a threat to quality assurance but as an opportunity for improving upon existing practice.


Sqa values4

SQA Values

Partnership

We understand that by working in partnership with SQA centres, we will achieve common goals of excellence and consistency in assessment.


Sqa values5

SQA Values

Service

We maintain a professional approach with SQA centres at all times, regardless of the challenges we might face in carrying out our work.


Preparing for qualification verification6

Preparing for Qualification Verification

More information on the preparation phase of

Qualification Verification can be found in the first

section of SQA’s New Approach to Quality

Assurance: Guidance on Visiting Verification For

HN Qualifications in China


Workshop

Workshop

Values Exercise


Planning for qualification verification

Planning for Qualification Verification

The Qualification Verifier role activities

Prepare for verification

Plan verification

Carry out verification

Communicate verification decision

Complete verification report


Planning

Planning

“Failing to plan is planning to fail”.

Alan Lakein


Visit planning sampling

Visit Planning: Sampling

SQA cannot always verify all HN Units, for all

candidates, on every visit to a centre, therefore a

sampling strategy has to be adopted.

Purpose of sampling:

To establish that there is a common interpretation of the standards within each centre and that the assessment and verification systems allow valid, reliable and fair assessment decisions to be made.


Visit planning applying a sampling frame

Visit Planning: Applying a sampling frame

SQA is responsible for systematically sampling on

a percentre basis:

Selected Units

Assessment decisions and practice

Methods/Instruments of assessment

Problematic/revised units

Locations where assessment takes place

Evidence of candidates across Units

Documented evidence of policies/procedures (implementation)


Visit planning

Visit Planning

SQA (China Office) agrees optimum date (s) for visiting verification. This is an agreement between SQA, each centre and each QV

SQA selects Units to be verified and communicates this to each centre (copying to SQA China Office)

SQA (China Office) confirms to each QV,

the centre(s) they should visit, the date of each visit, and the Units they should verify

Qualification Verifier contacts centre to confirm visit date(s) and travel arrangements


New approach to quality assurance

New Approach to Quality Assurance

Welcome Back

欢迎回来


Carry out qualification verification

Carry out Qualification Verification

The Qualification Verifier role activities

Prepare for verification

Plan verification

Carry out verification

Communicate verification decision

Complete verification report


Carry out qualification verification1

Carry out Qualification Verification

Discussion point

Consider the verification activities you perform

from the time of arrival until the end of the visit.

Contribute to the discussion by letting us know

what kind of techniques/approaches you have

used that you have found particularly beneficial.


Carry out the visit

Carry out the visit

Prepare yourself for the visit

Agree agenda and running order of visit

Conduct sampling activities

Judge the sufficiency of evidence


Conduct sampling activities

Conduct sampling activities

Complete Units (all Outcomes complete)

Incomplete Units (some Outcomes complete)

Evidence not yet internally verified

Assessment/verification decisions

Assessment and verification practice

Interviews with assessors/verifiers/candidates

Referencing of evidence to standards


Introducing the quality criteria

Introducing the Quality Criteria

Introducing the New Approach to Quality

Assurance Criteria

Introducing the Qualification Verification Guidance


Introducing the quality criteria1

Introducing the Quality Criteria

In your groups, discuss and familiarise yourselves with the criteria you have been allocated.

Formulate a list of summary points on flip chart paper to help you explain the purpose of each criterion to your colleague verifiers. The summary should also include possible sources of evidence.

In formulating your points, you are asked to use SQA’s New Approach to Quality Assurance: Guidance on Visiting Verification for HND Qualifications in China as your information source.

Timings:

30 minutes discussion and research

20 minutes flip-charting your points

Please appoint a spokesperson(s) to present the flip-charted points which will be presented in the first session tomorrow morning.

Each group will be assigned a facilitator to answer any questions/queries you may have.


Questions1

Questions


The new approach to quality assurance

The New Approach to Quality Assurance

Welcome Back

欢迎回来


The new approach to quality assurance1

The New Approach to Quality Assurance

Introducing the the new approach to quality assurance criteria

feedback


Criteria 4 2

Criteria 4.2

4.2

The centre must provide documented evidence to ensure that assessments are valid, reliable, equitable and fair.

Signed pre-delivery checklist.

Standardisation meeting minutes, internal audit, review records


Criteria 2 3

Criteria 2.3

2.3

Records must be maintained to provide evidence that the centre has sufficient competent staff who have the necessary qualifications, occupational experience and understanding to support the assessment and internal verification of qualifications being offered in the centre.

CVs/staff info sheets, CPD records, copies of relevant certificates, list of current assessors and internal verifiers, changes to the deployment of assessors/internal verifiers


Criteria 4 8

Criteria 4.8

4.8

Outcomes of External quality assurance must be disseminated to appropriate staff and any action points must be monitored against agreed timescales.

Signed distribution list, corrective action log/report, minutes of meetings


Criteria 3 2

Criteria 3.2

3.2

Candidates' development needs and prior achievements (where appropriate) must be matched against the requirements of the award. These must then be related to agreed personal action/training plans.

APL evidence, skills profile, training needs analysis, assessment plans, reviews


Criteria 4 6

Criteria 4.6

4.6

The centre must comply with requests for access to premises, records, information, candidates and staff for the purpose of external quality assurance.

Documented procedure for handling QA visits, roles and responsibilities, site selection checklists, permission for SQA QA representatives to obtain access


Criteria 2 7

Criteria 2.7

2.7

There must be evidence of initial and on-going reviews of accommodation, equipment and reference, learning and assessment materials.

Current Health & Safety certificates

itinerary checklists, procurement records, library contents, system for supporting e-assessment

Records of review


Criteria 4 3

Criteria 4.3

4.3

Evidence of candidates' work must be accurately and consistently judged by assessors against SQA's requirements.

Assessor reports, records of achievement, standardisation meeting minutes, IV reports, Qualification Verification reports


Criteria 4 1

Criteria 4.1

4.1

The centre's documented assessment and verification procedures must be implemented and the subsequent assessment/verification practices reviewed, the results of which must be recorded and actioned.

Minutes of assessor/internal verifier meetings, internal audit, review records


Criteria 4 7

Criteria 4.7

4.7

Candidate evidence must be retained in line with SQA requirements.

Candidate evidence, documented retention policy


Criteria 4 5

Criteria 4.5

4.5

The centre must take steps to ensure that assessment evidence is the candidate's own work.

Candidate disclaimer, Candidate and staff induction materials: assessor/IV roles and responsibilities


Criteria 5 6

Criteria 5.6

5.6

Comments/queries about the qualification specification, assessment guidance, qualification verification or related SQA matters must be resolved and recorded.

Correspondence file, emails, standardisation meeting minutes, notes of action points and resolution


Preparing for qualification verification7

Preparing for Qualification Verification

The Qualification Verifier role activities

Prepare for verification

Plan verification

Carry out verification

Communicate verification decision

Complete verification report


Making verification decisions

Making verification decisions

Deciding on the sufficiency of evidence can be likened to making an assessment decision

Making an assessment decision takes account of the overall strength of the evidence provided

One source of evidence will be sufficient at times. At other times, a number of sources will have to be considered, dependent on each centre’s approach to meeting quality assurance requirements and to some extent the structure of each criterion


Sqa s approach to quality assurance of assessment

One evidence source could be sufficient in some cases for this criterion:


Sqa s approach to quality assurance of assessment

More than one evidence source is likely to be considered for this criterion:


The main guiding principles for making verification decisions

The main guiding principles for making verification decisions

Decisions should be based:

on the sufficiency of evidence relating to each criterion

only on the criteria provided

The evidence examples given in the guidance

document are just that. They are not a shopping

list of evidence sources that all centres must have.


Sqa s approach to quality assurance of assessment

Own

standards

Awareness of things that could influence on our decisions

Attitude of centre staff

Own

experience

Own

mindset (closed)

Our confide-ence

Previous centre perform-ance


Insufficient no evidence action points

Insufficient/no evidence = action points

Sufficient evidence - this means that the centre has provided evidence that fully meets the criterion (no action points required)

No evidence – this means that the centre has not provided any evidence in support of the criterion (action points required)

Insufficient evidence – this means the centre can provide some evidence in support of the criterion (action points required)


Agreeing action points

Agreeing action points

Is there anything wrong with this action point?

“You should submit evidence of having

disseminated the results of Qualification

verification visits to relevant staff”


Action points should be smart

Action points should be SMART

Specific: Centres should be clear in terms of the action they need to take to close-off the action point.

Measurable: How will SQA or the centre know that the action point has been met, what will be the measure of success?

Achievable: The centre must have sufficient time in which to achieve the agreed action. Please factor in time it takes for SQA to receive, process and edit the report and send it to the centre.

Relevant: The action must directly relate to the criterion

Time bound: Agree a specific and realistic.

date for the action to be completed by.


Sqa s approach to quality assurance of assessment

Questions


New approach to quality assurance1

New Approach to Quality Assurance

Welcome Back

欢迎回来


The stages of qualification verification

The stages of Qualification Verification

Prepare for verification

Plan verification

Carry out verification

Communicate verification decision

Complete verification report


The stages of communicating verification decisions

The stages of communicating verification decisions

Plan to deliver balanced feedback

Explain your verification decisions


Plan to deliver balanced feedback

Plan to deliver balanced feedback

Plan to include:

  • Good practice

  • Recommendations

  • Actionpoints (where appropriate)

    Consider:

  • Location

  • Order of feedback and emphasis

  • Words and tone


Explaining your verification decisions

Explaining your verification decisions

Where action points are given, ensure you:

  • have explored all possible evidence sources

  • gather all information to support your decisions

  • can justify your decisions - clearly relating them

    to criteria requirements

  • Give the centre an opportunity to ask questions and clarify points


Communicating decisions

Communicating Decisions

Role play exercise:

David as a Qualification Verifier feeding back at the end of a Qualification Verification visit to Matthew who is the SQA Co-ordinator.

Please note your observations of the following on your observation sheet:

The content of the Feedback

Attitudes and behaviours of the Qualification Verifier and

the SQA Co-ordinator

Any possible improvements


The stages of qualification verification1

The stages of Qualification Verification

Prepare for verification

Plan verification

Carry out verification

Communicate verification decision

Complete verification report


The stages of completing the qualification verification report

The stages of completing the Qualification Verification Report

Ensure report content matches your feedback

Justify your verification decisions

Write clear action points

Check report prior to submitting to SQA


The qualification verification visit report

The Qualification Verification Visit Report


The criteria pages

The Criteria Pages


The required action log

The Required Action Log


The summary worksheet

The Summary Worksheet


Recommendations and good practice worksheet

Recommendations and Good Practice Worksheet


Writing effective verifier reports

Writing Effective Verifier Reports

SMART action points

Report writing guidance available

www.sqa.org.uk/files ccc/Writing for SQA part A.pdf

Write report as soon after visit as possible

Checking content thoroughly


Assignment report writing

Assignment: Report Writing

Assignment will be emailed to you by Thursday 20th September.

You will be asked to:

Review a Case Study

Complete a Qualification Verification Visit Report

Return to [email protected] by Friday 12th October


Assignment report writing1

Assignment: Report Writing

SQA will provide you with:

Guidance document on how to complete a qualification verification visit report

Feedback on each report to provide support

byFriday 26th October 2012


Summary

Summary

  • To introduce you to SQA’s New Approach to Quality Assurance

  • To help you understand the key changes to Quality Assurance process

  • To help you become familiar with:

    • The new Quality Assurance Criteria

    • The new Confidence Statements

    • How visits are planned, conducted and reported


Questions2

Questions


  • Login