slide1 n.
Download
Skip this Video
Download Presentation
‘Research into Quality Assurance of Training Provision for the BANKSETA’ - November 9 th 2005 -

Loading in 2 Seconds...

play fullscreen
1 / 38

‘Research into Quality Assurance of Training Provision for the BANKSETA’ - November 9 th 2005 - - PowerPoint PPT Presentation


  • 116 Views
  • Uploaded on

‘Research into Quality Assurance of Training Provision for the BANKSETA’ - November 9 th 2005 -. - Index - . The structure of this presentation is as follows …. The research map The challenges of the BANKSETA Quality Assurance from a process perspective

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '‘Research into Quality Assurance of Training Provision for the BANKSETA’ - November 9 th 2005 -' - joshua


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the structure of this presentation is as follows

- Index -

The structure of this presentation is as follows …
  • The research map
  • The challenges of the BANKSETA
  • Quality Assurance from a process perspective
  • BANKSETA’s view on the definition of Quality Assurance
  • Selection of findings and conclusions of the research
  • General conclusions on Quality Assurance

Research MapSlide 2

ChallengesSlide 3

Frame work Slide 4-12

Definition Slide 13-17

Findings Slide 18-35

Conclusions? Slide 36

the research followed the following steps partly in an iterative way

In depth view

Fit to challenges

Implementation

Acceptance

-

+

Definition

- Research map -

The research followed the following steps, partly in an iterative way …

Challenges

BANKSETA

Relevance (Filter)

‘Practice’ 1

Relevant

Practice 1

Practice 2

Literature research

Framework

‘Practice’ 2

Practice 3

Legislation /

ETQA functions

‘Practice’ ..

Interviews

Information

& Sources

‘Practice’ X

Challenges

Country

Sector

Legislation

Initial research

Best

Practice

a summary of the main challenges that the bankseta was facing at the start of the research

- Challenges BANKSETA -

A summary of the main challenges that the BANKSETA was facing at the start of the research …
  • Accreditation: What is a user-friendly process for service providers and is it accessible and manageable for small providers?
  • Programme evaluation/approval: Should this be done and if so, what is the most effective and efficient way to do so?
  • Overlap: Certain scenarios result in overlaps with other SETAs with different systems in use. How can potential conflict in roles and responsibilities be avoided or resolved? Are MoUs the appropriate mechanisms?
  • Audits: What is the best combination of internal and external audits and how can the internal audit process be streamlined?
the primary process the learning value chain

Programme Development- SP -

Certification/

Data recording- ETQA -

Needs Analysis- SP / Empl -

Learning- SP -

Assessment- Assessor -

- Framework -

The Primary Process: The Learning Value Chain

Primary

  • Characteristics
  • Volume driven process steps. The more programmes and the more learners, the more often these steps have to be executed
  • Steps are part of the Learning Value Chain.
  • If the activities are not (well) executed this has a direct effect on the Quality of Learning.
secondary process how do we check the quality of the primary process

Certification/

Data recording- ETQA -

Programme Development- SP -

Assessment- Assessor -

Learning- SP -

Needs Analysis- SP / Empl -

- Framework -

Secondary Process: How do we check the Quality of the primary process?

Primary

Programme review

Learning Evaluation

Moderation

Internal

Secondary

Programme review

Learning Evaluation

Moderation

External

  • Characteristics
  • Volume driven activities. The more programmes and the more learners, the more often these steps have to be executed
  • Activities look directly into the steps of the Learning Value Chain.
  • If the activities are not (well) executed this has an indirect effect on the Quality of Learning.
slide7

Programme Development- SP -

Certification/

Data recording- ETQA -

Needs Analysis- SP / Empl -

Learning- SP -

Assessment- Assessor -

- Framework -

Tertiary Process: The capacitating of the entities involved in the primary and secondary processes. Who do we need to capacitate?

Primary

Programme review

Learning Evaluation

Moderation

Internal

Secondary

Programme review

Learning Evaluation

Moderation

External

3

The Moderator

The ETQA registers Moderators.

1

Service Provider

The ETQA accredits, audits and empowers Service Providers

2

The Assessor

The ETQA registers Assessors.

slide8

- Framework -

Tertiary Process: How is the right capacity of Service Providers, Assessors and Moderators ensured by the ETQA?
  • Characteristics
  • Activities are not volume driven. Irrespective of the number of programmes or learners, the activities will take place according to a certain frequency.
  • Activities do not look directly into the steps of the Learning Value Chain but check the capacity of the different entities involved in executing the steps of the Learning Value Chain and the Secondary (QA) processes connected to it.
  • If the activities are not (well) executed this has an indirect effect on the Quality of Learning.

In take

Develop

Audit

Service Provider

Internal Audit

External Audit

Monitoring visits

Workshops

Accreditation

Tertiary

Assessor

Forums

Workshops

?

Registration

Moderator

Forums

Workshops

?

Registration

slide9

Certification

Data recording- ETQA -

Programme Development- SP -

Assessment- Assessor -

Learning- SP -

Needs Analysis- SP / Empl -

- Framework -

The total model is made up of Primary, Secondary and Tertiary Processes all working together to deliver quality for the learner.

Primary

Programme review

Learning Evaluation

Moderation

Internal

Secondary

Programme review

Learning Evaluation

Moderation

External

In take

Develop

Audit

Service Provider

Internal Audit

External Audit

Monitoring visits

Workshops

Accreditation

Tertiary

Assessor

Forums

Workshops

?

Registration

Moderator

Forums

Workshops

?

Registration

slide10

- Framework -

The first six of the ten ETQA functions can clearly be linked to the framework and fall within the scope of the QA research.

ETQA functions linked to framework

Accrediting Providers

Promoting Quality amongst Providers

Monitoring provision

Evaluating assessment and facilitating moderation amongst providers

Registering assessors

Certificating learners

ETQA functions outside the framework

Co-operation with relevant moderating bodies

Recommending new standards or qualifications (or modifications to existing standards and qualifications) to SGBs

Maintain a database

Submitting reports to SAQA

all the etqa functions are linked to three streams of the training quality assurance process

- Framework -

All the ETQA functions are linked to three streams of the Training & Quality Assurance process

6 Certificate learners

4 Evaluating assessment and facilitating moderation amongst providers

3 Monitoring Provision

1 Accrediting Providers

2 Promoting Quality amongst Providers

5 Registering Assessors

4 Evaluating assessment and facilitating moderation amongst providers

slide14

- Definition -

The more we focus on the BANKSETA itself, the more the ‘general knowledge’ needs to be extrapolated to the BANKSETA
  • Quality Assurance
  • Quality Assurance in training
  • Quality Assurance for SAQA
  • Quality Assurance for the BANKSETA

Research

Discussion

Scope

Precision

the general definitions of quality assurance

- Definition -

The General Definitions of Quality Assurance ….
  • ISO 9000:2000
  • “Part of quality management focused on providing confidence that quality requirements will be fulfilled.”
  • Six Sigma
  • “Work done to ensure that Quality is built into work products, rather than Defects.” This is by (a) identifying what “quality” means in context: (b) specifying methods by which its presence can be ensured; and (c) specifying ways in which it can be measured to ensure conformance.
  • Q-Africa 2004 Conference Paper
  • “The sum of activities/elements that assure the quality of products and services.”
quality assurance for training is considered

- Definition -

Quality Assurance for training is considered …
  • CHE
  • “The underlying objective of the HEQC with regard to quality assurance is: to ensure that institutions effectively and efficiently deliver education, training, research and community service which are of high quality and which produce socially useful end enriching knowledge…”
  • Engineering Council
  • “Quality assurance: in general, a process which examines the quality of a process and its product; when applied to an education and training programme, evaluating the programme and its assessment against standards using defined policies and procedures.”
  • Higher Education - Australia
  • “(The) systematic management and assessment procedures adopted to monitor performance and achievement and to ensure achievement of specified quality or improve quality.” (Harman & Meek, 2000, p. 11)
the view of saqa and the setas

- Definition -

The view of SAQA and the SETAs …
  • SAQA’s definitions
  • “Quality Assurance means the sum of activities that assure the quality of products and services at the time of production or delivery. Quality assurance procedures are frequently applied only to the activities and products associated directly with the goods and services provided to external customers”
  • “Quality Assurance means the process of ensuring that the degree of excellence specified is achieved”
  • “Implicit in SAQA’s implementation of the quality system is the understanding that quality assurance, quality management and accreditation are not things or products; rather, quality is a process.” (Quality is an ongoing event)
what does this mean specifically for the bankseta

Certification

Data recording- ETQA -

Programme Development- SP -

Learning- SP -

Assessment- Assessor -

Needs Analysis- SP / Empl -

- Definition -

What does this mean specifically for the BANKSETA?
  • The extrapolation to a definition of quality assurance for the BANKSETA:
  • = = = = = =
  • “Quality assurance refers to the integrated processes that evaluate the provision and outcomes of education, training and development, so that compliance with SAQA requirements is achieved and systems for continuous improvement are in place for the benefit of learners.”
  • = = = = = =

Primary

Outside Scope

In Scope

the following sources gave input to the findings

- Findings: Primary Process -

The following sources gave input to the findings
  • Literature
  • General literature on Quality Assurance
  • Skills Development Legislation (National & International)
  • SAQA documentation
  • ETQA Guidelines and Policies
  • Interviews
  • A total of 28 persons were interviewed representing the following groups: Employers, Service Providers, ETQA managers, SAQA officials and independent experts
  • Practices
  • 5 ETQAs were researched
  • 3 International Practices were researched (Aus, New Zealand, Malaysia/Scotland)
practices of the primary process

Programme Development- SP -

Certification /

Data recording- ETQA -

Needs Analysis- SP / Empl -

Learning- SP -

Assessment- Assessor -

- Findings: Primary Process -

Practices of the Primary process

Primary

Programme Approval by ETQAs

1) Yes, including SME and review committee

2) Yes, mainly as part of Accreditation

3) Yes, mainly as part of Accreditation

4) Yes, mainly as part of Accreditation

5) Yes, mainly as separate process, no SME needed

Programme Approval International

Australia: Use of nationally endorsed Training Packages

Programme ≠ Material

Material ∈ Programme

+

the view of experts on the primary processes

Programme Development- SP -

Certification /

Data recording- ETQA -

Needs Analysis- SP / Empl -

Learning- SP -

Assessment- Assessor -

- Findings: Primary Process -

The view of experts on the Primary processes

Primary

ETQAs:

Programme approval is …

- Very important

- Not so important

- SME needed

- SME not needed

Employers:

“Huge problem with time that assessments take”

Scope issue

ETQAs:

“There is a risk in the Employer – Assessor relationship”

Scope issue

Service Providers:

“Too much focus on Assessment and not on Training Needs affects the performance negatively”

“SPs tend more toward measuring business impact of training to improve themselves”

Employers:

“Business Impact will inform the quality better than assessments”

Independent Experts:

“We don’t know how to measure the impact of the NQF and with that the ETQA function”

slide22

- Findings: Primary Process -

Programme Approval is a gate keeper for the end result, Subject Matter Expertise can be applied depending on the context …

E X A M P L E S

  • Approval by committee

Safety

Thoroughness needed in Programme Approval

  • Material correct according to SME
  • Aligned with US according to SME

Potential Impact of errors

Compliance

Possibility of relying on self regulatory function of the market

Banking Sector

  • Aligned with US

Service

  • Outcomes based

Turn around time from sector in case of incorrect content of programme

conclusions on the primary process

Programme Development- SP -

Certification /

Data recording- ETQA -

Needs Analysis- SP / Empl -

Learning- SP -

Assessment- Assessor -

- Findings: Primary Process -

Conclusions on the Primary process

Primary

1

“Garbage in is Garbage out”

2

“Performance Management is needed”

3

“Needs analysis falls outside the scope”

way forward on the primary process

Programme Development- SP -

Certification /

Data recording- ETQA -

Needs Analysis- SP / Empl -

Learning- SP -

Assessment- Assessor -

- Findings: Primary Process -

Way forward on the Primary process

Primary

2

1

3

  • Suggestions
  • Introduce a gate keeper in the form of Programme Approval:- Minimise the use of Subject Matter Expertise- Take samples based on ‘smart’ selection
  • Do further research into whole process of assessments. Is it possible to address the challenge of the sector?
  • Use the Performance Auditing that will be introduced by SAQA to start measuring the performance of the ETQA function from a client perspective (the learner). This will enable continuous improvement of the ETQA processes.
practices of the secondary process

- Findings: secondary process -

Practices of the Secondary process

Programme review

Learning Evaluation

Moderation

Internal

Secondary

Programme review

Learning Evaluation

Moderation

External

Combined with Approval

Does not exist

Incidentally considered

Moderation by ETQAs

Is done incidentally, policy to minimise it.

Is not done

External moderation is done

Moderation International

Aus: No secondary processes are performed

Internal checks

Mostly directly or indirectly covered in Accreditation / QMS criteria

experts on the secondary process

- Findings: secondary process -

Experts on the Secondary process

Programme review

Learning Evaluation

Moderation

Internal

Secondary

Programme review

Learning Evaluation

Moderation

External

Experts:

“Quality Assurance Mechanisms are being set up for the sake of compliance rather than to improve Quality”

Independent Experts:

“Moderation and Program Evaluation should be built-in systems and part of self evaluation”

Independent Experts:

“Too much emphasis on assessment and moderation, we should be putting systems in place”

SAQA:

“The main degrees of freedom that occur where overlap between ETQAs take place, lie in the Secondary Processes”

Service Provider:

“Let me pass the (accreditation) mark and let me then run my own processes”

slide27

- Findings: secondary process -

The overlap between different SETAs leaves us with four combinations with many more possible applications (1) …
  • Service Provider Qualification Sector
  • Normal Cases
  • Case 1 I a, b, c A
  • Case 2 II i, j, k B
  • Case 3 III x, y, z C
  • Special Cases
  • Extra-sectoral I a B
  • Extra-qualification I i A
  • Extra-sectoral/qualification I I i B
  • Extra-sectoral/qualification II I x B
slide28

- Findings: secondary process -

The overlap between different SETAs leaves us with four combinations with many more possible applications (2) …

Service Provider Qualification Sector/ETQA

Special Cases

1. Extra-sectoral I a B

2. Extra-qualification I i A

3. Extra-sectoral/qualification I I i B

4. Extra-sectoral/qualification II I x B

Are these activities done? Who is doing them? Can they be outsourced to another ETQA?

conclusions on the secondary process

Programme Development- SP -

- Findings: secondary process -

Conclusions on the Secondary process

Programme review

Learning Evaluation

Moderation

Internal

Secondary

Programme review

Learning Evaluation

Moderation

External

1

° Freedom

“MoU challenge = secondary processes”

Approval

“Programme Review = Secondary Process”“Programme Approval = Primary Process”

2

Learning- SP -

Programme Review

3

Policing? / Redoing? / Empowering?

“Promotion of a Culture of Quality?”

way forward on the secondary process

- Findings: secondary process -

Way forward on the Secondary process

Programme review

Learning Evaluation

Moderation

Internal

Secondary

Programme review

Learning Evaluation

Moderation

External

  • Suggestions
  • Minimise the effort put into Secondary Processes
  • Rather focus on Tertiary Processes that enable Service Providers to perform internal Secondary Processes
  • Work in conjunction with SAQA towards clarity concerning the degrees of freedom that occur around the Secondary Processes in case of overlap between ETQAs
practices of the tertiary process

- Findings: Tertiary Processes -

Practices of the Tertiary process

In take

Develop

Audit

Service Provider

Internal Audit

External Audit

Monitoring visits

Workshops

Accreditation

Tertiary

Assessor

Forums

Workshops

?

Registration

Moderator

Forums

Workshops

?

Registration

ETQAs - Accreditation

Provisional option often usedAll different categories / Criteria within SAQA framework

International AccreditationAus: larger scope, includes advertising and marketing

Can, Scotland: incl. e.g. Marketing and Needs Analysis

ETQAs - Auditing

Internal Audit not always applicable

Mostly aligned with Accreditation

Mixed with Secondary Processes

International AuditingAus: 1-1 alignment with Accreditation

experts on the tertiary process

- Findings: Tertiary Processes -

Experts on the Tertiary process

In take

Develop

Audit

Service Provider

Internal Audit

External Audit

Monitoring visits

Workshops

Accreditation

Tertiary

Assessor

Forums

Workshops

?

Registration

Moderator

Forums

Workshops

?

Registration

ETQAs:

“Accreditation is the key-element to ensure performance”

Service Provider:

“Internal Audits can be a helpful tool to create an internal platform for change and performance improvement”

General:

“It is possible to get accredited and not perform”

Independent Experts:

“Accreditation process is often not fast enough, results in bottlenecks”

Service Provider:

“There is not always a clear link between Accreditation Criteria and how it can positively affect business processes”

Service Provider:

“Accreditation is still a cumbersome process”

slide33

Certification /

Data recording- ETQA -

Program Development- SP -

Needs Analysis- SP / Empl -

Assessment- Assessor -

Learning- SP -

Taking Corrective Actions

Act

Planning the Primary Process

Checking the results of the Primary Process

Plan

Check

Do

Primary Processes

- Findings: Tertiary Processes -

Although alignment is important, the Demming wheel suggests an accent on the “Act” phase when performing external audits

Accreditation

Int. Audit

Ext. Audit

=

What are the corrective actions based on Plan, Do, Check?

What is the business planning in terms of Programme Focus?

What is the procedure for Programme Evaluation?

What is the procedure for Programme Development?

slide34

Programme Development- SP -

Certification/

Data recording- ETQA -

Needs Analysis- SP / Empl -

Learning- SP -

Assessment- Assessor -

- Findings: Tertiary Processes -

The categories for accreditation/auditing are informed by the Primary and Secondary processes where the SP is involved.

Primary

4

1

2

3

Programme review

Learning Evaluation

Moderation

Internal

5

6

7

Secondary

8

e.g. overhead processes

SAQA

9

Sector Specific

x

accreditation and audit criteria could be summarised and structured as follows

- Findings: Tertiary Processes -

Accreditation and Audit Criteria could be summarised and structured as follows …
  • General QMS requirements, e.g.- Management responsibility- Follow-up systems
  • Specific Policies, Procedures and Processes, e.g.- Core Programme Development, Programme Review (+ requirements) Provision, Learning Evaluation (+ requirements) Assessment, Moderation - Overhead HR (+ requirements) Financial (+ requirements) …
  • Static Criteria, e.g.- Admin and Financial Capacity, Legal Entity, …
conclusions on the tertiary process

- Findings: Tertiary Processes -

Conclusions on the Tertiary process

In take

Develop

Audit

Service Provider

Internal Audit

External Audit

Monitoring visits

Workshops

Accreditation

Tertiary

Assessor

Forums

Workshops

?

Registration

Moderator

Forums

Workshops

?

Registration

1

“thorough processes are needed”

2

“simple and aligned processes”

way forward on the tertiary process

- Findings: Tertiary Processes -

Way forward on the Tertiary process

In take

Develop

Audit

Service Provider

Internal Audit

External Audit

Monitoring visits

Workshops

Accreditation

Tertiary

Assessor

Forums

Workshops

?

Registration

Moderator

Forums

Workshops

?

Registration

  • Suggestions
  • Establish rhythm of Accreditation, Internal Audit and External Audit
  • Align accreditation 1-on-1 with Audits and site visits
  • Focus on a Culture of Quality that goes beyond compliance. Look for ‘real’ evidence in the “Act” phase
  • The strength of the sector suggests that Provisional Accreditation is currently not a necessity. Rather focus on empowerment.
the following bullet points remain open for discussion

- Conclusions? -

The following bullet points remain open for discussion …
  • Can one size fit all?
  • Is a sector specific model the route to follow?
  • Given the degrees of freedom, is it possible to make sensible comparisons?
  • How close is Best Practice to Common Practice?
ad