Skip this Video
Download Presentation
Bruno Curvale President of the ENQA board Bologna expert Head of international affairs at AERES

Loading in 2 Seconds...

play fullscreen
1 / 26

Bruno Curvale President of the ENQA board Bologna expert Head of international affairs at AERES - PowerPoint PPT Presentation

  • Uploaded on

ENQA experiences with external reviews of agencies - Challenges -. Bruno Curvale President of the ENQA board Bologna expert Head of international affairs at AERES. 29th of May 2009 Vilnius. Outline of the presentation. Introduction I - European Quality Assurance Major steps

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Bruno Curvale President of the ENQA board Bologna expert Head of international affairs at AERES' - dai

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

ENQA experiences with external reviews of agencies

- Challenges -

Bruno Curvale

President of the ENQA board

Bologna expert

Head of international affairs at AERES

29th of May 2009


outline of the presentation
Outline of the presentation
  • Introduction
  • I - European Quality AssuranceMajor steps
  • II - The ESG and the ENQA membership criteria
  • Similarities and differences
  • III - ENQA experiences
  • ENQA coordinated reviews
  • The work of the board
  • IV - The issue of consistency
  • Interpretation
  • Notions
  • Cultural dimension
  • V - Improvements and remediation
  • Debriefing
  • Training
there are many challenges related to
There are many challenges related to:

The context

The objectives of ENQA

The link between the ESG and the ENQA membership criteria

The complexity of the two stages evaluation process

The nature of the judgements to be made by the review panels and the ENQA board

The methodology itself


Bologna process and quality assurance developments

Primary responsibility of HE institutions for quality

Evaluation of EQAR

Cooperation of QA agencies and HE institutions

European cooperation in quality assurance

European Standards and Guidelines

European Register



E4 Group














2006 Reco. 2006-143-CE

1998 Reco. 98-561-CE

Source : Colin Tück, E4, B. Curvale

II - The ESG and the ENQA membership criteriaENQA objectivesSimilarities and differences between the ESG and the ENQA criteria
enqa objectives
ENQA objectives

Promotion of agencies

ENQA full membership as a recognised sign of professionalism

A single external review of agencies for ENQA membership and EQAR listing

differences and similarities between the esg and the enqa criteria
Differences and similarities between the ESG and the ENQA Criteria

ENQA criteria

Principle based or rules based? Agencies shall follow the ESG and have to comply with the membership criteria

Correspond to part 3 of ESG

Integrate the part 2 of ESG (and by consequence the part 1 of the ESG)

Occasionally associate Standards (ENQA criteria 1)

Occasionally mix Standards and guidelines but keep the requirement in the conditional (ENQA criteria 4)

Occasionally go further than the standard or the guidelines (ENQA criterion 3, ENQA criterion 7)

Occasionally split Standards in different criteria (ENQA criteria 6i and 6ii)

Have additional requirements (ENQA criteria 8i, 8ii, 8iii)

But are fundamentally identical to the letter and the spirit of the ESG, unless it is the opposite

the experience
The experience
  • 7 ENQA coordinated reviews organised so far (ANECA, AQA, AQU, ACSUG, AGAE, ARACIS, EUA-IEP)
  • 2 to be done (SKVC for sure and ECCE)
  • 24 agencies (among 39 full members) are already externally reviewed (AAC, AQA, FHR, VLHORA, NEEA, EVA, ACQUIN, AQAS, ASIIN, FIBAA, GAC, ZEvA, HAC, HETAC, NQAI, NVAO, NOKUT, PKA, AGAE, ANECA, AQU, NAHE, OAQ, QAA)
  • 8 candidates members are preparing to the review
the types and the coordination of the external review process
The types and the coordination of the external review process

Type A (ENQA membership criteria / ESG)

Type B (also other aspects)

- statutory functions (national missions, …)

- ECA Code of Good Practice

- special context (e.g. bi-national character, NVAO)

- follow-up on previous review



ENQA(+ other non-national)

Sources: Tibor Szanto, Barcelona 17/03/09

the work of the panel
The work of the panel

Is the agency…

- fully compliant

- substantially compliant

- partially compliant

- non-compliant

… with the criterion/ESG standard

Overall judgement not required (but can be made if the panel wishes to do so)

Sources: Tibor Szanto, Barcelona 17/03/09

the work of the board
The work of the board

To answer the question: Can it follows the analysis and thejudgements made by the panel and decide to grant the full ENQA membership.

What the board expects from the report:

ESG Parts II and III fully covered

Each criteria discussed separately

Structure (under each criteria):

- evidence („description”, „facts and data”)

- analysis („consideration”)

- conclusion (judgement on compliance)

- recommendation (if any)

Sources: Tibor Szanto, Barcelona 17/03/09

consistency of judgement is a challenge
Consistency of judgement is a challenge

For the panels and for the ENQA board because of:

Situations have to interpreted

Notions used are difficult

Language issues

Cultural dimension

Implicit knowledge

the experience is gathered
The experience is gathered

Well organised review processes are determinant

Internal processes and external processes are equally important

ENQA is learning from what is done by the panels

Debriefing/brainstorming meeting for the training of experts for agency reviews, Barcelona, Spain, 17 March 2009

ENQA has started to train external reviewers

Training session/workshop for experts for agency reviews, 21-22 May 2009, Paris, France

the experience is shared
The experience is shared

ENQA develops its documentation and makes it available to all:

The Guidelines for national reviews of ENQA member agencies

The Briefing pack for review panel members of ENQA coordinated reviews

The Principles for ENQA coordinated reviews

are now merged in one single document:

The Guidelines for external reviews for the purpose of ENQA membership

ENQA will continue to develop the methodology of external reviews

The board has now a “book of precedents”, regularly updated by the ENQA secretariat


The European parliament and the council of the European Union…

Hereby recommend that member States:

4. Enable higher education institutions active within their territory to choose among quality assurance or accreditation agencies in the European register an agency which meets their needs and profile, provided that this is compatible with their national legislation or permitted by their national authorities

5. Allow higher education institutions to work towards a complementary assessment by another agency in the European Register, for example to enhance their international reputation.

Extract of the Recommendation of 15 February 2006 on further cooperation in QA in HE


EQAR Criteria

All agencies which comply substantially with the European Standards and Guidelines for Quality Assurance (ESG) can be admitted to the Register.

Substantial compliance with the European Standards and Guidelines is to be evidenced through an external review by independent experts. Such a review is coordinated either by a national authority or another organisation that is independent from the quality assurance agency under review.

Full ENQA membership, being also based on substantial compliance with the ESG, will normally constitute satisfactory evidence for inclusion in the Register.

Extract from the EQAR website


Standard 3.3

Guidelines 3.3

Standard 3.1

ESG 3.1, 3.3 / ENQA Criterion 1: Activities

“The Agency undertakes external quality assurance activities (at institutional or programme level) on a regular basis. These may involve evaluation, review, audit, assessment, accreditation or other similar activities and should be part of the core functions of the Agency. In undertaking its activities, the Agency should take into account the presence and effectiveness of the external quality assurance processes described in Part 2 of the European Standards and Guidelines (ESG”

Extract from the Application form for membership of ENQA


Standard 3.5

Guidelines 3.5

ESG 3.5 / ENQA Criterion 4: Mission statement

“The Agency should have clear and explicit goals and objectives for its work, contained in a publicly available statement. This statement should describe the goals and objectives of the Agency’s quality assurance processes, the division of labour with relevant stakeholders in higher education, especially the higher education institutions, and the cultural and historical context of its work. The statement should make clear that the external quality assurance process is a major activity of the Agency and that there exists a systematic approach to achieving its goals and objectives. There should also be documentation to demonstrate how the statement is translated into a clear policy and management plan.”

Extract from the Application form for membership of ENQA


Standard 3.4

ESG 3.4 / ENQA Criterion 3: Resources

“The Agency should have adequate and proportional resources, both human and financial, to enable it to organise and run its external quality assurance process(es) in an effective and efficient manner, with appropriate provision for the development of its processes, procedures and staff.”

Extract from the Application form for membership of ENQA


Standard 3.8

Guidelines 3.8

“are expected“ in the ESG guidelines

ESG 3.8 / ENQA Criterion 7: Accountability procedures

“The Agency should have in place procedures for its own accountability. These procedures are required to include the following:

i. a published policy for the assurance of its own quality, made available on its website;

ii. documentation which demonstrates that:

ii. a mandatory cyclical external review of its activities at least once every five years which includes a report on its conformity with the membership criteria of ENQA. Documentation could here either indicate that a review has taken place or is planned.”

Extract from the Application form for membership of ENQA

review and evaluation of the review
Review AND evaluation of the review

ENQA full


Inclusion in


Evaluation of:

- the review process

- the review outcomes

ENQA board

Register committee

Review of the agency

(ESG 3, 2 and 1)

+ specific ENQA criteria

External review

Procedures, “behaviour”


National regulations


Q Standards

Specific objectives