1 / 26

Bruno Curvale President of the ENQA board Bologna expert Head of international affairs at AERES

ENQA experiences with external reviews of agencies - Challenges -. Bruno Curvale President of the ENQA board Bologna expert Head of international affairs at AERES. 29th of May 2009 Vilnius. Outline of the presentation. Introduction I - European Quality Assurance Major steps

dai
Download Presentation

Bruno Curvale President of the ENQA board Bologna expert Head of international affairs at AERES

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ENQA experiences with external reviews of agencies - Challenges - Bruno Curvale President of the ENQA board Bologna expert Head of international affairs at AERES 29th of May 2009 Vilnius

  2. Outline of the presentation • Introduction • I - European Quality AssuranceMajor steps • II - The ESG and the ENQA membership criteria • Similarities and differences • III - ENQA experiences • ENQA coordinated reviews • The work of the board • IV - The issue of consistency • Interpretation • Notions • Cultural dimension • V - Improvements and remediation • Debriefing • Training

  3. There are many challenges related to: The context The objectives of ENQA The link between the ESG and the ENQA membership criteria The complexity of the two stages evaluation process The nature of the judgements to be made by the review panels and the ENQA board The methodology itself

  4. I - European Quality AssuranceMajor steps

  5. Bologna process and quality assurance developments Primary responsibility of HE institutions for quality Evaluation of EQAR Cooperation of QA agencies and HE institutions European cooperation in quality assurance European Standards and Guidelines European Register EQAR 04/2008 E4 Group 2009 Leuven Louvain-la-Neuve 1999 Bologna 2001 Prague 2003 Berlin 2005 Bergen 2007 London 2006 Reco. 2006-143-CE 1998 Reco. 98-561-CE Source : Colin Tück, E4, B. Curvale

  6. II - The ESG and the ENQA membership criteriaENQA objectivesSimilarities and differences between the ESG and the ENQA criteria

  7. ENQA objectives Promotion of agencies ENQA full membership as a recognised sign of professionalism A single external review of agencies for ENQA membership and EQAR listing

  8. Differences and similarities between the ESG and the ENQA Criteria ENQA criteria Principle based or rules based? Agencies shall follow the ESG and have to comply with the membership criteria Correspond to part 3 of ESG Integrate the part 2 of ESG (and by consequence the part 1 of the ESG) Occasionally associate Standards (ENQA criteria 1) Occasionally mix Standards and guidelines but keep the requirement in the conditional (ENQA criteria 4) Occasionally go further than the standard or the guidelines (ENQA criterion 3, ENQA criterion 7) Occasionally split Standards in different criteria (ENQA criteria 6i and 6ii) Have additional requirements (ENQA criteria 8i, 8ii, 8iii) But are fundamentally identical to the letter and the spirit of the ESG, unless it is the opposite

  9. III - ENQA experienceENQA coordinated reviewsThe work of the board

  10. The experience • 7 ENQA coordinated reviews organised so far (ANECA, AQA, AQU, ACSUG, AGAE, ARACIS, EUA-IEP) • 2 to be done (SKVC for sure and ECCE) • 24 agencies (among 39 full members) are already externally reviewed (AAC, AQA, FHR, VLHORA, NEEA, EVA, ACQUIN, AQAS, ASIIN, FIBAA, GAC, ZEvA, HAC, HETAC, NQAI, NVAO, NOKUT, PKA, AGAE, ANECA, AQU, NAHE, OAQ, QAA) • 8 candidates members are preparing to the review

  11. The types and the coordination of the external review process Type A (ENQA membership criteria / ESG) Type B (also other aspects) - statutory functions (national missions, …) - ECA Code of Good Practice - special context (e.g. bi-national character, NVAO) - follow-up on previous review … Co-ordination national ENQA(+ other non-national) Sources: Tibor Szanto, Barcelona 17/03/09

  12. The work of the panel Is the agency… - fully compliant - substantially compliant - partially compliant - non-compliant … with the criterion/ESG standard Overall judgement not required (but can be made if the panel wishes to do so) Sources: Tibor Szanto, Barcelona 17/03/09

  13. The work of the board To answer the question: Can it follows the analysis and thejudgements made by the panel and decide to grant the full ENQA membership. What the board expects from the report: ESG Parts II and III fully covered Each criteria discussed separately Structure (under each criteria): - evidence („description”, „facts and data”) - analysis („consideration”) - conclusion (judgement on compliance) - recommendation (if any) Sources: Tibor Szanto, Barcelona 17/03/09

  14. IV - The issue of consistencyConsistency of judgement

  15. Consistency of judgement is a challenge For the panels and for the ENQA board because of: Situations have to interpreted Notions used are difficult Language issues Cultural dimension Implicit knowledge

  16. V - Improvements and remediationDebriefingTraining

  17. The experience is gathered Well organised review processes are determinant Internal processes and external processes are equally important ENQA is learning from what is done by the panels Debriefing/brainstorming meeting for the training of experts for agency reviews, Barcelona, Spain, 17 March 2009 ENQA has started to train external reviewers Training session/workshop for experts for agency reviews, 21-22 May 2009, Paris, France

  18. The experience is shared ENQA develops its documentation and makes it available to all: The Guidelines for national reviews of ENQA member agencies The Briefing pack for review panel members of ENQA coordinated reviews The Principles for ENQA coordinated reviews are now merged in one single document: The Guidelines for external reviews for the purpose of ENQA membership ENQA will continue to develop the methodology of external reviews The board has now a “book of precedents”, regularly updated by the ENQA secretariat

  19. Thank you for your attention bruno.curvale@orange.fr www.enqa.eu

  20. The European parliament and the council of the European Union… Hereby recommend that member States: … 4. Enable higher education institutions active within their territory to choose among quality assurance or accreditation agencies in the European register an agency which meets their needs and profile, provided that this is compatible with their national legislation or permitted by their national authorities 5. Allow higher education institutions to work towards a complementary assessment by another agency in the European Register, for example to enhance their international reputation. … Extract of the Recommendation of 15 February 2006 on further cooperation in QA in HE

  21. EQAR Criteria All agencies which comply substantially with the European Standards and Guidelines for Quality Assurance (ESG) can be admitted to the Register. Substantial compliance with the European Standards and Guidelines is to be evidenced through an external review by independent experts. Such a review is coordinated either by a national authority or another organisation that is independent from the quality assurance agency under review. Full ENQA membership, being also based on substantial compliance with the ESG, will normally constitute satisfactory evidence for inclusion in the Register. Extract from the EQAR website http://www.eqar.eu

  22. Standard 3.3 Guidelines 3.3 Standard 3.1 ESG 3.1, 3.3 / ENQA Criterion 1: Activities “The Agency undertakes external quality assurance activities (at institutional or programme level) on a regular basis. These may involve evaluation, review, audit, assessment, accreditation or other similar activities and should be part of the core functions of the Agency. In undertaking its activities, the Agency should take into account the presence and effectiveness of the external quality assurance processes described in Part 2 of the European Standards and Guidelines (ESG” Extract from the Application form for membership of ENQA

  23. Standard 3.5 Guidelines 3.5 ESG 3.5 / ENQA Criterion 4: Mission statement “The Agency should have clear and explicit goals and objectives for its work, contained in a publicly available statement. This statement should describe the goals and objectives of the Agency’s quality assurance processes, the division of labour with relevant stakeholders in higher education, especially the higher education institutions, and the cultural and historical context of its work. The statement should make clear that the external quality assurance process is a major activity of the Agency and that there exists a systematic approach to achieving its goals and objectives. There should also be documentation to demonstrate how the statement is translated into a clear policy and management plan.” Extract from the Application form for membership of ENQA

  24. Standard 3.4 ESG 3.4 / ENQA Criterion 3: Resources “The Agency should have adequate and proportional resources, both human and financial, to enable it to organise and run its external quality assurance process(es) in an effective and efficient manner, with appropriate provision for the development of its processes, procedures and staff.” Extract from the Application form for membership of ENQA

  25. Standard 3.8 Guidelines 3.8 “are expected“ in the ESG guidelines ESG 3.8 / ENQA Criterion 7: Accountability procedures “The Agency should have in place procedures for its own accountability. These procedures are required to include the following: i. a published policy for the assurance of its own quality, made available on its website; ii. documentation which demonstrates that: … … ii. a mandatory cyclical external review of its activities at least once every five years which includes a report on its conformity with the membership criteria of ENQA. Documentation could here either indicate that a review has taken place or is planned.” Extract from the Application form for membership of ENQA

  26. Review AND evaluation of the review ENQA full membership Inclusion in EQAR Evaluation of: - the review process - the review outcomes ENQA board Register committee Review of the agency (ESG 3, 2 and 1) + specific ENQA criteria External review Procedures, “behaviour” Agency National regulations Purposes Q Standards Specific objectives 1 2

More Related