Quality Assurance in Italian Universities

1. EURAM 2010 Back to the Future 19-22 May 2010 – Tor Vergata, Rome Quality Assurance in Italian Universities Gianfranco Rebora, University Cattaneo – LIUC (Italy) Matteo Turri, University of Milan (Italy)

2. This paper aims to review the main events in the development of evaluation activities in Italian universities from 1993 to the present day, to explain the reasons of poor results coming from a relevant collective effort to find some relevant evidence also in the optics of pubblic management and governance

3. QA/ Evaluation • Not a very specific concept… • … comprehends different approaches, methodologies and practices referred to definition, development and assessment of quality, aiming to improve the ability of institutions, staff and students to meet HE goals (which are debated not fixed beforehand)

4. 2007 – 2009 Bologna ProcessStocktaking Report 2009 Report from working groups appointed by the Bologna Follow-up Group to the Ministerial Conference in Leuven/Louvain-la-Neuve 28-29 April 2009 • Stage of development of external QA system 3 • Level of student participation in QA 2 • Level of international participation in QA 2

5. 1993- 2009: Evaluation and QA activities and practices have been growing at a fast rate But results seem poor: Accountability ? Improvement ? Why do results seem poor?(of QA in Italian experience)

6. Conceptual framework Components of an evaluation system Institutional and Organisational Impact: IDEA Knowledge- Learning Inertia, opportunism and unforeseen or undesidered effects METHODS USE Improvement Accountability BODIES

7. The critical point: How evaluation is used • The “actual use” according to Patton (1997, p.20) is the best way to understand the value of evaluation activities and the efforts dedicated to it. We considered: • Generating knowledge • Improvement • Accountability • non-use, given that in some cases no use is made of evaluation output

8. The Italian QA story: three stages • 1993-1998: start up • 1999 – 2006: a surfeit of information and very few results • 2007-2009: stalemate

9. Bodies

10. How QA has been used: an overview Growing and diffused knowledge This use of evaluation “causes embarrassment” because it provides justification for university government actions that cannot be taken” Path of improvement has been interrupted (in research field) No use of evaluation prevails – risk is avoided

11. 1999-2006: a surfeit of information but very few results Universities interpreted this greater autonomy by increasing the number of degree courses and adopting opportunist behaviour such as increasing the number of competitions for lecturing positions, at times dealing with career opportunities in an underhand way

12. This university was admitted…

13. 2007-2009: stalemate • 2007: a law decided the merger of the CNVSU and CIVR and the setting up of the ANVUR • 2008: the new government modified the set-up of Anvur • 2009: the new rules are on the way of final approval, but the Agency is not yet beginning its activity • 2010: a new research evaluation exercise is now starting (2004-2008): CIVR is still in charge of it • Universities must now manage relevant cuts in their budgets with the risk that future rules about evaluation will increase the pressure for compliance and conformity

14. Lessons learned • To emphasize (to stress) the USE (of evaluation) helps to understand things • The field of Higher education is open /very sensitive to accept evaluation • International (European) drive • Availability of core competences • Culture/past experiences of research • Governance matters • University system governance • Institution level • Two different vision are in conflict: • Administrative/ bureaucratic • Professional

15. 3. Governance problems have negative impact on use of evaluation • University system level • HE Institutions level

16. At University system level • An administrative/ bureaucratic approach prevails on a more substantial and professional one • Minimum quality requisites have a very formal intepretation • Lack of resources/budget and political events influence the continuity of QA practices: es. CIVR/VTR after 2006

17. At HEIs level • A weak type of governance like the one in Italian universities is disinterested in evaluation because it involves making decisions that no one has the strength to make: • Several rectors saw evaluation as a stimulus and tool for governance but then had difficulty in finding the necessary consensus for re-election • A sudden reduction in the autonomy of evaluation units is however quite common after the election of a new rector • (MINELLI E, REBORA G., TURRI M. (2008). How can evaluation_fail? The case of Italian universities. QUALITY IN HIGHER EDUCATION. vol. 14)

18. Mismatch between macro level initiatives in QA and micro level experienced needs We can remove it by: • establishing a threshold in order to prevent the diffusion of weak higher education initiatives • abolishing rules that impose specific organisational patterns and limit strategies of differentiation • providing rules that university leaders (rectors, deans and the various coordinators of teaching activities) can use in order to validate and strengthen their strategic choices and their government structure • promoting autonomy and a more competent professional approach of central bodies and agencies operating in the field of HE

19. How QA has been used: an overview