Quality assurance in doctoral school programmes

1. Quality assurance in doctoral school programmes Dr. Thomas EkmanJørgensen 6 December 2013 ECOOM Colloquium, The use of indicators for research evaluation purposes Hasselt University

2. Doctoral Education in Europe • Doctoral education has expanded significantly over the last decade • 50%-100% increases in graduations are not uncommon • There has been increasing political attention • Inclusion in the Bologna Process 2003 • Salzburg Principles 2005 – Salzburg II 2010 • Increased importance for EU research policies • Innovation Union 2010 • Principles for Innovative Doctoral Training 2011 • National legislation

3. Universities have responded • Since 2005, we have seen a ’quiet revolution’ in doctoral education • Professional management: The Rise of the doctoral school • 30 % of universities had a doctoral school in 2007 • 65 % in 2009* • 85% EUA ERA survey 2013 • Reform of doctoral programmes • Transferable skills • Mobility components • Quality Assurance *TRENDS V, TRENDS 2010

4. EUA Council for Doctoral Education – a response to the changes • EUA – European University Association • Council for Doctoral Education (CDE) • 850 universities and rectors’ conferences in 47 countries • Developing evidence-based policies • Advocating these policies • Promoting development of universities as institutions • a membership service focused on doctoral education • Development of doctoral schools • Doctorate-specific policy development • 223 members in 35 countries (from Faro to Tomsk)

5. CDE activities Stakeholder dialogue - EU and global Recommendations and policy development Membership activities - Events, Doctoral Education Bulletin, networking and projects

6. Salzburg Principles and recommendations • Salzburg Principles from 2005 – outcomes of an EUA-led project and a Bologna seminar • The doctorate is research-based • Importance of institutional strategies • Diversity • Salzburg recommendations 2010 – from consultations with CDE members • Research as the ’basis and the difference’ from the other two cycles • Space for individual development • Autonomy for the institution to choose mission and strategy and to set up the appropriate structures

7. Salzburg II on QA • “It is necessary to develop specific systems for quality assurance [for doctoral education]... there is a strong link between the assessment of the research of the institution and the assessment of the research environments that form the basis of doctoral education.” • Development of systems that combine quality of research, quality of structures and take into account ”the professional development of the researcher as well as the progress of the research project.”

8. Convergence between doctoral education and QA • QA has been closely linked to the creation of the EHEA • Focus on 1st and 2nd cycle • Establishment of a quality culture (shared values and structures to enhance quality) • Doctoral Education has been reformed through doctoral schools as part of the professional management • Close to the research mission of universities • Procedures have been established, but not directly labelled as QA • Common basic elements: Accountability and quality enhancement

9. Quality Culture • We are in a period where quality culture in doctoral education is moving from a ’professional’ to an ’integrated culture’

10. Quality culture and indicators • KPIs are good for a quick overview • However, quality only makes sense in a context, and so do indicators • Indicators should reflect qualitative as well as quantitative characteristics • Importantly, they need to be part of a feedback loop for quality enhancement

12. Two sides of quality in doctoral educationAcademic quality • Critical mass of research (Salzburg II): • ”Europe’s universities have developed diverse strategies to assure critical mass and diversity, building their areas of strength through focused research strategies and engaging in larger research networks, collaborations or regional clusters” • Indicators could be bibliometrics, external funding, qualifications of staff (Poland), size of programmes (Italy), output/impact/environment (REF, UK)

13. Academic quality II • Academic quality needs an inclusive research environment • Doctoral candidates need to be part of the excellent research • Measured through student satisfaction (to be used cautiously!) • Supervision as the main factor • Good supervision is central to the quality culture • Most universities have rules and/or guidelines for good supervision • Professional development of supervisors is a priority

14. Indicators for academic quality (ARDE Survey)

15. Procedural quality • Linked to the professional management of doctoral education • Are the rules transparent and efficient? • Who gets admitted, how long should it take, how is the outcome evaluated? • Is there accountability? • If something does not work, who is responsible? • How do we fix it? • Indicators could be satisfaction, time to degree and completion rate

16. Satisfaction with procedures (ARDE Survey)

17. What do rules and guidelines for supervision contain? (ARDE Survey)

18. What are the sanctions? (ARDE Survey)

19. The ’life-cycle’ of doctoral education: Thesis defence

20. Indicators for procedural quality (ARDE Survey)

21. Conclusions • Universities have set up structures to deal with academic and procedural quality • We do not know much about how these are connected to quality enhancement • ... And do they promote a quality culture?

22. The road goes ever on...Upcoming and ongoing activities connected to doctoral education • Workshop: ”The Outcomes of Doctoral Education” – Dokuz Eylül University, Izmir Turkey 23-24 January 2014 • Conference: ”The Global Doctorate” – University of Liverpool, UK, 19-20 June 2014 • Publication on the European Research Area – Spring 2014

23. Thank you for your attention