1 / 14

Graduate surveys as an outcome evaluation

Institut für Forschungsinformation und Qualitätssicherung. Graduate surveys as an outcome evaluation Presentation EAIR Forum 2009, July 23-26, Vilnius, Lithuania Dr. René Krempkow Institute for Research Information and Quality Assurance, Bonn (Germany)

kathleen-slater
Download Presentation

Graduate surveys as an outcome evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Institut für Forschungsinformation und Qualitätssicherung Graduate surveys as an outcome evaluation Presentation EAIR Forum 2009, July 23-26, Vilnius, Lithuania Dr. René Krempkow Institute for Research Information and Quality Assurance, Bonn (Germany) (former project leader of Freiburg Graduate Surveys, University of Freiburg)

  2. Outline • Relevance of graduate surveys for Quality Management • Measuring outcomes of higher education – Influencing factors of professional success • Exemplary results • Interpretation of results • Prospects for analysis

  3. Graduate Surveys

  4. When integrating graduate surveys in quality management: • Adequate interpretation of results only through comparison with other HEIs and more general national data (see Teichler 2003) • Account for individual characteristics through institution-specific and programme-specific surveys (see QM-literatur) • Nationwide “Core Questionnaire” with institution-/programme-specific adaptations The Potential of graduate surveys for Quality Management I

  5. Analyse data to identify determinants of professional success to feed back to Quality Management system • Unit of observation: study programmes • Account for specific context by interpreting data taking into account influencing factors (e.g. regional economy) • Account for specific initial conditions, e.g. % of student parents, socioeconomic background of students (Schomburg/Teichler, 1998) The Potential of graduate surveys for Quality Management II

  6. The Potential of graduate surveys for Quality Management III • Source: Schomburg/ Teichler (1998: 165) • Two HEI stand out positively • To what degree can differences be attributed to different starting conditions of HEIs? This will be discussed on the basis of Freiburg Graduate Surveysand Dresden Graduate Surveys.

  7. Measuring Outcome with graduate surveys – what does professionell success depend on? • Multivariate analysis allows identifying factors influencing professionell success • This information is independant from (subjective) self-assessment of graduates (see Krempkow/Pastohr 2006) • Example: job experience, study times abroad and specific competences appear to be more important than study duration and grades – That leads to the question: Can and (if yes) What can HEIs do to improve professional success of graduate? • Multivariate (correlatory) analysis allows analysing relationships between professional success and aspects of study, including starting conditions and context!

  8. Influencing factors on professional success Objective and subjective criteria of professional success as indicators of HE outcomes (Teichler/Schomburg (1997: 248) • Objective indicators for professional success (e.g. income, hierarchical position) • Subjective indicators for professional success (e.g. job satisfaction, autonomy) • Objective indicators for transition HE-labour market(e.g. Time of job hunt) • Subjective assessment of goodness-of-fit HE – job (e.g. usefulness of study contents, employment adequacy). • Freiburg and Dresden Survey: Gross income, job satisfaction and employment adequacy fitting criteria for professional success.

  9. Basic model of influencing factors on professional success One time study: CHARACTERISTCS OF HIGHER EDUCATION (PROGRAM) INDIVIDUAL CHARACTERIS-TICS + SOCIO-ECONOMIC BACKGROUND LEARNING OUTCOMES (COMPETENCES) 1. threshold INDIVIDUAL STATE AT TIME OF SURVEY PROFESSIONAL SUCCESS / JOB SEARCH, TRANSITION PHASE, JOB ENTRY 2. threshold First repetition of study: INDIVIDUAL STATE AT TIME OF SURVEY 2. threshold PROFESSIONAL SUCCESS JOB SEARCH, TRANSITION PHASE, JOB ENTRY

  10. Example of results: stand. Regression coefficients for income Figure 3: Regression coefficients for the monthly income of current occupation; +/ */ **= significant on the 10%-/ 5%-/ 1%-level (“-“ = variable was not included in the explanatory model)

  11. Example of results: stand. Regression coefficients for income • Similar Results in Dresden Graduate Survey with the same analysis scheme, but the social background was more important Figure 3: Regression coefficients for the monthly income of current occupation; +/ */ **= significant on the 10%-/ 5%-/ 1%-level (“-“ = variable was not included in the explanatory model)

  12. Interpretation of results (Freiburg & Dresden) • Influencing income: Previous job training, GPA (Secondary school) and gender, in Medicine: Personal contacts and career oriantation are important • Influencing job satisfaction: Technical sciences: Socioeconomic background, previous job trainingEngineering: Tendendy for women to be less satisfied Humanities: Higher self-assessed expertise and family orientation: lower job satisfaction • Influencing job adequacy:Engineering: gender - women have less adequate jobs Prospects for analysis?

  13. Further information • IfQ Bonn: www.research-information.de • Freiburg Graduate Surveys: www.qm.uni-freiburg.de/projekte • Requirements of graduate surveys for measuring HE outcomes: „Leistungsbewertung, Leistungsanreize und die Qualität der Hochschullehre“ (2007) www.universitätsverlagwebler.de/krempkow.html, Onlinepublikation (2005) http://nbn-resolving.de/urn:nbn:de:swb:14-1129208825969-55860 • Determinants of professional success: Zeitschrift für Evaluation 1/2006 (www.zfev.de), S. 7-37, bzw. http://www.kfbh.de/downloads/Was_macht_Hochschulabsolventen_erfolgreich.pdf • Dresden Graduate Survey: www.kfbh.de/absolventenstudie

  14. Example of results: stand. Regression coefficients

More Related