Evidence based practice and evaluation in career guidance

1. Evidence based practice and evaluation in career guidance Professor Tristram Hooley Session for MA in Careers Education and Coaching

2. Overview

3. Overview

4. Discussion: What do you do that is based on evidence? • What evidence is it based on? • How confident are you in that evidence? • What stops you seeking more evidence?

5. Why evidence matters

6. Make time for evidence

7. How practitioners can engage with evidence?

8. Overview

9. “If I have seen farther than others, it is becauseI stood on the shoulders of giants.” Isaac Newton  (or possibly Bernard of Chartres) Research is an iterative process

10. Building up the evidence

11. The importance of literature • Google Scholar(https://scholar.google.co.uk/) • The Education Endowment Foundation teaching and learning toolkit (https://educationendowmentfoundation.org.uk/evidence-summaries/teaching-learning-toolkit/) • Education & Employershttps://www.educationandemployers.org/research-main/. • The Careers and Enterprise Companyhttps://www.careersandenterprise.co.uk/research. • The International Centre for Guidance Studieshttps://www.derby.ac.uk/research/icegs/.

12. The scientific method Reject Refine Confirm

13. What is a research question? A research question sets out: • the issue that you are going to be investigating; • your argument or thesis (what you want to prove, disprove, or explore); and • the limits of your research (i.e. what you are not going to be investigating).

14. An example • Demonstrating that career guidance works. • Does career guidance work? • Does career guidance in English schools work? • Does career guidance in English schools increase young people’s chance of choosing a STEM subject at A level?

15. Overview

16. The cycle of continuous improvement

17. Turning the cycle of continuous improvement into an evaluation cycle

18. Constructing a theory of change • A theory of change describes the steps that need to take place between what you do and the impact that you hope to achieve. • It helps you to think about what you are doing and why. • It also helps you to establish a framework for evaluation and quality assurance.

19. An example theory of change

20. Exercise: Construct your own theory of change

21. 4. Designing outcomes. Levels of impact All of these levels of impact matter! But as you move up the levels they can get more difficult to measure/assess

22. Do something with it! • Finish it • Write it up • Send it to people. • Put it on the agenda of a meeting. • Set up a meeting. • Publish it. • Produce bespoke summaries.

23. Overview

24. About the evidence base • There is a considerable evidence base on career guidance. • It demonstrates repeatedly that career guidance has a range of individual and social impacts. • The evidence is multi-disciplinary, international and based on a wide range of research methods. • There are also a number of literature reviews, systematic literature reviews and statistical meta-analyses. • In summary we know that it works (for some things) and we know quite a lot about how it works.

25. Impacts for individuals • Improves engagement with education • Enhances performance in the education system (increases motivation) • Support transitions between education and work (and other life stages) • Contributes to lifelong wellbeing and success

26. Career guidance systems should

27. Career guidance should be…. • Well designed • Learning focused • Context aware • High volume • Varied • Experiential • Led and coordinated by professionals • Involving employers and working people • Recognising the diversity of learners • Providing feedback and assessment

28. References • Andrews, D. & Hooley. (2018). The Careers Leader Handbook. Bath: Trotman. See also https://www.trotman.co.uk/CLH/. • Booth, Williams, & Colomb (2003). The craft of research. Chicago: The University of Chicago Press • Dent et al. (2014). Higher Education outreach To widen participation. Toolkits for practitioners: Evaluation. Bristol: HEFCE. • Early Intervention Foundation. (n.d.). EIF evidence standards. Available from http://www.eif.org.uk/eif-evidence-standards/ [Accessed 19th April 2018]. • Goldacre, B. Bad Science. http://www.badscience.net/ • Hooley, T. (2014). The evidence base on lifelong guidance. Jyväskylä, Finland: European Lifelong Guidance Policy Network (ELGPN). • Hooley, T. (2017). Moving beyond ‘what works’: Using the evidence base in lifelong guidance to inform policy making. In Schroder, K. and Langer, J. Wirksamkeit der Beratung in Bildung, Beruf und Beschäftigung (The Effectiveness of Counselling in Education and Employment) (pp.25-35). Bielefeld: WBV. • Pawson, R. and Tilley, N. (1997). Realistic Evaluation London. Sage • University of Leicester Student Learning Development. (n.d.). Planning and conducting a dissertation research project. Available from https://www2.le.ac.uk/offices/ld/resources/writing/writing-resources/planning-dissertation [Accessed 22nd October 2018]. • What works and other research publications are available from https://www.careersandenterprise.co.uk/research/publications

29. Conclusions • Evidence matters! • We know quite a lot about what works? • We are still learning all the time. • We need to evaluate our own practice and to use the research of others. • But none of it matters unless we all act on it.

30. About me Tristram Hooley Professor of Career Education Email:t.hooley@derby.ac.uk Twitter: @pigironjoe Blog: https://adventuresincareerdevelopment.wordpress.com/