Outputs, outcomes and impacts

1. Outputs, outcomes and impacts • Using Theory of Change to plan, develop and evaluate your initiative

2. Aims of the session • To make explicit the link between vision, planning and evaluation • To introduce you to Theory of Change as a tool for understanding, developing and evaluating your initiative • To help you prepare for your team planning time this afternoon

3. Introduction to Theory of Change • Theory of Change is… • “an outcomes-based, participatory method…for planning, evaluation, and organisational capacity-building”; it “defines all building blocks required to bring about a given long-term goal”(www.theoryofchange.org) • a development of ‘programme theory’ and ‘logic models’, adapted to complex change where both the path and the destination are evolving (Gamble 2008) • useful as a “bridging tool” that creates “provisional stability”, linking planning and evaluation (Saunders et al 2005)

4. Purpose of Theory of Change

5. Developing your Theory of Change • Identify your long-term goal. • Conduct ‘backwards mapping’ to identify the intermediate outcomes (or preconditions) necessary to achieve that goal. • Identify the activities that your initiative will undertake to create these outcomes. At the same time consider contextual factors, assumptions, resources needed etc…

6. Developing your Theory of Change A good Theory of Change should be: • Plausible – the logic of the theory is credible • Doable – achievable with the resources (and time) available • Meaningful – stakeholders see the goals as important and worthwhile • Testable – there are credible ways of discovering whether the predicted results occur (and how/why)

7. Activity • In teams, develop a simple Theory of Change for your initiative using the template provided (40 mins) • Consider: • Is your goal meaningful? • Does the logic of your theory make sense? • What assumptions are you making? • What resources will you need? • How will you test your theory?

8. Testing your Theory of Change

9. Sources of data • Data from evaluation activities • surveys, focus groups, interviews, case studies etc • time-consuming to collect and limited in scope • Data generated by initiative itself • planning documents, decision logs, network maps, meeting notes, emails etc • freely available but of limited worth externally • ‘Naturally occurring’ institutional data • NSS, module feedback, retention/achievement data etc • valued as impact data but difficult to show causality

10. Developing your evaluation strategy • What aspects of your initiative will you evaluate? • Not everything can or should be evaluated – what is both important and testable? • Why are you evaluating? • To meet performance indicators? To show impact? To understand, inform and develop your initiative? • Who are you evaluating for? • Senior managers? Funders? Staff? Students? The sector? Yourselves? • How will you evaluate? • What sources of data will you use? What methods will you use to collect and analyse the data?

11. Summary and next steps • Theory of Change makes explicit what is assumed or tacitly understood about your initiative • Theory of Change is a planning tool that helps you evaluate and an evaluation tool that helps you plan • Your Theory of Change is just that – a theory: it should be revisited and revised throughout your initiative • Use the team planning time this afternoon to develop your ToC further and consider your evaluation strategy (purposes, audiences, data and methods)

12. References and further reading Dozois, E, Langlois, M & Blanchet-Cohen, H (2010) DE201: A practitioner’s guide to developmental evaluation, J W McConnell Family Foundation. Gamble, J (2008) A developmental evaluation primer, J W McConnell Family Foundation. Rogers, P J (2008) Using Programme Theory for Complicated and Complex Programmes, Evaluation, 14(1), 29-48. Saunders, M, Charlier B, Bonamy, J (2005) Using evaluation to create ‘provisional stabilities’: Bridging innovation in higher education change processes, Evaluation, 11 (1), 37-54. Theory of Change Community, http://www.theoryofchange.org/