Advice on Data Used to Measure Outcomes

1. Advice on Data Used to Measure Outcomes Friday 20th March 2009

2. Today’s Workshop • Introduce myself & background • 2008/09 SOA – SBC experience • 2009/10 SOA – Context • How we developed the 2009/10 SOA • Consultation • Linking activities to SOA – the 4 areas PED leads • How to measure success • The outcome • What SBC needs to measure – a worked example • Issues in what to measure • Primary • Some conclusions • Discussion

3. Introduction • Planning & Economic Development • Business Information Unit has led development of Outcome Agreements • Own experience of developing evaluation frameworks contributing to PSA in England Local Authorities • Process of Logical Frameworks, Logic Chains etc

4. Area of activity • Regeneration – town centre, retail, events & tourism expertise • Property – economic development, infrastructure and property management expertise • Economic Development – tourism, inward investment, business support expertise • Rural Development – economic development, key sectors, funding, regeneration expertise • Business Gateway – key sectors, business support expertise

5. Our Experience • Scottish Government’s National Outcomes • Development for 2008/09 • Scottish Borders Council - Observations • New way of working • Community Planning Partnership = 20 partners • c 48 Outcomes • c 15+ under ‘economic’ • Mix of outcome, indicators, output, activity • Difficulties in performance management • Different times

6. 2009/10 SOA – Context • NO - We realise our full economic potential with more and better employment opportunities for our people • NO - We are better educated, more skilled and more successful, renowned for our research and innovation • NO - Our young people are successful learners, confident individuals, effective contributors and responsible citizens • NO - We live in well-designed, sustainable places where we are able to access the amenities and services we need

7. 2.1 – supporting & growing business activity in key sectors 2.2 – promoting self-employment & creating sustainable businesses 2.3 – maximising employment opportunities 10.3 – improving the socio-economic performance of our town centres & growing stronger communities 3.1 – maximising participation in education, training & employment amongst economically inactive 3.2 – existing workforce is highly skilled and responsive to the needs of employers 3.3 – the Scottish Borders Campus is a World Class Centre of Learning 4.2 – all young people will be in education, training or employment 4.3 – young people demonstrate achievement & attainment 9 Outcomes that relate to the ‘economy’

8. How did we decide? • Examined where SBC PED is responsible & accountable? • Probably 4 key areas • Looked at the nature of our contribution • Where SBC contributes with other Stakeholders? • Through Competitive Borders • Where SBC’s other departments are responsible & accountable • Skills, education, learning

9. Consultation • Internal consultation – SBC • Logic Chain Approach • External consultation • Partners eg SEn, VisitScotland, SDS, Sectoral Groups • Education sector • Business Planning

10. Objective of exercise • To understand the activities, projects we do • How these link to strategy, policy, objectives • And how these need to link to Single and National Outcomes • What is needed to deliver activities => And this is what we have come up with

11. Linking activities to SOA

12. Linking activities to SOA

13. Linking activities to SOA

14. Linking activities to SOA

15. But how do we measure success? • Outcome = result from outputs – why we should be spending ££ (cumulative = impact) • Output = The services delivered as a result of activities – measurable • Activities = The way(s) in which we deliver/ implement • Inputs = The resources used to deliver or produce activities (& outputs) eg human and financial resources

16. How to Measure the Outcome

17. And what is SBC measuring? • Outcome? Outputs? • Effectiveness in meeting Objectives? • Inputs - Project Managers must consider the causal link between activities & ultimate performance • So we use ‘if/ then’ methodology - at each level we can develop targets and indicators => For example, the data we need to collect

18. Example: Business Start Up Support

19. Example: Business Start Up Support

20. Example: Business Start Up Support

21. Example: Business Start Up Support

22. Regardless of ‘who measures’, there may be issues • What to monitor – hard & soft outcomes? • Baseline • Tracking - a data capture strategy; • Primary – Business or Beneficiary – survey? • Secondary – spatial availability? Sampling issues? Delay • Frequency – over what time period • Longitudinal • Attribution • Reference Case? Control Groups • Collection & Ownership • Who and at what cost? • Single point of capture? Shared assessment? Benefits?

23. Pros & Cons - Primary

24. Advice on data used will depend on • Why it needs to be collected; at which level of the ‘Logic Tree’; and by who • The audience – is it • Internal (eg Economic Development, budget holders, project managers) • External (eg partners, funders, public, policy makers) • Their different levels of interest • Achievements against objectives • Monitoring • Outcomes & Impacts • Value & Lessons • Strategic Added Value

25. Close Richard Sweetnam 01835 825 069 richard.sweetnam@scotborders.gov.uk