1 / 24

Collecting and Analysing Data

Collecting and Analysing Data. Chris Dayson Research Fellow Presentation to: Involve/CRESR Social Impact Masterclass 26th September 2013. 1. Collecting data. What types of data are there?. Primary data. This is the data that you have collected or could collect Examples include:

jory
Download Presentation

Collecting and Analysing Data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Collecting and Analysing Data Chris Dayson Research Fellow Presentation to: Involve/CRESR Social Impact Masterclass 26th September 2013

  2. 1. Collecting data

  3. What types of data are there? Primary data This is the data that you have collected or could collect Examples include: Monitoring information User surveys Outcome stars Case management information Project outputs Qualitative

  4. What types of data are there? Secondary data This is the data that others have collected that might help you Examples include: Area level data: Census Benefits claimants Deprivation Crime User level: Health Crime

  5. What types of data should you collect? Some tips Think about the types of data you already collect Could you use it more effectively? Does this tell you what you need to know about your outcomes and impact? What gaps are there: does it cover all key stakeholders? Think about the data that others might collect about your stakeholders Could you use it to demonstrate your impact better? Could it help explain context Can you access it? Consent and data protection Negotiation and mutual benefits

  6. 2. Collecting primary data

  7. Starting out Some considerations Map out what it is you need to know What outcomes are you trying to measure What would outcome change look like in practice What level/scale will you start with? The whole organisation A project A particular stakeholder group Think about your priorities Decide on the tools and methods you will use Who will be responsible for data collection? Surveys: face-to-face; postal; web/email Outcome Stars: practitioner/user led

  8. Starting out Some considerations How will collate you the data? Data entry onto a spreadsheet/database Bespoke platforms Who will be responsible for regular data entry How will you analyse and use the data? Frequency of reporting Who will be responsible for analysis and reporting? Think about the skills, capacity and resources Are there any skills gaps? Need to create time and space to do it well Can you build it into funding bids etc

  9. Developing outcome indicators Some key principles Don't reinvent the wheel: use existing measures where possible sources include: ONS, WikiVois, UK Data archive, similar organisations Measure distance travelled: baseline; during; post-intervention; to provide evidence of change and how long it lasts retrospective measurement possible but less effective capture evidence re additionality and impact Sampling: decide how many beneficiaries you need to measure/track need to expect attrition in the sample over time

  10. 3. Analysing data

  11. Analysing quantitative outcome data Key principles Key it simple and relevant What will you organisation find useful? What will funders etc expect to see? Focus on change How much change have you observed? how many/what proportion have improved? how much improvement has there been? have there been different levels of improvement within different groups?

  12. 4. A worked example

  13. Background info A volunteering employability programme Aim: to improve the employability of people 'disadvantaged in the labour market' through volunteering based interventions Stakeholder: individual beneficiaries of the programme 'Hard' outcomes: individuals undertake volunteering individuals move into employment employment is sustained 'Soft' outcomes: individuals move closer to the labour market individuals have improved health and well-being individuals are more involved in their communities

  14. Mapping the change Soft outcomes Overarching outcome: individuals move closer to the labour market Specific outcomes: Greater motivation to find work More confidence in holding down a job More skills and experience Better able to complete job applications More confident in attending interviews

  15. Analysing the data Distance travelled findings Distance travelled data: 200 participants - all completed a baseline 100 participants completed a distance travelled questionnaire after 4 months

  16. Analysing the data Interpreting the data to identify change We can now extrapolate for all service users Of 200 beneficiaries: 80 were more motivated to find work 74 were more confident in holding down a job 78 had more skills and experience 76 were better able to complete job applications 86 were more confident in attending interviews High levels of additionality for each outcome Qualitative interviews corroborate quantitative findings Next steps: Tracking beneficiaries for a longer period Valuing outcomes?

  17. 5. Taking it further: valuing outcomes

  18. Putting a value on the outcome Approaches to valuation/monetisation Towards social return on investment (SROI) Aim: assigning a value to something without a market price A range of options: Cost savings - to the public sector (and others) Real money - net gains in income Willingness to pay - how much would they pay for the outcome Revealed preference - build up the value from other market values Other proxies: Travel cost and household spending Don't forget the stakeholder's perspective

  19. Valuation in practice Returning to the worked example How would we value? Greater motivation to find work More confidence in holding down a job More skills and experience Better able to complete job applications More confident in attending interviews

  20. Valuation in practice The proxification conundrum Some considerations: Are we looking at more than one outcome... ...or a subset of outcomes linked to 'being better equipped to find work'? Is being better equipped to find work an interim outcome on the way to actually getting a job? From a valuation perspective, does getting a job usurp any outcome on the way to finding work Would we be 'double counting' if we valued each outcome for each participant?

  21. Valuation in practice Disentangling the proxification conundrum One solution: For each participant that finds work, identify a proxy value for that work For participants that do not find work, but are better able to find work, identify a proxy value for that outcome This approach ensures outcomes are not double-counted and that outcomes are not over-valued

  22. Collecting and Analysing Data Any questions? Chris Dayson Research Fellow Contact details: email: c.dayson@shu.ac.uk tel: 0114 225 3539 web: www.shu.ac.uk/cresr

More Related