1 / 20

Measuring the Outcomes of Interventions with Young Carers PRTC Conference, Manchester, 11/10/10

Measuring the Outcomes of Interventions with Young Carers PRTC Conference, Manchester, 11/10/10. Introduction. Defining terms Why measure outcomes? Development of outcome measurement tools for young carers Use of the tools Learning so far Planning considerations. Definitions.

elmo
Download Presentation

Measuring the Outcomes of Interventions with Young Carers PRTC Conference, Manchester, 11/10/10

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring the Outcomes of Interventions with Young CarersPRTC Conference, Manchester, 11/10/10

  2. Introduction • Defining terms • Why measure outcomes? • Development of outcome measurement tools for young carers • Use of the tools • Learning so far • Planning considerations

  3. Definitions • Aims the areas of change you intend to achieve through your • project and which stem directly from the needs of your clients. • Outcomes the changes and effects that actually happen as a result • of your work, expected or unexpected, welcome or unwelcome. • The outcomes you hope to see are all the specific changes that will • help you to achieve your aims. • Outcome change the difference between outcome levels at • different points in time. This requires the collection of pre • intervention (baseline) and post intervention data. • Outputs what the organisation does; the services it delivers. • Source: Cupitt, S. with Ellis, J. (2004) Your project and its outcomes

  4. Why measure outcomes? • To see if the intervention is making a difference • To identify what is working or not working • Value for money • Learning in order to replicate or revise interventions • Planning purposes • Being responsible and reflective practitioners • Holding ourselves to account • To satisfy commissioners and grant bodies

  5. The need for robust measures in the UK • Growth in awareness and recognition of young carers due to: • The work of third sector organisations in prompting public and • political awareness; • Identification through research; • Legislation, guidance and policy initiatives that recognise the rights and needs of young carers and their families. • Census 2001 UK: 175,000 young carers were identified. • 350 projects/services across the UK supporting approx 30,000 young carers. • “ A detailed evaluation of the different approaches and their impact on young carers and their outcomes is lacking” • (National Carers Strategy, HM Government 2008)

  6. Comic Relief/PRTC Grants Programme 2008/11 •  Programme aims: • Identify and reduce the extent of inappropriate or harmful caring responsibilities taken on by children; • Produce positive outcomes for children; • Produce useful learning for other organisations considering similar work; • Identification of an additional aim based on local need (optional); • Work should strengthen families while supporting young carers. • Projects required to provide direct support to young carers under 21 • where the intervention was targeted at the most vulnerable young • carers or those with the heaviest caring responsibilities or • under-represented communities or hard to reach children.

  7. The development process • Stage 1: Identification of items for inclusion in the tools • Stage 2 (1st pilot) Reducing the number of items • Identifying missing items • Feedback from services • Stage 3: Refinement of items using principal component analysis • Stage 4 (2nd pilot): Validation of the instruments • Obtaining preliminary normative data • Feedback from services • Stage 5: Design and publication of the tools and guidance notes re administration

  8. MACA-YC18 Multi-dimensional Assessment of Caring Activities Young Carers 18 item Scale MACA-YC45 Clinical assessment tool

  9. Multi-dimensional assessment of caring activities (MACA-YC18) • 18-item self-report measure to provide an overall summary score • (index) of the amount of caring activity. • Young carers indicate if they do a task: never/some of the time/ • a lot of the time and are then scored 0/1/2 respectively. • Index of caring = the total score. • 0No caring activity recorded • 1 - 9 Low amount of caring activity • 10 -13Moderate amount of caring activity • 14 -17High amount of caring activity • 18 plusVery high amount of caring activity • Six subscales: personal care, domestic tasks, emotional care, • household management, sibling care; financial/practical management.

  10. PANOC-YC20 Positive and Negative Outcomes of Caring – Young Carers 20 item Scale

  11. The positive and negative outcomes of caring (PANOC-YC20) • 20-item self-report measure to provide a score (index) of the subjective • cognitive and emotional impacts of caring. • There are two 10-item subscales: positive and negative impacts. • Young carers indicate if they do a task: never/some of the time/a lot of the time • and are then scored 0/1/2 respectively. • Scores on both scales have a range of 0 - 20. • Higher scores indicating greater positive and negative responses, respectively. • To calculate positive response score Sum items: 1, 2, 3, 4, 7, 8, 15,18,19, 20 • To calculate negative response score Sum items: 5,6,9,10,11,12,13,14,16,17 • Scores of less than 12 on the positive scale and/or greater than 8 on the • negative scale may be indicative of concern and along with other assessment • processes may indicate support is required for the young carer and/or their • family.

  12. Potential uses of the tools • Assessments of carers • Review of progress/interventions • Evaluation: pre intervention and post intervention (immediate) • post-post measuring (sustainability) • Research • Results can be collated to generate scores at three levels: • 1) individual • 2) project/service • 3) programme

  13. Example of use of pre-intervention data for a young carer • Katie, aged 13 • MACA 15 • domestic activity 3 • household management 3 • financial & practical 0 • personal care 1 • emotional care 4 • sibling care 2 • PANOC +ve 20 • PANOC -ve 4

  14. Example of pre-intervention data at a project level • Project ‘A’ Project ‘B’ • MACA 10.31 14.87 • PANOC negative 4.64 8.69 • PANOC positive 14.81 11.62 • Young carers at Project A have a lower MACA score than those at Project B, • indicating that they were involved in lower levels of caring compared to young • carers at Project B. • Young carers at Project B experienced much higher negative outcomes of • caring than young carers at Project A and their positive scores were also • lower, suggesting that they were experiencing less positive benefits of caring • than young carers at Project A. Low positive and high negative scores on the • PANOC indicate that there is a potential for concern regarding the outcomes • of caring for young carers at Project B.

  15. MACA-YC18 113 fully completed tools. Individual scores for young carers ranged from 1 - 26 that indicates a wide variability in the amount of care being undertaken. Mean MACA score: 12.06 ie higher end of the moderate band of the scale with girls (12.82) and boys (10.76). Source: University of Nottingham (2009) Interim Evaluation Report, Comic Relief/ PRTC Young Carers Grants Programme 2008/11 PANOC pre intervention results 118 fully completed tools. Individual scores ranged from 0 - 20 on both the positive and negative scale indicating wide variability in the cognitive and emotional consequences of caring. Mean pre-intervention PANOC score: 13.34 (positive) and 6.21 (negative). Hence across the 11 projects there is not a level of significant negative outcomes. Examples of pre-intervention data at a programme level (11 projects)

  16. Feedback comments • “As a worker we found the questionnaires very useful. Especially when a young carer has completed both pre and post intervention questionnaires we can easily recognise the difference if the young carer has any changes. It is very easy to analyse the data and it helps us to save a lot of time when we come to do evaluations”. • “The questionnaire has been a very valuable addition to the work that we are doing with young carers. It has enabled us to measure the interventions that we make in a much more systematic way including the impact of caring responsibilities on the young people”. • “When the young people have completed these correctly they did show clearly the extent of caring duties and where these were inappropriate. This information was useful in terms of influencing the decision of the local authority to provide a care package”.

  17. Challenges • Administration of the tools and the issue of trust • More ‘paperwork’ alongside other organisational requirements • Ensuring sufficient time and support for completion of the tools • Issues of literacy and language • Young carer resistance or suspicion • Opting out - not completing post intervention tools • Incomplete tools • Project anxiety

  18. Planning considerations • Staff briefing and exploration of any anxieties • Guidance notes • Integrate into existing paperwork and practice • Personalise the ‘Me and My Young Carers’ questionnaire to align with your project aims and objectives • Workload implications for staff to support young carers completing the tools • Administration to assist those responsible for co-ordinating and analysing the data • Troubleshooting • Translation of instruments • Using the results

  19. References • Stephen Joseph, Fiona Becker, Saul Becker (2009) Manual for • measures of caring activities and outcomes for children and young people • available: www.carers.org (young carers page on assessments). • Stephen Joseph et al., (2009) Assessment of caring and its effects in young • people: development of the Multidimensional Assessment of Caring Activities • Checklist (MACA-YC18) and the Positive and Negative Outcomes of Caring • Questionnaire (PANOC-YC20 for young carers’. Child: Care, Health and • Development,35 (4) p 510-520. • Honour Rhodes (2009) Knowing what you do works. Family and Parenting • Institute. • Sally Culpitt with Jean Ellis (2004) Your project and its outcomes, Charities • Evaluation Service. • For a range of reports and resources on young carers go to: www.saul.becker.co.uk

  20. Contactfbecker@nspcc.org.ukor saul.becker@nottingham.ac.uk

More Related