Outcome measurement system for cacs in tennessee
This presentation is the property of its rightful owner.
Sponsored Links
1 / 46

Outcome Measurement System for CACs in Tennessee PowerPoint PPT Presentation


  • 77 Views
  • Uploaded on
  • Presentation posted in: General

Outcome Measurement System for CACs in Tennessee . Agenda. Background Administrative Guide- Survey Administration Administrative Guide- Score Sheets Data Analysis Final questions. Materials. Hard Copies Administrative Guide Disk Administrative Guide Surveys Score sheets

Download Presentation

Outcome Measurement System for CACs in Tennessee

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Outcome measurement system for cacs in tennessee

Outcome Measurement System for CACs in Tennessee


Agenda

Agenda

  • Background

  • Administrative Guide- Survey Administration

  • Administrative Guide- Score Sheets

  • Data Analysis

  • Final questions


Materials

Materials

  • Hard Copies

    • Administrative Guide

  • Disk

    • Administrative Guide

    • Surveys

    • Score sheets

    • PowerPoint presentation


Background

Background

  • Membership Priority – Evaluation and Assessment

    CACTX maintains its commitment to an outcome based model and will initiate proactive efforts to conduct a comprehensive evaluation project designed to establish shared, meaningful outcomes for local CACs and for the statewide movement.

2006

2007

2008

2009

2010

2011


What is out there

What is out there?

  • NIJ Special Report

  • NCAC Cost-Benefit Analysis

  • UNH Multi-Site Evaluation of CACs

  • Various studies regarding individual components

2006

2007

2008

2009

2010

2011


Policy research project

Policy Research Project

2006

2007

2008

2009

2010

2011


Types of evaluation

Types of Evaluation

  • Program Monitoring Evaluation

  • Outcome Evaluation

  • Impact Evaluation


Logic model

Logic Model

Zone 1

Zone 2

Zone 3

INPUTS

ACTIVITIES

OUTPUTS

OUTCOMES

IMPACT

Accounting and Administration

Performance Management

Research

*From Peter Frumkin, RGK Center Director Power Point Presentation – CACTX Annual Conference 2009


Outcome measurement system for cacs in tennessee

Literature Review

T

H

E

M

E

S

I

N

D

I

C

A

T

O

R

S

Focus Group

Instrument Analysis

Project Activities


Project activities

Project Activities

  • Orientation:

    • Introduction to CAC world

    • Tour of local center

  • Literature Review:

    • CAC Model and Outcome Measurement

  • Focus Groups:

    • 3 Focus Groups with CAC Directors

  • Instrument Analysis:

    • National Institute of Justice Study

    • Collected tools from CACs

  • Outcome Measurement System:

    • Develop and Review

  • Report/Final Product:

    • Final Deliverable to CACTX


Literature review cac model

Literature Review – CAC Model

  • Importance of support networks, particularly non-offending caregiver for child recovery

  • Effectiveness of using MDT approach for handling child abuse cases

  • Implications of quality forensic interviews

  • Difficulty of comparing research on prosecution outcomes


Literature review outcome measurement

Literature Review-Outcome Measurement

Definition

Internal Uses

External Uses


Focus groups november 2008

Focus Groups – November 2008

Participants

  • Small CAC: 10

  • Mid-Size CAC: 12

  • Large CAC: 13


Questions

Questions

  • 3- 5 important indicators that you collect.

  • Indicators you wish you could collect.

  • Indicators you are collecting for other funders.

  • Any concerns/recommendation that the team should be aware of.


Assessment instrument analysis

Assessment Instrument Analysis


Example child friendly facility instruments

Example: Child Friendly Facility Instruments


Example child friendly facility instruments1

Example: Child Friendly Facility Instruments

6 instruments, 63 questions, 16 question categories


Local cac instruments

Local CAC Instruments

Total of 40 forms collected

Modified cross tabulation


Results of analysis

Results of Analysis

  • Identification of key themes and concepts

  • Support for indicator phrasing

  • Development of outcome statements


Indicator categories

Indicator Categories

  • MDT

  • Child & Family Satisfaction

  • Justice

  • Community

  • CAC Organization


Considerations

Considerations

  • Developing an effective system for all CAC types

  • Developing a system that could be implemented with little added stress

  • Developing a system within the real world constraints:

    • Time, resources, evaluating children over a period of years, nature of subject matter


Considerations1

Considerations

  • Keep the system simple, clear, uncluttered

  • Make administration as least taxing on resources as possible

  • Focus on programmatic level outcomes and the role of the CAC in the process

  • Trust in the aspects of the model verified by the Literature Review


Outcome measurement system

Outcome Measurement System

OMS


Children s advocacy center outcomes

Children’s Advocacy Center Outcomes

Primary Goals of CACs in Texas

  • Outcome #1: The Children’s Advocacy Center facilitates healing for the child and the caregivers.

     Minimize re-victimization of child victims and their supportive family members throughout the investigative and prosecutorial stages of their cases and beyond

  • Outcome #2: The team approach results in more collaborative and efficient case investigations.

     Facilitate prosecution of perpetrators through effective fact finding and strong case development.


Oms development

OMS Development

Client Focused Indicators

  • Client’s needs are being met

  • Client is satisfied with the services received

  • Client has a positive CAC experience


Oms development1

OMS Development

MDT Focused Indicators

  • CAC supports the needs of MDT members

  • MDT team is committed to the mission

  • CAC facilitates effective collaboration between team members


Oms development2

OMS Development


Outcome measurement system for cacs in tennessee

Validity

Validity is the strength of our conclusions, inferences or propositions. More formally, Cook and Campbell (1979) define it as the "best available approximation to the truth or falsity of a given inference, proposition or conclusion


Oms development3

OMS Development

Two Client Surveys

  • Initial Survey

  • Follow-up Survey

    Two Multi-Disciplinary Team Surveys

  • General Survey

  • Case Specific Survey

    One Needs Assessment Form

  • An output based form used for milestone analysis


Sample survey questions

Sample Survey Questions

Initial Survey

General Survey


Individual client needs assessment

Individual Client Needs Assessment


Outcome measurement system for cacs in tennessee

OMS

  • Importance of “Scoring”

    • Need to turn our outcomes into data so we can evaluate performance

    • Scores allow for internal target setting

    • Gives us information we can communicate to the outside


Does it work

Does it work?


The pilot test

The Pilot Test

  • Pilot Test Centers

    • Governed by two constraints:

      • Easily accessible from the Austin area

      • Number of clients served considered to ensure optimal level of data collection

    • Diverse centers to represent the differences between the 64 centers in Texas

2006

2007

2008

2009

2010

2011


Reliability

Reliability

  • Cronbach’s alpha

    • A statistical method to ensure the reliability of the instrument.

    • The data gathered from pilot test was used to ensure the reliability of the instrument.


Pilot rollout

Pilot Rollout

  • OMS surveys in English and Spanish and score sheets on CD

  • Administrative guide

    • Detailed instructions

    • Phone Interview Script

  • Technical Support for implementation

  • Agreement


Fy10 march august 2010

FY10 (March - August 2010)

2006

2007

2008

2009

2010

2011


Outcomes for cacs in texas

Outcomes for CACs in Texas

  • The Children’s Advocacy Center facilitates healing for the child and caregivers.

  • 90% of caregivers felt that CAC’s facilitated healing for the child and themselves.

  • The multidisciplinary approach results in more collaborative and efficient case investigations.

  • 95% of MDT members felt their approach resulted in more collaborative and efficient case investigations.


Fy11 sept 2010 august 2011

FY11 (Sept. 2010 - August 2011)

2006

2007

2008

2009

2010

2011


Caregiver initial survey results

Caregiver Initial Survey Results


Caregiver follow up survey results

Caregiver Follow Up Survey Results


Mdt general survey results

MDT General Survey Results


Individual client needs assessment1

Individual Client Needs Assessment


Implementation

Implementation

  • November training

  • Sign agreement

  • Implementation – December 2011/January 2012

  • Reporting-

    • July 15, 2012

    • January 15, 2013

    • July 15, 2013

    • January 15, 2014


Questions1

Questions?


  • Login