Developmental disabilities program independent evaluation ddpie project
1 / 30

Developmental Disabilities Program Independent Evaluation (DDPIE) Project - PowerPoint PPT Presentation

  • Uploaded on

Developmental Disabilities Program Independent Evaluation (DDPIE) Project. Jennifer Johnson Lynn Elinson Cynthia Thomas AUCD Annual Meeting October 31, 2006. Purpose of the Independent Evaluation. Demonstrate impact of DD Network programs on: Individuals Families Service providers

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Developmental Disabilities Program Independent Evaluation (DDPIE) Project' - iram

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Developmental disabilities program independent evaluation ddpie project

Developmental Disabilities Program Independent Evaluation (DDPIE) Project

Jennifer Johnson

Lynn Elinson

Cynthia Thomas

AUCD Annual Meeting

October 31, 2006

Purpose of the independent evaluation
Purpose of the Independent Evaluation (DDPIE) Project

  • Demonstrate impact of DD Network programs on:

    • Individuals

    • Families

    • Service providers

    • State systems

  • Provide feedback to ADD to help improve the effectiveness of its programs and policies

  • Promote positive achievements of DD Network programs

  • Promote accountability to the public

Ddpie project
DDPIE Project (DDPIE) Project

  • Independent evaluation

  • 2 phases

  • Phase 1 – development and testing of tools

  • Phase 2 – full-scale evaluation

  • Westat – contracted by ADD to implement Phase 1

Evaluation tools
Evaluation Tools (DDPIE) Project

  • Measurement matrices

    - standards

    - indicators (structures, processes, outputs,


    - performance levels

  • Data collection instruments

Evaluation (DDPIE) Project



What do we hope to achieve?

What do we observe

(measurement of indicators)?


Are there differences/discrepancies?

What is the nature and extent of the differences?

What action needs to be taken?

Framework of indicators rfp
Framework of indicators (RFP) (DDPIE) Project

To organize and guide the development of the performance standards and related measurement matrices, the following framework of indicators of program impact should be used:

- Structural indicators – adequate and appropriate settings and infrastructures, including staffing, facilities and equipment, financial resources, information systems, governance and administrative structures, etc.

- Process indicators – activities, procedures, methods, and intervention supporting practices

- Output indicators –results of the DD Network’s policies, procedures, and services

- Outcome indicators – intermediate results

Open systems model
Open Systems Model (DDPIE) Project









Measurement matrices
Measurement Matrices (DDPIE) Project

  • Tools in the evaluation that will organize indicators, standards, and performance levels for each key function within each DD Network program and collaboration

  • Developed and pilot-tested in Phase I

  • Used in Phase II

Basic evaluation approach
Basic Evaluation Approach (DDPIE) Project

  • Performance-based approach – interested in outcomes

  • Development of standards and indicators

  • Development of measurement matrices that contain standards, indicators, and performance levels

  • Collection of data

  • Measurement of indicators to determine level at which standards are being met

  • Determination of overall performance at the national level

Key assumptions
Key Assumptions (DDPIE) Project

  • State programs vary on their level of performance across the standards.

  • Consistently high performance across the standards is related to better outcomes.

  • Consistently low performance across the standards is related to poor outcomes.

Validation (DDPIE) Project

  • Advisory Panel

  • Working Groups

  • Validation Panels

  • Pilot Study

  • Further analysis

Role of advisory panel
Role of Advisory Panel (DDPIE) Project

To provide balance, impartiality, and expertise.

To provide advice on:

  • DDPIE process

  • Standards, indicators, performance levels, and data collection

  • Measurement matrices

  • Pilot study

  • Synthesis of findings and recommendations

Composition of advisory panel
Composition of Advisory Panel (DDPIE) Project

Individuals with expertise on:

  • DD population

  • Policies and services for the DD population

  • Evaluation research

  • DD Network programs

  • Other evaluations

Advisory panel
Advisory Panel (DDPIE) Project

  • Self-advocates

  • Family members

  • Representatives from 3 programs – Richard Carroll from Arizona UCEDD

  • Child/disability advocates

  • Evaluation expert

  • Federal representative (for PAIMI evaluation)

Working group members criteria for selection
Working Group Members: Criteria for Selection (DDPIE) Project

  • Have broad overview of all aspects of UCEDD (Director, Associate Director)

  • Rural/urban state

  • Geographic distribution

  • Placement of UCEDD in University

Validation panels
Validation Panels (DDPIE) Project

  • Role: To endorse the contents of the measurement matrices.

  • Composition:

    - Stakeholders (consumers, advocates)

    - DD Network program staff

    - DD Council members

    - ADD staff

    - Evaluation experts

Pre test and pilot study
Pre-test and Pilot Study (DDPIE) Project

  • States randomly selected

  • Pre-test – 1 state

  • Pilot Study – 4 states

Progress to date
Progress to Date (DDPIE) Project

  • Reviewed background materials.

  • Conducted preliminary/background interviews.

  • Established and met with Advisory Panel twice.

  • Established and met with Working Groups.

  • Identified key functions of each DD Network program.

  • Discussed structures, processes, outputs and outcomes of each key function.

  • Conceptualized measurement matrices.

  • Developed early drafts of standards and indicators for each key function.

Project tasks to do
Project Tasks to Do (DDPIE) Project

  • Complete draft matrices.

  • Share process with state programs.

  • Validate matrices (Validation Panels)

  • Develop data collection instruments.

  • Obtain OMB clearance (ADD).

  • Conduct pilot study.

  • Analyze and synthesize data.

  • Write report and recommendations.

Key functions
Key Functions (DDPIE) Project

  • A collection of activities that are intended to achieve particular results

  • Examples:

    - P&A – individual advocacy, outreach/public


    - DD Councils – systemic advocacy, development

    of community capacity

    - UCEDDs – training, community

    service/technical assistance, research,

    dissemination (from DD Act)

Ucedd working group members
UCEDD Working Group Members (DDPIE) Project

*Collaboration Working Group

Working group meetings
Working Group Meetings (DDPIE) Project

  • Orientation by telephone – March, 2006

  • Telephone meetings (full group) in spring

  • In-person meeting in spring, 2006 –coinciding with the national meeting

  • Telephone focus groups (teams) summer, 2006

  • In-person meeting in fall, 2006

Ucedd key functions
UCEDD Key Functions (DDPIE) Project

  • Provision of Interdisciplinary Pre-Service Preparation and Continuing Education of Students and Fellows

  • Provision of Community Services

  • Conduct of Basic or Applied Research

  • Dissemination

  • Governance and Management?

  • Priority Setting?

Members of ucedd working subgroups
Members of UCEDD Working Subgroups (DDPIE) Project

Pre-Service Preparation

  • Fred Palmer

  • Lucille Zeph

    Community Services

  • Tawara Goode

  • Fred Orelove

  • Basic or Applied Research

  • David Mank

  • Gloria Krahn

  • Dissemination

  • Carl Calkins

  • Gloria Krahn

Working group teams
Working Group Teams: (DDPIE) Project

  • Described goals for each key function

  • Explained the main activities

  • Identified outcomes

  • Discussed variability across UCEDDs

  • Provided Examples

  • Helped formulate SPOO tables, standards

Interdisciplinary pre service preparation
Interdisciplinary Pre-Service Preparation... (DDPIE) Project


Develop a cadre of individuals with knowledge, skills, attitudes and values to increase the capacity of states to provide services and supports in a culturally and linguistically competent manner for people with developmental disabilities and their families.

Interdisciplinary pre service preparation cont d
Interdisciplinary Pre-Service Preparation...(cont’d) (DDPIE) Project

Function includes:

  • Developing and teaching courses in the core curriculum

  • Developing and/or teaching disability content for courses in other departments

  • Developing and teaching classes that offer continuing education credits

    Outcomes Include:

  • “Graduates” from the preservice programs who demonstrate knowledge, skills, and attitudes and values consistent with …. the principles of the DD Act.

  • Students who complete courses with disability content and demonstrate attitudes and values ….

  • Trainees who complete continuing education classes …..

Following steps
Following Steps: (DDPIE) Project

  • Development of written descriptions of each key function

  • Development of draft indicators

  • Full working group reviews descriptions, standard, indicators