slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Results Not Demonstrated PowerPoint Presentation
Download Presentation
Results Not Demonstrated

Loading in 2 Seconds...

play fullscreen
1 / 51

Results Not Demonstrated - PowerPoint PPT Presentation

  • Uploaded on

AKA. Results Not Demonstrated. National Picture. Feb 2008 SPP and APR review Child Outcomes - Indicators C3 & B7 Family Outcomes - Indicator C4. Highlights from . . . State Approaches to Measuring Child Outcomes. All approaches have challenges . . . All approaches have challenges . . .

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Results Not Demonstrated' - keena

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Feb 2008 SPP and APR review

Child Outcomes - Indicators C3 & B7

Family Outcomes - Indicator C4

Highlights from . . .

number of children included in feb 08 spp apr data
Number of Children Included in Feb ‘08 SPP/APR Data

Part C (52)

Range: 1-5944

<30 = 19

30-99= 15

100-499= 11

500-999= 4

1000+ = 3

Preschool (53)

Range: 1-4249

<30 = 7

30-99 = 13

100-499= 14

500-999= 8

1000+ = 11

assessment tool trends
Assessment Tool Trends
  • Part C
    • HELP
    • BDI-2
    • AEPS
    • Carolina
    • ELAP
  • Preschool
    • Creative Curr
    • BDI-2
    • Brigance
    • AEPS
    • High Scope
    • WSS
populations included
Populations Included
  • Part C
    • 40 States statewide
    • 6 phasing in
    • 6 sampling
  • Preschool
    • 23 States statewide
    • 14 phasing in
    • 6 sampling
    • 5 included children in other EC programs
definitions of near entry
Definitions of “near entry”
  • Part C
    • Variety of starting points- initial IFSP the most common reference point
    • Earliest: as part of intake, or with eligibility determination
    • Latest: w/in 6 months of enrollment
  • Preschool
    • Wide variation
    • From 30 days to 4 months from entry
    • States using ongoing assessments used “fall” data point for entry

Definitions of “near exit”

  • Preschool
    • About two thirds provided definition
    • Ranged from 30 days to 6 months
    • Included spring assessment points and “end of the school year”
  • Part C
    • About half defined near exit
    • Typically within 30 to 60 days of exit

Criteria for same aged peers

  • COSF- 6 or 7 on the scale, by definition
  • Single tool statewide- variation in criteria across states; e.g. BDI 1.3 SD from mean for 2 states and 1.5 SD from mean for another state
  • Publishers analysis of data intended to correspond to COSF summary ratings
caution interpreting data
Only represents children who have entered and exited since outcome system put in place in states

In a typical state, data may represent children who participated in the program for 6 to 12 months

The quality of data collection usually increases over time as guidance gets clearer and practice improves the implementation

Caution – Interpreting Data
scanning your data for unusual patterns
First, focus on progress categories “a” and “e”

Should reflect the characteristics of the children served in the state (e.g. eligibility definition in Part C)

Should be fairly stable over time (when data are high quality and representative)

Scanning Your Data for Unusual Patterns
checking category a
Percents too high?

Should represent children with very significant delays or degenerative conditions (any improvement in functioning puts a child into “b”)

Why it may be too high

Decision rules based on different interpretation of “no progress”

Tools without enough discrimination to show small amounts of progress

Checking Category “a”
checking category e
Percents too high or low?

Should represent children functioning at age expectations at entry and exit in each outcome area

Do your patterns make sense for each outcome based on the children served in the state?

Why it may be too high or low

Decision rules based on different interpretation of “age expectations”

Checking Category “e”
the validity of your data is questionable if
The n is too small

The overall pattern in the data looks ‘strange’:

Compared to what you expect

Compared to other data

Compared to similar states

The data is not representative:

Of all areas of the state

Of all kinds of families

Of all kinds of children

The validity of your data is questionable if…
improvement activities
Almost all states (Part C and 619) are conducting training and professional development:

Assessment strategies

Data collection procedures

Data analysis and use

(and a little bit of) Practices to improve child outcomes

Improvement Activities
improvement activities1
Improving infrastructure for providing TA and support

Conducting Evaluation

Reviewing data for accuracy and quality

Reviewing and revising processes

Identifying successes and challenges in the implementation of the outcomes system

Improving data collection and reporting

Improvement Activities
part c tools for family outcomes
Part C Tools for Family Outcomes

28 (52%) reported that they provided translations and/or translation services to assist families

variations in target populations
Variations in Target Populations

5 states did not report on criteria for the population

trends in improvement activities
Clarifying and developing policies and procedures (40 states)

clarification of policies regarding family rights and family centered services

modifications to procedures related to the implementation of family surveys

Providing training and professional support (28 states)

to providers and service coordinators regarding family rights and procedural safeguards

effective practices relating to family centered services

understanding the procedures for implementing the measurement of family outcomes

and understanding and using the family survey data for program improvement

Trends in Improvement Activities
trends in improvement activities1
Conducting evaluation (27 states)

evaluating the processes used to implement family outcome measurement in FFY 2005 (including distribution methods, follow-up, methods of analysis

family focus groups or random interviews with families to validate outcomes data

Improving data collection and reporting (25 states)

developing strategies for improving the family survey response rates and representativeness of the data

Trends in Improvement Activities
preparing for the future
Setting Targets

Improving Data Quality

Training & TA Capacity

Written policies and procedures

Analysis and interpretation of the data

Quality Assurance / Monitoring

Improvement Planning – for better data collection and for improved outcomes

Preparing for the Future




Family Outcomes




Teacher/Provider Skills

Family Outcomes




Program/Classroom Quality

Teacher/Provider Skills

Family Outcomes





Program/Classroom Quality

Teacher/Provider Skills

Family Outcomes



themes of agenda sessions
Quality Assurance

Quality assessment data

Reliable use of tools

Quality of analysis and reporting

Training and TA (to address quality)


Part C and 619 Preschool

Across Early Care and Education

Themes of Agenda Sessions
themes of agenda sessions1
Challenges of particular approaches

Decision rules for “age expectations” and progress category assignment for states using one tool statewide

Consistent interpretation and use of the COSF

Outcomes from the local and family perspectives

Themes of Agenda Sessions
themes of agenda sessions2
Building outcomes into monitoring and accountability systems

Sampling issues and strategies

Family outcomes

Using data for improving family services and supports

Return rates and representative data

Themes of Agenda Sessions