Looking at data l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 58

Looking at Data PowerPoint PPT Presentation


  • 52 Views
  • Uploaded on
  • Presentation posted in: General

Looking at Data. Presented by The Early Childhood Outcomes Center. Revised January 2013. Using data for program improvement = EIA. E vidence I nference A ction. Evidence. Evidence refers to the numbers, such as “45% of children in category b” The numbers are not debatable. Inference.

Download Presentation

Looking at Data

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Looking at data l.jpg

Looking at Data

Presented by The Early Childhood Outcomes Center

Revised January 2013


Using data for program improvement eia l.jpg

Using data for program improvement = EIA

Evidence

Inference

Action

Early Childhood Outcomes Center


Evidence l.jpg

Evidence

Evidence refers to the numbers, such as

“45% of children in category b”

The numbers are not debatable

Early Childhood Outcomes Center


Inference l.jpg

Inference

How do you interpret the #s?

What can you conclude from the #s?

Does evidence mean good news? Bad news? News we can’t interpret?

To reach an inference, sometimes we analyze data in other ways (ask for more evidence)

Early Childhood Outcomes Center


Inference5 l.jpg

Inference

Inference is debatable -- even reasonable people can reach different conclusions from the same set of numbers

Stakeholder involvement can be helpful in making sense of the evidence

Early Childhood Outcomes Center


Action l.jpg

Action

Given the inference from the numbers, what should be done?

Recommendations or action steps

Action can be debatable – and often is

Another role for stakeholders

Early Childhood Outcomes Center


What can we infer l.jpg

What can we infer?

Poll results A:

Candidate I.M. Good 51%, Candidate R.U. Kidding 49% (+ or – 3%)

Poll results B:

Candidate I.M. Good 56%, Candidate R.U. Kidding 44% (+ or – 3%)

Early Childhood Outcomes Center


Program improvement where and how l.jpg

Program improvement: Where and how

At the state level – TA, policy

At the regional or local level – supervision, guidance

Classroom level -- spend more time on certain aspects of the curriculum

Child level -- modify intervention

Early Childhood Outcomes Center


Key points l.jpg

Key points

Evidence refers to the numbers and the numbers by themselves are meaningless

Inference is attached by those who read (interpret) the numbers

You have the opportunity and obligation to attach meaning

Early Childhood Outcomes Center


E i a jeopardy l.jpg

E – I – A Jeopardy

$100

$100

$100

$200

$200

$200

$300

$300

$300

Early Childhood Outcomes Center


Use of data activity l.jpg

Use of Data: Activity

Evidence-Inference-Action

Early Childhood Outcomes Center


Continuous program improvement l.jpg

Continuous Program Improvement

Reflect

Are we where we want to be?

Check

(Collect and analyze data)

Plan (vision)

Program characteristics

Child and family outcomes

Implement

Early Childhood Outcomes Center


Tweaking the system l.jpg

Tweaking the System

Is there a problem?

Reflect

Are we where we want to be?

Why is it happening?

Is it working?

What should be done?

Check

(Collect and analyze data)

Plan (vision)

Program characteristics

Child and family outcomes

Implement

Is it being done?

Early Childhood Outcomes Center


Continuous means l.jpg

Continuous means…

….the cycle never ends.

Early Childhood Outcomes Center


Outcome questions for program improvement e g l.jpg

Outcome questions for program improvement, e.g.

Who has good outcomes =

Do outcomes vary by

Region of the state?

Level of functioning at entry?

Services received?

Age at entry to service?

Type of services received?

Family outcomes?

Education level of parent?

Early Childhood Outcomes Center


Examples of process questions l.jpg

Examples of process questions

Are ALL services high quality?

Are ALL children and families receiving ALL the services they should in a timely manner?

Are ALL families being supported in being involved in their child’s program?

What are the barriers to high quality services?

Early Childhood Outcomes Center


Working assumptions l.jpg

Working Assumptions

There are some high quality services and programs being provided across the state.

There are some children who are not getting the highest quality services.

If we can find ways to improve those services/programs, these children will experience better outcomes.

Early Childhood Outcomes Center


Numbers as a tool l.jpg

Numbers as a tool

Heard on the street

“Why are we reducing children to a number?”

So why do we need numbers?

Early Childhood Outcomes Center


Slide19 l.jpg

Early Childhood Outcomes Center


Slide20 l.jpg

Early Childhood Outcomes Center


Slide21 l.jpg

Early Childhood Outcomes Center


Slide22 l.jpg

Early Childhood Outcomes Center


Examining cos data at one time point l.jpg

Examining COS data at one time point

One group - Frequency Distribution

Tables

Graphs

Comparing Groups

Graphs

Averages

Early Childhood Outcomes Center


Distribution of cos ratings in fall l.jpg

Distribution of COS Ratings in Fall

We are using

fake data

for illustration

Early Childhood Outcomes Center


Frequency on outcome 1 fall l.jpg

Frequency on Outcome 1 - Fall

Early Childhood Outcomes Center


Frequency on outcome 1 fall26 l.jpg

Frequency on Outcome 1 - Fall

Early Childhood Outcomes Center


Comparison of two classes fall l.jpg

Comparison of two classes - Fall

Early Childhood Outcomes Center


Frequency on outcome 1 fall28 l.jpg

Frequency on Outcome 1 - Fall

Early Childhood Outcomes Center


Frequency on outcome 1 class 1 l.jpg

Frequency on Outcome 1 – Class 1

Early Childhood Outcomes Center


Average scores on outcomes by class fall 2008 l.jpg

Average Scores on Outcomes by Class – Fall, 2008

Early Childhood Outcomes Center


Average scores on outcomes by class fall 200831 l.jpg

Average Scores on Outcomes by Class – Fall, 2008

Early Childhood Outcomes Center


Average scores on outcomes by class fall 200832 l.jpg

Average Scores on Outcomes by Class – Fall, 2008

Early Childhood Outcomes Center


Looking at change over time l.jpg

Looking at change over time

Extent of change on rating scale

The OSEP categories

Developmental trajectories

Maintaining

Changing

Early Childhood Outcomes Center


Extent of change on rating scale time 1 to time 2 l.jpg

Extent of change on rating scale: Time 1 to Time 2

Early Childhood Outcomes Center


Osep progress categories l.jpg

OSEP progress categories

Looking at information across time

Reducing the information to fewer categories to allow easier comparisons

Early Childhood Outcomes Center


Working with data l.jpg

Working with data

Different levels of analysis are required for different levels of questions

Aggregation will work for you – but loses detail about individual children.

50 assessment items on 20 children in 5 classes in Fall and Spring

50 x 20 x 5 x 2 = 10,000 pieces of information

Early Childhood Outcomes Center


Using assessment data at the classroom level l.jpg

Using assessment data at the classroom level

Looking at the data by child

At a single point in time

Over time

Looking at data for areas that cut across children

At a single point in time

Over time

Early Childhood Outcomes Center


Slide42 l.jpg

Example: Item Results for 5 Imaginary Children

A=Accomplished; E= Emerging; NY= Not yet

Early Childhood Outcomes Center


Slide43 l.jpg

Example: COS Outcome Ratings for Class 3c by Child

Early Childhood Outcomes Center


Slide44 l.jpg

Example of an Aggregated Report for Program:

Percentage of Children Scoring 5 or Higher on COS by Class

What do you see in these data?

Early Childhood Outcomes Center


Outcome questions for program improvement e g45 l.jpg

Outcome questions for program improvement, e.g.

Who has good outcomes =

Do outcomes vary by

Region of the state?

Level of functioning at entry?

Services received?

Age at entry to service?

Type of services received?

Family outcomes?

Education level of parent?

Early Childhood Outcomes Center


Looking at data by region l.jpg

Looking at Data by Region

Percentage of Children Who Changed

Developmental Trajectories

After One Year of Service

Possible inference?

Early Childhood Outcomes Center


Looking at data by age at entry l.jpg

Looking at Data by Age at Entry

Percentage of Children Who Changed

Developmental Trajectories

After One Year of Service

Possible inference?

Early Childhood Outcomes Center


Take home message l.jpg

Take Home Message

You will want to look at your data in lots of different ways

You will want to think about the possible inferences

You may need other information to decide among possible inferences

Act on what you have learned

Early Childhood Outcomes Center


Tweaking the system49 l.jpg

Tweaking the System

Is there a problem?

Reflect

Are we where we want to be?

Why is it happening?

Is it working?

What should be done?

Check

(Collect and analyze data)

Plan (vision)

Program characteristics

Child and family outcomes

Implement

Is it being done?

Early Childhood Outcomes Center


How will might these data be used l.jpg

How will/might these data be used?

Federal level

Overall funding decisions (accountability)

Resource allocation (e.g., what kind of TA to fund?)

Decisions about effectiveness of program in individual states

State level

Program effectiveness??

Program improvement??

Local level

Program improvement??

Early Childhood Outcomes Center


Need for good data l.jpg

Need for good data

Encompasses all three levels: federal, state, local

Depends on how well local programs are implementing procedures

Early Childhood Outcomes Center


Many steps to ensuring quality data l.jpg

Many steps to ensuring quality data

Early Childhood Outcomes Center


Take home message53 l.jpg

Take Home Message

If you conclude the data are not (yet) valid, they cannot be used for program effectiveness, program improvement or anything else.

Inference = Data not yet valid

Action = Continue to improve data collection and quality assurance

Early Childhood Outcomes Center


Data exploration l.jpg

Data Exploration

Examine the data to look for inconsistencies

If and when you find something strange, look for some other data you have that might help explain it. Is the variation caused by something other than bad data?

Early Childhood Outcomes Center


Obtaining good data l.jpg

Obtaining good data

Focus on addressing the threats to good data

Local providers do not understand the procedures

Local providers do not follow the procedures

And others…..

Identify and address the threats

Early Childhood Outcomes Center


How far along is our state l.jpg

How far along is our state?

Early Childhood Outcomes Center


Slide57 l.jpg

High quality services for children and families that will lead to good outcomes.

Keeping our eye on the prize:

Early Childhood Outcomes Center


Slide58 l.jpg

For more information….

www.the-eco-center.org

The Early Childhood Outcomes Center


  • Login