How to Explain the Numbers:
This presentation is the property of its rightful owner.
Sponsored Links
1 / 45

Batya Elbaum, NCSEAM Pam Roush, West Virginia Part C OSEP National Early Childhood Meeting PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on
  • Presentation posted in: General

How to Explain the Numbers: Helping Staff, Parents, and Other Stakeholders Understand the Results of the NCSEAM Surveys for Part C and 619. Batya Elbaum, NCSEAM Pam Roush, West Virginia Part C OSEP National Early Childhood Meeting Arlington, VA, December 2007. Purpose of the session.

Download Presentation

Batya Elbaum, NCSEAM Pam Roush, West Virginia Part C OSEP National Early Childhood Meeting

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Batya elbaum ncseam pam roush west virginia part c osep national early childhood meeting

How to Explain the Numbers:Helping Staff, Parents, and Other Stakeholders Understand the Results of the NCSEAM Surveys for Part C and 619

Batya Elbaum, NCSEAM

Pam Roush, West Virginia Part C

OSEP National Early Childhood Meeting

Arlington, VA, December 2007


Purpose of the session

Purpose of the session

  • To provide participants with strategies for explaining the measures and percentages that come from the NCSEAM rating scales addressing Indicators C4 and B8.


Part c indicator 4

Part C Indicator #4

“Percent of families participating in Part C who report that early intervention services have helped the family a) know their rights, b) effectively communicate their children’s needs, and c) help their children develop and learn.”


Part b indicator 8

Part B Indicator #8

“Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities.”


Wv part c indicator 4

WV Part C Indicator 4

  • 4A – Know rights

    • 200676.5%

    • 200773.8%

  • 4B – Communicate

    • 200672.8%

    • 200771.0%

  • 4C – Help child

    • 200686.3%

    • 200778.7%


Change in mean measure by region

Change in mean measure by region


Percent on 4a by region in 2006 and 2007

Percent on 4A by Region in 2006 and 2007


Why use a measurement analysis

Why use a measurement analysis?


Why use a measurement analysis1

Why use a measurement analysis?

  • Use of a measurement framework ensures that measures will mean the same thing, regardless of how many items, or which specific items, are administered.


Why use a measurement analysis2

Why use a measurement analysis?

  • We can’t assume that all survey items are equally agreeable.

  • A measurement analysis gives us a measure of each item’s overall agreeability.


Order of impact on family items from least to most agreeable

Order of Impact on Family items from least to most agreeable


Why use a measurement analysis3

Why use a measurement analysis?

  • Use of a measurement framework allows us to test whether all the items are measuring the same thing.


Measuring the 3 subindicators

Measuring the 3 subindicators

  • NCSEAM’s measurement analysis of pilot data from thousands of families showed that items that are related to the three OSEP outcomes all fit into a single scale of family outcomes.


Measuring the 3 subindicators1

Measuring the 3 subindicators

  • Families appear to achieve these outcomes in a very consistent order.

    • Families who report that EI helped them know their rights also report that EI also helped them help their child develop and learn.

    • Families who report that EI helped them effectively communicate their children’s needs also report that EI also helped them know their rights and help their child develop and learn.


Location of key items related to indicator c4

Location of key items related to Indicator C4

Indicator 4b: Effectively communicate their children’s needs.

IFS Item: “Communicate more effectively with the people who work with my child and family.” [556]

556

Indicator 4a: Know their rights.

IFS Item: “Know about my child's and family's rights concerning Early Intervention services.” [539]

539

Indicator 4c: Help their children develop and learn.

IFS items: “Understand my child's special needs.” [516] “Do things with and for my child that are good for my child's development.” [498]

516


Why use a measurement analysis4

Why use a measurement analysis?

  • We can’t assume that the response choices (very strongly disagree, strongly disagree, disagree, agree, strongly agree, very strongly agree) indicated by equidistant circles or numbers on a page are really equidistant.


Spacing of response choices

Spacing of response choices


Spacing of response choices1

Spacing of response choices


The logic of distributions

The logic of distributions


Example distribution of height

Example: Distribution of height


Distribution of measures of reading achievement 1

Distribution of Measures of Reading Achievement - 1


Distribution of measures of reading achievement 2

Distribution of Measures of Reading Achievement - 2


West virginia results 2006

West Virginia results 2006


West virginia results 2007

West Virginia results 2007


Wv without extreme cases 2006

WV without extreme cases 2006


Wv without extreme cases 2007

WV without extreme cases 2007


Instrumentation and approach to data analysis make a difference

Instrumentation and approach to data analysis make a difference


Choice of instruments part b

Choice of instruments: Part B

  • NCSEAM K-12 and 619 9

  • NCSEAM K-1221

  • Customized NCSEAM survey11

  • State-developed or adapted18

  • ECO Family Outcomes Survey 1


States reported baseline data on indicator 8b

States’ reported baseline data onIndicator 8B


Choice of instruments part c

Choice of instruments: Part C

  • NCSEAM Survey 25

  • ECO Family Outcomes Survey 18

  • State-developed or adapted10

  • Combination 1


Analysis of states baseline data indicator c4

Analysis of states’ baseline data:Indicator C4


Analysis of states baseline data indicator c41

Analysis of states’ baseline data:Indicator C4


Batya elbaum ncseam pam roush west virginia part c osep national early childhood meeting

APR Family Outcome Results – Part C States Using NCSEAM Survey and RASCH analysisAPRs submitted February 2007


Reference to a standard

Reference to a standard


Percent of measures above the adopted standards

Percent of measures above the adopted standards

standards


Batya elbaum ncseam pam roush west virginia part c osep national early childhood meeting

What is the relationship between % at or above the NCSEAM standard and % agreement on the “threshold item?”


Sample descriptive interpretation related to indicator 4a

Sample descriptive interpretation related to Indicator 4A

  • Approximately 90-95% of families agreed, with approximately two-thirds of families expressing strong or very strong agreement, that early intervention helped them:

  • Understand their child’s special needs.

  • Do things with and for their child that are good for their child’s development.

  • Feel that they can handle the challenges of parenting a child with special needs.


Part c example wv data

Part C Example: WV data


Q44 know about my child s and family s rights

Q44 - know about my child's and family's rights


Q42 communicate more effectively

Q42 - communicate more effectively


Q46 understand my child s special needs

Q46 - understand my child's special needs


Part b example

Part B Example


The ncseam standard is a stringent standard

The NCSEAM standard is a stringent standard

  • Cut score for B at 600

  • Cut scores for C at 539, 516, 556

  • The standard is set so as to ensure that approximately 95% or more of the responses will be in one of the agree categories on the item designated as the “threshold item”

  • Simple agreement with the “threshold item” is not enough


How precise are the percentages that states are reporting

How precise are the percentages that states are reporting?


Error in estimation

Error in estimation

  • Whenever data are based on a sample (whether owing to sampling or to a less than 100% return rate), there is some amount of error in generalizing to the population.

  • “Error,” or imprecision, can be represented by a confidence interval.

  • As sample size decreases, error of estimate - and its representation as a confidence interval - increases.

  • The decrease is not linear!


  • Login