1 / 40

The Current Status of States' Early Childhood Outcome Measurement Systems

The Current Status of States' Early Childhood Outcome Measurement Systems. Kathy Hebbeler, SRI International Lynne Kahn, FPG Child Dev Inst October 17, 2009 DEC's 25th Annual International Conference on Young Children with Special Needs and Their Families. We’ve come a long way, BABY!.

aaron-robinson
Download Presentation

The Current Status of States' Early Childhood Outcome Measurement Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Current Status of States' Early Childhood Outcome Measurement Systems Kathy Hebbeler, SRI International Lynne Kahn, FPG Child Dev Inst October 17, 2009 DEC's 25th Annual International Conference on Young Children with Special Needs and Their Families

  2. We’ve come a long way, BABY!

  3. In the Beginning- Federal Accountability • 1994 Government Performance and Results Act (GPRA) • 2002 Performance Assessment Rating Tool (PART) • Both Part C and Part B Preschool categorized as “Results Not Demonstrated” due to lack of outcome data. • OMB recommended OSEP develop a strategy for collecting outcome data

  4. The Development of Outcome Statements • 2004-early 2005: ECO generates discussion and gathers input on child and family outcomes • Summer 2005: OSEP announces the child and family outcomes States must report on through their SPP/APRs Early Childhood Outcomes Center

  5. Three Child Outcomes Percent of children who demonstrate improved: • Positive social emotional skills (including positive social relationships) • Acquisition and use of knowledge and skills (including early language/ communication [and early literacy]) • Use of appropriate behaviors to meet their needs Early Childhood Outcomes Center

  6. Key Concepts to Communicate • Outcomes are functional– term has multiple definitions • Each outcome is integrated across domains • Challenge: Reporting language does not clearly define the outcomes Early Childhood Outcomes Center

  7. Reporting Child Progress • December 2005: States submit plans via SPP on how they will collect outcome data • September 2006: OSEP finalizes the child outcome reporting categories Early Childhood Outcomes Center

  8. Child Progress- the % of children who a. did not improve functioning b. improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers c. improved functioning to a level nearer to same-aged peers but did not reach it d. improved functioning to reach a level comparable to same-aged peers e. maintained functioning at a level comparable to same-aged peers Early Childhood Outcomes Center

  9. Key Concepts to Communicate • Progress categories require 2 data points for each child, • are based on growth trajectories, • compare a child to him or herself over time, and also compare each child to age expectations Early Childhood Outcomes Center

  10. Entry Exit Early Childhood Outcomes Center

  11. States Submit Data to OSEP in their Performance Reports • Feb 2007: “status at entry” data • Feb 2008: child progress data for children who exited 7/1/06 through 6/30/07 • Feb 2009: child progress data for children who exited 7/1/07 through 6/30/08 Early Childhood Outcomes Center

  12. State Approaches to Measuring Child Outcomes

  13. Number of Children Included in Feb ‘09 SPP/APR Data Part C (56) Range: 5-6452 <30 = 3 30-99= 10 100-499= 25 500-999= 6 1000+ = 12 Preschool (59) Range: 3-10157 <30 = 3 30-99 = 9 100-499= 14 500-999= 10 1000+ = 22

  14. Key Concepts to Communicate • Meaningfulness of the progress categories • Establishing expectations for how the percentages “should” look • Exploring relationships between state eligibility definitions and expectations for outcomes Early Childhood Outcomes Center

  15. Setting Targets for Improving Child Outcomes • 2008 and 2009: State input into summary statements to be the basis of target setting • Currently: summary statements have been out for public comment, and are in draft form in SPP Measurement tables • February 2010: Baseline data and target setting on summary statements

  16. Summary Statement Data • Required Summary Statement 1: Of those children who entered or exited the program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they exited the program. • Required Summary Statement 2: The percent of children who were functioning within age expectations in each Outcome by the time they exited the program.

  17. Key Concepts to Communicate • The meaning of the summary statements • Ensuring that data is clean before making program changes based on it • Choosing/ developing strategies that will improve child outcomes Early Childhood Outcomes Center

  18. So where are the states now?

  19. States and Outcomes Measurement • Many states have embraced the ideas of measuring outcomes • Want information for own purposes • Program improvement • The power of GOOD outcomes data is widely recognized

  20. States and Outcomes Measurement • States have invested resources and time in building an outcomes measurement system. • Involving stakeholders • Identifying an approach • Training providers in approach • Building a data system And collecting data!

  21. States have made incredible progress • States started at various times • Have varying amounts of data • Examining their data (and procedures) • The Quality Question?? This is VERY DIFFICULT work

  22. Some of the Challenges • Need for good assessment tools • Need for more providers skilled in assessment • Need strong grounding in typical child development • Adequate training statewide in data collection procedures • Understanding and commitment • Resources for new data systems • Capability to analyze, report and interpret data • Generating reports for local programs • Resources in general

  23. Next steps: Putting it all together

  24. Making Good Use of Data: Some Necessary Ingredients Set of good questions Data set with the required elements Analytic capability to analyze the data to address the question Commitment to use the information for program improvement Early Childhood Outcomes Center 28

  25. Examining and Tweaking the Service System Is there a problem? Reflect Are we where we want to be? Why is it happening? Is it working? What should be done? Check (Collect and analyze data) Plan (vision) Program characteristics Child and family outcomes Implement Is it being done?

  26. Information Infrastructure: Data Needed for Program Improvement WHO SERVICES OUTCOMES QUALITY COST Early Childhood Outcomes Center 30

  27. Where is your state (or program) now? WHO SERVICES OUTCOMES What do you have? How much is linked? Early Childhood Outcomes Center 31

  28. WHO WHO WHO WHO SERVICES SERVICES SERVICES SERVICES OUTCOMES OUTCOMES OUTCOMES OUTCOMES QUALITY QUALITY QUALITY QUALITY COST COST COST COST Grade 2 Tracking outcomes over time Grade 1 Kindergarten Preschool

  29. An outcomes measurement system is more than hardware, software, and data

  30. Framework Development • ECO is working with 7 partner states over the next two years to develop a framework for outcomes measurement. • Framework provides the basis for a self assessment to allow states to track their own progress in building an outcomes measurement system.

  31. Vision for What Is to Be Learned Data Collection and Transmission Analysis Information Sharing Action Areas of DRAFT Outcomes Measurement System Framework Evaluation

  32. Data Collection and Transmission • State has a method capable of accurately capturing outcomes information • State has effective procedures for professional development around data collection. • State has effective procedures for moving data from the local to the state level • Etc.

  33. State Self Assessment • Is each area of the outcomes measurement system implemented? • Is the area implemented in a way that reflects high quality?

  34. We’ve come a long way, BABY! 39

  35. …and we still have a long way to go… ...but won’t it be great when we get there?

  36. For more information (and for a copy of this presentation) www.the-eco-center.org

More Related