Development of Child Outcomes and Indicators

1. Development of Child Outcomes and Indicators NH Cornerstone Project Preschool Subcommittee Meeting February 15, 2005 Martha Diefendorf Early Childhood Outcomes Center

2. Cornerstone Project focuses on developing an outcomes measurement system BUT • Outcomes data alone are of very limited usefulness for program improvement • Outcomes need to be part of an evaluation system that also looks at implementation issues (e.g., program quality, services received, procedural issues, satisfaction)

3. Accountability Alignment(horizontal & vertical) • DESIRED OUTCOMES • OR STANDARDS • Dept. of Ed • Health & Human Services • Child Care Bureau • Head Start & Early HS • birth – 2 years • preschool • K – 3rd grade • school age PROGRAM QUALITY Features Services Curriculum Processes Practices ASSESSMENT SYSTEM Measurements School readiness

4. Terminology Child outcomes are also called: = Early learning guidelines = Performance standards = Readiness standards = Desired results = Widely held expectations = Building blocks

5. WHY does NH want data on outcomes? • For APR reporting so that OSEP can monitor implementation of IDEA across the nation and provide data to OMB and Congress about program effectiveness to justify funding • For state level decision-making regarding program improvement and to provide effectiveness data to the legislature and other potential funding sources • For local program decision-making and program improvement

6. Steps in Development of an Outcomes System • Identify overall purpose for the system (including priorities) • Identify outcome areas for children and families • Develop outcome statements • Formulate the evidence statements • Identify measurement approaches (e.g., select instruments) • Identify approaches for analyzing the data

7. Steps (continued) • Build a state infrastructure for collecting and reporting • Collect data (and submit to OSEP) • Analyze the data • Report the data • Refine the approach based on what was learned

8. DEFINITION: OUTCOME An outcome is: • a benefit experienced as a result of services received; something that happens because services are provided An outcome is NOT: • the receipt of services • satisfaction with services

9. About outcome statements • Outcomes in an accountability system are global statements of what we are trying to do for children and families • Not the same as outcomes on an IFSP which are very specific outcomes for an individual child or family

10. Criteria for outcome statements • Be consistent with IDEA • Reflect what the EI/ECSE programs are trying to do, their overall vision • Reflect what is known about development and learning • Be compatible with best practice • Be readily understood • Have potential to influence policy and practice in a positive way

11. How should we think about childoutcomes? • Domains (language, cognitive, adaptive, etc.) • Functional skills (feeding, dressing, mobility, etc.) • Underlying functional capacities (self-regulation, engagement, knowledge acquisition skills, etc.) • Pre-academic skills (early reading, math, science, etc.)

12. DIRECTION FROM OSEP ON CHILD OUTCOMES • Decision: Functional outcomes • Decision: Global, not specific • Decision: Status wording, not progress wording • Decision: OSEP will not require that all children be assessed with the same instrument

13. ECO’s proposed preface for child outcome statements The ultimate goal of early intervention and early childhood special education programs is: to enable young children to be active and successful participants during the early childhood years and in the future in a variety of settings– in their homes with their families, in child care, preschool or school programs, and in the community.

14. Child Outcomes(most current version) • Children have positive social relationships • Children acquire and use knowledge and skills • Children take appropriate action to meet their needs

15. Considerations • Should child outcomes apply to children for the entire birth through 5 age span? • Should child outcomes have a relationship to family outcomes?

16. Family Outcomes(most current version) • Families understand their children’s strengths, abilities and special needs. • Families know their rights and effectively communicate their children’s needs. • Families help their children develop and learn. • Families have adequate social support. • Families are able to access services and activities that are available toall families in their community.

17. Linkages Child outcome “X” encompasses a child’s ability to: [bulleted list of sub-outcomes] Families can better support a child in Outcome “X” when: [bulleted list of family outcomes/sub-outcomes] EI & ECSE programs can assist families and promote positive outcomes for children in Outcome “X” by: [bulleted list of program features/supports/services]

18. DEFINITION:SUB-OUTCOMES (aka “indicators”) • Specific behaviors that demonstrate child growth and development and describe what children should know and be able to do to demonstrate progress toward attainment of the intended outcome • How best to explain or elaborate on the outcomes

19. DEFINITION: EVIDENCE STATEMENT • A statement the incorporates a statistic and provides evidence as to whether or not an outcome has been achieved. Note: We use evidence statements to convince others of the benefits of programs or services to garner support and justify their continuation.

20. Categories of Evidence Statements • Status statement (achievement of a common benchmark): % of families who have achieved the outcome at the end of participation in Part C and/or Part B preschool program • Progress statement (change relative to earlier status): % of families who have made progress from the beginning to the end of their participation in Part C and/or 619 • Perception of progress: % of families who are satisfied with the progress their family has made over a specified period of time. • Achievement of individual family goals on IFSP

21. DEFINITION: MEASUREMENT • Tools and techniques to quantify the child’s status on aspects of functioning; for example: • Norm-based assessments • Curriculum-based assessments • Rating scales

22. Subcommittee’s Task[large group discussion] • Agree on set of preschool outcomes • Consider the 3 ECO outcome statements • Review the new ECO “examples” to get a better understanding of what is meant by each outcome statement • Develop outcome statements for NH or endorse ECO’s

23. Subcommittee’s Task[small group discussions] • Write “sub-outcomes” for NH for each preschool outcome statement to fully describe the outcome • Brainstorm as a group without looking at reference materials • Make an exhaustive list that covers the broad outcome statement • Refine wording; OK to look at examples

24. Subcommittee’s Task • Conduct a crosswalk of the “sub-outcomes” for each NH preschool outcome statement to: • NH Early Learning Standards • Other programs’ standards (e.g., Head Start) ?? • National initiatives ??

25. Subcommittee’s Task 4. Winnow down the list of sub-outcomes • Rate one or two sub-outcomes using the rating tool [individually] • Review proposed rating tool and give feedback about its utility [large group] • Make a plan to finalize list of sub-outcomes 5. Give final product to Task Force who will: • Look at relationship to infant/toddler and family outcomes • Coordinate work of all 3 subcommittees • Send back to subcommittees for review/critique