Measuring Early Childhood Outcomes Idaho Early Childhood Special Education Fall 2009

1. Measuring Early Childhood Outcomes Idaho Early Childhood Special Education Fall 2009 Idaho’s Early Childhood Outcomes System (Idaho ECOS) Updated

2. Agenda Measuring Early Childhood Outcomes: Why, What, When, Who & How? Review Child Outcome Summary Form, (COSF-R) Tools & Resources Reporting the data on the Idaho early Childhood Outcome Collection System Practice Exercises

3. Training Objectives Participants will : Learn how to accurately identify, record, and report a child’s outcome data using the Idaho Child Outcome Summary Form-Revised (Idaho COSF-R) Increase valid and reliable data reporting on the ECO’s – timely and accurate

4. Goal of Early Childhood Special Education “…To enable young children to be active and successful participants and learners during the early childhood years in a variety of settings – in their homes with their families, in child care, in preschool or school programs, and in the community.” Early Childhood Outcome Center

5. Why Collect Outcome Data? • Federal government is the driving force behind the move to collect outcome data, however it is not the only reason… • Data on outcomes are also valuable • To document program effectiveness • To improve district programs • To ultimately better serve children

6. WHY collecting outcome data makes sense for Idaho Improve current assessment practices and program development Promote progress toward age-expected skills and behavior throughout the child’s day Demonstrate value of Early Childhood Special Ed (Part B) programs Support DEC/NAEYC best practice guidelines Meet Federal Reporting Requirements -Office of Special Education Programs (OSEP)

7. OSEP’s Solution Document the number of children for whom the program has changed their developmental trajectories

8. Thinking about how children are doing with regard to each outcome. Movement away from age-expected Age-expected skills & behavior Movement toward age-expected

9. OSEP Indicators – categories of progress Percentage of children who: a. Did not improve functioning b. Improved functioning, but not sufficient to move nearer to functioning comparable to same age peers c. Improved functioning to a level nearer to same-age peers but did not reach it d. Improved functioning to reach a level comparable to same aged-peers e. Maintained functioning at a level comparable to same-aged peers

10. OSEP Indicators & Measurement Categories maintained or reached typical 5 year old level made progress but.. did not make progress 2 year old level Enry Exit

11. How do we document change in developmental trajectories while in services ? Obtain and compare a child’s assessment data at program entry and exit.

12. What happens with this data and why is accuracy and timeliness is so important? • Individual student data is aggregated at state level • States are required to report outcome data to Feds according to % of children exiting ECSE services – • State must set targets and report progress toward meeting targets: • % who entered program below age expectations in each outcome and substantially increased their rate of growth • % who were functioning within age expectations in each outcome by time they exited

13. So, who do we collect outcome data on and when… All children enrolled in ECSE must have EC Outcome Entry data collected within 45 calendar days of initial consent for placement/IEP start date (if they will be in services for at least 6 months). For children transitioning from Part C – Part C outcome exit data may be used for Part B entry data, if an approved anchor tool was administered.

14. Who and when …. EC Outcome Exit data will be collected within 30 days of the end of the school year (including ESY) for all children who: Have been in Idaho receiving services for at least 6 months, and Are 5 years old on or before Sept 1st of the next school year (cut-off date for kindergarten entry), no matter if they are staying in a preschool setting or going into kindergarten.

15. EC Outcome Exit data must also becollected… Within 30 days of determining a child no longer meets eligibility criteria (prior to age 5 and been in program at least 6 months) 30 days prior to a child moving out of state If child leaves before 5 without notice and prior to administering the anchor assessment, complete the COSF-R with the information available.

16. Who is responsible to collect outcome data? Preschool teachers/SLP’s should take a lead role in gathering the necessary data/information from multiple people and measures through a team process Related service providers also provide assessment data, observation, etc.

17. What are the OSEP’s Early Childhood Outcomes?

18. OSEP’s 3 Child Outcomes Children have positive social-emotional skills (including social relationships) Children acquire and use knowledge and skills (including communication and early literacy) Children use appropriate behaviors to meet their needs

19. Outcome 1 1. Children have positive social-emotional skills (including positive social relationships) Relating with adults, and other children For older children: following rules related to groups or interacting with others Attachment/separation/autonomy Responding to others, expressing emotions, turn-taking, using appropriate social greetings, play skills or having friendships with same-age peers, etc.

20. Outcome 2 2.Children acquire and use knowledge and skills (including communication and early literacy) Using symbols and language Thinking, reasoning, remembering, problem-solving Understanding physical and social worlds Early concepts – symbols, pictures, numbers, classification, spatial relationships Imitation, object permanence Expressing thoughts and ideas, listening to and enjoying stories and books, or learning new ways to do things, etc.

21. Outcome 3 3. Children use appropriate behaviors to meet their needs Taking care of basic needs Getting from place to place Using tools like fork, toothbrush, crayon In older children, contributing to their own health and safety Self help skills (dressing, feeding, toileting, grooming, household responsibility) Acting on world to get what he needs

22. The Child Outcome Areas… Represent critical functional outcomes necessary and meaningful in every day activities and routines Not domains based – not trying to separate child development into discrete areas (communication, gross motor, etc.) and single behaviors Emphasize how the child is able to integrate series of behaviors or skills across domains to achieve outcomes

23. Functional Outcomes What does a child typically do at this age level? Actual performance across settings and situations How child uses his/her skills to accomplish tasks Not the child’s capacity to function under ideal circumstances Not necessarily the child’s performance in a structured testing situation:" noncompliant”

24. So…how do we demonstrate that our programs produce good outcomes for children?…

25. Data will demonstrate that… Children have positive outcomes regardless of, their delays, disabilities, and functioning when they entered ECSE services. Individual child outcomes are better than they would have been without ECSE services.

26. Complicating Issues Many children progress with no intervention (maturing with age). Children with disabilities’ show diverse progress and under the best of programs, will experience different outcomes. Children receive different amounts, types and quality of services.

27. Quality Assessment  Quality Service  Better Outcomes

28. “Much of developmental psychology (early childhood testing) as it now exists is the science of the strange behavior of children with strange adults in strange settings for the briefest possible periods of time.” (Bronfenbrenner, 1979)

29. What is assessment? “Early childhood assessment is flexible, collaborative decision-making process in which teams of parents and professionals repeatedly revise their judgments and reach consensus about the changing developmental, educational, medical, and mental health services needs of young children and their families.” Bagnato and Neisworth, 1991 Quoted in DEC Recommended Practices, 2005

30. DEC Recommended Practices for Assessment Involves multiple sources (e.g., families, professional team members, service providers, caregivers) Involves multiple measures (e.g., observations, criterion-curriculum-based instruments, interviews, curriculum-compatible norm-referenced scales, informed clinical opinion, work samples)

31. Uses and limitations of assessment tools for measuring child outcomes… Used to compare a child’s skills and behaviors to those of his same-age peers. Most are organized around domains. Not designed for direct measurement of the three outcomes. Some call for standardized assessment items to be administered in a setting other than the child’s natural environment, making it difficult to use the information to determine whether a child uses this skill in everyday life. Teams may have different information that differs, thus “overriding” assessment tool results.

32. Critical Assumptions Related to the Three Child Outcomes • Measuring achievement of outcomes is based on comparison with age expectations • Children of different ages will demonstrate achievement in different ways, so expectations change with age • There are many pathways to functioning for children with atypical development (using sign language, wheelchair, etc.)

33. What information is gathered? Anchor Assessment- Norm-referenced, criterion-referenced or curriculum-based assessments Multiple sources of data are used to rate a child’s functioning Informed Professional Judgment & Observation Parent Input

34. Which anchor assessment tools will we use in Idaho? The following tools have been adopted for completing the outcome rating scale. BDI – II (Battelle) HELP (Hawaii) Creative Curriculum AEPS Carolina OUNCE Work Sampling High Scope (COR) Brigance Developmental Assessment of Young Children (DAYC) Bayley III (ITP program only)

35. Informed professional judgment Teachers, paraprofessionals, related service providers Anecdotal records, documented observations and data, progress reports, work samples, portfolios Parental/Caregiver input Best practice Provides critical information about how a child functions across the day and in a variety of settings Information to gather…

36. So, our challenge is… How do we take what we know about assessment and apply it to measuring the 3 outcomes… …when there is no single assessment tool that assesses all 3 outcomes directly.

37. “Crosswalks” were developed by National ECO Center to… Show how the skills assessed by most commonly used published instruments map to the three outcomes. Provide comparisons across instruments to see their various strengths and weaknesses with regard to the three outcomes.

38. Early Childhood Outcomes Center 38

39. So, how do we put it all together… .

40. Child Outcomes Summary Form (COSF-R) • Produces and compares data from programs across the state that are using different assessment tools • A 7-point scale for summarizing information related to a child’s progress on each of the three child outcome areas • Rating based on child’s functioning: • Across settings and situations • Compared with what skills are expected given the child’s age

41. Child Outcome Summary Form Key Features Is NOT an Assessment Is NOT about eligibility determination Does NOT rate or summarize Info about services provided to child Family’s satisfaction with services Information for planning services for child Blends diverse data from multiple sources into a consistent, reliable unit of measurement that can be aggregated and tracked over time

42. Child Outcome Summary Form Key Features Uses diverse information for global view of a child Compares a child’s functioning to same-aged peers Documents progress and movement toward typical development Considers functional abilities across diverse settings (not under ideal circumstances) Provides information about child’s overall sense of functioning in the three outcome areas

43. COSF Key Points • Assumption: Children can be described with regard to how close they are to age-expected functioning for each of the three outcomes • By definition, most children in the general population demonstrate the outcome in an age-expected way • Over time, some children will move farther away from age-expected functioning (skills at older ages are more demanding) • By providing services and supports, programs are trying to move children closer to age-expected functioning • Some children will never achieve this

44. Essential Knowledge for team members completing the COSF Know about the child’s functioning across settings and situations Understand age-expected child development Understand the content of the three child outcomes Know how to use the rating scale Understand age expectations for child functioning within the child’s culture Early Childhood Outcomes Center

45. Child Outcomes Summary Form Early Childhood Outcomes Center

46. The Form Cover page + three outcome pages On each outcome page: Two questions per outcome – one at entry and two at exit Space to document the summary of relevant results (and examples of age-appropriate, immediate foundational and/or foundational skills), which will be the basis for the rating Early Childhood Outcomes Center

47. The Two COSF Questions for each outcome a. To what extent does this child show behaviors and skills related to this outcome appropriate for his/her age across a variety of settings and situations? (Rating: 1-7) b. Has the child shown any new skills or behaviors related to [this outcome] since the last outcomes summary? (Yes-No) Early Childhood Outcomes Center

48. Summary Ratings Reflect Global Functioning Ratings on each outcome are a snapshot of: The whole child Status of the child’s current functioning based on the various types of information from a variety of people who are familiar with the child across a variety of typical settings and situations Team consensus to ensure validity and reliability Rather than reflective of : Skill by skill, in one standardized way, or split by domains Early Childhood Outcomes Center

49. The Basis for the Ratings Scale runs from 1-7 with a 6 or 7 indicating age appropriate functioning for the outcome Lower numbers indicate distance from age appropriate functioning for the outcome Resources – “Definitions for Outcome Ratings” and “Decision Tree for Summary Rating Discussions” Early Childhood Outcomes Center

50. Important Terms… • Foundational skills are…. Skills and behaviors that occur earlier in development and serve as the foundation for later skill development Teachers and interventionists often use foundational skills to help children move to the next level developmentally • Immediate foundational skills are… Skills that are conceptually linked to later skills and immediately precede the later skills developmentally Example: Children play alongside one another before they interact in play