Aligning Outcomes Measurement With Early Learning Guidelines/Standards

1. Aligning Outcomes Measurement With Early Learning Guidelines/Standards Jennifer Olson (OR) Birth-5 Ruth Littlefield (NH) Birth-5 April 2006 – Albuquerque, NM

2. Questions to Guide the Session: What was the context in the state that led to the choice to align with the Early Learning Standards/Guidelines? How has the alignment with standards/guidelines influenced your selection of assessment tools? How will you report back on standards/guidelines and/or outcomes to local programs and to OSEP? What other implications have you seen as a result of deciding to align with your standards/guidelines? What do you feel are the pros and cons for the decision to align with your state standards/ guidelines?

3. Oregon Early Childhood Foundations and Assessment Project OREGON DEPARTMENT OF EDUCATION, PORTLAND STATE UNIVERSITY, UNIVERSITY OF OREGON AND EDUCATORS FROM OREGON EARLY INTERVENTION/ EARLY CHILDHOOD SPECIAL EDUCATION, PRE-KINDERGARTEN/HEAD START AND CHILD CARE PROGRAMS

4. PROJECT CONTACT INFORMATION Portland State University Joel Arick: 503-725-4251, arickj@pdx.edu Ruth Falco: 503-725-4486, falcor@pdx.edu Helen Young: 503-725-5207, youngh@pdx.edu Oregon Department of Education Nancy Johnson-Dorn: 503-378-3600 X2339, nancy.johnson-dorn@state.or.us Nancy Latini: 503-378-3600 X2361, nancy.latini@state.or.us Jennifer Olson: 503-378-3600, x2338, jennifer.olson@state.or.us

5. New Hampshire Early LearningCurriculum GuidelinesandPreschool Child Outcomes forYoung Children with Disabilities Ages 3-5 New Hampshire Department of Education/Bureau of Special Education/Preschool Special Education Office

6. Question 1. What was the context in the state that led to the choice to align with the Early Learning Standards/Guidelines?

7. Context for Alignment – Federal Requirements • National focus on accountability, standards-based reform, & measurement of child outcomes (GPRA, 1993; IDEA, 1997 & 2004; NCLB, 2001) • Good Start, Grow Smart (2002) required research-based, voluntary state guidelines for literacy, language, pre-reading, and numeracy, indicating what young children birth to 5 should know & be able to do prior to kindergarten

8. Context for Alignment – ODE Initiatives • Oregon Department of Education (ODE) initiated development of child-based outcome standards (expectations for what children should know and be able to do prior to kindergarten) • Guidelines in standards development • Link preschool outcome standards to statewide, Oregon K-12 Standards • Focus on standards for all children, 0 – 5, including children in Part C, Part B 619, OPK, Head Start, & child care programs • Add domains essential to early childhood to the domains in the K-12 standards (e.g., Approaches to Learning and Social Emotional)

9. Context for Alignment • Oregon Department of Education (ODE) formed advisory group of over 100 early childhood professionals to develop initial draft of Oregon Early Childhood Foundations (birth to 5) that align with K – 12 standards • Oregon Early Childhood Foundations reviewed (Scott-Little, 2005) • Revised Foundations drafted (ODE, 2005)

10. Context for Alignment • Oregon Early Childhood Foundations developed in 8 domains Approaches to Learning Social Emotional Development Language and Literacy Development Mathematics Science Social Studies Physical Education and Health The Arts

11. Context for Alignment – Legislature Requirements • Needed easily understood, aggregated child outcome data to present to Oregon state legislators to support funding for Early Intervention/Early Childhood Special Education (EI/ECSE) and Oregon Pre-Kindergarten (OPK) Programs/Head Start • Oregon legislature charged Oregon Department of Education (ODE) with developing statewide assessment for EI/ECSE and OPK/Head Start • This Oregon Early Childhood (EC) Assessment needed to be linked to standards for school-aged children and provide the ability to monitor overall progress of children from pre-K through school age

12. Context for Alignment in NH • Preschool Special Educator’s Questions • What is the general preschool curriculum? • What standards/guidelines exist for us to follow?

13. Context for Alignment in NH • Creation of the Early Learning Curriculum Guidelines (ELCG) Project • 619 funded • Create informed dialog across early childhood community • Strengthen connections between preschool special education and K-12 system • Develop shared language and frames of reference for communication • Enhance educational and developmental outcomes for young children with disabilities • Support access to general early childhood curriculum and appropriate preschool activities for young children with disabilities

14. Context for Alignment in NH • NH Department of Health & Human Services: Child Development charge to create early learning standards for children birth through 5 • Domain Categories • Indicators of progress • GSEG Grant: NH Cornerstones Project • Subcommittee on preschool outcomes

15. Question 2: How has the alignment with standards/guidelines influenced your selection of assessment tools?

16. Alignment with Standards and Assessment Selection – Addressing Federal and State Requirements • Office of Special Education Programs (OSEP) requires reporting of • (a) % of children who reach or maintain functioning at level comparable to same-age peers • (b) % of children who improve functioning (but were not included in a) • (c) % of children who do not improve functioning • State requires • Assessment aligned with Early Childhood Foundations • Child outcome data the Legislature will easily understand

17. Alignment with Standards and Assessment Selection – Current Program Practices • ODE currently requires EI/ECSE Programs to use one of four different curriculum-based assessments (CBAs) annually for each child • Each OPK/Head Start Program uses one of at least two different curriculum-based/criterion-referenced assessments with all children • ODE continues to value the use of comprehensive CBAs to link assessment to program planning

18. Alignment of Standards and Assessment Selection – Concerns with CBAs • Not feasible to aggregate data from the various CBAs • Not feasible to require all EI/ECSE and all OPK/Head Start Programs to use a common CBA • CBAs • Not designed to provide aggregated data on developmental progress • Not designed to compare child progress to that of typical peers • Previous attempts to use CBA data to share outcomes with Legislature were unsatisfactory • CBAs do not hold a clear relationship to Oregon Early Childhood Foundations and do not include all Foundation Domains or Areas

19. Alignment of Standards with Assessment - Oregon Early Childhood Assessment • ODE professionals decided federal and state needs will most effectively be met through development of the Oregon Early Childhood Assessment System, developed to be compatible with current CBAs while aligning with Oregon Early Childhood Foundations and Oregon K-12 Standards • ODE partnered with researchers at Portland State University (PSU) and University of Oregon (UofO) to develop the system

20. Alignment of Guidelines with Assessment Tools • Tools not yet selected: Assessment Fair on May 18th • NH Early Learning Guidelines (Good Start Grow Smart • Framework, not developmental skills checklist or curriculum • Articulate a common visions and commitment to children • Provide a map of what children should know, understand and be able to do • Basis for what adults working with young children should know, understand and be able to do

21. Alignment of Guidelines with Assessment Tools • Created as a companion document: NH Early Learning Curriculum Guidelines and Preschool Outcomes • Guiding Principals • Written in educational rather than early childhood terms • Mirrors Early Learning Guidelines • Preschool Outcomes presented with sub-outcomes • Expands on preschool indicators and what you may observe • Connects Outcomes and sub-outcomes to Domain areas

22. Question 3. How will you report back on standards and/or outcomes to local programs and to OSEP?

23. Process to Report on Outcomes:Statewide EC Assessment System Portland State University & University of Oregon are collaborating with the Oregon Department of Education to develop an assessment that measures the skills and abilities identified in the Early Childhood Foundations for all programs that serve birth to age 5 children This assessment system will be used to report back to local programs and OSEP

24. Development of Assessment Items • Research regarding early childhood development and important skills in each of the foundation domains and areas was reviewed • Numerous research-based assessments (both standardardized and non-standardized) were reviewed to identify important skills across development from birth to 6 years old • 128 items across 16 foundation areas were developed based on the research to assess a sample of skills or abilities from the foundation areas • Items were selected that are: • Observable • Functional • Developmentally appropriate (Sampling skills from 6 months – 6 years of age)

25. Criteria Used by Developers to Design the Assessment • Developmentally appropriate • Conducted in natural environments • Activity-based assessment (play, table-time, snack, group) • Measure progress over time • Aggregate/combine data across children/programs • Relate to currently used CBAs (e.g., AEPS, HELP, Carolina, Galileo, Creative Curriculum) • Valid and reliable • Easy for staff to use

26. EC Assessment System • On-Line assessment system • Uses combination of Checklist Rating Scale and Direct Observation of child during typical routines • Provides users with a student report • Interactive computer functions make data entry easy and provide immediate information to staff

27. PILOT STUDY: Demonstration of how data can be reported to local programs and OSEP

28. Pilot Study • Pilot Study to “try-out” the assessment was conducted from February 2005 to September 2005 • Educators gave children a pre-assessment and a post-assessment (each child’s pre and post-assessments were 2-4 months apart)

29. Pilot Study Purposes • To determine if assessment items would measure progress of children over time in several settings • To get feedback on assessment items, assessment process, and the on-line system • To conduct preliminary validity and reliability studies • To develop initial process for reporting on child outcomes to OSEP

30. DemographicsPilot Study: Preliminary Data Number of Children Assessed: Child Care (no-disabilities) 239 EI/ECSE 140 Head Start (non-EI/ECSE) 108 Total: 487

32. Internal Consistency ReliabilityAll Subjects in Pilot Study less than 60 months

33. Demographics: EI/ECSE

34. Federal Outcomes Data for Part C & B: Monitoring Criteria • (a) Percent of children who reach or maintain functioning at a level comparable to same-aged peers • (b) Percent of children who improve functioning (not included in a) • (c) Percent of children who did not improve functioning

35. Pilot Study Results: A) Comparison with Same-aged Peers • Preliminary Aggregate Data • Comparable to same-aged peer= within one standard deviation of same age peers (child care subjects) • Pre-post intervention period was 4 months * Post data collection was not available for all subjects at this time

39. Teacher Perceptions of Online Assessment System Survey Results (N=48)

40. What We Learned From the Pilot Study • All agencies (Preschools/Child Care, Head Start, and EI/ECSE) were supportive of the project • The on-line system efficiently allowed the assessment to be implemented across programs • All programs had access to the online system • In 4 months or less, all children in EI/ECSE, Head Start and Childcare programs showed progress • Child Care/preschool sites were very excited about being included in this project • Preliminary pilot data appears to validate the use of the assessment to monitor child development

41. NH Planning to Report Back • Still under construction • Strong stakeholder and state commitment to make this useful at local, state and federal level • Working with publishers of Assessment tools to: • minimize labor, • maximize usefulness of information to staff, • provide state level data that can easily be reported to OSEP, • be a cost-effective as possible • NHSES Statewide special education data system

42. Question 4. What other implications have you seen as a result of deciding to align with your standards?

43. Implications of Alignment of Standards and Assessment • Assessment data will be more valued by Oregon Legislature as well as OSEP • Increased knowledge of quality in EC settings • Linking the EC Assessment to K-12 standards and assessments may enable longitudinal progress monitoring

44. Implications of Alignment with Guidelines • Family input indicates this is useful for parents to see their child’s progress • Promotes better quality community child care/preschool options for all children • Strengthens the discussion with policy makers in the educational community • Reinforces the bridge between Part C and 619

45. Implications of Alignment with Guidelines • Brought preschool special education to the child care table in a very positive manner • Provides a foundation and framework for the conversations about the two burning questions for preschool special educators: • What is the general preschool curriculum? • What standards/guidelines exit for us to follow?

46. Question 5. What do you feel are the pros and cons for the decision to align with your state standards?

47. Pros • Assessment will meet both Federal and State needs • Assessment will address common curriculum goals for infants, toddlers and preschool children across EI/ECSE, OPK/Head Start, and child care • Assessment will link EI/ECSE, OPK/Head Start, and child care outcomes to outcomes in K-12 school programs • GSEG grant, in addition to State funding, will provide opportunity to further field-test unique measure to meet Federal and State needs

48. Cons • Perception that assessment is not as credible as other assessments • Perception that CBM could be used instead of adding another asessment tool. • Development of a valid and reliable measure, aligned with Oregon Early Childhood Foundations, requires time and funding

49. EC ASSESSMENT SYSTEM • If time or for those interested, we can view the on-line Oregon EC Assessment system

50. Pros • Resource documents have more stakeholder buy-in • Final products are more comprehensive and useful • More useful for a wider variety of players (families, early care and special education providers, administrators, etc) • One framework for all rather than parallel systems