Leadership in Using NeSA Data

1. NEBRASKA STATE ACCOUNTABILITY Leadership in Using NeSA Data

2. Nebraska schools have many sources of data: • Classroom assessments • Diagnostic district assessment • Norm-referenced • and now . . . • Nebraska State Accountability • Each assessment tool has a purpose and a role in the big picture of Continuous Improvement.

3. Nebraska schools should use NeSA data to . . . • Provide feedback to students, parents and the community • Inform instructional decisions. • Inform curriculum development and revision. • Measure program success and effectiveness. • Promote accountability to meet state and federal requirements.

4. Understanding NeSA Data • What is NeSA? • How do we access and interpret NeSA data? • How do we use NeSA data?

5. What is NeSA?

6. NeSA is . . . • A criterion-referenced summative test. • A measurement of the revised Nebraska Reading Standards specific to vocabulary and comprehension. • A tool including 45 to 50 multiple-choice items. • A test administered to students online OR paper/pencil.

7. NeSA is . . . • Administered during the spring of the year. • Based on Tables of Specification and Performance Level Descriptors. • Built upon the best thinking of Nebraska educators, national experts, and a worthy partner – Data Recognition Corporation.

8. What are . . . Tables of Specification

9. What are . . . Performance Level Descriptors

10. Where do we find the content components of NeSA? • www.education.ne.gov/Assessment • Tables of Specification • Performance Level Descriptors • Accommodations Guides • Webb’s DOK documents

11. NeSA . . . • Produces a raw score that converts to a scale score of 0-200. • Allows for students to be classified into one of three categories: Below the Standards, Meets the Standards, Exceeds the Standards. • Provides comparability across Nebraska school buildings and districts.

12. How do we access and interpret NeSA data?

14. What is the DRS? (Data Reporting System) • Secure Site – through portal • Public Site – NDE website http://drs.education.ne.gov

15. Interpreting NeSA-R Data Reports • District/building level information • Individual student level information • Subgroup information • Indicator information

16. Use the Reports Interpretive Guide! http://www.education.ne.gov/Assessment/documents/NESA.Read.InterpretiveGuide.pdf

17. PERFORMANCE LEVELS - three possible categories of student performance on NeSA ~NeSA Terminology~

18. How are performance levels determined? • Cut score processes: • Contrasting Group Method – 400+ teachers • Bookmark Method – 100+ teachers • State Board of Education Reviewed • Examined results of both processes • Examined NAEP and ACT results for Nebraska • Made decisions within recommended range at public meeting

19. RAW SCORE – the number of items a student answers ‘right’ on NeSA-R on NeSA Reports on Conversion Chart ~NeSA Terminology~

20. SCALE SCORE – a student’s transformed version of the raw score earned on NeSA ~NeSA Terminology~

21. What is the difference between a raw score and a scale score? What is a raw score? A raw score is the number of correct items. Raw scores have been typically used in classrooms as percentages: 18/20= 90% correct. ~NeSA Terminology~

22. What is a scale score? A scale score is a “transformation” of the number of items answered correctly to a score that can be more easily interpreted between tests and over time. The scale score maintains the rank order of students (i.e., a student who answers more items correctly gets a higher scale score). For NeSA, we selected 0-200 and will use it for all NeSA tests, including writing. ~NeSA Terminology~

23. Why convert raw scores to scale scores? Raw scores are converted to scale scores in order to compare scores from year to year. Raw scores should not be compared over time because items vary in difficulty level. Additionally, raw scores should not be compared across different content area tests. Scale scores add stability to data collected over time that raw scores do not provide. ~NeSA Terminology~

24. On score reports why is the . . . SCALE SCORE CONVERTED TO PERCENTILE RANK? The percentile rank was placed on the score reports because our Technical Advisory Committee felt that parents would want to know their child’s position in relation to other test takers. A percentile rank of 84 means the child scored better than 84% of the students who took the test that year. ~NeSA Terminology~

25. NeSA (CRT) vs. NRT ? • --Differences-- • Purposes: • NeSA is intended to match and measure identified standards and instruction. • NRT is not intended to measure any state’s standards. The intention is to compare students to each other. • Item Development: • NeSA items with exact match to the standards – NDE had to prove the match with an independent alignment study • NRT – No standards to match – matches inherent and previous knowledge, enriched homes, pre-skills.

26. NeSA (CRT) vs. NRT ? • --Similarities— • All the psychometric steps – standard setting (Bookmark, Angoff, Contrasting Group) • Reliabilities – KR 20-21 / Inter-rater Reliabilities • Descriptive Statistics (Item P-values, Dif-analysis) • Administration: • Both standardized – are generally administered the same way.

27. NeSA REPORTS • Individual Student Report • School Student Roster • School Indicator Summary • School Performance Level Summary • District Reading Indicator Summary • District Performance Level Summary • District Report of School Performance

28. Questions for Consideration • What can we learn from this report? • Do we have other data to support these results? • What are the implications of this report?

29. Individual Student Report

30. School Student Roster

31. School Indicator Summary

32. School Performance Level Summary

33. District Reading Indicator Summary

34. District Performance Level Summary

35. District Report of School Performance

36. How do we use NeSA data?

37. Use NeSA Data to inform . . . • Curriculum alignment process: • Are the tested indicators in our curriculum? -- Where? • When are they taught? • How are they instructed? • At what DOK (Depth of Knowledge) level? • By whom?

38. Use NeSA Data to inform . . . Test preparation processes. • Examine PLDs and Tables of Specification. • Do our students have opportunity to learn the tested indicators? • How are they performing on the indicators on a day-to-day basis? • Are we assessing them locally?

39. Use NeSA Data to inform . . . • Practice Tests • Have our students used practice tests? • Are our students familiar with the testing tools? • Are we familiar with appropriate accommodations?

40. NeSA results ARE an important data source! When combined with other information, these data can support curricular, instructional, and learning support decision making. --It’s all about the Continuous Improvement Process!