Using the 2011 NECAP Science Results

1. New England Common Assessment Program Using the 2011 NECAP Science Results

2. Dr. Kevon Tucker-Seeley Assessment Specialist – NECAP Manager Office of Instruction, Assessment & Accountability 1-401-222-8494 Peter McLaren Science and Technology Specialist Office of Instruction, Assessment & Accountability 1-401-222-8454 Jennifer Golenia Science and Technology Specialist Office of Instruction, Assessment & Accountability 1-401-222-8406 Lindsay Wepman Assessment Specialist Office of Instruction, Assessment & Accountability 1-401-222-8254 Welcome and RIDE Introductions

3. Measured Progress Introductions 3

4. Measured Progress Introductions Kellie Beaulieu: NECAP Program Assistant 603-749-9102 ext. 2156 beaulieu.kellie@measuredprogress.org Mellicent Friddell: NECAP Program Assistant 603-749-9102 ext. 2355 friddell.mellicent@measuredprogress.org Alison Cady: NECAP Program Assistant 603-749-9102 ext. 2149 cady.alison@measuredprogress.org NECAP Service Center: 1-877-632-7774

5. Goals of the Workshop • Review the different types of NECAP Science reports and share State results • Demonstrate a simple data analysis protocol using NECAP Science Data • Share some observations from the NECAP benchmarking process

6. Workshop Reference Materials

7. Types of NECAP Science Reports Student Report (Confidential)Information for Parents Item Analysis Report(Confidential)School level by student Results Report (Public)School and District level Summary Report (Public)District/State level Student Level Data Files(Confidential)Excel/csv files by grade on district and school confidential site 7

8. Accessing Your Confidential Reports http://iservices.measuredprogress.org 8

9. Accessing Your Confidential Reports Select “Interactive” to view Interactive Reports Select “Reports” to view Static Reports 9

10. NECAP Analysis and Reporting System (NARS) Account Creation Hierarchy Password Assistance: Use the following list to determine whom to contact for assistance with your User Name and Password: Superintendents – Contact NECAP Service Center at 877-632-7774 Principals – Contact NECAP Service Center at 877-632-7774 Teachers – Contact School Principal http://www.ride.ri.gov/assessment/Results.aspx

11. Using NECAP Science Data • Focus is improving student learning • The goal is to increase student achievement • Engage in collaborative discourse about data • Using relevant, timely data to influence educational decisions is considered a best practice • Discussion of data requires a structured approach • Protocols, ground rules, and shared talk time • Remember, NECAP Science data is for assessing school-level achievement, NOT individual student achievement • Look for trends and patterns

12. NECAP Science Data Today we’ll take a look at: • Demo District Results • Demo School Results • Test Items and Item Analysis Reports • State Results

13. Using NECAP Science Data Phases of Collaborative Inquiry Adapted from N. Love

14. Using Data―A Simple Approach • Observe the data • Look at the data with your table. • Make objective statements about the data. Avoid statements like, “It seems…” or “I think…” • What do you notice?

15. Using Data―A Simple Approach 2. Discuss • Talk with your colleagues at your table. • Why do think the data might look this way?

16. Using Data―A Simple Approach 3. Action steps • What are some other sources that could confirm or refute this data? • What does the data mean for my school regarding curriculum and instruction? • How can I dig deeper?

17. Using Data―A Simple Approach 4. Reflection (time permitting) • What did you learn from this activity? • Are there new things that you’d like to bring back to your school?

18. Tying it Together • Limitations • This is one source of data. You can’t make major programmatic changes from this alone. • Look to other sources of data to confirm or refute. • Think about curricular and instructional action steps. • Look for professional development around data use in the 2012–13 and 2013–14 school years.

19. Percentage of Rhode Island Students by Achievement Level

20. 10/11 Comparison of NECAP States by Achievement Level Note: Vermont’s data is embargoed until 9/28. RI will do a public release on 9/27

21. Benchmarking Short Answer and Constructed Response Items • Short answer items receive a score from zero to two. • Constructed response items receive a score from zero to three or zero to four.

22. Preparation for Benchmarking • The work in preparation for scoring student responses included: •  Development of scoring guides (rubrics) • Content specialists from the NH, RI, and VT Departments of Education • Measured Progress’s test developers • Selection of “benchmark” responses • Examples of student work at different score points for each item • Used in training and continuous monitoring of scorer accuracy

23. Scorer Training • Scorer training consisted of: • Review of each item and its related content and performance standard • Review and discussion of the scoring guide and multiple sets of benchmark responses for each score point • Qualifying rounds of scoring in which scorers needed to demonstrate a prescribed level of accuracy.

24. NECAP Benchmarking Process

25. Lessons Learned from NECAP Benchmarking • What does it mean • to identify? • to describe? • to explain? • How does this look over grade levels? • What does it mean to compare and contrast?

26. Lessons Learned Grade 4 • What is a characteristic? • What observations of information can be gained from charts and diagrams? • What patterns can be distinguished from graphs? • How can students better use evidence from data in their explanations?

27. Lessons Learned Grade 8 • When should line graphs be used? When should bar graphs be used? • How can students better support or refute their predictions/hypotheses using evidence from data? • Why is it important to use multiple trials in an investigation?

28. Lessons Learned Grade 11 • What are typical sources of error? • How can they affect the outcomes of an investigation? • How can skills in the analysis of data be demonstrated more deeply? • By 11th grade, students should know about the use of proper units of measurement.

29. Involvement of Local Educators in NECAP Science • Development of Assessment Targets • Participation in annual test item review committee & bias and sensitivity review committee • Use of classroom teacher judgment data • Participation in standard setting panels • Technical Advisory Committee work

30. Preparing for 2012 NECAP Science • Use the data • Share progress with parents • Identify gaps in curriculum, instruction, and assessment • Identify students in need of accommodations • Anticipate types of accommodations • NimbleTools will be an option – Spring 2012 • Plan for administration now • Review what went well and what went not so well during the 2011 administration • Begin preparing your students • Share the reference sheets • Released Items documents • Preparing Students for NECAP: • Tips for Teachers to Share with Students 30

31. Questions? Peter McLaren Science and Technology Specialist Office of Instruction, Assessment & Accountability Peter.mclaren@ride.ri.gov 1-401-222-8454 Jennifer Golenia Science and Technology Specialist Office of Instruction, Assessment & Accountability Jennifer.golenia@ride.ri.gov 1-401-222-8406 31