Unpacking PAT Tests from the NZCER Marking Website

1. Unpacking PAT Tests from the NZCER Marking Website Graeme Cosslett NZCER 2010

2. Building a greater understanding of the online reporting • Reporting at the: • Individual Student level • Class level • Schoolwide level • Understanding: • Scale scores, Stanines, Box plots, Measurement error, Interpreting reports, Generating the reports

3. Class of 2010 How are they doing?

4. Test scores

5. 0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40-44 Grouped test scores

6. Ordered test scores

7. It depends! How well are these students doing?

8. What kind of knowledge and skill do students have at different achievement levels? How much better is a student doing who gets 35 compared to 25? How do scores on this test compare with scores on a harder or easier test? What score should I expect the students to get in a year’s time? Questions left unanswered

9. We can start with a measurement scale. How do we answer these kinds of questions?

10. Allow you to convert a test score to a location on an equal interval scale. You can then see how one score compares with another, no matter which PAT test was used. You compare a student’s location on the scale with national norms for different year levels. Measurement scales

11. You can locate the difficulties of each test question on the same scale. We can then compare a student’s location on the scale with the location of different questions. Measurement scales

12. By Year 6, 50% of students have reached this level Questions involve longer passages containing abstract ideas and more complex vocabulary Measuring progress More comprehension skills Less comprehension skills

13. Create your own reading comprehension and mathematics scales that cater for achievement from Year 4 to 10. Describe the characteristics of tasks at different points on each scale. Tasks that can be done using counting strategies.

18. Put the tasks in order from least difficult to most difficult. Locate each task somewhere on your scale Which tasks are the most difficult? Why? How would a typical Year 7 do on each of the tasks? On each of your scales locate where you think the average Year 7 student would be How difficult?

19. 75 patm 66 patm Year 10 Student Year 7 Student 34 patm 20 patm

20. Students will typically answer correctly: 50% of the test items that are at the same location on the scale as they are more than 50% of the test items that are below their own location on the scale less than 50% of the test items that are above their own location on the scale. Comparing students and items

22. As students improve in comprehension they demonstrate increasing skill in: using abstract information using separated information using multiple pieces of information using implied information rejecting competing information using vocabulary using grammatical structures. Increasing skill as a reader

23. Vocabulary demands Percentage of words in word list one to 10

24. The items in PAT Reading Comprehension according to text

27. Unpacking grammatical structures

29. Average progression in reading vocabulary for New Zealand students Year 10 Year 9 Year 8 Year 7 Year 6 Year 5 Year 4

30. Average progression in mathematics for New Zealand students PAT:Mathematics Scale Year 10 Year 9 Year 8 Year 7 Year 6 Year 5 Year 4 Year 3

31. The PAT:Mathematics Scale

32. Stanines

33. Stanines

34. Stanines

35. Two students score 50% on PAT Mathematics tests. Student one has sat Test 5 and student two has sat Test 6. Test 6 is harder than Test 5. Who will be located higher on the scale? Where on the scale?

36. What Tom got right

39. Tom six six 28 55.9 ± 3.6 seven six

43. Which national year group is Tom being compared with? How good is stanine 7? How many Geometry and Measurement questions did Tom get wrong? Which was the easiest Geometry and measurement question? Was Tom expected to get question 30 correct? Toms’ Maths Report

44. Sam Year 6 March, 2008 20 44.1 3.5 5 Year 6

46. Retrieval Comprehension without needing to infer Local inference Comprehension of implied information from parts of the text Global inference Comprehension of implied information from across the text as a whole Question types

47. What does the poet compare herself with in the last verse? (Inferring the meaning of a simile from 1 line.) Why did Glen shrug? (Inferring a character’s intention from implied information across 6 sentences.) What was Adam imagining when he was playing with the water cannon? (Inferring a character’s thoughts from information in 4 paragraphs.) Local inference

48. What is the main purpose of this text? Which sentence would the author most likely agree with? What impression of the brain does this text create? What will probably happen next in the story? Global inference

50. PAT:Vocabulary items