Data Basics Data Basics Clif St. Germain, Ph.D.

1. Data Basics Data BasicsClif St. Germain, Ph.D.

2. One of the most critical problems our schools face is the fragmentation overload and incoherence resulting from uncritical and uncoordinated acceptance of anything that seems to work.

3. Here is the problem: Selecting instructional programs and processes “that work” is not enough to warrant inclusion as a viable strategy for improving the academic performance of our students.

4. What teachers and principals need to guarantee sustainable progress is a process for identifying strategies and programs that create a “multiplier effect” in that they impact academic performance beyond what can be expected by average progress.

5. This is where the appropriate use of data can be quite powerful.

6. If our goal is to create classrooms in which all students can improve their learning potential and demonstrate learning progress on end of year standardized tests then…

7. Priority one must be to disaggregate relevant testing data to determine the level of impact learning support programs have upon different performance groups in existing student population.

8. Session One

9. Seminar objectives: • Revisit the basic principles of data diagnostics • Compare and contrast types of tests and data reporting scales • Practice reading comprehensive testing histories • Review promising learning support strategies for disaggregated performance sub-groups • Work with sample testing histories to practice creating Within Group Learning Support Profiles for classroom teachers

10. Staying Clear About the Purposes of Data Disaggregation

11. Getting Out Ahead of Test Results Experts suggest that test data, appropriately used, provide schools (and school leaders) with a reliable mechanism for selecting appropriate learning support programs. The decisions schools make in this regard significantly impact their ability to show aggregate gains on standardized measures of achievement. Knowing the appropriate intervention to select is priceless.

12. Getting Out Ahead of Test Results For example, if a school’s data indicated a large number of students in the Special Needs group and that school did not provide extensive reading support it would be reasonable to expect that school to show no gain. Similarly, if a school provided reading support for all students, even Gifted students, the effort would be largely wasted.

13. Getting Out Ahead of Test Results The message here is knowing how to select and implement differentiated learning support programs is vital to showing across the board learning gains on state-mandated measures of achievement.

14. Data As A Steering Mechanism for Accelerating Student Academic Progress • Honor the natural variance of student abilities • Emphasize progress over attainment • Align school-wide learning support with within-group population distributions (i.e. students needing reading support) • Monitor within group year to year student progress

15. Measuring Success

16. What is value-added assessment? Value-added assessment is a new way of analyzing test data that can measure teaching and learning. Based on a review of students' test score gains from previous grades, researchers can predict the amount of growth those students are likely to make in a given year.

17. What is value-added assessment? Thus, value-added assessment can show whether particular students - those taking a certain Algebra class, say - have made the expected amount of progress, have made less progress than expected, or have been stretched beyond what they could reasonably be expected to achieve.

18. What is value-added assessment? Using the same methods, one can look back over several years to measure the long-term impact that a particular teacher or school had on student achievement.

19. How is value-added assessment different from traditional measures of student performance? Student performance on assessments can be measured in two very different ways, both of which are important.

20. How is value-added assessment different from traditional measures of student performance? In the past, students and schools have been ranked solely according to attainment. The problem with this method is that attainment is highly linked to the socioeconomic status of a student's family. For example, according to Educational Testing Service, SAT scores rise with every $10,000 of family income. This should not be surprising since all the variables that contribute to high-test scores correlate strongly with family income.

21. How is value-added assessment different from traditional measures of student performance? In contrast, value-added assessment measures growth and answers the question: how much value did the school staff add to the students who live in its community? How, in effect, did they do with the hand society dealt them? If schools are to be judged fairly, it is important to understand this significant difference. (Source: Theodore Hershberg Ph.D.)

22. NRT Data Reporting Scales A normal distributionis a very important statistical data distribution pattern occurring in many natural phenomena, such as height, blood pressure, lengths of objects produced by machines, etc. Certain data, when graphed as a histogram (data on the horizontal axis, amount of data on the vertical axis), creates a bell-shaped curve known as a normal curve, or normal distribution.

23. NRT Data Reporting Scales

24. NRT Data Reporting Scales Normal distributions are symmetrical with a single central peak at the mean (average) of the data. The shape of the curve is described as bell-shapedwith the graph falling off evenly on either side of the mean. Fifty percent of the distribution lies to the left of the mean and fifty percent lies to the right of the mean.

25. National Percentile Rank (NPR)

26. National Percentile Rank (NPR) The percentile rank, reported in units that range from 1 to 99, is perhaps the most useful and readily understood score for interpreting student achievement. It describes performance in small fairly precise units. These units clearly state performance relative to the norm group.

27. National Percentile Rank (NPR) For example, if Kara earned a percentile rank score of 72 on the Vocabulary test, it means that she scored the same as or higher than 72 percent of the students in the norm group who were in the same grade and took the same test. It also means that 28 percent of the students scored higher than Kara.

28. National Percentile Rank (NPR) The percentile rank magnifies small differences in the middle raw scores while reducing differences in very high and very low raw scores. For example, a difference of only three raw score points might be necessary to move from a percentile rank of 50 to a percentile rank of 55, while a difference of seven raw score points might be necessary to move from a percentile rank  

29. National Stanine (NS) Stanines (Standard-nine) express test results in nine equal steps ranging from 1 (lowest) to 9 (highest). • The average stanine is a score of 5. • Stanines, which are similar to percentile ranks, are relatively easy to use because they are all one-digit numbers. • The national stanine (NS) shows a student’s standing within the group of students in the same grade who took the test at the same time of year.

30. National Stanine (NS) Stanines for groups are useful for broadly identifying areas of academic strengths and weaknesses that might be represented by a set of test scores. A group of students with a stanine of 4 for Reading Total for example, shows that these students share a reading level that is slightly below average.

31. Within Group Distributions

32. National Stanine (NS) Stanines are less precise than percentile rank scores. For example, percentile ranks of 24 and 40 both represent a stanine of 4. However, PRs of 23 and 24 are consecutive PRs that represent stanines of 3 and 4, respectively.

33. Reading Comprehension Sub-test In the absence of an aptitude measure (I.Q.) the reading comprehension sub test score is the best predictor of potential for academic success.These tests are all greatly influenced by reading ability. In fact, 88% of the variance in 11th grade performance is predicted by the 3rd grade reading comprehension sub test.

34. Standard Score (SS) Standard scores(SSs) are produced from a single, equal-interval scale of scores that is continuous from kindergarten through grade 12. Standard scores on The Iowa Tests range from 80 for kindergarten through 400 for grade 12. The range of possible standard scores is different for each grade level.

35. Standard Score (SS) The major use of the standard score is to measure achievement growth of students or groups of students from year to year. Within a school or district, successively higher standard scores from grade to grade would be expected.

36. See handout for Sample IOWA NRT Survey Battery Score (Overall) /Year to Year Expected Progress

37. Questions: • What do you know about a student in 5th grade with a Standard Score of 263? • Can you determine this student’s NPR, Stanine? • Is this student a high, average or low performer? • Why do you think there are some spaces with empty scores?

38. Questions: • What do you know about a 6th grade student with a standard score of 216? • Can you determine this student’s predicted level for 7th grade? • What questions do you have about reading this score progression chart?

39. Comprehensive Testing History: The Louisiana State Department of Education publishes student testing histories for all students. These histories list the entire testing history of an individual student. In those instances where the most recent testing data do not provide ample diagnostic information to make an informed judgment of appropriate learning support it is recommended that teachers print a completehistory.

40. Comprehensive Testing History:

41. Session Two

42. Within Group Learning Support Profiles Your students can be disaggregated into five performance groups according to standardized academic indicators and overall literacy scores (survey battery).

43. A Note to Principals and Teachers Each performance group generally represents students with similar academic needs. Similar to the Response to Interventions Model, the performance groups represented here can be expected to improve their collective performance with appropriate instructional support and progress monitoring.

44. A Note to Principals and Teachers Note Well: Instructional interventions suggested for one group are not necessarily appropriate for others, nor are they intended to be substitutes for regular classroom learning experiences. Performance Groups with substantial percentages should be given considerable attention as you formulate your school improvement priorities.

45. Questions you might ask include the following: • Is the percentage of special needs students similar to the number of students presently approved for special academic services? • Students in the targeted assistance group are predicted to experience much higher levels of failure without additional time to complete basic school tasks. How can we arrange for these students to pair their reading, language and/or math classes with regular support linked to these subjects?

46. Questions you might ask include the following: • Students in the maximum impact group, because they have gaps in their prior knowledge due to poor organizational skills, or limited active working memory capacity, tend be underachievers. How can we guarantee that these students learn how to become more efficient learners? • Students in the gifted alert group are students who learn rapidly but whose development is asynchronous (uneven). In these children there is a great gap between mental abilities and physical and emotional development. For this reason these students are more vulnerable.

47. Promising Learning Support Disaggregated by Performance Groups Special Needs Group: These students score in the well-below average range on the iLeap Survey Battery Score/Over All (NPR 1-10)

48. A check-list of “best practice” strategies known to enhance the performance of special needs students includes the following:

49. A check-list of “best practice” strategies known to enhance the performance of special needs students includes the following:

50. A check-list of “best practice” strategies known to enhance the performance of special needs students includes the following: It cannot be avoided that these students require intense elemental help. If their test results are accurate, expecting these students to achieve significant increases in performance in one year of school is not supportable. The goal with these students is long-term, incremental progress in all areas with intense emphasis on reading and basic levels of literacy. The social and emotional well-being of these students is also important to monitor. Providing on-going counseling support for these students is a must.