Presentation to the House Education Committee January 19, 2007 Robert (Barney) Barnoski, Ph.D.

1. Performance on the 10th-Grade WASL: Summary of Findings to Date Presentation to the House Education Committee January 19, 2007 Robert (Barney) Barnoski, Ph.D. Washington State Institute for Public Policy

2. Who We Are Washington State Institute for Public Policy • Created by the 1983 Legislature • Conducts non-partisan research on topics assigned by the Legislature or board of directors Representative Helen Sommers Robin Arnold-Williams, DSHS Representative Fred Jarrett Victor Moore, OFM Representative Phyllis Kenney Sandra Archibald, University of Washington House position pending Andrew Bodman, Western Washington University Senator Karen Fraser Les Purce, The Evergreen State College Senator Jeanne Kohl-Welles Robert Rosenman, Washington State University Senator Pam Roach Ken Conte, House Office of Program Research Senator Mark Schoesler Richard Rodger, Senate Committee Services

3. Study Direction From the 2006 Legislature Conduct a statistical analysis of WASL data to:  Increase understanding of students who did not meet standard,  Identify the characteristics of those students, and  Identify possible barriers to success. [SSB 6618] Evaluate the Promoting Academic Success program for students who did not meet standard. [ESSB 6386]

4. Outline 1. Overall WASL findings  Strands  Subject-area results  Open-ended vs. multiple-choice 2. Characteristics of students who did not meet standard on the spring 2006 WASL 3. Evaluation of Promoting Academic Success (PAS) in summer 2006  Participation in PAS  Results from the August retake

5. Percentage of StudentsAchieving Proficiency in MATH Strands 65% Content 1 Content 2 60% Content 3 Content 4 Content 5 Process 1 Process 2 Process 3 55% 50% 45% 40% 35% 30% 1999 2000 2001 2002 2003 2004 2005 2006

6. Subject-Area Strand Results • The percentage of students who are proficient in reading and math strands varies considerably from one year to the next. • This variability diminishes the use of strand results to diagnose areas in need of improvement. • Variability in strand-level results does not diminish the overall reliability of the WASL.

7. Correlations Among Reading, Writing, and Math Subject-Area Scores The correlations among subject-area results are strong: READING and MATH 0.69 WRITING and MATH 0.62 1.0 Perfect Association .90 .40 .80 .60 .50 .30 .70 .10 .20 0 No Association READING and WRITING 0.64

8. Average Math Score by Reading Score Students who do well in math also do well in reading and writing, but the converse is not necessarily true: 550 500 Math Score = Reading Score 450 Math Score 400 10 point increase in reading = 7.5 point increase in math 350 300 300 350 400 450 500 550 Reading Score

9. Correlations Between Raw Scores on Open-Ended and Multiple-Choice Questions MATH 0.83 READING 0.65 .90 .40 .80 .60 .50 .30 .70 .10 .20 0 No Association 1.0 Perfect Association

10. Performance on Open-Ended and Multiple-Choice Questions • Most students who do well on multiple-choice questions also do well on open-ended questions, and vice versa. • That is, multiple-choice and open-ended questions require a similar ability level, especially in math.

11. Summary: Overall WASL Findings • Strand results are unreliable and cannot be used to diagnose student performance • Reading, writing, and math results are strongly correlated • Raw scores on open-ended and multiple-choice questions are strongly correlated

12. Factors Affecting WASL Performance Student Characteristics Extra Assistance Student Performance on WASL Classroom Instruction School Environment Unknown Factors Curriculum Alignment

13. Student Characteristics Associated With WASL Performance Higher met-standard rates • Asian American • European American • Grade point average • Parents’ educational attainment • Male students (math) Lower met-standard rates • Non-Asian minorities • Poverty • English language learners • Students with disabilities • Male students (reading and writing)

14. WASL Met-Standard Rates by Race/Ethnicity 100 Reading Writing Math 91 88 88 88 80 79 76 75 75 70 60 67 63 61 40 37 30 20 27 0 Asian European Hispanic African Native (7.9%) (71.6%) (10.3%) (4.8%) (2.5%)

15. WASL Met-Standard Rates by Poverty 100 Reading Writing Math 91 89 80 73 71 60 63 40 34 20 0 Not free or reduced lunch Free or reduced lunch (70.5%) (29.5%)

16. WASL Met-Standard Rates by Primary Language Spoken 100 Reading Writing Math 88 80 86 82 81 60 66 65 63 57 53 49 40 37 20 17 0 English Spanish Other (2.3%) Asian (1.9%) (91.4%) (4.4%)

17. Reading Writing Math -31 -29 -27 -18 -17 -16 -9 -7 -13 -4 -3 -9 -22 -21 -7 -6 -5 -5 -9 -25 +7 +2 +6 +5 +23 +21 +26 Relative Association Between WASL Met-Standard Rates and Students Grouped by Characteristics All Three Disabilities -27 Poverty -16 African American -13 Hispanic American • 9 Non-English • 8 Native American • 5 Male +3 Asian +5 Parents’ Education +26

18. Factors Affecting WASL Performance Student Characteristics Extra Assistance Student Performance on WASL Classroom Instruction School Environment Unknown Factors Curriculum Alignment

19. Participation in Summer 2006 PAS by Level of WASL Performance 35,000 32,144 Eligible (did not meet standard) 30,000 Level 2 PAS participants 25,000 Level 1 PAS participants 20,000 15,000 11,447 10,137 10,000 5,000 0 Reading Writing Math

20. Summer 2006 PAS: Reading Retake August 2006 (532) Summer 2006 PAS (897) Met Std. August 2006 (221) 8.8% of eligible students 59.3% of PAS participants 41.5% of WASL retakes Did not meet standard in spring 2006 (10,137) Retake August 2006 (1,333) No summer 2006 PAS (9,240) Met Std. August 2006 (586) 91.2% of eligible students 14.4% of non-participants 44.0% of WASL retakes

21. Summer 2006 PAS: Writing Retake August 2006 (602) Summer 2006 PAS (1,035) Met Std. August 2006 (394) 9.0% of eligible students 58.2% of PAS participants 65.4% of WASL retakes Did not meet standard in spring 2006 (11,447) Retake August 2006 (1,655) No summer 2006 PAS (10,412) Met Std. August 2006 (1,032) 91.0% of eligible students 15.9% of non-participants 62.4% of WASL retakes

22. Summer 2006 PAS: Math Retake August 2006 (2,986) Summer 2006 PAS (4,218) Met Std. August 2006 (843) 13.1% of eligible students 70.8% of PAS participants 28.2% of WASL retakes Did not meet standard in spring 2006 (32,144) Retake August 2006 (5,817) No summer 2006 PAS (27,926) Met Std. August 2006 (1,412) 86.9% of eligible students 20.8% of non-participants 24.3% of WASL retakes

23. Percentage Who Met Standard on the August 2006 WASL Retake 100 PAS participants 80 Non-participants * 60 65% 59% 40 42% 42% 28% 20 22% 0 Reading (n.s.) Writing (p<.01) Math (p<.01) * Met-standard rates for non-participants adjusted using logistic regression

24. 72 72 64 47 47 45 39 31 31 24 24 17 15 15 4 3 Percentage of Students By Level Who Met Standard on the August 2006 WASL Retake 100 Level 1 80 Level 2 60 40 20 0 Not-PAS Not-PAS PAS PAS PAS Not-PAS Reading Writing Math

25. Summary: Summer 2006 Promoting Academic Success • Few students who were eligible for PAS in summer 2006 participated (approximately 1 in 10). • Writing and Math: PAS participants outperformed non-participants by 6 percentage points on the August retake. • Reading: PAS participants have same performance as non-participantson the August retake.