Alan Spicciati, Ed.D. Seattle Pacific University, Class of 2008 spicciad@hsd401

1. Measuring the Link between Elementary Teachers and Student AchievementA Presentation of the Dissertation:“Elementary Teachers and the Mathematics Achievement of Urban Students” Alan Spicciati, Ed.D. Seattle Pacific University, Class of 2008 spicciad@hsd401.org

2. Research Findings on the Variability of Student Achievement by Teacher • The difference between teachers one SD above and below the mean is one year’s worth of achievement (Hanushek, 1992) • Teacher effects are cumulative; three years with top vs. bottom quintile teachers opens a 54 percentile gap (Sanders & Rivers, 1996) • Rowan, Correnti, & Miller’s (2002) comprehensive study of teacher measurement methodology concluded 52%-72% of student mathematics variance lies between classrooms, with the rest between students and between schools. • A one SD increase in teacher effectiveness is equal to a reduction in class size from 25 to 15 (Nye, Konstantopoulos, & Hedges, 2004)

3. Important Findings on Teacher Characteristics Experience • Experience has a curvilinear relationship with achievement. • Achievement rises with experience for between 2 and 5 years, with “on-the-job training”, then levels off (Ferguson, 1991; Darling-Hammond, 2000; Rockoff, 2004; Rivkin, Hanushek, Kane, 2005).

4. Important Findings on Teacher Characteristics Advanced Degrees • Master’s degrees are important in mathematics and science in secondary (Goldhaber & Brewer, 1997; Wenglinsky, 2000). • Findings on advanced degrees are split for elementary. • Many studies find that advanced degrees do not relate to elementary mathematics achievement...(Hanushek, 1986; Rivkin, Hanushek, Kain, 2005; Clotfelter, Ladd, & Vigdor, 2007). • However, some reputable studies find a positive, significant relationship (Ferguson & Ladd, 1996; Greenwald, Hedges, Laine, 1996; Nye, Konstantopolous, & Hedges, 2004).

5. Important Findings on Teacher Characteristics College Selectivity • A teacher’s academic ability, particularly verbal ability, is among the most established teacher variables in relation to student achievement (Hanushek, 1986; Rice, 2003). • College selectivity, often measured by Barron’s rankings, is a proxy for academic ability that is moderately related to student achievement (Wayne & Youngs, 2003).

6. Important Findings on Teacher Characteristics Mathematics Courses • Mathematics content knowledge, as measured by tests of teachers, relates to achievement (Harbison & Hanushek, 1992; Hill, Rowan, & Ball, 2005). • Mathematics courses relate to math achievement in secondary (Monk & King, 1994). • However, Hill, Rowan, & Ball (2005) found there is little empirical evidence examining math courses and achievement at the elementary level, and their findings were not significant.

7. Definition • Teacher effectiveness. The present study is focused on “teachers,” as opposed to “teaching.” In this context, “teacher effectiveness” is defined by the mathematics achievement of a teacher’s students, as measured by growth on the Measures of Academic Progress (MAP) test, compared to expected growth. While teacher effectiveness is a term used in the literature, this will be a correlational study and will not imply effects.

8. Research Questions • In terms of descriptive statistics, what is the distribution of achievement growth at the classroom level? • Is there a significant relationship between advanced degrees, experience, college selectivity, or total mathematics courses taken at the university level and growth in mathematics achievement? • What combinations of the above teacher variables best explain the variance in student growth? • Since poor and minority communities generally attract and retain less qualified and experienced teachers than other communities, would the achievement of diverse classes be significantly higher if they had equal or even equitable access to teachers with experience and advanced degrees?

9. Participants • 3,558 students • 70.7% of all students in grades 3-6 • 84.2% of all students with complete scores, excluding self-contained classes • 156 teachers • 68.7% of all teachers in grades 3-6 • 89.7% of all eligible teachers • Required teacher variable data was located for all teachers

10. Instrument • Measures of Academic Progress (MAP) • Published by Northwest Evaluation Association (NWEA) • Computer adaptive; item response theory • Multiple choice; typically 40 items • Measures the content strands found on the math WASL • Administered fall, winter, and spring • Reliability and Validity • Test-retest reliability: r = .88 to r = .93 • Marginal reliability: r = .94 • Concurrent validity (with state tests): r = .79 to r = .89

11. Procedures • Permission granted by superintendent and SPU Institutional Review Board • Gathered existing data • MAP scores accessed in “raw” format from district database • Teacher data accessed from Human Resources • Degree database contained universities and degrees • Highly Qualified Teacher database contained record of course taking • Samples double checked against actual transcripts

12. Variables Independent Variables • Demographic • Class Percent Non White (CPNW) • School Free and Reduced Lunch Percentage (SFRL) • Class Percent of English Language Learners (CPEL) • Teacher • Experience (EXP) • Experience Dichotomized (EXPDI) • Degree (DEGR) • College Selectivity (COLL) • Number of Mathematics Courses, Content and Pedagogy (MC) • Math Courses Dichotomized (MCDI) Dependent Variable • Class Percent of Expected Growth (CPEG) • Fall to spring student level MAP growth, divided by NWEA expected (normal) growth, aggregated to class level

13. Statistical Procedures • Descriptive Statistics • Overall • Disaggregated by quartile level of diversity • Correlation • Multiple Regression • Identification of best model for this dataset • Regression equation used to estimate results with various staffing scenarios

14. Descriptive Statistics

15. Means of variables, disaggregated by class percent non-white (CPNW) quartile Performance • Least diverse quartile grew most Demographics • Poverty and ELL highly related to diversity Teachers • Low diversity classes taught by more experienced teachers • Other variables have weaker relationships

16. Scatter of Classrooms by Diversity Level and CPEG • Diversity level only explains 9% of growth • Large range of growth at every level of diversity • Many highly diverse classes outperform expected growth

17. Performance by diversity quartile, and growth quartile within diversity quartile Explanation • Each color represents a diversity quartile • Each bar represents 9 or 10 classrooms, grouped by growth, with average CPEG shown Interpretation • Top classrooms in every diversity quartile outperform the average non diverse class

18. Intercorrelation of demographic, teacher, and classroom achievement variables

19. Multivariate linear regression, preliminary/full model • Diversity explains 9% of CPEG scores • Advanced degrees and experience explain an additional approximately 9% • Experience does not significantly explain CPEG scores beyond advanced degrees

20. Multivariate linear regression, reduced model • The reduced model includes only diversity and advanced degrees • Advanced degrees explain more than 9% of variance in CPEG scores beyond what diversity explains • The model as a whole explains about 18% of the variance in CPEG scores

21. Estimated achievement based on various scenarios of teacher assignment Explanation • Using Beta weights from the multiple regression equation, achievement levels are simulated using different allocation methods Interpretation • An equitable approach could close achievement gap between Q1 and Q4 from 21% (in the status quo model) to 7% • See limitations • This approach would be more powerful with a stronger measure of teacher quality or a characteristic that varies more greatly across schools

22. Discussion • Q1: Distribution of Achievement by Classroom • 1 SD of classroom effectiveness = nearly 3 months of growth • Q2: Teacher Characteristics and Math Achievement Growth • Advanced degrees • Different findings may be attributable to small “n” of colleges, local bargaining context, or methodology that cannot link individual teachers with their characteristics. • Experience • Findings herein consistent with research. • College selectivity • Data lacks variability to show results. • Mathematics courses • Content knowledge appears to matter, based on Hill, Rowan, & Ball (2005), but coursework is a poor proxy. • Q3: Combinations of Variables • No significant interactions. • Q4: Equal or Equitable Distribution of Teachers • Simulations of this nature may be needed to encourage policy.

23. Implications for Practice • Knowing and acting on data • Disaggregating achievement data by classroom • Using responsible and ethical assessment and HR practices • Engaging in courageous conversations and leadership actions • Teacher distribution and assignment • Monitor teacher characteristics data to prevent neediest schools from having disproportionately inexperienced/less qualified teachers • Referee student assignment to avoid repeated exposure to low performing classrooms (Sanders)

24. Limitations • Methodology • Gain scores • Small student “n” size per teacher • Multiple regression vs. HLM • Internal Validity • Does measuring classrooms = measuring teachers? • Unidentified covariates • External Validity • Ability to generalize • Assumptions that teachers would perform similarly in different situations

25. Suggestions for Future Research • A multi-state study. • A study of teachers that lasted more than one year. • A study of other forms of mathematics content acquisition. • A study that includes variables for teachers who took a remedial mathematics course or who failed a mathematics course. • A qualitative study of teachers whose students significantly outperform.

26. Measuring the Link between Elementary Teachers and Student AchievementA Presentation of the Dissertation:“Elementary Teachers and the Mathematics Achievement of Urban Students” Alan Spicciati, Ed.D. Seattle Pacific University, Class of 2008 spicciad@hsd401.org