TEACHER ANALYSIS AND USE OF INTERIM ASSESSMENT INFORMATION IN MATHEMATICS

1. TEACHER ANALYSIS AND USE OF INTERIM ASSESSMENT INFORMATION IN MATHEMATICS A Presentation for the Northern California Educational Leadership Research Symposium February 15, 2014 Kathy Pon, Ed. D.

2. Introduction Use of student assessment data has been heralded to improve instruction and student achievement… BUT Teachers remain challenged when it comes to linking data to actions that improve student performance and schools!

3. Purpose of Study This study’s purpose was to deepen our knowledge about • interim mathematics assessments ability to predict student performance on high stakes tests • how assessment results* are used to modify or improve instruction through pedagogical and curricular conversations of teachers • the elements of conversations and behaviors related to instructional decisions that inform and extend current data-informed decision-making theory. *Analysis of this data can lead to the creation of “actionable knowledge” that is, decisions and actions by teachers that modify and improve instruction for students (Halverson, 2010).

4. District A – Contributions of Interim Assessments to CST Math Scores By Grade Level

5. District B – Contributions of Interim Assessments to CST Math Scores By Grade Level

6. Quantitative Findings Using a Regression Analysis Districts A and B respectively, both had significant and strong, positive correlations to student performance on the California State Tests for grades 4-6. The results show that for grades 7-10, student performance in both assessments had significant and moderate, positive correlations to student performance to student performance on the California State Tests. • These results support findings from Clune and White (2008) suggesting that there may be a strong alignment between the taught and tested curriculum. This alignment is evidenced in relationship between benchmark scores and CSTs at the elementary levels, but less so, as both districts’ students progress into higher math courses. The assessment tools at the secondary level appeared to be less useful both districts in predicting student performance on CSTs, and as such, for instructional purposes.

7. Qualitative Component of Study Focused on Behaviors and Actions that Support Data Informed Decision Making

8. Weighting of Data Analysis Behaviors

9. Five Areas Impacting Analysis and Use of Data

10. Assessment (Item) Selection Phase Highly reflective conversations- Algebra teachers discussing test items whereby students are not just identifying or regurgitating the rules of exponents, but instead must inductively come up with a rule after seeing exponents expanded. Conversations lacking depth whereby items did not support the conceptual teaching of mathematics – Algebra teachers examining a multiple choice question adding negative exponents, and discuss a choral strategy they use that the “fraction on the bottom is the root and roots grow down, so the number on the top raises the power”.

11. Analysis Phase What contributes to higher levels of analysis: • Cycle of Inquiry • Protocols • Coaches who support and especially model data results and implications to instruction through probing questions

12. Supporting High Levels of Analysis “The math coach from the District Office sat in on meetings and helped me structure questions, and gave me feedback afterwards. I was trying not to directly point it out, but if no one else in the meeting noticed, I’d step in with “What do you see in number 1? They were more about leading questions. I needed to learn to lead the discussion. Those Cycle of Inquiry discussions were where we could talk in depth about strategies, not just the percent of kids who did well on questions, but about the actual teaching. That’s the part that’s the most helpful, to get to the place where you’re talking about strategies.”

13. Reflection Phase • In almost all instances, pedagogical discussions addressed the teaching of procedures. • Evidence of need to identify “formulas” to ensure students had a higher probability of solving word problems. • Reflection related to teachers’ capacity for discussing pedagogy as it related to the teaching of mathematical concepts.

14. Planning Phase • High levels = Teachers placing students in an intervention group and re-teaching in a different manner: “With algebra last year, we saw kids weren’t doing well. We put interventions into place and then measured if they made a difference…” • Basic levels = Teachers discussing how to support English Learners who weren’t showing results using a “checking for understanding” strategy or teaching academic vocabulary for the “bubble kids”. • Low levels (Predominantly used) = Teachers agreeing to review items in a warm up activity the following day or again before the next test.

15. Leadership Practices • Evidence of “defined autonomy” in leadership (Marzano & Waters, 2009) - the presence of central office systems of data collection, but site discretion about how to support data-informed decision-making. • District A’s co-occurrences of “Cycle of Inquiry” (COI) process with “strategic district leadership” were weighted as “3s”

16. The Presence and Power of Distributed Leadership Leaders who link visions of continuous improvement to data use and who engage others to use the data to collaboratively make decisions (Goodnow&Wayman, 2009) are “transformational”. This was evidenced in both districts by site and district level administrators. • Setting expectations • Release time • Autonomy to make decisions and purchase needed resources • Culture of data use

17. Conclusions Related to Mathematics Instruction from Qualitative Findings: There are critical knowledge bases for practitioners in mathematics: • content knowledge and understanding of items as they represent mathematical concepts; • an understanding of analysis processes that best facilitate deep analysis and understanding of student thinking; • knowledge of protocols that support reflection, especially on how to instructionally mediate misconceptions; • Ideas about how to best to modify instruction as needed.

18. Overall, What Does the Evidence Suggest about Assessment Practices in Mathematics Instruction? • Quantitative evidence suggests that interim assessments do play a part in increasing student achievement. • Qualitative evidence suggests that the teacher analysis of data resulting from these assessments has contributed to an increase in the alignment of taught and tested curriculum, more than to improved mathematics instruction. • There is not strong evidence in the qualitative results that data analysis, reflections or decisions made in the data-informed decision-making model result in stronger mathematics lessons, modifications to instruction that address student misunderstandings, concept development or an increased ability of students to apply skills.

19. Implications • In the end, the administration and analysis of interim assessments provides frequent a measurement tool which fulfills the “jump higher theory of improvement”, that is, if educators measure student performance enough, and do nothing differently pedagogically, then students will surely increase their scores (Elmore, R., personal communication, May 19, 2012).

20. Recommendations for Practitioners: • Deepen content knowledge in mathematics, especially for multiple subject teachers; • Construct interim assessments using better items: (enhanced multiple choice, constructed response and performance tasks) that inform instruction; • Use inquiry protocols that help teachers analyze students’ learning levels and misconceptions; • Provide structures that support teachers in the collection, reflection and connection of student learning data to their own teaching and delivery of instruction; • Identify how teachers can build re-engagement lessons and offer better support for students after a lesson; • Continue to support professional learning with distributed leadership structures that enable practitioners to do engage in this work.

21. Going Forward Additional research should identify assessment and data analysis practices for the tasks provided by the Smarter Balanced Assessment Consortium for mathematics. Additionally, research that explores those protocols that best elicit conversations that promote deeper conversations about pedagogy and instructional decisionssupporting Common Core will be invaluable to the field. The Predictive and Instructional Value of Interim Assessments Author(s): Pon, Kathleen Date: 2013 February Type: Dissertation URI: http://scholarworks.csustan.edu/handle/011235813/110