Purpose & Context

2. Purpose & Context • This work, particularly around MoSL, is very nuanced and complex. There are places where some policy decisions are still being made. • Throughout the day we want you to capture your reflections and thinking to help you process and distinguish between what will become feedback to the DOE and what you need to move the work forward in your schools. • The goal for today is to understand, so that you are ready to lead the work in your schools. • The day is packed with content, but we will be making connections throughout the day and the institute so that you can see how to pull it all together for your schools.

3. DOE Resources and Training • The following resources will assist principals and the School Local Measures Committee throughout the decision-making process and implementation: • Measures of Student Learning Guide: You can find more information on assessment and measurement options, as well as sample selections in the Guide, found at http://www.learndoe.org/dhr/recording-mosl-overview/. • Interactive Selection Tool: The DOE will release a tool to select assessments and measurements in August. • MOSL Specialists: Experts to help networks and schools implement in the fall. • Additional training opportunities: • MOSL Specialists will be providing additional support to networks, who will in turn support schools, as the school year begins.

4. MoSL Options Template • Required assessments are checked and shaded in gray. • State Measures decisions made by principal are color-coded in orange and Local Measures decisions made by the school committee are color-coded in blue (same as MoSL guides). • For State Measures, teachers who teach both ELA & math must have an ELA and math component for their State Measure, but not their Local Measure. The Local Measure can be any subject they teach. For schools that departmentalize, teachers are only responsible for the content they teach. • Pending policy decisions, for Local Measures, school committees may select up to 2 measures per group of similar teachers, i.e. same subject and grade. (2 measures = 1 assessment with 2 populations, or 2 assessments) • For example: • 1 assessment, 2 populations: ELA state exam – school and school lowest third • 2 assessments: ELA state exam – school, ELA performance tasks – individual

5. MoSL: Measures of Student Learning (40%) • All teachers will receive: • Two different Measures of Student Learning (40%) • State Measures • Local Measures • Initial planning conference and summative end of year conference to include discussion of student outcomes (Will be covered on Aug. 8th) • Multiple measures provide a more valid, robust picture of teacher performance, providing teachers with multiple sources of feedback

6. State and Local Measures Although principals make decisions about State Measures and the School Local Measures Committee makes decisions about Local Measures, these two decisions go hand in hand. • There are many reasons why it is important for the two measures to be determined in collaboration: • Teachers will receive the best picture of student growth with multiple measures, each telling a slightly different story of student performance. • For example:Principal reinforces a focus on school-wide learning by selecting a school-wide measure on a State Assessment for a given group of teachers’ State Measure. School Local Measures Committee then decides to base Local Measure on individual teacher performance on a 3rd party assessment (where possible) to include a variety of assessment perspectives. • Per law, teachers may not have the same exact measure for both State and Local (e.g., 4th grade teacher cannot have an individual classroom growth measure on ELA/Math state tests count for 40% of overall rating. Her Local Measure would have to measure another population either the lowest 1/3 or group measures such as school or grade). • The more alike the State and Local Measures are to each other, the more likely a teacher will receive the same rating on each. This may have positive or negative consequences.

7. Long-term Vision MORE OPTIONS LEARN WITH EXPERIENCE • Schools re-select options each year, learning with experience • More assessment options each year

8. Terminology – Common Mix Ups • State assessments can be used for both state and local measures. When we are talking about state tests, it doesn’t always mean we are talking about state measures. • The state made decisions about requirements on both the state and local side. Principals have a role in both as well. • Local assessments – NYC performance assessments – are required for different grades on both the state and local side. Therefore, when we talk about local assessments it does not always mean we are talking about local measures.

9. Local State State ELA Grade 4/5

10. Terminology – Common Mix Ups For each measure, the following three items need to be considered: • Even though there are only 3 main decisions that need to be made for each measure, decisions impact each other and all options are not available in all cases.

11. NYC Assessment Options • Information on assessments available for specific grades/subjects is available in the Measures of Student Learning Guides • (http://www.learndoe.org/dhr/recording-mosl-overview/). • The following parameters guided the selection of assessments available for each grade and subject: • State rules regarding allowable assessments: SED released rules about what assessments are allowed in each grade and subject, including specific rules about 3rd Party and NYC Performance Assessments. • Assessment quality: NYC assessment options were evaluated to ensure instructional value, validity and reliability. • Security and comparability: Assessments must be available across all NYC schools and students cannot have access to them prior to administration. School-created assessments are not allowed for outcome assessments.* • Existing practice in schools: Whenever possible, NYC assessment options build upon existing assessment practices in NYC schools. *Where baseline selections need to be made, schools may use school-based/created assessments as baselines. Schools may NOT use school-based/created assessments for final outcomes.

12. NYC Assessment Options  NYC • Only state-approved 3rd Party Assessments can be used. Not all NYC assessments are on this list. A final list will be approved on August 1. **NYC is not using all 3rd Party Assessments that were previously approved for use throughout the state.** NYCDOE is funding 3rd Party Assessments used for teacher evaluation. • 3rd Party & NYC Performance Assessments: • Include both pre- and post-tests. • Teachers (per law) can score their own baselines/pre-tests, but CANNOT score their own post-test results.

13. Activity: Preview NYC Performance Task • Review the NYC Performance Task and aligned Rubric sample. • Use the template to guide your reflections and thinking by considering the following prompts: • 3 key points you learned from the NYC Performance Task and Rubric. • 2 questions or implications of the assessment. • 1 message to present to school/local measures committee.

14. Baseline Assessments • A baseline assessment is one given in the beginning of the year to assess student starting points. • NYC Performance Assessments and approved 3rd Party Assessments come with baseline (pre) assessments. • The principal must select a baseline assessment when none is included or where there is a State Assessment without historical data (e.g., 3rd grade). • There are only a few instances where baselines will need to be selected for Local Measures. Where this is the case, the principal, not the committee, selects the baseline. • Schools that select school-based/created baselines must use goal setting for that measure.

15. Target Population Options • Where are they available? • All 3 options – Individual, Grade, and School – are available for the state tests for testing grade teachers. For non-testing grades teachers, the only options available for state tests are grade and school. • The only population available for NYC Performance Assessments is individual. • For 3rd party assessments, the only options are individual and school (this does not mean the assessment must be administered to all students in the school, it only means that the teacher will be evaluated based on everyone in the school who took the assessment). • Nuances: • In some cases these distinctions overlap. For example, for a 4th grade science cluster teacher, her individual student population will also be grade and school if she teachers the entire 4th grade. • Requirements: • For required State and Local Measures, the target population will always be the individual. • Teachers who do not give approved individual classroom-level assessments will use either a grade- or a school-wide measure. • If an assessment is repeated across measures, the target population must differ. If a school wants to use the same target population with the assessment, the Local Measures must focus on the lowest third of performers. You may only select the lowest 1/3 if the larger population was already selected as a measure.

16. Measurement: Growth or Goal Setting

17. Measurement: Growth Model • Calculated by the state for 4-8 ELA & math; calculated by the city for all other assessments including 3rd grade ELA & math, science, NYSESLAT, regents, etc. • Puts students in buckets according to 13 characteristics. Similar students are compared to each other to determine SGP. • Some characteristics describe the student and some describe the classroom. • Controls for similar characteristics as the Progress Report, but in different ways. • The Progress Report controls for classroom characteristics in their peer grouping and comparison. For growth scores, classroom characteristics are embedded in the student comparison “buckets”. • The Progress Report accounts for student characteristics by adding weights to student scores, For growth scores, student characteristics are accounted for by putting students in buckets to compare them only to similar students.

18. Measurement: Growth Model

19. Activity: Growth Model Understanding Check • Turn and Talk to your colleagues and discuss the two scenarios: • A teacher says to you: “I appreciate that the new evaluation system aims to measure growth, but my kids face so many challenges and just don’t grow as fast as other kids. I feel like I’m going to be disadvantaged by this new system.” In 2 minutes or less, explain how using a growth model will address her concern. • A teacher says to you: “I mean, we get these numbers called Student Growth Percentiles (SGPs), and we have no idea what they refer to. If Jose’s SGP is 65, does that mean he got 65% right on the test?” In one minute or less, explain the misconception this teacher has about SGPs, and what this number actually refers to.

20. MGP Lower Limit Upper Limit Confidence Range MGPs and Statistical Confidence 87 • NYSED will provide a 95% confidence range, meaning we can be 95% confident that an educator’s “true” MGP lies within that range. Upper and lower limits of MGPs will also be provided. • An educator’s confidence range depends on a number of factors, including the number of student scores in their MGP and the variability of student performance in the classroom.

21. State Growth Model: Scoring and Results Most teachers scored Effective on the state growth score in 2011-12. The results for NYC were slightly higher than the rest of the state: Note: SED has stated that it expects the future distribution of ratings to remain similar to what it was in 2011-12.

22. Goal-Setting Process Baseline assessment administered (not required for all assessments). Report baseline assessment results. • Administer baseline assessment • DOE sends predicted student targets • Teachers review DOE predicted targets • Principals approve or adjust targets • Administer end of year assessment • Teachers’ Ratings DOE sends predictions for how individual students will perform. Predictions are based on baseline performance, student achievement history, and student demographic characteristics. Teachers may choose to adjust these targets based on additional information about their students. Teachers submit student targets to principal. Principal (or designee) report finalized student targets. End-of-year assessment administered to students. Teachers’ HEDI ratings are calculated with a conversion chart based on students’ performance on outcome assessments relative to their targets.

23. Goal-Setting Considerations • This is not the same as the goal-setting you may typically see in schools. • Goals are scored against a state conversion chart which makes the target-setting process difficult and non-intuitive. • Before considering goal-setting, make sure you understand the work and additional training this entails.

24. 50% Rule • All teachers who have individual assessments (assessments for their students and subject) available must have State Measures that cover at least 50% of their students. Therefore, you may need to add additional measures to get to 50%. • Begin by accounting for courses that end in state exams that result in state growth scores (4-8 ELA & Math) • Next, add courses that end in state exams but do not lead to state growth scores. You must begin with the largest (highest enrollment) courses first. • Finally, add additional courses beginning with largest (highest enrollment) courses first. • 50% of students covered is the minimum, where principals have choice, you may choose to cover a larger percentage.

26. DEFAULT • Default only affects the Local Measure and applies for the entire school. • When you default, all local measure decisions and required assessments (ELA NYC performance assessments) are eliminated and replaced with the default. • The local measure default is: K-5 • SED provided growth scores on 4-5 ELA & Math state tests •  Lowest 1/3 growth scores for teachers who have school-wide measures for their State Measure K-8 Schools • SED provided growth scores on 4-8 ELA & Math state tests •  Lowest 1/3 growth scores for teachers who have school-wide measures for their State Measure K-3 & K-2 Schools • An average of all measures on the State side. • Defaulting DOES NOT affect the State Measure. Principal’s are still required to make decisions highlighted in orange. All required assessments for the State Measure are still required.

27. DEFAULT - Nuances • Pending upcoming policy decisions  • If a committee does not make a decision for a particular grade and subject and it leaves it blank, the default option is automatically filled in and eliminates the required assessments. • For grades without required Local Measure options, the committee may select the default options. • If the committee selects the default options for a grade with required assessments, that means they are doing both the required assessments and the default.

28. Activity: Committee Decision Check • Review the committee’s choices for assessment and target population for errors. • Circle any selections that are not allowable within the guidelines. • Use your graphic organizer to note the error, why it is an error (reference the rule), and what suggestions you have for making changes. • Review the committee’s choices for assessment and target population for broader implications. • Draw a box around any selections that ARE allowable, but might have broader implications that are worth discussing before selections are finalized. • Use your second graphic organizer to note the selection and what broader implications this selection might have.

29. Activity: Problem Solving Scenarios – Part 1 • Read and select 1-2 scenarios to discuss with your colleagues. Use the template to guide your conversation. For each scenario: • Describe the problem • Indicate the problem type (technical, cultural, contextual, indecision) • Brainstorm possible problem solving strategies • Identify 3 next steps for the school

30. Activity: State Measures Decisions • Practice using the MoSL Options Template to make your decisions for state measures for your teachers. • Use your Table of Organization: • First, cross off any Pre-K teachers and teachers who do not teach 40% of the time. Make a note that these teachers will be rated S or U. • Second, write the names of your teachers in the appropriate place in the left column of the MoSL Options Template. • Exception: For teachers who do not have 50% or more of their students in a particular grade and subject, make a note to include them on the last page of the template. • Check off the decisions you have made or are considering for your teachers. • Discuss your thinking and reasoning with your colleagues.

31. Activity: Connecting MoSL - Elevator Speech – Take 1 • More than ever, the success of the school will be dependent on the clarity of school leaders vision and communication. In light of the new Principal Evaluation System, school leaders ability to clearly and concisely voice their “theory of action” will largely impact their Measures of Leadership Practice. • Over the course of the Summer Institute and throughout the year we will revisit the idea of the elevator speech as a way of working through how to best articulate the work we do. • Now, as a way to synthesize our thinking around MoSL and next steps for our school we will begin to draft our elevator speech regarding assessment in our school. • Use the elevator speech template to draft your thinking around the following prompts: • Goal & Action Steps: We are working on…by…. • Data & Purpose: because…so that….. • Monitoring & Next Steps: The evidence we are looking for….based on this data our next steps will be

32. Activity: Connecting MoSL - Considering our Community • Even though we may hold particular beliefs and a vision for assessment in our schools, depending on how your committee and teachers are processing this work, it may make sense to modify our decisions to honor/account for their perspective. • Turn and Talk to your colleagues and share: • How your teachers are reacting to or may react to this work • Does that align or not with where you are in this work • How might that impact your final decisions

33. Activity: Connecting MoSL - Elevator Speech - Take 2 • Based on your considerations for your community, revise your elevator speech in “take 2” on the template. • How might this message impact your… • Meetings with your MoSL Committee • September faculty conferences • Initial planning conferences • Turn, Talk & Share with your colleagues.

34. Activity: Problem Solving Scenarios – Part 2 • Use the Consultancy Protocol to discuss a challenge you are having at one of your schools. • Protocol overview: • Presentation: Presenter provides overview of work and 1-3 questions (3 minutes) • Clarifying Questions: Colleagues ask presenter clarifying questions (3 minutes) • Probing Questions: Colleagues ask presenter probing questions (3 minutes) • Non-presenter Discussion: Colleagues discuss the presenters work and questions without the presenter (5 minutes) • Presenter Response and Open Discussion: Presenter responds and discusses work with colleagues (5 minutes) • Reflection/Debrief: Group reflects on the process (5 minutes)