Teacher Observations and Student Learning Objectives: An Integrated Approach -

1. Teacher Observations and Student Learning Objectives: An Integrated Approach - Module 6 Mr. Fred Cohen Dr. Valerie C. D’Aguanno Dr. Robert Greenberg Mrs. Laverne Mitchell

2. 9 Required Elements of 30-2.9 for Lead Evaluator Certification: • NYS Teaching Standards and the ISLLC, 2008 Leadership Standards • Evidence-based observation techniques • Application and use of the student growth and value-added growth model • Application and use of State-approved teacher/principal rubrics • Application and use of any assessment tools you intend to use (e.g., portfolios, surveys, goals) • Application and use of any State-approved locally developed measures of student achievement you intend to use • Use of the Statewide Instructional Reporting System • The scoring methodology used by the department and/or your district • Specific considerations in evaluating teachers and principals of English language learners and students with disabilities • *** Other: While not listed as a training component, districts will need to ensure inter-rater reliability for both their teacher and principal evaluation systems.

3. Instructional Planning-Delivery-Reflection move the cogs in a 100% system

4. Module 6 Training Objectives • Understand the connections between teacher evaluation and student learning outcomes • Advance our common language around teacher evaluation and student learning outcomes • Understand expectations for evidence collection and assessment of skills for teacher evaluation • Understand how to interpret evidence against criteria to assess teacher practice • Gain direct experience with and learning from crafting an SLO • Collaborate with colleagues • [Required Elements 1, 2, 3, 4, 5, 6, 9, IRR]

5. Working Agreement • Be present: minds and hands on all day • Respect time boundaries • Recognize the need for quiet while working • Use electronics respectfully and appropriately when prompted • Return to large group attention when signaled

6. TLS Evidence Rubric Element 2

7. OAR(S) Objective Aligned Representative Sufficient

8. Evidence is a Factual Reporting of EventsElement 2 • It includes teacher and student actions and/or behaviors • Teacher presented the content from the front of room. • It includes statements made and questions posed by the teacher and the students • “Bring your white boards, markers and erasers to the carpet and sit on your square.” • It includes artifacts prepared by the teacher, students, or others. • Task cards, journals, lesson plans, etc.] • It includes quantifiable information about time, student participation, resource use, etc. • 9:14 – 9:29- Warm-up. 8 of 22 Ss finished at 9:20, sat still until 9:29 • It includes an observed aspect of the environment • Desks were arranged in groups of four

9. Trends in January Evidence Samples • A preponderance of the submitted evidence samples contained • Summary statements • “Provided relevant examples and scaffolding to discuss how we "measure your foot" or "blood pressure" driving home the point that we need "different tools for different problems" In this way the Clinometers and meter stick were introduced before directions were given for outside task.” • Conclusions/judgments • “Students were engaged during small group activities.” • A lack student evidence • T: Who has a prediction? • T: What is your method? • T: Why did you use tangent?

10. Trends in January Evidence Samples • Vague information • “Teacher circulates” • Misalignment to the criteria

11. Evidence Exemplars From the Field • Review the Evidence Exemplars • Use OAR(S) to examine and review the exemplar. • Discuss with your table mates.

12. Evidence Analysis • With an elbow partner examine the evidence collection activity sheet • Identify the reason why the evidence is problematic • E.g., “teacher circulates” is vague because it does not provide a context. Is the teacher monitoring student understanding or student behavior? Does the teacher talk with students? If so, what is the conversation? How can an evaluator determine the level of effectiveness of the teacher’s walking among the students? • Be prepared to share your work with the larger group

13. Expectations for Demonstrating IRR before Turn-Key Training “Other” expectation: IRR • Achieving inter-rater reliability is a process… • GOAL: to demonstrate acceptable IRR after calibration • March/April: First official assessment of the observation process, receive feedback on three areas of evidence and trends in accuracy • April/May: Demonstrate proficiency in the 3 evidence areas [objectivity, alignment, representativeness], receive feedback on evidence and trends in accuracy • May/June: Demonstrate proficiency in the 3 evidence areas and in at least 2 of the 3 accuracy measures

14. Training Platform • For training purposes we will be using a platform developed by True North Logic • Please log on to the following site: • http://engage.truenorthlogic.com

15. Log-In Enter your user name Click to Log-In Enter your password

16. Welcome to the Home Page Select “Observations:”

17. Starting the Observation Select the rubric Select the teacher

18. Observation Practice: Pre-ObservationElements 1, 2, 4, 6, 9, IRR • Individually • Review the lesson plan and pre-conference notes for Ms. Rios • Align the evidence to the appropriate elements/indicators [smallest grain size] • Check your work with a partner who is using the same rubric • Did you align your evidence to the same element/indicator? • Discuss your rationale for your alignment • Be prepared to share some examples of your aligned evidence collection

19. Observation Practice: Classroom ObservationElements 1, 2, 4, 6, 9, IRR • Individually • Watch the video of Ms Rios • Collect evidence for the observable elements/indicators of your rubric • Align your evidence to the appropriate elements/indicators • Check your work with a partner who is using the same rubric • Did you capture similar evidence? • Did you align your evidence to the same elements/indicators? • Discuss your rationale for your alignment. • Be prepared to share some examples of your aligned evidence collection

20. Video • Bright/Rios

21. Observation Practice: Post ObservationElements 1, 2, 4, 6, 9, IRR • Individually • Review the teacher’s reflection and post-observation notes • Align the evidence to the appropriate elements/indicators • Check your work with a partner who is using the same rubric • Did you align the evidence to the same elements/indicators? • Discuss your rational for your alignment • Be prepared to share some examples of your aligned evidence collection

22. Observation Practice: ScoringElements 1, 2, 4, 6, 9, IRR • Individually • Score the teacher’s practice at the element/indicator level for • NYSUT Standards III and IV • ASCD Standards 2 and 3 • Be sure your judgments are based on your aligned evidence • Reference the language of the rubric to determine the level of performance • Be sure to Save Changes frequently and at the end • Once you finish scoring and exit the True North Logic site, your evidence and scores will be reviewed by the TLS evaluators.

23. SLO Information • 100-Point Evaluation System • Learning Content • Evidence • Student Population, Baseline, and Target(s) • HEDI • Interval and Rationale • Participant SLO Sharing • NY SLO Development Guide • Why SLOs? • Assignment, Survey, and Questions

24. Multiple Hats for the Day Throughout the day, we are engaging in the work at multiple levels: • Learner: What is the content and rationale for an SLO? • Teacher: How does this work support and impact teachers? • District: What decisions need to be made and by whom? • Trainer: How will I convey the importance and content of this work to districts, schools, and other stakeholder groups?

25. 100-Point Evaluation System for Teachers Student Learning Objectives (focus of this training)

26. 5 District Decisions by 5/30/2012 • Assess and identify priorities and academic needs. • Identify who will have State-provided growth measures and who must have SLOs as “comparable growth measures.” • Determine District rules for how specific SLOs will get set. • Establish expectations for scoring SLOs and for determining teacher ratings for the growth component. • Determine District-wide processes for setting, reviewing, and assessing SLOs in schools. 3/01 4/16 5/30 From NY DOE engageny.org Introduction to SLO Deck

27. Discussing District Decisions • Decision 1: Assess and identify district priorities and academic needs. • Decision 5: Determine district-wide processes for setting, reviewing, and assessing SLOs in schools. • Recognize how district priorities and academic needs can inform learning content selections. • Determine at what level your district(s) should consider resting the decision for learning content selections (district, school, teacher, etc.).

28. 100-Point Evaluation System: State 20% Three types of teachers:

29. Discussing District Decisions • Decision 2: Identify who will have State-provided growth measures and who must have SLOs as “comparable growth measures.” • Using the three types of teachers, discuss who will have SLOs as “comparable growth measures.” • Refer to the SLO materials provided on EngageNY.org as needed. • Policy/State questions: slohelp@mail.nysed.gov

30. NYSED SLO Framework • A student learning objective is an academic goal for a teacher’s students that is set at the start of a course. • It represents the most important learning for the year (or, semester, where applicable). • It must be specific and measurable, based on available prior student learning data, and aligned to Common Core, State, or national standards, as well as any other school and district priorities. • Teachers’ scores are based upon the degree to which their goals were attained. • Source: Page 4 of Guidance on the New York State District-Wide • Growth Goal Setting Process: Student Learning Objectives

31. NYSED SLO Framework

32. Student Population and Learning Content Team Activity: (http://www.p12.nysed.gov/ciai/cores.html) As a team, discuss and select: • Student Population – • These are the students included in the SLO. • Learning Content – • a particular course name • the course’s source of standards (Common Core, etc.) • the exact items (indicators, standards, etc.) to be taught, learned, and assessed.

33. Learning Content This is the content to be taught in the SLO. What is being taught? CCSS/National/State standards? Will this goal apply to all standards applicable to a course or just to specific priority standards? Task: Identify the course name and source of standards (Common Core, national, state, local) associated with this SLO, and specify the exact standards, performance indicators, etc., that will be taught, learned, and assessed.

34. Learning Content – Training SLO • Training SLO Context • Course: 7th Grade Visual Arts • Standards Source: New York State Learning Standards • Standard 1 – Creating, Performing, and Participating in the Arts • Standard 3 – Responding to and Analyzing Works of Art. • Standard 4 – Understanding the Cultural Dimensions and Contributions of the Arts • Performance • Indicators: 1c, 1d, 3a, 3c, 4c • Common Core Writing Standards for Literacy in …Technical Subjects: WHST.6-8.4, WHST.6-8.8

35. Learning Content – Training SLO (1c) [Students] use the elements and principles of art to communicate specific meanings to others in their art work. (1d) During the creative process [students ] reflect on the effectiveness of selected mediums or techniques to convey intended meanings. (3a) [Students] discuss and write their analyses and interpretations of their own works of art and the art of others, using appropriate critical language. (3c) [Students] compare the ways ideas and concepts are communicated through visual art with the various ways that those ideas and concepts are manifested in other art forms.

36. Learning Content – Training SLO • (4c) [Students] create art works that reflect a particular historical period of a culture. • (WHST.6-8.4) [Students] produce clear and coherent writing in which the development, organization, and style are appropriate to task, purpose, and audience. • (WHST.6-8.8) [Students] gather relevant information from multiple print and digital sources, using search terms effectively; assess the credibility and accuracy of each source; and quote or paraphrase the data and conclusions of others while avoiding plagiarism and following a standard format for citation.

37. Interval of Instructional Time • Interval of Instructional Time • This is the timeframe within which the learning content will be taught. (This is generally one academic year unless the course is set as a semester, quarter, etc.) • Interval of Instructional Time Task: • Specify when the teaching for this learning content will begin and end. (Rationale is required if less than the typical year-long interval is set.)

40. Discussing District Decisions • Decision 3: Determine district rules for how specific SLOs will get set. • Review the NYSED Assessment Decision Chart to assist in planning. • Decide on assessments to be used with SLOs.

41. Evidence These are the assessments used for determining students’ levels of learning. Task: As a team discuss what pre- and summative assessments best measure the selected learning content for the course and standards/performance indicators you selected. • (If none is available for your learning content, describe the characteristics of a high-quality and valid assessment for the identified learning content, how it would be developed, by whom, when, etc.?) (Districts and BOCES must verify comparability and rigor for any assessments they develop.)

42. Evidence – Training SLO (Summative Rubric) • Note on Rubric: Each student’s work will be scored by two other district middle school visual arts teachers using the following rubric, with both teachers’ scores being averaged to yield the final student scores.

43. Evidence – Training SLO (Summative Rubric)

44. Evidence – Training SLO (Summative Rubric)

45. Evidence – Training SLO (Summative Rubric)

46. Discussing District Decisions • Decision 4: Establish expectations for scoring SLOs and for determining teacher ratings for the growth component. • Decision 5: Determine district-wide processes for setting, reviewing, and assessing SLOs in schools. • Discuss district implications for establishing baselines and setting targets for the student population. (Considerations include common target expectations, where decisions for these selections will rest, etc.)

47. Baseline and Target(s) Baseline This is the level of students’ knowledge and skill in the targeted learning content at the beginning of the interval of instructional time. Target(s) This is the level of knowledge and skill that students are expected to achieve at the end point of the interval of instructional time.

48. Baseline and Target(s) Baseline: Describe how students performed on the identified pre-assessment(s) for the learning content. (Actual baseline scores for each student are required.) Target(s): Define numerical growth goals for student performance on identified summative assessment(s) which measure student knowledge and skill in the learning content. (Actual final scores for each student are required.)

49. Student Population, Baseline and Target(s) Training SLO Student Population: All 105 students across 4 sections of 7th grade visual arts classes (names included on the Excel file) Training SLO Baseline: Students scored between 0-30% on the grade 7 visual arts pre-assessment

50. Student Population, Baseline, and Target(s) • Training SLO Target Approach 1: Set a common growth target. • 90% of students, including special populations, will grow by 60 percentage points or more on their summative assessment compared to their pre-test for the standards. (e.g., Student E’s target is 60 more than 30, or 90.)