1 / 30

Measuring Student Growth in the Classroom Together Educating America’s Children July 12, 2011

Measuring Student Growth in the Classroom Together Educating America’s Children July 12, 2011. This workshop will explore what it means to incorporate student learning measures into a teacher development and evaluation system. Through NYSUT’s Innovation Initiative

africa
Download Presentation

Measuring Student Growth in the Classroom Together Educating America’s Children July 12, 2011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring Student Growth in the ClassroomTogether Educating America’s Children July 12, 2011 This workshop will explore what it means to incorporate student learning measures into a teacher development and evaluation system. Through NYSUT’s Innovation Initiative five locals are identifying locally developed measures that will be part of their school’s teacher development and evaluation system.  Presenter:Ellen Sullivan, NYSUT Educational Services

  2. Measuring Student Growth in the Classroom Today’s presentation will address: • Setting the context: New evaluation regulations for New York State • Taking the lead: NYSUT Innovation Initiative • Understanding the work : Local student growth measures • What we learned: District/Teacher perspective • Challenges and Considerations

  3. Context for Developing New York State’s Evaluation System • In New York state, the Regent’s modified the annual professional performance review (APPR) through regulation, in an effort to improve the state’s RTTT application. • NYSUT leveraged the limited timeline imposed by the RTTT application to negotiate a new evaluation system focused on professional growth that ensures student test scores will not be the sole determinant of teacher effectiveness. • The parameters for the new evaluation system are established through section 3012-c of Education law. (Chapter 103 of the Laws of 2010.)

  4. Chapter 103 Laws of 2010Summary of Provisions of 3012-C • APPR – the law builds on the current APPR regulations to include: measures of student achievement; a 100 pt composite score based on multiple measures; four rating categories; and, requires that performance reviews be a significant factor in key employment decisions (tenure, retention, termination sup compensation) and professional support. • Teacher/Principal Improvement Plans (TIP) – teachers rated as developing and ineffective must be given a clearly defined improvement plan that includes a timeline, how improvements will be assessed and individualized professional development. • Expedited 3020a - requires a locally established appeals procedure for evaluations and that hearings be conducted within a specific timeframe, with limited adjournments, before a single hearing officer.

  5. APPR Composite Score 100 points Phase-In Begins in 2011-2012 School Year. • 60 points on multiple measures of professional practice –locally developed APPR for NYS Teaching Standards using multiple measures • 40 points multiple measures of student achievement • 20 points on student growth on state assessments or a comparable measure • 20 points on locally selected measures determined to be rigorous and comparable across classrooms, as defined by regulations • (Approx 30% of teachers use state assessments) The Other 69% Center for Educator Compensation Reform (Prince et al.2009

  6. NYSUT’s Innovation InitiativeA collaborative labor-management approach to improving teacher effectiveness and student learning through a comprehensive system of teacher evaluation and professional development. NYSUT work on developing teacher evaluation to improve instruction began long before RTTT. • NYSUT received AFT grant for the development of a comprehensive teacher evaluation program that is based on: • Clear and comprehensive teaching standards • Accurate and effective evaluation process • Targeted professional support, including peer assistance and review

  7. Innovation Initiative: Key components • Labor/management collaboration • Teaching standards • Multiple measures of teacher effectiveness • Inclusion of student achievement data in teacher evaluation • Trained evaluators • Growth model linking professional development to evaluation

  8. Innovation Initiative: Labor-management teams • Research on collaborative decision making around teacher evaluation • Five labor/management design teams from Albany, Hempstead, Marlboro, North Syracuse, and Plattsburgh • NYSUT leading the way on developing process, practice, and protocols that are inclusive and respectful of the knowledge and experience of both teachers and administrators

  9. Student growth means the change in student achievement for an individual student between two or more points in time. Door Frame – to measure child height/same date every year Pediatrician Visit – percentile of growth as compared to other children in same age band Definition of Student Growth(Race To The Top)

  10. NYSUT Innovation InitiativeI3 AFT Educator Evaluation for Excellence in Teaching and Learning Pilot (E3TL) • Innovation Initiative design team members played leading role in the development of New York State Teaching Standards adopted January 11, 2011: • Innovation Initiative developed the NYSUT Teacher Practice Rubric aligned with the state standards, and with NYS APPR regulations, describes levels of performance (highly effective, effective, developing, ineffective), and works for observation and other measures of teacher effectiveness • Innovation Initiative has developed a Comprehensive Teacher Evaluation and Development System using multiple measures for teacher observation evaluation • E3TL Consortium (NYS and Rhode Island) piloting the evaluation system over the next four years

  11. Comprehensive Teacher Evaluation and Development System • working toward a complete and fully developed evaluation system, • design considerations at the district level • ensures alignment across all of the component parts of the system • comprehensive, serving all classroom educators • available for any district to choose

  12. Innovation Initiative: Inclusion of student learning data in teacher evaluation • Innovation Initiative working to develop process for developing measures of student achievement, exemplars of measures across subject areas and grade levels, and a template for local use in developing measures of student achievement • Begin by reviewing current practice in district • What assessments are available already? • How are they administered? • Is the superintendent likely to certify the assessment as rigorous and comparable

  13. Teachers are the experts and they already measure student learning progress for many reasons “harvest the work” • Working with Dr. Laura Goe design teams met in subject/grade alike teams to identify measures that would be appropriate to assess students progress toward the appropriate standards. ( Nov 2010 and March 2011 and ongoing). Trained table facilitators – 170 participants • A review process (handout) was developed to assess the measure against the criteria: 1. Measure must show growth between two points in time 2. Measure must be comparable across classrooms 3. Measures must be rigorous 4. Measures should improve teaching and learning

  14. Working Definitions of Key Terms(Regents Task Force on Teacher/Principal Evaluation) Comparable Across Classrooms: Standardized or standardized system, meaning that all teachers use the assessment in exactly the same way. • Valid, meaning that it measures what is intended. • Recorded, meaning that student progress can be compared across classrooms and schools. • Meets the criteria for comparable Comparable Measure of Student Growth: • Measures the state learning standards in the content area; • Are as rigorous as state assessments; • Are consistent with technical criteria for validity, reliability, and freedom from bias; • Administered and the results are interpreted by appropriately qualified school staff in accordance with described standards. Rigorous: • Three significant components of rigor in education are: setting high expectations, supporting students and teachers so they can reach those expectations, and accountability for reaching high expectations. (Blackburn, 2008)

  15. Local Measures to be negotiated must : • Align with NYS standards related to the teacher’s subject/grade level • To extent practical be valid and reliable, as defined by the testing standards • Administer/score in a comparable manner across a subject and/or grade level. • NYSED Regulations allow: • 1. Student achievement on State Assessments, Regents examination and/or SED approved alternative examinations. • 2. School-wide, group or team metric • 3. Commercially developed student assessments approved by SED. • (locally used standardized test e.g., Dibels, DRA, Iowa, Terra Nova) • 4. District, regional or BOCES developed assessments • (curriculum based assessment, project or performance assessments) • 5. District-wide student growth goal setting process to be used with student assessment, or teacher created assessments. • (school/teacher created e.g., pre/post tests, curriculum-based assessments, portfolio of student work)

  16. Green Handout • Step 1: Determine what measures should be considered/what measures are currently in place: • show growth in student achievement (two points across time) • are or can be standardized (administered and scored in a standardized fashion) if not, can they be? • are valid (are appropriate measures for the purposes of teacher evaluation and student growth) • are recorded and able to be compared across classrooms (data collected and stored at the student level) •  Step 2: Determine if measures have enough variety to: • capture a wide range of growth (some measures do not have floor/ceiling effects) • context effects (factors that are beyond the teachers’ control such as class size, attendance, or lack of non-academic supports) • be sensitive to varied student growth trajectories (not all students learn at the same pace) • modified for different schools or grade levels

  17. Step 3: Determine parameters to ensure: • alignment to standards • scoreable • resources and training needed • implementation considerations

  18. Statements from staff… • I didn’t sign up for this. • Kids will be tested to death. • This will force all of us to do the same things. • This has great potential once it is formalized. • When are we going to have time to do all this and teach? • Who will be collecting all the data and how will it be used? • This has forced me to look at my practice very differently. • Wows and Worries

  19. Review of the Measure • Over 150 draft measures in development • Review process : Rigorous, Comparable, Standards based? Growth over time? Improve Teaching –does measure contribute to teachers understanding of student needs? Inform Professional Growth?

  20. Multiple Measures of student achievement (harvesting the work) • Local and district wide achievement tests • Portfolio of student work that shows evidence of student growth • Curriculum based assessments and subject matter assessments • Learning goals developed by educators • Performance and Project-based assessments • Formative and summative student assessments

  21. Considerations: Are we measuring what’s important? Is there a consensus on the competencies students should achieve in this content area? • After lots of discussion: • Focus on proficiency: experts and practitioners define the knowledge, concepts, and skills students should acquire for each subject and grade level based on content standards • Content standards form the basis on which measures can be either identified or developed

  22. Considerations: • What assessments/measures can be used to reliably measure these competencies with validity? • After lots of discussion sorted measures into categories: • Existing measures • Rigorous new measures • Portfolios/products/performance/projects • Student learning objectives

  23. Existing measures • Strengths of this Measure • Alignment with content • Pre-tests could be possible • Challenges for this Measure • Validity is a concern whenever a measure is used in a way that was not intended

  24. New Measures • Strengths of this Measure • Tests can be made to match specific grade/ subject standards Challenges for this Measure • Time and cost intensive approach • Paper-and-pencil tests may not be appropriate as the solemeasure, particularly in subjects requiring students to demonstrate knowledge and skills (art, music, etc.)

  25. Use products/portfolio/performance/projects • Strengths of this Measure • Evidence of growth can be documented over time using performance rubrics • Portfolios and projects can reflect skills and knowledge that are not readily measured by paper-and-pencil tests. Challenges for this Measure • Training for inter-rater reliability • Logistical challenge for group raters

  26. Challenges (mechanical): • Teacher of Record Issues: Should teachers be held to the same level of accountability if a student: -is only in classroom for a portion of the year? -has a high rate of school absence? -fails to complete assessments that will be used for determining teachers’ contribution to student growth? Which teacher should be held accountable in a co-teaching situation or push in/pull out?

  27. . Scoring considerations – locals must negotiate a conversion chart to convert multiple measures to the 20 points basis. • Ms. Rivera’s class averages on multiple assessments: Midterm = 85 Performance assessment = 50 Final assessment = 82 Final research paper = 71 • Total = 288 288 / 4 (number of assessments) = 72 A conversion chart would establish the scoring bands for rating catgories i.e. 56 – 85 = 14 which falls within the effective range for this sub-component.

  28. Challenges (implementation) • What resource and human capacity limitations and strengths need to be factored into the decision on measurements? • Does the district have the human capacity to implement these assessments with fidelity? • What are the training needs? • What type of resources are required to ensure implementation fidelity?

  29. Challenges • Do all districts have the same school calendar? • Do various types of educators in all districts across the state have the same job descriptions • How will we identify a “comparable measure” of assessment if a state assessment is unavailable? • How will we identify “locally selected measures” that are “rigorous and comparable across classrooms”? • Does ourdistrict use district-wide assessments by grade and subject? • Does ourdistrict use school-wide assessments by grade and subject? • Do they include measures that are “rigorous and comparable across classrooms”? • Do they include measures that show learning growth “between two points in time”? • Do they include measures that are aligned with and focused on grade level and subject standards? • Do they allow teachers from all subjects to be evaluated with evidence of student learning growth? • Will using this model help improve teaching and learning?

  30. Feedback from pilots: • Lesson 1 –Establish time for teachers to collectively examine student work and come to a consensus on performance and a shared understanding of the process. • Lesson 2 - Develop rubrics for the assessments with explicit instructions and examples of proficiency within levels. • Lesson 3 – Provide professional development for teachers to determine how assessments should be given, how often and how to record results in standardized formats. • Lesson 4 - Identify implementation challenges and guidance for factors such as scoring, teacher of record, etc. • Guidance regulations due July 2011

More Related