teacher development and support system n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Teacher Development and Support System PowerPoint Presentation
Download Presentation
Teacher Development and Support System

Loading in 2 Seconds...

play fullscreen
1 / 87

Teacher Development and Support System - PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on

Teacher Development and Support System. Feedback from SIG schools July, 2011. Context for feedback. As part of developing the teacher evaluation and support system for SIG schools, the district has been soliciting feedback from participants on various aspects of the system

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Teacher Development and Support System' - tuyen


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
teacher development and support system

Teacher Development and Support System

Feedback from SIG schools

July, 2011

context for feedback
Context for feedback
  • As part of developing the teacher evaluation and support system for SIG schools, the district has been soliciting feedback from participants on various aspects of the system
  • Approximately 650 teachers and staff participated in this work
    • 75% completion rate as of June 24
  • The feedback is still in the process of being gathered and analyzed; those areas with the most feedback to date have been analyzed
  • This feedback will inform the further development of the measures and the piloting of the evaluation system in school year 2011-12
the multiple measures of teacher effectiveness
The Multiple Measuresof Teacher Effectiveness

Observation of Practice

Observing teaching & review artifacts of practice (e.g. lesson and unit plans, student work)

By Administrators

&

By Teachers

Development & Support

Self-Reviews

Individual Growth Plans

Multiple Measures

Multiple Measures

Stakeholder Feedback

Parent Surveys

Student Surveys

Contributions to School Community

TBD

Contributions to Student Learning Outcomes

Academic Growth over Time

Analysis of Supplemental Close-Ended and Open-Ended Assessments

Differentiated Compensation & Recognition

Teaching & Learning Framework

A common foundation for effective teaching

key questions for teachers feedback
Key questions for teachers’ feedback

How is effective teaching best measured?

Observation of Practice

Observing teaching & review artifacts of practice (e.g. lesson and unit plans, student work)

By Administrators

&

By Teachers

How is effective teaching best supported?

Development & Support

Self-Reviews

Individual Growth Plans

Multiple Measures

Multiple Measures

Stakeholder Feedback

Parent Surveys

Student Surveys

Contributions to School Community

TBD

Contributions to Student Learning Outcomes

Academic Growth over Time

Analysis of Supplemental Close-Ended and Open-Ended Assessments

How are decisions related to promotion, compensation, and intervention aligned to support teacher effectiveness?

Differentiated Compensation & Recognition

Teaching & Learning Framework

A common foundation for effective teaching

How are changes to teacher support

and development best implemented?

activities that informed each component of the system
Activities that informedeach component of the system

Observation of Practice

Teaching & Learning Framework (TLF): Workshop I & II

Teaching & Learning Framework: Book Study Group

Total Effectiveness Results: Small Group

Total Effectiveness Results: Individual Feedback

Listening Sessions

Development & Support

TLF: Workshop II

Listening Sessions

Multiple Measures

Multiple Measures

Stakeholder Feedback

Contributions to School Community

Contributions to Student Learning Outcomes

Differentiated Compensation & Recognition

Stakeholder Input: Survey Feedback

Total Effectiveness Results: Small Group & Individual

Listening Sessions

Total Effectiveness Results: Small Group & Individual

AGT: School Level Results

TER: Small Group & Individual

Listening Sessions

Diff Comp: Small Group

Teaching & Learning Framework

A common foundation for effective teaching

Implementation

Listening Sessions

what are the benefits of the teaching and learning framework
What are the benefits of the Teaching and Learning Framework?

Listening Sessions:

  • Virtually all teachers seemed to like the TLF and praised it for providing “common language” for all. Comments included:
    • “really good guide on how to be a good teacher”
    • “not McStandards”
    • “good foundation”
    • “job description” that includes “all aspects” of being a teacher.
  • In two separate schools, teachers expressed their pleasure that the new framework seemed to be a move away from evaluation based on “red flags” such as “standards on the board.”

Book Study:

  • Predominantly, participants felt that the TLF will benefit teachers as a roadmap and guide for reflection and self review.
  • Participants also felt that the TLF will promote collaboration between colleagues by providing a common language and clear expectations.
what are the concerns with the teaching and learning framework
What are the concerns with the Teaching and Learning Framework?

Listening Sessions:

  • The most prevalent concern was principals’ capacity to use the framework to evaluate teachers due to the time, knowledge and skill required by such a complex and rigorous tool. They acknowledged the demands that the TLF placed on evaluators
  • Several other concerns were mentioned, though less frequently:
    • Concerns about favoritism (real or perceived)
    • Potential for distorted reports or poor interpretation of the framework’s intention
    • Time required to complete might lead to falsifying the document (as has been known to happen with the STULL evaluation).

Book Study:

  • Given the complexity and size of the TLF, participants were concerned about the implementation and whether the district will provide enough time and support to teachers and administrators.
perceptions about the teaching and learning framework workshops
Perceptions about the Teaching and Learning Framework Workshops

Workshop 1:

Over 90% of participants agreed or strongly agreed that

    • They learned useful information about the TLF
    • Colleagues were interested in the workshop content
    • They were interested in the workshop content

Workshop 2:

  • 87% of participants agreed or strongly agreed that they learned useful information about the TLF self review process and lesson design template.
  • 88% of participants agreed or strongly agreed that they were interested in the workshop content.
  • 90% of participants agreed or strongly agreed that their colleagues were interested in the workshop content
feedback on the tlf lesson design p rocess and template
Feedback on the TLF lesson design process and template

Workshop 2:

  • Benefits of the lesson design template (link):
    • considers all components of teaching, very detailed
    • can be used as a guide to organize lesson
    • pushes evaluator to look at various aspects
    • lets teacher know what's being observed
  • Concerns about the lesson design template (link):
    • the design template is overwhelming and will take too much time to complete
    • the observers, especially the administrators, will not understand the design template and adhere to the observation protocol
    • teachers will have difficult meeting the expectations of the design template
how would you weight observation of practice towards a total effectiveness result
How would you weight Observation of Practice towards a Total Effectiveness Result?

Total Effectiveness Result: Teacher Results:

  • On average, participants felt that Observation of Practice should count towards 55% of the Total Effectiveness Result, with 35% coming from Administrator Observation and 20% coming from Teacher Leader Observation (link).
  • 80% and 62% of responses wanted Administrator Observation and Teacher Leader Observation, respectively, to count towards teacher evaluations (link).
benefits of and concerns with using observation of practice as part of teacher evaluations
Benefits of and concerns with using Observation of Practice as part of teacher evaluations

Total Effectiveness Result: Teacher Results:

feedback on stakeholder surveys1
Feedback on Stakeholder Surveys

Stakeholder Input Small Group Activity:

Participants felt the pilot student survey was clear and organized, had a good coverage of topics, and would be useful for informing teaching practice

Many also felt the survey was too lengthy and could be subject to student bias

Listening Session:

Few participants mentioned the role of surveys. Of those who did, most thought they would be useful and had the potential to engage others (students and parents) in the work of school improvement. One teacher thought that disgruntled students would use them as a means to get back at strict teachers.

how would you weight stakeholder input towards a total effectiveness result
How would you weight Stakeholder Input towards a Total Effectiveness Result?

Total Effectiveness Result: Teacher Results:

    • On average, participants felt that Stakeholder Input: Student Surveys should count towards 5% of the Total Effectiveness Result (link), with 51% wanting to use it as feedback only, and 14% not count all (link).
  • The benefit of Stakeholder Surveys: provides opportunity for learners to weigh-in (link).
  • The concerns: subjective, could conflict with discipline and grading efforts (link).
how would you weight contributions to school community towards a total effectiveness result
How would you weight Contributions to School Community towards a Total Effectiveness Result?

Contributions to School Community Activity

Unfortunately, due to time constraints, we were not able to conduct the activity for this component.

Total Effectiveness Result Small Group Activity:

On average, participants felt that Contributions to School Community would count towards 8% of the Total Effectiveness Result, with 33% wanting to use it as feedback only, and 21% not count all.

The benefit of using this measure: supports collaboration (link)

The concern: depends on definition of measure (link)

what are the benefits of using academic growth over time agt as a measure of teacher effectiveness
What are the benefits of using Academic Growth over Time (AGT) as a measure of teacher effectiveness?

AGT Small Group Activity:

Gives teachers motivation and direction to improve

Can be used as benchmark and comparison to other schools

Focus on improvement and not just achievement

Total Effectiveness Result Small Group Activity:

Provides objective measure of student learning

Supports educators to seek best practice to improve growth

Supports collaboration within school

Listening Sessions:

Some teachers appeared to look forward to the opportunity to demonstrate their effectiveness via test scores

what concerns do you have about agt
What concerns do you have about AGT?

AGT Small Group Activity:

External factors/variables not factored into score

Incomplete/limited results by subject, grade level, subgroups

Accuracy, Validity, Reliability of AGT

Total Effectiveness Result Small Group Activity:

Could focus efforts too narrowly on test prep

Test scores are due to many variables

Can be outside of individual teacher’s control

Listening Sessions:

“…participants named as a fear the possibility that test scores would come to dominate the evaluation process…”

how would you weight agt towards a total effectiveness result
How would you weight AGT towards aTotal Effectiveness Result?

66% and 53% of participants supported the inclusion of individual and school AGT, respectively, in the TER score (link).

On average, student outcomes (individual and school AGT) weighted 28% of Total Effectiveness, second to observations (link).

Variance shown in how much AGT should count towards TER score (link).

Source: SIG Total Effectiveness Results: Teacher Results small group activity 2011; N= 133 small groups; blank answers excluded and answers adjusted to total 100%.

how should we support effective teaching
How should we support effective teaching?

Support teachers through FEEDBACK

  • While overall teachers liked the framework as a document, how the document is used was equally important:
    • Use “as a tool, not a weapon”
    • Use as an “appraisal,” as a step in developing teachers not, “punitively.”
  • They wanted to know that the process would be “sufficient” and “fair,” providing multiple rounds of constructive feedback and allowing for opportunities to improve.

Support teachers through CULTURE OF COLLABORATION

  • The dominant theme among all groups was the importance of collaboration. Collaboration was indicated as a way to benefit both individual teachers and their school overall.
  • A few teachers noted that collaboration is not necessarily intrinsic and that it must be developed. Several more expressed concerns that the EE work has the potential to be divisive, if it becomes competitive rather than collaborative.

Source: SIG Listening Small Group Sessions 2011; 17 sessions; 76 participants.

how should we support effective teaching1
How should we support effective teaching?

Support teachers through PROFESSIONAL DEVELOPMENT

Not surprisingly, teachers repeatedly noted the importance of high quality professional development. Several specified that it should be specifically linked to an element of the framework, including opportunities to learn further about the framework itself and to strategize with peers about putting it into action

Several teachers noted the importance of “real” time – not just the minutes left over during their weekly meeting after the school logistics and announcements are completed.

Support teachers through FLEXIBILITY

In numerous sessions, teachers spoke about the flexibility inherent in the TLF. They appreciated that the framework appears to focus on the quality of instruction rather than adherence to a particular way of teaching.

Some fear that the TLF will be misinterpreted as “one and only one way to implement.”

Source: SIG Listening Small Group Sessions 2011; 17 sessions; 76 participants.

how should we support effective teaching2
How should we support effective teaching?

Support teachers through SELF-ACCOUNTABILITY

In nearly every listening session, teachers talked about their hopes for their own growth as teachers, many very eloquently. Many appreciated the role that the TLF could play in encouraging “self-reflection,” “accountability for self” and “personal growth.” Interestingly, most of these comments were made prior to participating in the second TLF workshop that included the self-review process.

Source: SIG Listening Small Group Sessions 2011; 17 sessions; 76 participants.

what kinds of incentives should be used and on what basis
What kinds of incentives should be used and on what basis?

Monetary incentives were, by far, the most frequently proposed form of compensation, followed by career pathways (link).

Academic Growth was the most frequently mentioned standard of measurement for receiving incentives (link).

Most respondents preferred at least 1 incentive (link).

how should the differentiated compensation system be structured
How should the differentiated compensation system be structured?

Majority of respondents preferred fixed incentives, such as bonuses (link).

Respondents preferred individual and school-wide incentives over group (team/department) incentives (link).

Variance shown in the weighting of individual, versus group versus school-wide incentives (link).

Respondents tended to weight individual incentives more than group and school incentives (link and reasons for weighting: individual, group, school).

how should the differentiated compensation system be structured1
How should the differentiated compensation system be structured?

Majority of respondents say teachers who are more effective should be compensated more. However, there was concern about divisiveness and less collaboration (link).

Respondents also felt that multiple indicators, and not just CSTs and graduation rates, should be used as a measure of achievement (link).

what additional factors should we consider in implementation
What additional factors should we consider in implementation?

Student Achievement

In every single listening session, teachers named improved student learning and achievement as a hoped for outcome from the Educator Effectiveness work.

Communal Responsibility

Teachers spoke about the role that other factors play into successful teaching and learning. One asserted that “academic success of a student is not solely the responsibility of one teacher [but rather] a collection of things.”

Several teachers believed that peer observations and collaborative work focused on the framework could be used to hold each other accountable and to encourage everyone to “do a little extra” thereby raising expectations for all.

Some teachers stressed the role of collaboration among school staff in the form of “collective accountability for students with each other.” Others talked about seeing parents and students as “pieces of the puzzle” and discussed how to engage parents as partners in their children’s success.

Source: SIG Listening Small Group Sessions 2011; 17 sessions; 76 participants.

what additional factors should we consider in implementation1
What additional factors should we consider in implementation?

Student Accountability

Several groups of teachers spoke about the need for student accountability as well as teacher accountability.

“Teacher Bashing”

In most sessions, teachers expressed concerns about the trend toward “teacher bashing” and fears that the “entire burden” for student and school success would be placed on them. One teacher (who incidentally likes the TLF and expressed strong support for its use) noted that there is an “underlying attitude that teachers are not working hard enough.”

Turnover

In about half of the sessions, teachers talked about the disruptive and demoralizing effects of turnover. While concerns about turnover appeared repeatedly, some thought this work would increase it, while others thought it would lead to a more stable teaching force.

Source: SIG Listening Small Group Sessions 2011; 17 sessions; 76 participants.

what additional factors should we consider in implementation2
What additional factors should we consider in implementation?

Alignment

Aligning TLF with Beginning Teacher Support and Assessment (BTSA) standards, or finding a way to use TLF in its place, would simplify the work for beginning teachers and those who work with them.

Organization and Communication

The nature of the pilot phase (i.e. compressed timeline to complete 50-hours of professional development, school calendars that were established prior to pilot phase, and inconsistent information pathways) led to teachers’ concerns about organization and communication. This created an “air of anxiety” in many schools. Teachers stressed repeatedly that “transparency,” “good communication” and “consistency” would significantly improve the implementation of the EE work.

“Stick With It”

In all schools, there were teachers who expressed the need to persist with the Educator Effectiveness work beyond the 50-hour commitment. Many noted that that it requires substantial learning and will take time to succeed. They expressed hopes that their colleagues will “step up.”

Source: SIG Listening Small Group Sessions 2011; 17 sessions; 76 participants.

year 1 sig activities teacher effectiveness
Year 1 SIG Activities:Teacher Effectiveness

Click the hyperlinks below to jump to any of the activities:

Teaching & Learning Framework: Workshop I—Introduction to the Framework

Teaching & Learning Framework: Workshop II—Application of the Framework

Teaching & Learning Framework: Book Study Group Activity

Academic Growth over Time: School Level Results Overview

Total Effectiveness Results: Small Group—Teacher Results

Total Effectiveness Results: Individual Feedback Form

Differentiated Compensation: Small Group

Stakeholder Input: Survey Feedback—Small Group Activity or Focus Group

Listening Sessions

teaching learning framework workshop 1
Teaching & Learning Framework: Workshop 1

Click for last

viewed slide

The first Teaching & Learning Workshop introduced participants to the Teaching and Learning Framework and rubric.

Participants filled out feedback forms at the end of their session, 495 responses in total.

Click for

Activities List

feedback on workshop 1 structure
Feedback on Workshop 1 Structure

Click for last

viewed slide

  • Overall, over 90% of the participants Agreed or Strongly Agreed that
    • The sessions were well organized
    • Activities were well-paced and appropriately varied
    • Presenters had strong knowledge
    • They learned useful information about the TLF
    • Colleagues were interested in the content
    • They were interested in the content
  • Overall, 75-77% of participants  Agreed or Strongly Agreed that
    • They have a clear understanding of how the session relates to their SIG work back at school
    • They understand the next steps related to the SIG work

Click for

Activities List

Source: SIG TLF Workshop 1 feedback forms 2011; N= 239; Blank answers excluded.

feedback on workshop 1 content
Feedback on Workshop 1 Content

Click for last

viewed slide

Overall, Wisdom of Practice and Memorable Moment were the most engaging for the teachers (79-80% selecting highly engaging or engaging)

Give One-Get One, Evidence for Standards 4 & 5, and Reflective Writing were the least engaging (60%, 54%, and 55% selecting highly engaging or engaging, respectively)

As a whole, participants had a clearer understanding of “Establishing a Culture for Learning” (Q18), the difference between Developing and Effective (Q19), and Cognitive Engagement (Q21).

Less than 50% of participants selected the correct answer for “Ineffective Practice” (Q20) and 21st Century Skills (Q22).

Click for

Activities List

Source: SIG TLF Workshop 1 feedback forms 2011; N= 239; Blank answers excluded.

feedback from participants
Feedback from Participants

Click for last

viewed slide

About 20% of the surveys had comments

Participants were most confused about applying the framework to their teaching, understanding the purpose of the framework, implementing the framework at the classroom, schools, and district level given the complexity, and the next steps in the SIG process

Participants thought it would be most helpful to review and study the content on their own or with peers, see real-life examples of the framework being used in the classroom, and receiving more PD like this one.

Click for

Activities List

Source: SIG TLF Workshop 1 feedback forms 2011; N= 239; Blank answers excluded.

workshop 1 participant make up
Workshop 1 Participant Make-Up

Click for last

viewed slide

Over half were middle school teachers, 15% were elementary teachers, and 19% were high school teachers.

2% were principals

Compared to the High School teachers, Elementary teachers had a more favorable opinion about the workshop structure (on average +8% response rate for agree and strongly agree) and content (on average +13% response rate for engaging and highly engaging).

High school teachers and middle school teachers had similar ratings of the workshop

Compared to the Elementary school teachers, principals had a more favorable opinion about the structure but a less favorable opinion about the content.

Click for

Activities List

Source: SIG TLF Workshop 1 feedback forms 2011; N= 239; Blank answers excluded.

teaching learning framework workshop 2
Teaching & Learning Framework: Workshop 2

Click for last

viewed slide

The second Teaching & Learning Workshop showed participants how to apply the Teaching and Learning Framework and rubric to self review and lesson study.

239 feedback forms inputted to date with approximately 400 remaining

Click for

Activities List

feedback on workshop 2 structure
Feedback on Workshop 2 Structure

Click for last

viewed slide

  • Overall, over 90% of the participants Agreed or Strongly Agreed that
    • The sessions were well organized
    • Activities were well-paced and appropriately varied
    • Presenters had strong knowledge
    • Colleagues were interested in the content
  • Overall, 82-88% of participants  Agreed or Strongly Agreed that
    • They learned useful information about the Self-Review process
    • They have a clear understanding of how the session relates to their SIG work back at school
    • They learned useful information about the Lesson Design Template
    • They were interested in the content that was discussed in the workshop

Click for

Activities List

Source: SIG TLF Workshop 2 feedback forms 2011; N= 239; Blank answers excluded.

feedback on workshop 2 content
Feedback on Workshop 2 Content

Click for last

viewed slide

Reflecting on My Instruction and Completing the Self Review were most engaging (77% selecting highly engaging or engaging)

Reviewing the TLF, Examining a Sample Lesson Plan Using the Lesson Design Template, and Applying the Lesson Design Template to My Own Content were the least engaging (71%, 71%, and 68% selecting highly engaging or engaging, respectively)

Click for

Activities List

Source: SIG TLF Workshop 2 feedback forms 2011; N= 239; Blank answers excluded.

slide44
Feedback on using the Lesson Design Template when preparing for a formal classroom observation by administrator or another teacher

Click for last

viewed slide

Click for

Activities List

Source: SIG TLF Workshop 2 feedback forms 2011; N= 239; Blank answers excluded.

slide45
Feedback on using the Lesson Design Template when preparing for a formal classroom observation by administrator or another teacher

Click for last

viewed slide

Click for

Activities List

Source: SIG TLF Workshop 2 feedback forms 2011; N= 239; Blank answers excluded.

slide46
Feedback on using the Lesson Design Template when preparing for a formal classroom observation by administrator or another teacher

Click for last

viewed slide

Click for

Activities List

Source: SIG TLF Workshop 2 feedback forms 2011; N= 239; Blank answers excluded.

teaching learning framework book study
Teaching & Learning Framework: Book Study

Click for last

viewed slide

Participants formed book study groups to read and discuss Charlotte Danielson’s Enhancing Professional Practice, culminating in written reflections about the Teaching and Learning Framework and the SIG educator effectiveness work in general.

631 people participated of which 100 reflections were sampled

Click for

Activities List

teaching learning framework book study1
Teaching & Learning Framework: Book Study

Click for last

viewed slide

Predominantly, participants felt that the Teaching and Learning Framework (TLF) will benefit teachers as a roadmap and guide for reflection and self review.

Participants also felt that the TLF will promote collaboration between colleagues by providing a common language and clear expectations.

Given the complexity and size of the TLF, participants were concerned about the implementation and whether the district will provide enough time and support to teachers and administrators.

Click for

Activities List

teaching learning framework book study2
Teaching & Learning Framework: Book Study

Click for last

viewed slide

Click for

Activities List

general comments about the tlf evaluations and educator effectiveness
General comments about the TLF, evaluations, and educator effectiveness

Click for last

viewed slide

Click for

Activities List

general comments about the benefits of the tlf
General comments about the benefits of the TLF

Click for last

viewed slide

Click for

Activities List

concerns with the tlf and teacher evaluation
Concerns with the TLF and teacher evaluation

Click for last

viewed slide

Click for

Activities List

general comments about the implementation of the tlf and teacher evaluation
General comments about the implementation of the TLF and teacher evaluation

Click for last

viewed slide

Click for

Activities List

academic growth over time
Academic Growth over Time

Click for last

viewed slide

Participants were introduced to the Academic Growth over Time School Report through a webinar and then asked to analyze their school report.

155 small group responses, representing 562 participants

Click for

Activities List

what are the benefits of using academic growth over time agt as a measure of teacher effectiveness1
What are the benefits of using Academic Growth over Time (AGT) as a measure of teacher effectiveness?

Click for last

viewed slide

Click for

Activities List

Source: SIG Academic Growth over Time: School Results small group activity 2011; N= 155 small groups; Blank answers excluded.

what concerns do you have about agt1
What concerns do you have about AGT?

Click for last

viewed slide

Click for

Activities List

Source: SIG Academic Growth over Time: School Results small group activity 2011; N= 155 small groups; Blank answers excluded.

what questions do you have about agt
What questions do you have about AGT?

Click for last

viewed slide

Click for

Activities List

Source: SIG Academic Growth over Time: School Results small group activity 2011; N= 155 small groups; Blank answers excluded.

comments about agt
Comments about AGT

“This data is certainly helpful in monitoring the effects we are having in getting our students caught up to other schools in LAUSD. We like that this “does not mean a lowering of expectations for any grouping of students addressed by a control variable”. This gives a very clear picture of whether the school is doing its job of getting students up to speed despite some environmental challenges. Without this data, we think it becomes very easy to blame everything on our kids’ home life and pass the responsibility from ourselves. This data is at least meant to isolate our effect on our students. The question then becomes, how effective and accurate is it at isolating the data. Our main concern is that measures of growth are still based solely on CST scores. We have grave concerns about the validity of the results of this one test. Our concerns about the CST (or any high stakes testing model) are numerous and echoed throughout academia. We acknowledge that this is the situation that we are confronted with and AGT is a superior lens through which to view the results of the test than just comparing teacher effectiveness on a completely level field. Our other concern is the effectiveness of the isolation of data as mentioned in question #1. We see the 9 control variables presented but don’t know yet how well they work. We are especially concerned because students advance to the next grade level, whether or not the students’ ability advances from year to year. If a student has not adequately learned addition one year, how could they be expected to “grow” under 8th grade standards of algebra? Perhaps the student advanced 2 grade levels under a teacher, learning addition, subtraction, multiplication, division, etc. but did not get to algebra. That school is now being judged as not adequately meeting the standards of growth for a student who probably actually advanced two years under their teacher...”

Source: SIG Academic Growth over Time: School Results small group activity 2011; N= 155 small groups; Blank answers excluded.

comments about agt1
Comments about AGT

“AGT is an invaluable tool to utilize in assessing the effectiveness of schools as it is useful in tracking the progress or growth of individual students over time, rather than the percentage of students that meet an absolute target or standard. This allows for true comparisons of effectiveness. Additionally, AGT enables us to identify, study, and share the practices of schools who are achieving effective results. These results are based upon a common standardized measure of growth, so that when we compare growth in one school to another, we know that we are using a term with one definition.

Although AGT models are obviously beneficial, they also represent a narrow way to assess the effectiveness of educators and schools. The results should be considered within the context of other metrics – both qualitative and quantitative. Like all measures related to student learning, this approach is subject to limitations, which is why (i) it is important to look at multiple indicators and results and (ii) this approach should be used as one amongst multiple measures. Further, the effectiveness of this analysis is lost if it is not linked to ways to improve practice or to learn from those with clear positive effects on student learning.

Further, AGT models are complex statistical models, which can be easily misread or misinterpreted. Moreover, there are factors external to our schools and classrooms that can impact student learning rates, that may not be reflected in AGT models. Additionally, it only evaluates how students perform on a standardized test, which measures only a small part of what happens in a classroom during the academic year.

Will LAUSD be offering Learning Zone classes to supplement the AGT primer?

The AGT model follows the premise that students can be fairly evaluated by how they perform on a standardized test, but such tests only measure a fraction of what happens in a classroom during the academic year. Although the AGT model is constructed to evaluate a school's growth, it may well be a way to link student scores to teacher evaluations.”

Source: SIG Academic Growth over Time: School Results small group activity 2011; N= 155 small groups; Blank answers excluded.

comments about agt2
Comments about AGT

“Our group thinks that the data may be helpful for us as teachers because it allows for us to examine our student’s data and make predictions on their expected growth for that academic year. Furthermore, if we examine our students more closely we should be able to compare our student’s actual achievement growth with their predicted achievement growth. Moreover, the school can benefit from the AGT because the results are suppose account for the environmental variables therefore making the AGT more reliable, fair, and accurate when calculating student growth.

The concerns that we have about AGT is how does it truly differs from any other testing model or tool used. It appears that many teachers when analyzing student data utilize the student’s previous performance on standardized test. This means we as teachers are already taking into account the student’s previous performance or academic ability. A second concern that is relevant to our group which was not mentioned in environmental variables are the students previous teachers or schools that have contributed towards the students success or failure in achieving a predicted growth. For example, a student who attends a low API school, and had received their instruction predominantly from substitute teachers versus a student who doesn’t have these environmental variables, how do they account for these factors when creating a fair result. The video uses two gardeners as an analogy to demonstrate AGT and it states the difference between Gardner A and Gardner B are their strategies, however the strategies were never fully discussed what was discussed was the uncontrolled variables such as soil richness, rain, etc. Factors which neither Gardner can truly control. Whereas the strategies that were not examined in regards to how often the soil was turned, cutting dead leaves off, talking to our tree, etc. AGT didn’t seem to focus on the controlled strategies, which is what we as classroom teachers provide our students.

A wish that has not been included is actually walking us through the AGT process with real students and teachers or a panel. A visual presentation could have been more useful. Second, making the material more personal to my particular department in which I teach in such as Special Education, Math, Science, ELA, and History. I don’t think AGT is explained as easily as it could be for your teacher who just wants to know or see how to apply this to their work.”

Source: SIG Academic Growth over Time: School Results small group activity 2011; N= 155 small groups; Blank answers excluded.

total effectiveness result teacher results
Total Effectiveness Result:Teacher Results

Click for last

viewed slide

Participants were asked to read articles about teacher evaluation systems and then determine the best weighting for each measure of teacher effectiveness towards a Total Effectiveness Result.

133 small group responses, representing 521 participants

Click for

Activities List

total effectiveness result teacher results1
Total Effectiveness Result:Teacher Results

Click for last

viewed slide

  • Greatest support for observations (esp. by administrators) followed by student outcomes (esp. at individual level)
  • On average, observations weighted 55% of total effectiveness and student outcomes 28%
  • For nearly all measures, the distribution of teacher responses reflects the full range, from “Not Count” to 100% of the weighting
    • Observations had highest weightings with lowest indication of “feedback-only” or “not counting”
    • Stakeholder Input received low weightings, with many teachers indicating that the surveys should be feedback-only or not count
  • Teachers’ qualitative comments illuminated both benefits and concerns of particular measures and their inclusion in the total effectiveness result

Click for

Activities List

slide63

On average, teachers indicated observations should count for the majority (55%) with student outcomes (28%) accounting for the next largest component

Click for last

viewed slide

Click for

Activities List

Source: SIG Total Effectiveness Results: Teacher Results small group activity 2011; N= 133 small groups; blank answers excluded and answers adjusted to total 100%.

slide64
Overall, most teachers supported the inclusion of observation and student outcomes measures; fewer supported other measures

Click for last

viewed slide

Click for

Activities List

Source: SIG Total Effectiveness Results: Teacher Results small group activity 2011; N= 133 small groups; blank answers excluded.

distribution of teacher perspectives across measures shows variance
Distribution of teacher perspectives across measures shows variance

Click for last

viewed slide

Click for

Activities List

Source: SIG Total Effectiveness Results: Teacher Results small group activity 2011; N= 133 small groups, blank answers excluded.

teachers comments illuminate benefits and concerns of the measures under consideration
Teachers’ comments illuminate benefits and concerns of the measures under consideration

EXAMPLES

Click for last

viewed slide

Click for

Activities List

Source: SIG Total Effectiveness Results: Teacher Results small group activity 2011; N= 133 small groups.

teachers comments illuminate benefits and concerns of the measures under consideration cont
Teachers’ comments illuminate benefits and concerns of the measures under consideration (cont.)

EXAMPLES

Click for last

viewed slide

Click for

Activities List

Source: SIG Total Effectiveness Results: Teacher Results small group activity 2011; N= 133 small groups.

slide68
More teachers thought more measures should count when giving small group feedback as compared to individual feedback

Source: SIG Total Effectiveness Results: Teacher Results small group and individual activity 2011; 133 small groups; N= 333 individuals.

slide69
Variation in weightings for teachers* between individual feedback and small group feedback concentrated on surveys

*Teachers with AGT results.Source: SIG Total Effectiveness Results: Teacher Results small group and individual activity 2011; 133 small groups; N= 333 individuals.

differentiated compensation
Differentiated Compensation

Click for last

viewed slide

Participants were asked to select and read articles about differentiated compensation and incentives, and then suggest the type of incentives they would like to have, if any

132 small group responses, representing 515 participants

Click for

Activities List

monetary incentives and career pathways were the most frequently proposed incentives
“Monetary incentives” and “career pathways” were the most frequently proposed incentives

Click for last

viewed slide

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

academic growth was the most frequently mentioned standard of measurement for receiving incentives
“Academic Growth” was the most frequently mentioned standard of measurement for receiving incentives

Click for last

viewed slide

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

most respondents preferred at least 1 incentive
Most respondents preferredat least 1 incentive

Click for last

viewed slide

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

a majority of respondents prefer fixed monetary incentives such as bonuses
A majority of respondents prefer fixed monetary incentives, such as bonuses

Click for last

viewed slide

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

slide75

Individual and school incentives were weighted more heavily than group incentives, with individual and school incentives sharing equal weight

Click for last

viewed slide

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

slide76
Variance shown in the relative amount of compensation going towards individual, group, and/or school incentives

Click for last

viewed slide

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

respondents tended to weight individual incentives higher than group and school incentives
Respondents tended to weight individual incentives higher than group and school incentives

Click for last

viewed slide

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

reasons for using individual i ncentives
Reasons for usingindividual incentives

Click for last

viewed slide

Teachers have the greatest impact on student outcomes

Motivation and retention of teachers

Group rewards may dilute incentive

Cannot control what happens in other classrooms; don’t want to be penalized by the performance of ineffective teachers

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

reasons for using group team department incentives
Reasons for usinggroup (team/department) incentives

Click for last

viewed slide

Promotes team building

Share common goals

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

reasons for using s chool wide i ncentives
Reasons for usingschool-wide incentives

Click for last

viewed slide

Promotes collaboration and team building

Avoids division and envy

Prevents favoritism and cheating

Everyone at the school site contributes to student outcomes, not just teachers

Not all teachers have the same type of students; some students are harder to teach than others

Some teachers do not have test scores

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

slide81

Should everyone have a chance at getting the reward (greater chance, less amount), or should there be a scaled system that gives more money for making a greater difference in student outcomes (less chance, greater amount)?

Click for last

viewed slide

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

slide82
Should the incentive system use just CSTs and graduation rates as indicators of achievement, or other indicators as well?

Click for last

viewed slide

Click for

Activities List

Source: SIG Differentiated Compensation small group activity 2011; 132 small groups; blank answers excluded.

stakeholder survey feedback small group activity
Stakeholder Survey Feedback: Small Group Activity

Click for last

viewed slide

Participants gave feedback on the Stakeholder Surveys (Student and Staff) that were piloted at the SIG schools

Additional feedback on the surveys has been collected by the researchers, who will present their findings at a later time.

111 small group responses, representing 330 participants

Click for

Activities List

general comments about student survey
General comments about student survey

Click for last

viewed slide

Click for

Activities List

Source: SIG Stakeholder Survey small group activity 2011; N= 111 small groups; Blank answers excluded.

general comments about student survey cont
General comments about student survey (cont.)

Click for last

viewed slide

Click for

Activities List

Source: SIG Stakeholder Survey small group activity 2011; N= 111 small groups; Blank answers excluded.

recommendations for improving student survey
Recommendations for improving student survey

Click for last

viewed slide

Click for

Activities List

Source: SIG Stakeholder Survey small group activity 2011; N= 111 small groups; Blank answers excluded.

listening sessions
Listening Sessions

To access the full report, end slideshow (if you haven’t already) and double-click the image to the right 

Click for last

viewed slide

Click for

Activities List