Using MEAP
Download
1 / 31

Using MEAP To Evaluate Teachers MERA Fall 2011 Conference Dave Treder - PowerPoint PPT Presentation


  • 156 Views
  • Uploaded on

Using MEAP To Evaluate Teachers MERA Fall 2011 Conference Dave Treder. Study Methods Looked at MEAP growth/change over two consecutive years (grade 4 MEAP to grade 5 MEAP): Both Fall 07 to Fall 08 and Fall 08 to Fall 09. Counts. -- 20 of the 21 school districts In Genesee ISD.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using MEAP To Evaluate Teachers MERA Fall 2011 Conference Dave Treder' - dobry


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Using meap to evaluate teachers mera fall 2011 conference dave treder

Using MEAP

To Evaluate

Teachers

MERA Fall 2011 Conference

Dave Treder


Using meap to evaluate teachers mera fall 2011 conference dave treder

  • Study Methods

  • Looked at MEAP growth/change over two consecutive years

  • (grade 4 MEAP to grade 5 MEAP):

  • Both Fall 07 to Fall 08 and Fall 08 to Fall 09

Counts

-- 20 of the 21 school districts In Genesee ISD


Using meap to evaluate teachers mera fall 2011 conference dave treder

Description of Study Data

  • 4th Grade Teachers

  • 20 of the 21 Districts In Genesee ISD

  • Based on the Districts’ SIS system

  • Business rules:

  • - If schedules weren’t used in the district, the homeroom teacher was coded as the math and reading teacher – in the districts that used schedules in the elementary grades, math and reading classes were used to designate the math and reading teacher

  • - Teachers were included if they were the teacher of record at the beginning of the school year (1st semester grades) and the end of the school year (last semester grades)

  • - Students included if they had the same teacher at the beginning of the school year (1st semester grades) and the end of the school year (2nd semester grades)


Using meap to evaluate teachers mera fall 2011 conference dave treder

Number of 4th Grade Teachers, by District:

F07-08, F08-09, and in the District Both Years

74%


Using meap to evaluate teachers mera fall 2011 conference dave treder

What was the size of the classrooms, for the teachers in this study?

(2007-08 School Year)

F07-08 Class Size Frequency

Class Size


Using meap to evaluate teachers mera fall 2011 conference dave treder

Study Methods this study?

Method 1

1) Compute by-student residuals

(non-linear regression, student predicted - actual scale score)

2) Compute Mean Teacher residual

Method 2

1) Compute by-student Performance Level (PL) Change Index

2) Compute Mean Teacher PL change index

  • For each method, compute indices for F07-F08 and F08-F09.

  • Look at the relationship between the two years.

Also Tried:

1) Converting Scales Scores to Z Scores

2) Compute by-student ‘’change”: Year2 Z_Score – Year1 Z_Score

3) Compute mean teacher change


Using meap to evaluate teachers mera fall 2011 conference dave treder

Computing By-Student Residuals (Predicted – Actual) this study?

Math F07-F08

Scale Score, Grade 5

Scale Score, Grade 4


Using meap to evaluate teachers mera fall 2011 conference dave treder

Method this study? 1 – General Results

How Much of the Variance can be Explained at the Teacher Level?

By Teacher, Mean Residual, Fall07-08

Math: 15%

Reading: 1.5%

What is the Relationship Between Teacher Level Mean Residuals,

F07-08 & F08-09?

Correlation:

Math: .32

Reading: .19


Using meap to evaluate teachers mera fall 2011 conference dave treder

How do Teachers’ Ranks Compare, F07-08 to F08-F09? this study?

Math –Teacher Ranks (Mean Residual), F07-08 X F08-F09

Rank – F08 – F09

Rank – F07 – F08


Using meap to evaluate teachers mera fall 2011 conference dave treder

How do Teachers’ Ranks Compare, this study?Within Districts, F07-08 to F08-F09?

MATHWithin District Relationship - Quintile Rank – SS residual


Using meap to evaluate teachers mera fall 2011 conference dave treder

How do Teachers’ Ranks Compare, F07-08 to F08-F09? this study?

Reading –Teacher Ranks (Mean Residual), F07-08 X F08-F09

Rank – F08 – F09

Rank – F07 – F08


Using meap to evaluate teachers mera fall 2011 conference dave treder

How do Teachers’ Ranks Compare, this study?Within Districts, F07-08 to F08-F09?

READINGWithin District Relationship - Quintile Rank – SS residual


Using meap to evaluate teachers mera fall 2011 conference dave treder

Method 2 this study?

1) Compute by-student Performance Level (PL) Change Index

2) Compute Mean Teacher PL change index

-- Computed with the New Cut Score Sub-Level Cut Scores --

2011 Top-to-Bottom PL Change Business Rules

State Top-to-Bottom Ranking

Performance Level Change Index

FOR EACH THE SCHOOL, THE PERFORMANCE LEVEL CHANGE SCORES ARE SUMMED ACROSS STUDENTS AND AN AVERAGE IS TAKEN TO CREATE THE TWO-YEAR AVERAGE PERFORMANCE LEVEL CHANGE INDEX. THE INDEX FOR EACH SCHOOL IS THEN COMPARED TO THE REST OF THE SCHOOLS IN THE STATE TO CREATE A SCHOOL-LEVEL PERFORMANCE LEVEL CHANGE Z-SCORE. THAT Z-SCORE IS THEN MULTIPLIED BY 1/4 TO CONTRIBUTE TO THE OVERALL SCHOOL INDEX IN THE CONTENT AREA.


Using meap to evaluate teachers mera fall 2011 conference dave treder

Top-to-Bottom List*- School-Level this study?Correlation - Reading, Grds 3-8

Relationship between:

PL Change/Growth F07-to-F08 & PL Change/Growth F08-to-F09

N = 2500

r = .032

p = .11

F07 to F08

F08 to F09

*Original Top-to-Bottom Rules: (N Imp – N Dec )/ N Matched



Using meap to evaluate teachers mera fall 2011 conference dave treder

Method 2 – General Results this study?

How much of the Variance can be Explained at the Teacher Level?

By Teacher, Mean Residual, Fall07-08

Math: 15%

Reading: 4%

What is the Relationship Between Teacher Level Mean Residuals, F07-08 & F08-09?

Correlation

Math: .42

Reading: .15


Using meap to evaluate teachers mera fall 2011 conference dave treder

Relationship Between this study?Teacher PL Change Index, F07 to F08 X F08 to F09

Math –By Teacher PL Change Index, F07-08 X F08-F09

R = .42

PL Change Index – F08 – F09

PL Change Index – F07 – F08


Using meap to evaluate teachers mera fall 2011 conference dave treder

What’s the Relationship Between a Teacher’s Classroom Starting Point (Mean Scale Score) and Their PL Change Index?

MATH –By Teacher, Mean MEAP Scl Score Z– F07 X PL Change IndexF07-08

R = -.46

PL Change Index – F07 – F08

Mean MEAP Scl Score Z– F07


Using meap to evaluate teachers mera fall 2011 conference dave treder

2011 State Top-to-bottom Math – Pl Change X Achievement Status

(Old Cut Scores, New Rules)

R = - .81

PL Change Z score

Achievement Status z score


Using meap to evaluate teachers mera fall 2011 conference dave treder

How do Teachers’ Ranks Compare, F07-08 to F08-F09? Status

MATH –Teacher Ranks (PL Change Index), F07-08 X F08-F09

Rank – F08 – F09

Rank – F07 – F08


Using meap to evaluate teachers mera fall 2011 conference dave treder

How do Teachers’ Ranks Compare, StatusWithin Districts, F07-08 to F08-F09?

MATH –Within District Relationship quintile rank - PL Change Index


Using meap to evaluate teachers mera fall 2011 conference dave treder

How do Teachers’ Ranks Compare, F07-08 to F08-F09? Status

READING –Teacher Ranks (PL Change Index), F07-08 X F08-F09

Rank – F08 – F09

Rank – F07 – F08


Using meap to evaluate teachers mera fall 2011 conference dave treder

How do Teachers’ Ranks Compare, StatusWithin Districts, F07-08 to F08-F09?

READING –Within District Relationship quintile rank - PL Change Index


Using meap to evaluate teachers mera fall 2011 conference dave treder

Relationship Between StatusTeacher Mean Residual & PL Change Index

READING –By Teacher, Mean Residual X PL Change Index (F07to 08)

R = .88

PL Change Index – F07 – F08

Mean Residual


Using meap to evaluate teachers mera fall 2011 conference dave treder

Relationship Between StatusTeacher Mean Residual & PL Change Index

MATH –By Teacher, Mean Residual X PL Change Index (F07to 08)

R = .77

PL Change Index – F07 – F08

Mean Residual – F07 – F08


Using meap to evaluate teachers mera fall 2011 conference dave treder

…Error rates for teacher-level analyses will be about 26 percent if three years of data are use for estimation. This means that in a typical performance measurement system, more than 1 in 4 teachers who are truly average in performance will be erroneously identified for special treatment, and more than 1 in 4 teachers who [exceed] average performance by will be overlooked.


Using meap to evaluate teachers mera fall 2011 conference dave treder

Is Your PL Change Index Helped (or Hurt), based on the Grade You Teach? (Or, On the Grade Levels in Your School)?

MEAP PL Change Index, Math & Reading, F06 -- F10 (GISD Students)

Math

PL Change Index

Reading

Gr3-Gr4 Gr4-Gr5 Gr5-Gr6 Gr6-Gr7Gr7-Gr8

Original Top-to-Bottom Rules: (N Imp – N Dec )/ N Matched


Using meap to evaluate teachers mera fall 2011 conference dave treder

Is Your PL Change Index Helped (or Hurt), based on the Grade You Teach? (Or, On the Grade Levels in Your School?

(OLD Cut Scores, NEW PL Index Rules)

MEAP PL Change Index, Math & Reading, 08-09 & 09-10 (GISD Students)

Math

PL Change Index

Reading


Using meap to evaluate teachers mera fall 2011 conference dave treder

Old & New Cut Scores You Teach? (Or, On the Grade Levels in Your School?

MATH

READING


Using meap to evaluate teachers mera fall 2011 conference dave treder

Is Your PL Change Index Helped (or Hurt), based on the Grade You Teach? (Or, On the Grade Levels in Your School?

(NEW Cut Scores, NEW PL Index Rules)

MEAP PL Change Index, Math & Reading, 08-09 & 09-10 (GISD Students)

Math

Reading

PL Change Index

Reading

School-Level Std Dev- Math: .21 (08-09) .27 (09-10) Reading: .16 (08-09) .18 (09-10)


Using meap to evaluate teachers mera fall 2011 conference dave treder

Conclusion You Teach? (Or, On the Grade Levels in Your School?

Developing a Growth/Progress Measure, Based on a Single Summative Assessment, that:

1) is stable across years;

2) doesn’t punish high performing students/ classes/schools;

3) incorporates the need to expect low performing students/classes/schools to “grow” at a greater rate than low performing students/classes/schools.

Ain’t easy