slide1
Download
Skip this Video
Download Presentation
Using MEAP To Evaluate Teachers MERA Fall 2011 Conference Dave Treder

Loading in 2 Seconds...

play fullscreen
1 / 31

Using MEAP To Evaluate Teachers MERA Fall 2011 Conference Dave Treder - PowerPoint PPT Presentation


  • 155 Views
  • Uploaded on

Using MEAP To Evaluate Teachers MERA Fall 2011 Conference Dave Treder. Study Methods Looked at MEAP growth/change over two consecutive years (grade 4 MEAP to grade 5 MEAP): Both Fall 07 to Fall 08 and Fall 08 to Fall 09. Counts. -- 20 of the 21 school districts In Genesee ISD.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Using MEAP To Evaluate Teachers MERA Fall 2011 Conference Dave Treder' - dobry


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Using MEAP

To Evaluate

Teachers

MERA Fall 2011 Conference

Dave Treder

slide2

Study Methods

  • Looked at MEAP growth/change over two consecutive years
  • (grade 4 MEAP to grade 5 MEAP):
  • Both Fall 07 to Fall 08 and Fall 08 to Fall 09

Counts

-- 20 of the 21 school districts In Genesee ISD

slide3

Description of Study Data

  • 4th Grade Teachers
  • 20 of the 21 Districts In Genesee ISD
  • Based on the Districts’ SIS system
  • Business rules:
  • - If schedules weren’t used in the district, the homeroom teacher was coded as the math and reading teacher – in the districts that used schedules in the elementary grades, math and reading classes were used to designate the math and reading teacher
  • - Teachers were included if they were the teacher of record at the beginning of the school year (1st semester grades) and the end of the school year (last semester grades)
  • - Students included if they had the same teacher at the beginning of the school year (1st semester grades) and the end of the school year (2nd semester grades)
slide4

Number of 4th Grade Teachers, by District:

F07-08, F08-09, and in the District Both Years

74%

slide5

What was the size of the classrooms, for the teachers in this study?

(2007-08 School Year)

F07-08 Class Size Frequency

Class Size

slide6

Study Methods

Method 1

1) Compute by-student residuals

(non-linear regression, student predicted - actual scale score)

2) Compute Mean Teacher residual

Method 2

1) Compute by-student Performance Level (PL) Change Index

2) Compute Mean Teacher PL change index

  • For each method, compute indices for F07-F08 and F08-F09.
  • Look at the relationship between the two years.

Also Tried:

1) Converting Scales Scores to Z Scores

2) Compute by-student ‘’change”: Year2 Z_Score – Year1 Z_Score

3) Compute mean teacher change

slide7

Computing By-Student Residuals (Predicted – Actual)

Math F07-F08

Scale Score, Grade 5

Scale Score, Grade 4

slide8

Method 1 – General Results

How Much of the Variance can be Explained at the Teacher Level?

By Teacher, Mean Residual, Fall07-08

Math: 15%

Reading: 1.5%

What is the Relationship Between Teacher Level Mean Residuals,

F07-08 & F08-09?

Correlation:

Math: .32

Reading: .19

slide9

How do Teachers’ Ranks Compare, F07-08 to F08-F09?

Math –Teacher Ranks (Mean Residual), F07-08 X F08-F09

Rank – F08 – F09

Rank – F07 – F08

slide10

How do Teachers’ Ranks Compare, Within Districts, F07-08 to F08-F09?

MATHWithin District Relationship - Quintile Rank – SS residual

slide11

How do Teachers’ Ranks Compare, F07-08 to F08-F09?

Reading –Teacher Ranks (Mean Residual), F07-08 X F08-F09

Rank – F08 – F09

Rank – F07 – F08

slide12

How do Teachers’ Ranks Compare, Within Districts, F07-08 to F08-F09?

READINGWithin District Relationship - Quintile Rank – SS residual

slide13

Method 2

1) Compute by-student Performance Level (PL) Change Index

2) Compute Mean Teacher PL change index

-- Computed with the New Cut Score Sub-Level Cut Scores --

2011 Top-to-Bottom PL Change Business Rules

State Top-to-Bottom Ranking

Performance Level Change Index

FOR EACH THE SCHOOL, THE PERFORMANCE LEVEL CHANGE SCORES ARE SUMMED ACROSS STUDENTS AND AN AVERAGE IS TAKEN TO CREATE THE TWO-YEAR AVERAGE PERFORMANCE LEVEL CHANGE INDEX. THE INDEX FOR EACH SCHOOL IS THEN COMPARED TO THE REST OF THE SCHOOLS IN THE STATE TO CREATE A SCHOOL-LEVEL PERFORMANCE LEVEL CHANGE Z-SCORE. THAT Z-SCORE IS THEN MULTIPLIED BY 1/4 TO CONTRIBUTE TO THE OVERALL SCHOOL INDEX IN THE CONTENT AREA.

slide14

Top-to-Bottom List*- School-Level Correlation - Reading, Grds 3-8

Relationship between:

PL Change/Growth F07-to-F08 & PL Change/Growth F08-to-F09

N = 2500

r = .032

p = .11

F07 to F08

F08 to F09

*Original Top-to-Bottom Rules: (N Imp – N Dec )/ N Matched

slide16

Method 2 – General Results

How much of the Variance can be Explained at the Teacher Level?

By Teacher, Mean Residual, Fall07-08

Math: 15%

Reading: 4%

What is the Relationship Between Teacher Level Mean Residuals, F07-08 & F08-09?

Correlation

Math: .42

Reading: .15

slide17

Relationship Between Teacher PL Change Index, F07 to F08 X F08 to F09

Math –By Teacher PL Change Index, F07-08 X F08-F09

R = .42

PL Change Index – F08 – F09

PL Change Index – F07 – F08

slide18

What’s the Relationship Between a Teacher’s Classroom Starting Point (Mean Scale Score) and Their PL Change Index?

MATH –By Teacher, Mean MEAP Scl Score Z– F07 X PL Change IndexF07-08

R = -.46

PL Change Index – F07 – F08

Mean MEAP Scl Score Z– F07

slide19

2011 State Top-to-bottom Math – Pl Change X Achievement Status

(Old Cut Scores, New Rules)

R = - .81

PL Change Z score

Achievement Status z score

slide20

How do Teachers’ Ranks Compare, F07-08 to F08-F09?

MATH –Teacher Ranks (PL Change Index), F07-08 X F08-F09

Rank – F08 – F09

Rank – F07 – F08

slide21

How do Teachers’ Ranks Compare, Within Districts, F07-08 to F08-F09?

MATH –Within District Relationship quintile rank - PL Change Index

slide22

How do Teachers’ Ranks Compare, F07-08 to F08-F09?

READING –Teacher Ranks (PL Change Index), F07-08 X F08-F09

Rank – F08 – F09

Rank – F07 – F08

slide23

How do Teachers’ Ranks Compare, Within Districts, F07-08 to F08-F09?

READING –Within District Relationship quintile rank - PL Change Index

slide24

Relationship Between Teacher Mean Residual & PL Change Index

READING –By Teacher, Mean Residual X PL Change Index (F07to 08)

R = .88

PL Change Index – F07 – F08

Mean Residual

slide25

Relationship Between Teacher Mean Residual & PL Change Index

MATH –By Teacher, Mean Residual X PL Change Index (F07to 08)

R = .77

PL Change Index – F07 – F08

Mean Residual – F07 – F08

slide26

…Error rates for teacher-level analyses will be about 26 percent if three years of data are use for estimation. This means that in a typical performance measurement system, more than 1 in 4 teachers who are truly average in performance will be erroneously identified for special treatment, and more than 1 in 4 teachers who [exceed] average performance by will be overlooked.

slide27

Is Your PL Change Index Helped (or Hurt), based on the Grade You Teach? (Or, On the Grade Levels in Your School)?

MEAP PL Change Index, Math & Reading, F06 -- F10 (GISD Students)

Math

PL Change Index

Reading

Gr3-Gr4 Gr4-Gr5 Gr5-Gr6 Gr6-Gr7Gr7-Gr8

Original Top-to-Bottom Rules: (N Imp – N Dec )/ N Matched

slide28

Is Your PL Change Index Helped (or Hurt), based on the Grade You Teach? (Or, On the Grade Levels in Your School?

(OLD Cut Scores, NEW PL Index Rules)

MEAP PL Change Index, Math & Reading, 08-09 & 09-10 (GISD Students)

Math

PL Change Index

Reading

slide30

Is Your PL Change Index Helped (or Hurt), based on the Grade You Teach? (Or, On the Grade Levels in Your School?

(NEW Cut Scores, NEW PL Index Rules)

MEAP PL Change Index, Math & Reading, 08-09 & 09-10 (GISD Students)

Math

Reading

PL Change Index

Reading

School-Level Std Dev- Math: .21 (08-09) .27 (09-10) Reading: .16 (08-09) .18 (09-10)

slide31

Conclusion

Developing a Growth/Progress Measure, Based on a Single Summative Assessment, that:

1) is stable across years;

2) doesn’t punish high performing students/ classes/schools;

3) incorporates the need to expect low performing students/classes/schools to “grow” at a greater rate than low performing students/classes/schools.

Ain’t easy

ad