Agenda
This presentation is the property of its rightful owner.
Sponsored Links
1 / 73

Agenda PowerPoint PPT Presentation


  • 73 Views
  • Uploaded on
  • Presentation posted in: General

Agenda. Student Learning Growth Teacher VAM School Growth PYG Area Scorecards. PYG, and other Performance Indicators. Florida Department of Education’s “Value-Added Model (VAM)”. Marc Baron, Chief, Performance Accountability Mark Howard, Director, Research, Evaluation, and Assessment.

Download Presentation

Agenda

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Agenda

Agenda

  • Student Learning Growth

    • Teacher VAM

    • School Growth

  • PYG

  • Area Scorecards. PYG, and other Performance Indicators


Florida department of education s value added model vam

Florida Department of Education’s “Value-Added Model (VAM)”

Marc Baron, Chief, Performance Accountability

Mark Howard, Director, Research, Evaluation, and Assessment


Fldoe value added model established by student success act sb 736

FLDOE Value-Added ModelEstablished by Student Success Act (SB 736)

  • Evaluation Criteria for All Instructional Personnel and School Administrators

    • Professional Practice

    • Student Learning Growth

      • FCAT Classroom Teachers: 40%-50%

      • Non FCAT Classroom Teachers: 30%-40%

      • Non Classroom Teachers: 20%-30%

      • Principals: 50%

  • JTEC & Principal Evaluation Committee


Sdpbc must classify instructional personnel and school administrators into these four categories

SDPBC Must Classify Instructional Personnel and School Administratorsinto these Four Categories

  • Highly Effective

  • Effective

  • Needs Improvement

  • Unsatisfactory


Vam student learning growth

VAM & Student Learning Growth

  • Contribution to a change in a student’s achievement

  • Calculated from a measure of student learning growth


What is the student learning growth score

What is the Student Learning Growth Score?

The difference between

Current test score and

Predicted test score

What is the Predicted Student Score?


What is the predicted student score

What is the Predicted Student Score?

It is score you would predict a student to achieve based on the student’s performance on prior tests and other information available on the student


Fldoe value added model variables determining student learning growth

FLDOE Value-Added ModelVariables determining student learning growth

  • Two or more years of prior achievement scores

  • Gifted status

  • Student Attendance (Days)

  • Mobility (number of transitions)

  • Difference from modal age in grade (indicator of retention)

  • Class size

  • The number of subject-relevant courses in which the student is enrolled

  • Homogeneity of entering test scores in the class


Fldoe value added model variables determining student learning growth1

FLDOE Value-Added ModelVariables determining student learning growth

  • Students with Disabilities (SWD) status

    • Language impaired

    • Hearing impaired

    • Visually impaired

    • Emotional/behavioral

    • Specific learning disability

    • Dual sensory impaired

    • Autism spectrum disorder

    • Traumatic brain injury

    • Other health impaired

    • Intellectual disability

  • English Language Learner (ELL) status

    • LY


Fldoe value added model variables not determining student learning growth

FLDOE Value-Added ModelVariables not determining student learning growth

Gender

Race

Ethnicity

Socioeconomic status


Fldoe value added model variables determining student learning growth2

FLDOE Value-Added Model Variables determining student learning growth

TERMS

District Data Base of All Student Records


What is the predicted student score1

What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the predicted student score2

What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

900

800

CURRENT TEST

700

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the predicted student score3

W

What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

900

800

CURRENT TEST

700

600

500

400

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the predicted student score4

What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

Answer:

Average of

current scores

of similar students

900

800

CURRENT TEST

700

PREDICTED SCORE

600

600

500

400

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


Student learning growth is the amount above or below predicted score

Student Learning Growth is the Amount Above or Below Predicted Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

900

800

ACTUAL SCORE

CURRENT TEST

Above Predicted Score

700

PREDICTED SCORE

600

600

500

Below Predicted Score

400

ACTUAL SCORE

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the predicted student score based on prior achievement

What is the Predicted Student Score? Based on Prior Achievement

Based on Prior Achievement Only

Based on Prior Achievement & ELL

(FY11 FCAT)

…100 200 300 400 500 600 700 800 900 1000…

10 20 30 40 50 60 70 80 90 100…

Each prior year test score has a Predicted current year test score.

90

800

CURRENT TEST

70

CURRENT TEST

600

50

400

10 20 30 40 50 60 70 80 90 100…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

PRIOR TEST


Fldoe value added model variables determining student learning growth3

FLDOE Value-Added ModelVariables determining student learning growth

Two or more years of prior achievement scores

Attendance

Mobility (number of transitions)

Difference from modal age in grade (indicator of retention)

Class size

The number of subject-relevant courses in which the student is enrolled

Homogeneity of entering test scores in the class

LY

Gifted status

Language impaired

Hearing impaired

Visually impaired

Emotional/behavioral

Specific learning disability

Dual sensory impaired

Autism spectrum disorder

Traumatic brain injury

Other health impaired

Intellectual disability


Reading grade 7 indicators ranked by statistical significance

Reading Grade 7Indicators Ranked By Statistical Significance


Illustrative growth points all other variables being equal

Illustrative Growth Points All other variables being equal

*adapted from AIR materials presented at August 2, 2011,meeting

These are illustrative values, not actual values


What is the predicted student score5

What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

Based on Prior Achievement Only

Based on Prior Achievement & ELL

(FY11 FCAT)

10 20 30 40 50 60 70 80 90 100…

90

800

CURRENT TEST

70

CURRENT TEST

600

50

400

10 20 30 40 50 60 70 80 90 100…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

PRIOR TEST


What is the predicted student score6

What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

Based on Prior Achievement Only

Based on Prior Achievement & ELL

(FY11 FCAT)

(FY11 FCAT)

900

800

CURRENT TEST

700

CURRENT TEST

600

500

400

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

(FY10 FCAT)

PRIOR TEST


What is the predicted student score7

What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

Based on Prior Achievement Only

Based on Prior Achievement

& Low Attendance

(FY11 FCAT)

(FY11 FCAT)

800

700

CURRENT TEST

CURRENT TEST

600

500

400

300

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

(FY10 FCAT)

PRIOR TEST


How to calculate the student learning growth score

How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

The diagonal line is the predicted growth observed among similar students.

(FY11 FCAT)

CURRENT TEST

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How to calculate the student learning growth score1

How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How to calculate the student learning growth score2

W

How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

100

550

450

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How to calculate the student learning growth score3

Y

How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

? ? ? ?

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How to calculate the student learning growth score4

How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How do you calculate the teacher growth score

How do you calculate the teacher growth score?

  • Identify the students assigned to the teacher

  • Determine the learning growth of each of the assigned students

  • Calculate the average learning growth of the students

TERMS


Teacher growth score

Teacher Growth Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

Typical

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Teacher growth score1

Teacher Growth Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

Above

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Teacher growth score2

Teacher Growth Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

Below

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Teacher growth

Teacher Growth

School

Growth

Teacher

Growth

Avg Growth = (A+B+C+D)/4


Teacher effect

Teacher Effect

School

Growth

Teacher

Effect


Calculating teacher value added score part 1

Calculating Teacher Value-Added ScorePart 1

Teacher Effect

(

)

-

=

School

Growth

Score

Teacher

Growth

Score

DRE


How do you calculate the school learning growth score

How do you calculate the school learning growth score?

  • Identify the students assigned to the school

  • Determine the learning growth of each of the assigned students

  • Calculate the average learning growth of the students


School growth score

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

School Growth Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Calculating teacher value added score part 1 aka teacher effect

Calculating Teacher Value-Added ScorePart 1: AKA Teacher Effect

Tcheffect = (Tchgrowth score – Schgrowth score )

DRE


Agenda

What is a Teacher Value-Added Score?

and

How is it calculated?

DRE


Fldoe teacher value added score conceptual

FLDOE Teacher Value-Added Score Conceptual

Teacher Effect

)

(

)

(

-

+

.5

X

=

School

Growth

Score

School

Growth

Score

Teacher

Growth

Score

Tchvas = (Tchgrowth score – Schgrowth score ) + ( .5 x Schgrowth score)

DRE


Teacher value added score

Teacher Value-Added Score

Tchvas = (Tchgrowth score – Schgrowth score ) + ( .5 x Schgrowth score)

DRE


Agenda

State Average Student Learning Growth

Teacher

Growth = 20

The greater a teacher’s growth score is from the state’s average and the school’s growth score, the higher a teacher’s VAS

VAS = 15

VAS = 20

VAS = 25

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


Agenda

State Average Student Learning Growth

Schoo

l

Growth = 0

At a school with a growth score of 0, the teacher’s VAS equals the teacher’s growth

VAS = 20

VAS = 0

VAS = -20

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


Agenda

State Average Student Learning Growth

Schoo

l

Growth = 10

What is the significance of the green line?

At a high VAS school, a teacher’s growth must exceed half the school’s growth to have a positive VAS

VAS = 15

VAS = 5

VAS = 0

VAS = -5

VAS = -10

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


Agenda

State Average Student Learning Growth

Schoo

l

Growth = -10

At a low VAS school, a teacher’s growth must exceed half the school’s growth to have a positive VAS

VAS = -15

VAS = -5

VAS =0

VAS = 5

VAS = 10

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


A real picture is worth a thousand words

A Real Picture is Worth a Thousand Words

=


Teacher vam

Teacher VAM


Sdpbc teacher vam grade 5 subject reading

SDPBC Teacher VAM (Grade 5 Subject: Reading)

Mean = 3.3

N = 561

Class >= 10


State teacher vam grade 5 subject reading

State Teacher VAM (Grade 5 Subject: Reading)

Mean = 0.1

N = 12182


Teacher vam distribution for reading

Teacher VAM Distribution for Reading


School growth

School Growth


Sdpbc school growth grade 5 subject reading

SDPBC School GrowthGrade 5 Subject: Reading

Mean = 4.0

N = 120


State school growth grade 5 subject reading

State School GrowthGrade 5 Subject: Reading

Mean = 0.02

N = 2082


School component distribution for reading

School ComponentDistribution for Reading


How precise are vam measures

How precise are VAM measures?


Standard error of teacher vams

Standard Error of Teacher VAMs

  • The standard error gives the uncertainty (error band) surrounding a teacher’s VAM score

  • It can be used to prevent categorizing teachers when that categorization would be uncertain


Standard error of teacher vams1

Standard Error of Teacher VAMs

Teacher Score

Minus

one SE

Plus

one SE


Using se to categorize teachers

Using SE to Categorize Teachers

State Average Score

Teacher 2

Teacher 1

Teacher 3

Teacher 4


Agenda

Teacher VAMs Similar to State

State

Mean

Sig from State

# of

Stdts

VAM

No sig higher &

No sig diff

No sig higher &

No sig diff

No sig higher &

No sig diff


Agenda

Some teacher VAMs Higher than State

Sig from State

# of

Stdts

State

Mean

VAM

1 sig higher &

No sig diff

2 sig higher &

No sig diff

4 sig higher &

sig differences


Agenda

Some teacher VAMs Lower than State

Sig from State

# of

Stdts

State

Mean

VAM

1 sig lower

No sig diff

2 sig lower &

sig differences

3 sig lower &

No sig diff


Agenda

Some Teacher VAMs Lower/Higher than State

Sig from State

# of

Stdts

State

Mean

VAM

1 sig higher

2 sig lower &

sig differences

1 sig higher

1 sig lower &

sig differences


Summary vam and school learning growth

Summary: VAM and School Learning Growth

  • State Mandated

  • Student Learning Growth

    • Difference between current score and predicted score

    • Based on prior achievement and selected student characteristics

  • Teacher VAM includes both teacher effect and school learning growth

  • School component is school learning growth

  • Consider standard error when using Teacher VAM and school learning growth


Sdpbc must classify instructional personnel school administrators into these four categories

SDPBC Must Classify Instructional Personnel/School Administratorsinto these Four Categories

  • Highly Effective

  • Effective

  • Needs Improvement

  • Unsatisfactory


Proportion of a year s growth school growth score

Proportion of a Year’s Growth& School Growth Score


Agenda

PYG

FY10 FY11 FY11

FCAT FCAT -FY10

…100 200 300 400 500 600 700 800 900 1000…

One Year’s Worth of Growth

M

O

R

E

(FY11 FCAT)

L

E

S

S

CURRENT TEST

80

100

125

150

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


A real picture is worth a thousand words1

A Real Picture is Worth a Thousand Words

=


Agenda

SDPBC

Mean


Agenda

11

10

09

08

07


Area scorecard and pyg

Area Scorecard and PYG

To review area and school scorecards, go to the EDW.


Summary pyg and scorecard

Summary: PYG and Scorecard

  • PYG

    • Difference between current score and predicted score

    • Highly correlated with school learning growth

    • Use PYG reports and scorecard to see how you would have done?


  • Login