Agenda
This presentation is the property of its rightful owner.
Sponsored Links
1 / 73

Agenda PowerPoint PPT Presentation


  • 77 Views
  • Uploaded on
  • Presentation posted in: General

Agenda. Student Learning Growth Teacher VAM School Growth PYG Area Scorecards. PYG, and other Performance Indicators. Florida Department of Education’s “Value-Added Model (VAM)”. Marc Baron, Chief, Performance Accountability Mark Howard, Director, Research, Evaluation, and Assessment.

Download Presentation

Agenda

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Agenda

  • Student Learning Growth

    • Teacher VAM

    • School Growth

  • PYG

  • Area Scorecards. PYG, and other Performance Indicators


Florida Department of Education’s “Value-Added Model (VAM)”

Marc Baron, Chief, Performance Accountability

Mark Howard, Director, Research, Evaluation, and Assessment


FLDOE Value-Added ModelEstablished by Student Success Act (SB 736)

  • Evaluation Criteria for All Instructional Personnel and School Administrators

    • Professional Practice

    • Student Learning Growth

      • FCAT Classroom Teachers: 40%-50%

      • Non FCAT Classroom Teachers: 30%-40%

      • Non Classroom Teachers: 20%-30%

      • Principals: 50%

  • JTEC & Principal Evaluation Committee


SDPBC Must Classify Instructional Personnel and School Administratorsinto these Four Categories

  • Highly Effective

  • Effective

  • Needs Improvement

  • Unsatisfactory


VAM & Student Learning Growth

  • Contribution to a change in a student’s achievement

  • Calculated from a measure of student learning growth


What is the Student Learning Growth Score?

The difference between

Current test score and

Predicted test score

What is the Predicted Student Score?


What is the Predicted Student Score?

It is score you would predict a student to achieve based on the student’s performance on prior tests and other information available on the student


FLDOE Value-Added ModelVariables determining student learning growth

  • Two or more years of prior achievement scores

  • Gifted status

  • Student Attendance (Days)

  • Mobility (number of transitions)

  • Difference from modal age in grade (indicator of retention)

  • Class size

  • The number of subject-relevant courses in which the student is enrolled

  • Homogeneity of entering test scores in the class


FLDOE Value-Added ModelVariables determining student learning growth

  • Students with Disabilities (SWD) status

    • Language impaired

    • Hearing impaired

    • Visually impaired

    • Emotional/behavioral

    • Specific learning disability

    • Dual sensory impaired

    • Autism spectrum disorder

    • Traumatic brain injury

    • Other health impaired

    • Intellectual disability

  • English Language Learner (ELL) status

    • LY


FLDOE Value-Added ModelVariables not determining student learning growth

Gender

Race

Ethnicity

Socioeconomic status


FLDOE Value-Added Model Variables determining student learning growth

TERMS

District Data Base of All Student Records


What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

900

800

CURRENT TEST

700

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


W

What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

900

800

CURRENT TEST

700

600

500

400

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

Answer:

Average of

current scores

of similar students

900

800

CURRENT TEST

700

PREDICTED SCORE

600

600

500

400

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


Student Learning Growth is the Amount Above or Below Predicted Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

900

800

ACTUAL SCORE

CURRENT TEST

Above Predicted Score

700

PREDICTED SCORE

600

600

500

Below Predicted Score

400

ACTUAL SCORE

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the Predicted Student Score? Based on Prior Achievement

Based on Prior Achievement Only

Based on Prior Achievement & ELL

(FY11 FCAT)

…100 200 300 400 500 600 700 800 900 1000…

10 20 30 40 50 60 70 80 90 100…

Each prior year test score has a Predicted current year test score.

90

800

CURRENT TEST

70

CURRENT TEST

600

50

400

10 20 30 40 50 60 70 80 90 100…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

PRIOR TEST


FLDOE Value-Added ModelVariables determining student learning growth

Two or more years of prior achievement scores

Attendance

Mobility (number of transitions)

Difference from modal age in grade (indicator of retention)

Class size

The number of subject-relevant courses in which the student is enrolled

Homogeneity of entering test scores in the class

LY

Gifted status

Language impaired

Hearing impaired

Visually impaired

Emotional/behavioral

Specific learning disability

Dual sensory impaired

Autism spectrum disorder

Traumatic brain injury

Other health impaired

Intellectual disability


Reading Grade 7Indicators Ranked By Statistical Significance


Illustrative Growth Points All other variables being equal

*adapted from AIR materials presented at August 2, 2011,meeting

These are illustrative values, not actual values


What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

Based on Prior Achievement Only

Based on Prior Achievement & ELL

(FY11 FCAT)

10 20 30 40 50 60 70 80 90 100…

90

800

CURRENT TEST

70

CURRENT TEST

600

50

400

10 20 30 40 50 60 70 80 90 100…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

PRIOR TEST


What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

Based on Prior Achievement Only

Based on Prior Achievement & ELL

(FY11 FCAT)

(FY11 FCAT)

900

800

CURRENT TEST

700

CURRENT TEST

600

500

400

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

(FY10 FCAT)

PRIOR TEST


What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

Based on Prior Achievement Only

Based on Prior Achievement

& Low Attendance

(FY11 FCAT)

(FY11 FCAT)

800

700

CURRENT TEST

CURRENT TEST

600

500

400

300

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

(FY10 FCAT)

PRIOR TEST


How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

The diagonal line is the predicted growth observed among similar students.

(FY11 FCAT)

CURRENT TEST

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


W

How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

100

550

450

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Y

How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

? ? ? ?

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How do you calculate the teacher growth score?

  • Identify the students assigned to the teacher

  • Determine the learning growth of each of the assigned students

  • Calculate the average learning growth of the students

TERMS


Teacher Growth Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

Typical

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Teacher Growth Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

Above

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Teacher Growth Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

Below

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Teacher Growth

School

Growth

Teacher

Growth

Avg Growth = (A+B+C+D)/4


Teacher Effect

School

Growth

Teacher

Effect


Calculating Teacher Value-Added ScorePart 1

Teacher Effect

(

)

-

=

School

Growth

Score

Teacher

Growth

Score

DRE


How do you calculate the school learning growth score?

  • Identify the students assigned to the school

  • Determine the learning growth of each of the assigned students

  • Calculate the average learning growth of the students


FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

School Growth Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Calculating Teacher Value-Added ScorePart 1: AKA Teacher Effect

Tcheffect = (Tchgrowth score – Schgrowth score )

DRE


What is a Teacher Value-Added Score?

and

How is it calculated?

DRE


FLDOE Teacher Value-Added Score Conceptual

Teacher Effect

)

(

)

(

-

+

.5

X

=

School

Growth

Score

School

Growth

Score

Teacher

Growth

Score

Tchvas = (Tchgrowth score – Schgrowth score ) + ( .5 x Schgrowth score)

DRE


Teacher Value-Added Score

Tchvas = (Tchgrowth score – Schgrowth score ) + ( .5 x Schgrowth score)

DRE


State Average Student Learning Growth

Teacher

Growth = 20

The greater a teacher’s growth score is from the state’s average and the school’s growth score, the higher a teacher’s VAS

VAS = 15

VAS = 20

VAS = 25

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


State Average Student Learning Growth

Schoo

l

Growth = 0

At a school with a growth score of 0, the teacher’s VAS equals the teacher’s growth

VAS = 20

VAS = 0

VAS = -20

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


State Average Student Learning Growth

Schoo

l

Growth = 10

What is the significance of the green line?

At a high VAS school, a teacher’s growth must exceed half the school’s growth to have a positive VAS

VAS = 15

VAS = 5

VAS = 0

VAS = -5

VAS = -10

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


State Average Student Learning Growth

Schoo

l

Growth = -10

At a low VAS school, a teacher’s growth must exceed half the school’s growth to have a positive VAS

VAS = -15

VAS = -5

VAS =0

VAS = 5

VAS = 10

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


A Real Picture is Worth a Thousand Words

=


Teacher VAM


SDPBC Teacher VAM (Grade 5 Subject: Reading)

Mean = 3.3

N = 561

Class >= 10


State Teacher VAM (Grade 5 Subject: Reading)

Mean = 0.1

N = 12182


Teacher VAM Distribution for Reading


School Growth


SDPBC School GrowthGrade 5 Subject: Reading

Mean = 4.0

N = 120


State School GrowthGrade 5 Subject: Reading

Mean = 0.02

N = 2082


School ComponentDistribution for Reading


How precise are VAM measures?


Standard Error of Teacher VAMs

  • The standard error gives the uncertainty (error band) surrounding a teacher’s VAM score

  • It can be used to prevent categorizing teachers when that categorization would be uncertain


Standard Error of Teacher VAMs

Teacher Score

Minus

one SE

Plus

one SE


Using SE to Categorize Teachers

State Average Score

Teacher 2

Teacher 1

Teacher 3

Teacher 4


Teacher VAMs Similar to State

State

Mean

Sig from State

# of

Stdts

VAM

No sig higher &

No sig diff

No sig higher &

No sig diff

No sig higher &

No sig diff


Some teacher VAMs Higher than State

Sig from State

# of

Stdts

State

Mean

VAM

1 sig higher &

No sig diff

2 sig higher &

No sig diff

4 sig higher &

sig differences


Some teacher VAMs Lower than State

Sig from State

# of

Stdts

State

Mean

VAM

1 sig lower

No sig diff

2 sig lower &

sig differences

3 sig lower &

No sig diff


Some Teacher VAMs Lower/Higher than State

Sig from State

# of

Stdts

State

Mean

VAM

1 sig higher

2 sig lower &

sig differences

1 sig higher

1 sig lower &

sig differences


Summary: VAM and School Learning Growth

  • State Mandated

  • Student Learning Growth

    • Difference between current score and predicted score

    • Based on prior achievement and selected student characteristics

  • Teacher VAM includes both teacher effect and school learning growth

  • School component is school learning growth

  • Consider standard error when using Teacher VAM and school learning growth


SDPBC Must Classify Instructional Personnel/School Administratorsinto these Four Categories

  • Highly Effective

  • Effective

  • Needs Improvement

  • Unsatisfactory


Proportion of a Year’s Growth& School Growth Score


PYG

FY10 FY11 FY11

FCAT FCAT -FY10

…100 200 300 400 500 600 700 800 900 1000…

One Year’s Worth of Growth

M

O

R

E

(FY11 FCAT)

L

E

S

S

CURRENT TEST

80

100

125

150

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


A Real Picture is Worth a Thousand Words

=


SDPBC

Mean


11

10

09

08

07


Area Scorecard and PYG

To review area and school scorecards, go to the EDW.


Summary: PYG and Scorecard

  • PYG

    • Difference between current score and predicted score

    • Highly correlated with school learning growth

    • Use PYG reports and scorecard to see how you would have done?


  • Login