Agenda
Download
1 / 73

Agenda - PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on

Agenda. Student Learning Growth Teacher VAM School Growth PYG Area Scorecards. PYG, and other Performance Indicators. Florida Department of Education’s “Value-Added Model (VAM)”. Marc Baron, Chief, Performance Accountability Mark Howard, Director, Research, Evaluation, and Assessment.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Agenda' - lucine


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Agenda
Agenda

  • Student Learning Growth

    • Teacher VAM

    • School Growth

  • PYG

  • Area Scorecards. PYG, and other Performance Indicators


Florida department of education s value added model vam

Florida Department of Education’s “Value-Added Model (VAM)”

Marc Baron, Chief, Performance Accountability

Mark Howard, Director, Research, Evaluation, and Assessment


Fldoe value added model established by student success act sb 736
FLDOE Value-Added Model (VAM)”Established by Student Success Act (SB 736)

  • Evaluation Criteria for All Instructional Personnel and School Administrators

    • Professional Practice

    • Student Learning Growth

      • FCAT Classroom Teachers: 40%-50%

      • Non FCAT Classroom Teachers: 30%-40%

      • Non Classroom Teachers: 20%-30%

      • Principals: 50%

  • JTEC & Principal Evaluation Committee


Sdpbc must classify instructional personnel and school administrators into these four categories
SDPBC Must Classify Instructional Personnel and School Administratorsinto these Four Categories

  • Highly Effective

  • Effective

  • Needs Improvement

  • Unsatisfactory


Vam student learning growth
VAM & Student Learning Growth Administrators

  • Contribution to a change in a student’s achievement

  • Calculated from a measure of student learning growth


What is the student learning growth score
What is the AdministratorsStudent Learning Growth Score?

The difference between

Current test score and

Predicted test score

What is the Predicted Student Score?


What is the predicted student score
What is the AdministratorsPredicted Student Score?

It is score you would predict a student to achieve based on the student’s performance on prior tests and other information available on the student


Fldoe value added model variables determining student learning growth
FLDOE Value-Added Model AdministratorsVariables determining student learning growth

  • Two or more years of prior achievement scores

  • Gifted status

  • Student Attendance (Days)

  • Mobility (number of transitions)

  • Difference from modal age in grade (indicator of retention)

  • Class size

  • The number of subject-relevant courses in which the student is enrolled

  • Homogeneity of entering test scores in the class


Fldoe value added model variables determining student learning growth1
FLDOE Value-Added Model AdministratorsVariables determining student learning growth

  • Students with Disabilities (SWD) status

    • Language impaired

    • Hearing impaired

    • Visually impaired

    • Emotional/behavioral

    • Specific learning disability

    • Dual sensory impaired

    • Autism spectrum disorder

    • Traumatic brain injury

    • Other health impaired

    • Intellectual disability

  • English Language Learner (ELL) status

    • LY


Fldoe value added model variables not determining student learning growth
FLDOE Value-Added Model AdministratorsVariables not determining student learning growth

Gender

Race

Ethnicity

Socioeconomic status


Fldoe value added model variables determining student learning growth2

FLDOE Value-Added Model Administrators Variables determining student learning growth

TERMS

District Data Base of All Student Records


What is the predicted student score1
What is the Predicted Student Score? Administrators

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the predicted student score2
What is the Predicted Student Score? Administrators

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

900

800

CURRENT TEST

700

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the predicted student score3

W Administrators

What is the Predicted Student Score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

900

800

CURRENT TEST

700

600

500

400

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the predicted student score4
What is the Predicted Student Score? Administrators

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

Answer:

Average of

current scores

of similar students

900

800

CURRENT TEST

700

PREDICTED SCORE

600

600

500

400

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


Student learning growth is the amount above or below predicted score
Student Learning Growth is the Amount AdministratorsAbove or Below Predicted Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

900

800

ACTUAL SCORE

CURRENT TEST

Above Predicted Score

700

PREDICTED SCORE

600

600

500

Below Predicted Score

400

ACTUAL SCORE

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)


What is the predicted student score based on prior achievement
What is the Predicted Student Score? AdministratorsBased on Prior Achievement

Based on Prior Achievement Only

Based on Prior Achievement & ELL

(FY11 FCAT)

…100 200 300 400 500 600 700 800 900 1000…

10 20 30 40 50 60 70 80 90 100…

Each prior year test score has a Predicted current year test score.

90

800

CURRENT TEST

70

CURRENT TEST

600

50

400

10 20 30 40 50 60 70 80 90 100…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

PRIOR TEST


Fldoe value added model variables determining student learning growth3
FLDOE Value-Added Model AdministratorsVariables determining student learning growth

Two or more years of prior achievement scores

Attendance

Mobility (number of transitions)

Difference from modal age in grade (indicator of retention)

Class size

The number of subject-relevant courses in which the student is enrolled

Homogeneity of entering test scores in the class

LY

Gifted status

Language impaired

Hearing impaired

Visually impaired

Emotional/behavioral

Specific learning disability

Dual sensory impaired

Autism spectrum disorder

Traumatic brain injury

Other health impaired

Intellectual disability


Reading grade 7 indicators ranked by statistical significance
Reading Grade 7 AdministratorsIndicators Ranked By Statistical Significance


Illustrative growth points all other variables being equal
Illustrative Growth Points AdministratorsAll other variables being equal

*adapted from AIR materials presented at August 2, 2011,meeting

These are illustrative values, not actual values


What is the predicted student score5
What is the Predicted Student Score? Administrators

…100 200 300 400 500 600 700 800 900 1000…

Based on Prior Achievement Only

Based on Prior Achievement & ELL

(FY11 FCAT)

10 20 30 40 50 60 70 80 90 100…

90

800

CURRENT TEST

70

CURRENT TEST

600

50

400

10 20 30 40 50 60 70 80 90 100…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

PRIOR TEST


What is the predicted student score6
What is the Predicted Student Score? Administrators

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

Based on Prior Achievement Only

Based on Prior Achievement & ELL

(FY11 FCAT)

(FY11 FCAT)

900

800

CURRENT TEST

700

CURRENT TEST

600

500

400

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

(FY10 FCAT)

PRIOR TEST


What is the predicted student score7
What is the Predicted Student Score? Administrators

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

Based on Prior Achievement Only

Based on Prior Achievement

& Low Attendance

(FY11 FCAT)

(FY11 FCAT)

800

700

CURRENT TEST

CURRENT TEST

600

500

400

300

…100 200 300 400 500 600 700 800 900 1000…

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

(FY10 FCAT)

PRIOR TEST


How to calculate the student learning growth score
How to calculate the student learning growth score? Administrators

…100 200 300 400 500 600 700 800 900 1000…

The diagonal line is the predicted growth observed among similar students.

(FY11 FCAT)

CURRENT TEST

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How to calculate the student learning growth score1
How to calculate the student learning growth score? Administrators

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How to calculate the student learning growth score2

W Administrators

How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

100

550

450

300

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How to calculate the student learning growth score3

Y Administrators

How to calculate the student learning growth score?

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

? ? ? ?

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How to calculate the student learning growth score4
How to calculate the student learning growth score? Administrators

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


How do you calculate the teacher growth score
How do you calculate the Administratorsteacher growth score?

  • Identify the students assigned to the teacher

  • Determine the learning growth of each of the assigned students

  • Calculate the average learning growth of the students

TERMS


Teacher growth score
Teacher Growth Score Administrators

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

Typical

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Teacher growth score1
Teacher Growth Score Administrators

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

Above

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Teacher growth score2
Teacher Growth Score Administrators

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

Below

FY10 FY11 FY11 FY11

FCAT FCAT FCAT FCAT

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Teacher growth
Teacher Growth Administrators

School

Growth

Teacher

Growth

Avg Growth = (A+B+C+D)/4


Teacher effect
Teacher Effect Administrators

School

Growth

Teacher

Effect


Calculating teacher value added score part 1
Calculating Teacher Value-Added Score AdministratorsPart 1

Teacher Effect

(

)

-

=

School

Growth

Score

Teacher

Growth

Score

DRE


How do you calculate the school learning growth score
How do you calculate the Administratorsschool learning growth score?

  • Identify the students assigned to the school

  • Determine the learning growth of each of the assigned students

  • Calculate the average learning growth of the students


School growth score

FY10 FY11 FY11 FY11 Administrators

FCAT FCAT FCAT FCAT

School Growth Score

…100 200 300 400 500 600 700 800 900 1000…

(FY11 FCAT)

CURRENT TEST

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


Calculating teacher value added score part 1 aka teacher effect
Calculating Teacher Value-Added Score AdministratorsPart 1: AKA Teacher Effect

Tcheffect = (Tchgrowth score – Schgrowth score )

DRE


What is a Teacher Value-Added Score? Administrators

and

How is it calculated?

DRE


Fldoe teacher value added score conceptual
FLDOE Teacher Value-Added Score Conceptual Administrators

Teacher Effect

)

(

)

(

-

+

.5

X

=

School

Growth

Score

School

Growth

Score

Teacher

Growth

Score

Tchvas = (Tchgrowth score – Schgrowth score ) + ( .5 x Schgrowth score)

DRE


Teacher value added score
Teacher Value-Added Score Administrators

Tchvas = (Tchgrowth score – Schgrowth score ) + ( .5 x Schgrowth score)

DRE


State Average Student Learning Growth Administrators

Teacher

Growth = 20

The greater a teacher’s growth score is from the state’s average and the school’s growth score, the higher a teacher’s VAS

VAS = 15

VAS = 20

VAS = 25

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


State Average Student Learning Growth Administrators

Schoo

l

Growth = 0

At a school with a growth score of 0, the teacher’s VAS equals the teacher’s growth

VAS = 20

VAS = 0

VAS = -20

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


State Average Student Learning Growth Administrators

Schoo

l

Growth = 10

What is the significance of the green line?

At a high VAS school, a teacher’s growth must exceed half the school’s growth to have a positive VAS

VAS = 15

VAS = 5

VAS = 0

VAS = -5

VAS = -10

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


State Average Student Learning Growth Administrators

Schoo

l

Growth = -10

At a low VAS school, a teacher’s growth must exceed half the school’s growth to have a positive VAS

VAS = -15

VAS = -5

VAS =0

VAS = 5

VAS = 10

-20 -15 -10 -5 0 5 10 15 20

Tchvas = (Tchgrowth - Schgrowth ) + ( .5 x Schgrowth)


A real picture is worth a thousand words
A Real Picture is AdministratorsWorth a Thousand Words

=


Teacher vam

Teacher VAM Administrators


Sdpbc teacher vam grade 5 subject reading
SDPBC Teacher VAM Administrators (Grade 5 Subject: Reading)

Mean = 3.3

N = 561

Class >= 10


State teacher vam grade 5 subject reading
State Teacher VAM Administrators (Grade 5 Subject: Reading)

Mean = 0.1

N = 12182


Teacher vam distribution for reading
Teacher VAM AdministratorsDistribution for Reading


School growth

School Growth Administrators


Sdpbc school growth grade 5 subject reading
SDPBC School Growth AdministratorsGrade 5 Subject: Reading

Mean = 4.0

N = 120


State school growth grade 5 subject reading
State School Growth AdministratorsGrade 5 Subject: Reading

Mean = 0.02

N = 2082


School component distribution for reading
School Component AdministratorsDistribution for Reading


How precise are vam measures

How precise are AdministratorsVAM measures?


Standard error of teacher vams
Standard Error of Teacher VAMs Administrators

  • The standard error gives the uncertainty (error band) surrounding a teacher’s VAM score

  • It can be used to prevent categorizing teachers when that categorization would be uncertain


Standard error of teacher vams1
Standard Error of Teacher VAMs Administrators

Teacher Score

Minus

one SE

Plus

one SE


Using se to categorize teachers
Using SE to Categorize Teachers Administrators

State Average Score

Teacher 2

Teacher 1

Teacher 3

Teacher 4


Teacher VAMs Similar to State Administrators

State

Mean

Sig from State

# of

Stdts

VAM

No sig higher &

No sig diff

No sig higher &

No sig diff

No sig higher &

No sig diff


Some teacher VAMs Higher than State Administrators

Sig from State

# of

Stdts

State

Mean

VAM

1 sig higher &

No sig diff

2 sig higher &

No sig diff

4 sig higher &

sig differences


Some teacher VAMs Lower than State Administrators

Sig from State

# of

Stdts

State

Mean

VAM

1 sig lower

No sig diff

2 sig lower &

sig differences

3 sig lower &

No sig diff


Some Teacher VAMs Lower/Higher than State Administrators

Sig from State

# of

Stdts

State

Mean

VAM

1 sig higher

2 sig lower &

sig differences

1 sig higher

1 sig lower &

sig differences


Summary vam and school learning growth
Summary: VAM and School Learning Growth Administrators

  • State Mandated

  • Student Learning Growth

    • Difference between current score and predicted score

    • Based on prior achievement and selected student characteristics

  • Teacher VAM includes both teacher effect and school learning growth

  • School component is school learning growth

  • Consider standard error when using Teacher VAM and school learning growth


Sdpbc must classify instructional personnel school administrators into these four categories
SDPBC Must Classify AdministratorsInstructional Personnel/School Administratorsinto these Four Categories

  • Highly Effective

  • Effective

  • Needs Improvement

  • Unsatisfactory


Proportion of a year s growth school growth score

Proportion of a Year’s Growth Administrators& School Growth Score


PYG Administrators

FY10 FY11 FY11

FCAT FCAT -FY10

…100 200 300 400 500 600 700 800 900 1000…

One Year’s Worth of Growth

M

O

R

E

(FY11 FCAT)

L

E

S

S

CURRENT TEST

80

100

125

150

…100 200 300 400 500 600 700 800 900 1000…

PRIOR TEST

(FY10 FCAT)

DRE


A real picture is worth a thousand words1
A Real Picture is AdministratorsWorth a Thousand Words

=


SDPBC Administrators

Mean


11 Administrators

10

09

08

07


Area scorecard and pyg

Area Scorecard and PYG Administrators

To review area and school scorecards, go to the EDW.


Summary pyg and scorecard
Summary: PYG and Scorecard Administrators

  • PYG

    • Difference between current score and predicted score

    • Highly correlated with school learning growth

    • Use PYG reports and scorecard to see how you would have done?


ad