pvaas overview evaluating growth projecting performance webinar n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
PVAAS Overview: Evaluating Growth, Projecting Performance Webinar PowerPoint Presentation
Download Presentation
PVAAS Overview: Evaluating Growth, Projecting Performance Webinar

Loading in 2 Seconds...

play fullscreen
1 / 69

PVAAS Overview: Evaluating Growth, Projecting Performance Webinar - PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on

PVAAS Overview: Evaluating Growth, Projecting Performance Webinar. PVAAS Statewide Core Team Fall 2011. Goal of Session. To provide educators with an understanding of the Pennsylvania Value-added Assessment System and the meaning and uses of the following reports. Web-based Scatter Plots

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'PVAAS Overview: Evaluating Growth, Projecting Performance Webinar' - yamin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
pvaas overview evaluating growth projecting performance webinar

PVAAS Overview:Evaluating Growth,Projecting PerformanceWebinar

PVAAS Statewide Core Team

Fall 2011

goal of session
Goal of Session

To provide educators with an understanding of the Pennsylvania Value-added Assessment System and the meaning and uses of the following reports.

  • Web-based Scatter Plots
  • District/School Value-Added Report
  • Value-added Summary Report
  • District/School Performance Diagnostic Report
  • Individual Student Projection Report
  • Projection Summary Reports
agenda
Agenda

What is PVAAS?

Scatterplots

PVAAS Value-added Reports

Going Deeper with Value-added

PVAAS Projections

Lines of Inquiry

Resources and Support

pvaas is
PVAASIS:
  • Analysis of existing assessment data
    • Longitudinal PSSA scores in available content areas
      • Math, Reading, Science, Writing
    • Provides reliable estimates
  • A measure of a school’s effectiveness
  • Critical Relationship:

Growth Data vs. Achievement Data

pvaas
PVAAS:
  • Is not another test

Critical concept

    • Is not a comparison of single test scores
review of literature
Review of Literature

Is there a relationship between…

  • Demographics vs. Achievement?

YES

  • Demographics vs. Growth?

NO

Demographics are inherently controlled for in PVAAS due to the use of all longitudinal data on students!

slide8

Current Data Collection

Approaches

High Achievement

Average Achievement

Low Achievement

Achievement is based

on each student’s ending point.

slide9

PVAAS Adds Another Variable To Achievement…GROWTH!

Below

Growth

Standard

At or Above

Growth

Standard

Growth is based

on each student’s starting point!!

scatter plot quadrants and their meanings
Scatter Plot Quadrants and Their Meanings

Quad 2 - Slipping

Adequate Achievement

Negative Growth

Quad 1 - Excelling

Adequate Achievement

Positive Growth

Quad 4 - Improving

Below Adequate Achievement

Positive Growth

Quad 3 - Underperforming

Below Adequate Achievement

Negative Growth

new web based scatter plots1
NEW! Web-based Scatter Plots!
  • Users may choose to plot different variables against achievement and/or growth!
  • This includes demographics such as:
    • % Tested Economically Disadvantaged
    • % Tested Limited English Proficiency
    • % Tested Special Education
    • % Tested Minority
two types of pvaas information
TWO Types of PVAAS Information

Looking Forward/Planning…

PVAAS Projection Reports For Individual Students and Groups of Students

Looking Back/Evaluation…

Value-added Growth Reports For Groups of Students

Today

value added
Value-added

Wouldn’t it be great

to know if your district/school met the standard for

PA Academic Growth for students tested in grades 4-8?

And in Grades 9-11?

a child s growth chart
A Child’s Growth Chart

This growth chart shows a little girl who is at the 15th percentile on her 1st birthday.

Her growth is typical if she stays at the 15th percentile in later years.

now at 3 years old
Now at 3 years old…

Growth Spurt!

X

TypicalGrowth

X

X

Growth Concern?

a challenge comparing scores across years math reading grades 4 through 8
A Challenge Comparing Scores Across YearsMath & Reading – Grades 4 through 8
  • 0 10 20 30 40 50 60 70 80 90 100

NCE (Normal Curve Equivalent) Units

1000 1100 1200 1300 1400 1500

Scaled Score Units

what value added growth measure means growth standard methodology math reading grades 4 through 8
What Value-Added Growth Measure Means!Growth Standard MethodologyMath & Reading – Grades 4 through 8

Achievement

6th grade

52.7

Achievement

5th grade

47.6

0 10 20 30 40 50 60 70 80 90 100

NCE Scale

State Mean = 50

PVAAS Growth Measure ≈ Change in position from 5th to 6th grade

≈ 52.7 – 47.6 ≈ 5.1 NCE units

slide26

Example of Growth Measure=0 for a

low-achieving district/school

Example of Growth Measure=0 for a

high-achieving district/school

predictive methodology math reading grades 11 writing grades 5 8 11 science grades 4 8 11
Predictive MethodologyMath & Reading – Grades 11Writing – Grades 5, 8, 11Science – Grades 4, 8, 11
pvaas value added reports writing science grade 11 math and reading
PVAAS Value-added ReportsWriting, Science & Grade 11 Math and Reading

Predictive Methodology

  • A predicted score for each student;
  • Mean of predicted scores for the group of students;
  • Mean of observed scores for the group of students.

Growth Measure ≈ Observed Mean - Predicted Mean

pvaas growth reporting
PVAAS Growth Reporting
  • District/School Value-added Report
  • Value-added Summary Report
  • District/School Performance Diagnostic Report
  • Performance Diagnostic Report for subgroups
district school value added report grades 4 through 8 math reading
District/School Value-added ReportGrades 4 through 8: Math & Reading

Question for Educators:

Are all grade levels meeting the standard for PA Academic Growth?

slide33

District/School Value-added ReportGrades 9-11 Math & Reading; Science; & Writing

Question for Educators:

Are all grade levels meeting the standard for PA Academic Growth?

district value added summary report grades 4 through 8 math reading
District Value-added Summary ReportGrades 4 through 8: Math & Reading

Question for Educators:

Which schools at which grade levels are exceeding the standard for PA Academic Growth?

district value added summary report grades 9 11 math reading science writing
District Value-added Summary ReportGrades 9-11 Math & Reading; Science; & Writing

Question for Educators:

Which schools at which grade levels are exceeding the standard for PA Academic Growth?

lines of inquiry
Lines of Inquiry
  • Main Questions Regarding PVAAS Reports
    • Are we serving all students adequately?
    • Are we serving all students equitably?
  • Questions to Ask Regarding Value-added Reports
    • Are students in all grade levels across all schools meeting or exceeding the standard for PA Academic Growth in all subjects?
    • Is the pattern of growth at the same grade level consistent from school to school? Or, does growth differ from school to school within the same grade level?
    • For each school, is the pattern of growth at the same grade level consistent from year to year? Or, has growth differed within the same grade level across the last three years?
going deeper performance diagnostic report
Going DeeperPerformance Diagnostic Report

Missing Bar – Insufficient Number of Students

Blue Bar – Current Year

Gold Bar – Previous Years

Whisker – Margin of Error on Growth Value

Question for Educators:

Are all groups of students making adequate growth?

pvaas performance diagnostic growth descriptors interpretation
PVAAS Performance Diagnostic Growth Descriptors - Interpretation

What the whiskers tell us…

Exceeded the standard for PA Academic Growth

Blue

Met the standard for PA Academic Growth

Green

Zero (0) Line

Growth

Did not meet the standard for PA Academic Growth

Pink

let s look at our example to practice interpretation
Let’s Look at Our Example to Practice Interpretation

Questions for Educators:

  • Are all types of learners showing a year’s growth or meeting expectations?
  • Are we serving all students adequately?
  • Are we serving all students equitably?
slide40

Links to students in selected predicted PSSA performance category

Link to all students reported for this subject.

student lists from performance diagnostic reports
Student Lists from Performance Diagnostic Reports

Current PSSA Performance level for each student

slide42

Patterns of Growth

  • Questions for Educators:
  • What patterns of growth do we see at the various grade levels?
  • What implications does this have for future achievement levels of students?
  • What implications does this have for closing he achievement gap?

A

C

B

lines of inquiry1
Lines of Inquiry
  • Questions to Ask Regarding Grade Level Performance Diagnostic Reports
    • Are students at all predicted PSSA performance levels meeting or exceeding the standard for PA Academic Growth in all grades and subjects?
    • What type of growth pattern do we see for each grade level? And, what are the implications for achievement if this growth pattern continues?
    • Where are we doing well? Which groups are showing positive growth?
    • Where are we facing challenges? Which groups are not showing positive growth?
    • Is there a change from previous years?
    • Are other buildings within the district seeing similar growth patterns?
    • What does this tell us about the standards-aligned system, specifically around curriculum, instruction, resources, and interventions?
best to compare subgroup to entire grade level
Best to Compare Subgroup to Entire Grade Level

Same School – Same Grade – Same Subject

Entire grade level

Subgroup

Question for Educators:

How do our subgroups compare to the entire grade level?

lines of inquiry2
Lines of Inquiry
  • Questions to Ask Regarding Diagnostic Reports for Subgroups
    • Did the students in the selected subgroup achieve acceptable growth? What about those students predicted to be Below Basic? Predicted to be Basic? Predicted to be Proficient? Predicted to be Advanced?
    • Is the growth pattern for subgroups similar to the growth pattern for all students in the same grade level?
    • How do these growth patterns compare to subgroup growth patterns in other grade levels and subject areas?
    • Is there a change from previous years?
    • Are other buildings within the district seeing similar growth patterns with subgroups?
    • What does this tell us about the standards-aligned system, specifically around curriculum, instruction, resources, and interventions?
two types of pvaas information1
TWO Types of PVAAS Information

Looking Forward/Planning…

PVAAS Projection Reports For Individual Students and Groups of Students

Looking Back/Evaluation…

Value-added Growth Reports For Groups of Students

Today

student projections
Student Projections

Wouldn’t it be great

to know the likelihood

that a student will

be proficient on a

future PSSA?

student projection methodology
Student Projection Methodology
  • Is this the same methodology as the one used to estimate growth?
    • No, the projection methodology is a separate modeling process that focuses on individual students.
  • What data are used in this methodology?
    • ALL available longitudinal data in both Reading and Math are used in projection calculations for all four subjects.
  • What students have projections? In what subjects?
    • Projections are available for all students who have a minimum of 2 years of historical data. The next slide details the grade levels and subjects in which projections are reported.
pvaas fall 2011 reporting projections to basic proficient advanced
PVAAS Fall 2011 Reporting: Projections to Basic, Proficient, Advanced

Grade PSSA

Last Taken

Grade PSSA

Last Taken

Grade PSSA

Last Taken

pvaas projection reporting
PVAAS Projection Reporting
  • Student Projection Report
  • District/School Projection Summary Report
more examples
More examples

Report A

Report B

the quality of projections for decision making
The Quality of Projections for Decision Making

The projection is precise, and is created using a model that has been reviewed and approved by four different peer review panels and the GAO (US Government Accountability Office).

PA’s growth model proposal to USDOE includes information regarding the statistical model and projection reliability study. (http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html)

Recent studies have confirmed that the PVAAS projections (even as far as 3 years into the future) are more reliable at looking at the future performance of a student than the most recent PSSA score.

slide57

PVAAS Projections for Groups of Students

Student Search

  • Question for Educators: What are students’ probabilities of reaching Proficient or Advanced levels on the next PSSA?
slide58

Identified Students

Meeting Search Requirements

  • Question for Educators: How can we better plan for this year now that we know the achievement probabilities of these students?
student projection summaries
Student Projection Summaries

Wouldn’t it be great

to have a summary of all of the probabilities that students in one school or in the entire district will be proficient on a future PSSA?

projection summary reports
Projection Summary Reports
  • What are they?
    • This is a report that summarizes the numbers and percentages of students in various likelihood ranges of performing at a proficient level on a future PSSA exam.
  • How might a school use these reports?
    • Intervention Planning
    • Resource Allocation
    • Strategic Planning
    • School Improvement Planning
  • Cautions
    • This report provides ONE indicator about likelihood of future performance.
    • Additional data should be used with the projection for better decision-making.
district school single grade projection summary report
District/School (Single Grade) Projection Summary Report
  • Questions for Educators:
  • Do we have a high percentage of our students likely to reach proficiency?
  • Do we have interventions and enrichment opportunities available for the appropriate students?
lines of inquiry3
Lines of Inquiry
  • Questions to Ask Regarding Projections
    • Where are we headed as a district or school?
    • Do we have a high percentage of our students likely to reach proficiency?
    • Do we have large numbers of students at risk of not reaching proficiency?
    • Do we have interventions and enrichment opportunities available for the appropriate students?
    • What is the schedule of courses? Is it fitting students’ needs consistently across schools?
    • How are students placed in courses? AP courses? Enrichment? Support?
    • Are students accessing the appropriate interventions and resources?
    • What does this tell us about the standards-aligned system, specifically around curriculum, instruction, resources, and interventions?
two types of pvaas information2
TWO Types of PVAAS Information

Looking Forward/Planning…

PVAAS Projection Reports For Individual Students and Groups of Students

Looking Back/Evaluation…

Value-added Growth Reports For Groups of Students

Today

fall professional development opportunities
Fall Professional Development Opportunities

Check out what is being offered this school year by visiting the PVAAS login page!

pa s intent for pvaas
PA’s Intent for PVAAS

Another Tool for Continuous Improvement

Measure of Growth to Add to Status/Achievement Measures

Not to Be Used in Isolation

slide68

Questions:PVAAS Materials or Statewide ImplementationEmail: pdepvaas@iu13.orgPhone: 717-606-1911PVAAS Reporting Web Sitehttps://pvaas.sas.com