pvaas f2f overview evaluating growth projecting performance n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
PVAAS F2F Overview: Evaluating Growth, Projecting Performance PowerPoint Presentation
Download Presentation
PVAAS F2F Overview: Evaluating Growth, Projecting Performance

Loading in 2 Seconds...

play fullscreen
1 / 117

PVAAS F2F Overview: Evaluating Growth, Projecting Performance - PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on

PVAAS F2F Overview: Evaluating Growth, Projecting Performance. Fall 2013. Welcome!. IU Facilitators Who is here today? District level administrators? School level administrators? Teachers? Instructional coaches? Guidance counselors? Other?. Partners & Teams for Session Activities.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'PVAAS F2F Overview: Evaluating Growth, Projecting Performance' - leroy


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
welcome
Welcome!
  • IU Facilitators
  • Who is here today?
    • District level administrators?
    • School level administrators?
    • Teachers?
    • Instructional coaches?
    • Guidance counselors?
    • Other?

F2F Overview 2013

partners teams for session activities
Partners & Teams for Session Activities
  • Some activities throughout the session will be completed in pairs or small groups, while others will be done independently.
  • Take a few minutes now to divide yourselves into:
    • Partners (groups of 2), AND
    • Small groups (groups of 4-6).

F2F Overview 2013

goal of session
Goal of Session

By the end of the session, participants will be able to answer the following questions:

  • What is PVAAS?
  • How does PVAAS measure growth?
  • What does PVAAS tell us about growth in our district and school(s)?
  • What does PVAAS tell us about how students are projected to perform on a future state assessment?
  • Which reports are most meaningful for school level planning, course level planning, grade level planning, and student level planning?

F2F Overview 2013

what is pvaas
What Is PVAAS?

F2F Overview 2013

what is pvaas1
What is PVAAS?
  • Longitudinal analysis of existing state assessment data
  • Reliable estimates of academic growth & projections of future performance
  • Effectiveness of the educational system

F2F Overview 2013

what is pvaas2
What is PVAAS?

Grade 5

Grade 6

  • Growth of a group of students over time
  • NOT a simple comparison of test scores

Grade 3

Grade 4

F2F Overview 2013

pvaas addresses complex issues
PVAAS Addresses Complex Issues!

Research-based Statistical Modeling

F2F Overview 2013

demographics and growth
Demographics and Growth

As long as you can sufficiently account for the prior achievementof the student,…

…there is NO significant relationship between demographics and student progress, or growth!

F2F Overview 2013

prior achievement
Prior Achievement

How can we adequately account for prior achievement?

  • The SAS EVAAS team has found that the only way to adequately account for prior achievement is to use ALL available test data for each student to dampen the effects of measurement error.
  • PVAAS does NOT need to adjust for the demographic status of students directly since it is accounting for students’ prior achievement.
    • In other words, demographics are inherently controlled for in PVAAS by the use of all longitudinal data on all students.

F2F Overview 2013

activity
Activity

Work with a partner to decide if each statement is true or false.

F2F Overview 2013

for more information
For More Information

Virtual Learning Module

PVAAS Misconceptions:

Clarifying Questions and Misunderstandings

F2F Overview 2013

slide14

What is PVAAS?

F2F Overview 2013

slide15

LOOKING AHEAD

LOOKING BACK

F2F Overview 2013

measuring academic growth
Measuring Academic Growth

Students don’t all start the year at the same place academically

F2F Overview 2013

measuring academic growth1
Measuring Academic Growth
  • Standard for PA Academic Growth
    • Regardless of their entering achievement level, students should not lose ground academically
  • Is it not a reasonable goal to maintain the achievement level of a group of students?
    • Not necessarily increase their achievement level, but maintain their achievement level.

F2F Overview 2013

pvaas growth measure color indicators
PVAAS Growth Measure &Color Indicators
  • The Growth Measure represents the amount of growth a group of students has made.
  • The Color Indicators represent the level of evidence we have in the data to say whether the group of students:
    • Did NOT meet the standard for PA Academic Growth
    • Met the standard for PA Academic Growth
    • Exceeded the standard for PA Academic Growth

PVAAS yields a Growth Measure AND a Growth Color Indicator.

F2F Overview 2013

slide21

Concept of Measuring Growth in

PVAAS

Prior Achievement for Group

Higher Achievement

Lower Achievement

F2F Overview 2013

slide22

Concept of Measuring Growth in PVAAS

Prior Achievement for Group

Higher Achievement

Lower Achievement

  • For grades 4-8, Math and Reading - represents the mean achievement level of the group of student
  • For Science, Writing, & Keystones- represents the mean predicted and observed/actual achievement level of the group of students

F2F Overview 2013

slide23

Concept of Measuring Growth in PVAAS

Most Recent School Year for Group

Prior Achievement for Group

Higher Achievement

Higher Achievement

Lower Achievement

Lower Achievement

  • For grades 4-8, Math and Reading - represents the mean achievement level of the group of student
  • For Science, Writing, & Keystones- represents the mean predicted and observed/actual achievement level of the group of students

F2F Overview 2013

slide24

Concept of Measuring Growth in PVAAS

Most Recent School Year for Group

Prior Achievement for Group

Higher Achievement

Higher Achievement

Lower Achievement

Lower Achievement

  • For grades 4-8, Math and Reading - represents the mean achievement level of the group of student
  • For Science, Writing, & Keystones- represents the mean predicted and observed/actual achievement level of the group of students

F2F Overview 2013

slide25

Concept of Measuring Growth in PVAAS

Most Recent School Year for Group

Prior Achievement for Group

Higher Achievement

Higher Achievement

Lower Achievement

Lower Achievement

  • For grades 4-8, Math and Reading - represents the mean achievement level of the group of student
  • For Science, Writing, & Keystones- represents the mean predicted and observed/actual achievement level of the group of students

F2F Overview 2013

slide26

Concept of Measuring Growth in PVAAS

Most Recent School Year for Group

Prior Achievement for Group

Higher Achievement

Higher Achievement

Lower Achievement

Lower Achievement

  • For grades 4-8, Math and Reading - represents the mean achievement level of the group of student
  • For Science, Writing, & Keystones- represents the mean predicted and observed/actual achievement level of the group of students

F2F Overview 2013

slide27

Concept of Measuring Growth in PVAAS

Most Recent School Year for Group

Prior Achievement for Group

Higher Achievement

Higher Achievement

Lower Achievement

Lower Achievement

  • For grades 4-8, Math and Reading - represents the mean achievement level of the group of student
  • For Science, Writing, & Keystones- represents the mean predicted and observed/actual achievement level of the group of students

F2F Overview 2013

regardless of the entering achievement level of the group1
Regardless of the entering achievement level of the group…
  • Students should not lose ground academically

F2F Overview 2013

regardless of the entering achievement level of the group2
Regardless of the entering achievement level of the group…
  • Students should not lose ground academically
  • A reasonable goal would be to at least maintain the achievement level of the group

F2F Overview 2013

regardless of the entering achievement level of the group3
Regardless of the entering achievement level of the group…
  • Students should not lose ground academically
  • A reasonable goal would be to at least maintain the achievement level of the group
  • All groups of students can make growth

F2F Overview 2013

practice
Practice!
  • Turn to a partner
  • Explain the concept of growth

Current School Year

Previous School Year

Higher Achievement

Higher Achievement

Lower Achievement

Lower Achievement

F2F Overview 2013

for more information1
For More Information

TWO Virtual

Learning Modules

Introduction to Measuring Academic Growth

Concept of Growth: Connections to Value-Added Reporting

F2F Overview 2013

questions
Questions

F2F Overview 2013

pvaas growth measure color indicators1
PVAAS Growth Measure &Color Indicators
  • The Growth Measure represents the amount of growth a group of students has made.
  • The Color Indicators represent the level of evidence we have in the data to say whether the group of students:
    • Did NOT meet the standard for PA Academic Growth
    • Met the standard for PA Academic Growth
    • Exceeded the standard for PA Academic Growth

PVAAS yields a Growth Measure AND a Growth Color Indicator.

F2F Overview 2013

measuring growth in pvaas
Measuring Growth in PVAAS
  • Remember, PVAAS does NOT compare the achievement of one group of students in a particular subject and grade with the previous year’s students in the same subject and grade.
  • PVAAS does NOT use the percentages of students at various academic performance levels to measure growth.
  • PVAAS uses students’ scale scores within those performance levels to better estimate growth.

F2F Overview 2013

growth measures not reported for prior years
Growth Measures Not Reportedfor Prior Years
  • PA’s state assessment system is transitioning to include Keystone end-of-course exams and statewide assessments aligned to the PA Common Core over the next several years.
  • Growth Measures in prior years will NOT be included in fall 2013 reporting (SY12-13).
  • Anticipated that Growth Measures from up to 3 previous years with 3-year averages will be yielded and reported beginning with fall 2015 reporting (SY14-15).

F2F Overview 2013

pvaas value added growth measures
PVAAS Value-AddedGrowth Measures
  • For Reading and Math in grades 4-8, Growth is measured in Normal Curve Equivalent (NCE) units as scale scores cannot be compared from grade to grade and year to year.
  • The Growth Measure is NOT reported in years and cannot be referenced in terms of number of year’s growth.
  • The Growth Measure is NOT reported in percentages and cannot be referenced in terms of an increase or decrease in percentage points.

F2F Overview 2013

rescaling grades 4 8 math reading
RescalingGrades 4-8: Math & Reading

1000 1100 1200 1300 1400 1500

Scale Score Points

  • 0 10 20 30 40 50 60 70 80 90 100

NCE Units

F2F Overview 2013

school value added reports science writing keystone content a reas
School Value-Added ReportsScience, Writing, & Keystone content areas

School A

School B

F2F Overview 2013

pvaas value added growth measures1
PVAAS Value-AddedGrowth Measures
  • For Science, Writing and Keystone content areas, Growth is measured in scale score points.
  • The Growth Measure is NOT reported in years and cannot be referenced in terms of number of year’s growth.
  • The Growth Measure is NOT reported in percentages and cannot be referenced in terms of an increase or decrease in percentage points.

F2F Overview 2013

for more information2
For More Information

TWO Virtual

Learning Modules

Value-Added Report

F2F Overview 2013

performance diagnostic report
Performance Diagnostic Report

Missing Bar – Insufficient Number of Students

Blue Bar – Current

Year

Whisker – Margin of Error on Growth Value

F2F Overview 2013

pvaas performance diagnostic growth descriptors interpretation
PVAAS Performance Diagnostic Growth Descriptors - Interpretation

What the whiskers tell us…

Exceeded the standard for PA Academic Growth

Met the standard for PA Academic Growth

Zero (0) Line

Growth

Did not meet the standard for PA Academic Growth

F2F Overview 2013

performance diagnostic patterns
Performance Diagnostic Patterns

Reverse Shed Pattern

Shed Pattern

System benefits

higher-achieving students

System benefits

lower-achieving students

F2F Overview 2013

performance diagnostic patterns1
Performance Diagnostic Patterns

Tent Pattern

V Pattern

System benefits

middle level learners

System benefits

higher and lower achieving students

F2F Overview 2013

performance diagnostic patterns2
Performance Diagnostic Patterns

Uniform I Pattern

Uniform 2 Pattern

System is not supporting growth of any students

System supports growth of

all students similarly and adequately

F2F Overview 2013

performance diagnostic patterns3
Performance Diagnostic Patterns

Opportunity Gap Pattern

System benefits all types of learners except those right below proficiency

F2F Overview 2013

performance diagnostic patterns4
Performance Diagnostic Patterns

Desirable Pattern

System benefits all types of learners with more growth occurring with the lower achieving students

F2F Overview 2013

activity check for understanding
Activity: Check for Understanding

Sketch a sample performance diagnostic bar graph that fits each description.

F2F Overview 2013

best to compare subgroup to entire grade or course level
Best to Compare Subgroup to Entire Grade or Course Level

Same District/School – Same Grade/Course

Entire grade or course

Subgroup

F2F Overview 2013

slide60
ActivityWhat conclusions can you make about the growth of different groups of students in 6th grade Reading?

F2F Overview 2013

purpose of value added performance diagnostic reports
Purpose of Value-Added & Performance Diagnostic Reports
  • Assessing curricular needs across grades, subjects, and Keystone content areas
  • Evaluating instructional needs of students at various achievement levels and in different subgroups
  • Ensuring instructional coherence/alignment
    • Curriculum Instruction Assessment
    • Supports, including scheduling and staffing needs
  • Monitoring implementation and effectiveness of grade-level or course-specific programs

F2F Overview 2013

for more information3
For More Information

Virtual

Learning Module

Performance Report

F2F Overview 2013

slide63

Many Reports

F2F Overview 2013

new pvaas launchpad
NEW! PVAAS Launchpad
  • See multiple SCHOOL reports in a consolidated format
  • Provides a user-friendly visual representation of school level data

F2F Overview 2013

new pvaas launchpad1
NEW! PVAAS Launchpad

Provides the Big Picture

  • Across PSSA grades and Keystone content areas
  • Across subject areas

F2F Overview 2013

new pvaas launchpad2
NEW! PVAAS Launchpad

F2F Overview 2013

new pvaas launchpad3
NEW! PVAAS Launchpad

F2F Overview 2013

launchpad demo
Launchpad Demo

F2F Overview 2013

remember the launchpad
Remember…the Launchpad
  • Provides a comprehensive view showing multiple reports on a new single report
  • Allows you to see patterns by grade and subject
  • Provides you with growth patterns that speak to the strength of your overall core program
    • deeper analysis is needed, such as looking at subgroups

F2F Overview 2013

for more information4
For More Information

WebEx Sessions held:

  • August 1st, 9:30am-10:30am
  • August 5th, 2:30pm-3:30pm

COULDN’T JOIN THE LIVE SESSION?

A recording of the WebEx session will be available on the eLearning page within the PVAAS web-based system.

https://pvaas.sas.com

F2F Overview 2013

pvaas scatterplots
PVAAS Scatterplots

Vertical Axis

Percent Proficient or Advanced

Standard for PA Academic Growth

Horizontal Axis

Average Growth Index

F2F Overview 2013

scatterplot quadrants and their meanings
Scatterplot Quadrants and Their Meanings

Quad 2 - Slipping

Adequate Achievement,

Negative Growth

Quad 1 - Excelling

Adequate Achievement,

Positive Growth

Quad 3 - Underperforming

Below Adequate Achievement,

Negative Growth

Quad 4 - Improving

Below Adequate Achievement,

Positive Growth

F2F Overview 2013

scatterplot demo
SCATTERPLOT Demo

F2F Overview 2013

interpreting scatterplots
Interpreting Scatterplots
  • Work with your small group to review BOTH scatterplots on the next 2 slides, one for Math and one for Reading.
  • Discuss strengths and weaknesses of both scatterplots. Remember, each “dot” represents a school!
  • We will discuss as a group…“What can you conclude by looking at both?”

F2F Overview 2013

activity district scatterplot
Activity: District Scatterplot

MATH

Note: Each dot on the scatter plot represents a school from this district.

F2F Overview 2013

activity district scatterplot same schools different subject
Activity: District ScatterplotSame Schools, Different Subject

READING

Note: Each dot on the scatterplot represents a school from this district.

F2F Overview 2013

for more information5
For More Information

TWO Virtual

Learning Modules

Public Scatterplots

Restricted-View Scatterplots

F2F Overview 2013

questions1
Questions

F2F Overview 2013

slide80

LOOKING AHEAD

LOOKING BACK

F2F Overview 2013

what are projections
What Are Projections?

Given a specific set of circumstances…

…what’s the most likely outcome?

F2F Overview 2013

setting expectations
Setting Expectations

How do teachers set expectations for their students?

  • Information
    • What do they know about this student?
  • Experience
    • How have students like this one performed in the past?

F2F Overview 2013

pvaas projections what are they based on
PVAAS ProjectionsWhat are they based on?

Expectations based on what we know:

About this student:

  • Prior test scores
  • Across subjects

F2F Overview 2013

pvaas projections what are they based on1
PVAAS ProjectionsWhat are they based on?

About other students:

  • Prior test scores across subjects
  • Scores on the test to which we are projecting

F2F Overview 2013

what are pvaas projections
What Are PVAAS Projections?

Given this student’s testing history, across subjects…

…what is the student likely to score on an upcoming test, assuming the average schooling experience?

F2F Overview 2013

student projection methodology
Student Projection Methodology

What data are used in calculating projections?

F2F Overview 2013

student projection methodology1
Student Projection Methodology

Which students have projections? In what subjects?

  • Projections are available for all students who have a minimum of 3 data points. The exception to this are projections to grade 4 where only 2 data points from grade 3 are needed.
  • The next two slides detail the grade levels, subjects, and Keystone content areas in which projections are reported.

F2F Overview 2013

pvaas projection reporting projections to pssa basic proficient advanced
PVAAS Projection Reporting: Projections to PSSA (Basic, Proficient, Advanced)

Grade PSSA

Last Taken

Grade PSSA

Last Taken

Grade PSSA

Last Taken

F2F Overview 2013

pvaas projection reporting projections to keystones basic proficient advanced
PVAAS Projection Reporting: Projections to Keystones (Basic, Proficient, Advanced)
  • Anticipating projections to future Keystone exams will be available to students who last tested in grade 5
  • Remember, projections to a Keystone exam are NOT grade specific
    • The projection probability indicates the probability of a student reaching the indicated performance level on a future Keystone exam – not necessarily at a specific grade level.

F2F Overview 2013

slide95

PVAAS Projections for

Groups of Students

Student Search

F2F Overview 2013

slide96

Identified Students

Meeting Search Requirements

F2F Overview 2013

how often did students with pvaas probabilities of 70 or higher score at proficient or advanced
How often did students with PVAAS probabilities of 70% or higher score at proficient or advanced?

F2F Overview 2013

how often did students with pvaas probabilities of 70 or higher score at proficient or advanced1
How often did students with PVAAS probabilities of 70% or higher score at proficient or advanced?

A document containing more information can be found on the PVAAS Login Page.

F2F Overview 2013

projection summary reports
Projection Summary Reports

What are they?

  • Summarizes the numbers and percentages of student probabilities to a selected performance level on a future state assessment
  • Summary available for projections to a Proficient level or higher, or to an Advanced level

Cautions

  • ONE indicator
  • Additional data should be used

F2F Overview 2013

how might educators use projection summary reports
How Might Educators Use Projection Summary Reports?

Administrators can use for program planning:

  • Intervention Planning
  • Resource Allocation
  • Course Availability & Placement Decisions
  • Grade-level and Course-level Planning
  • School-level Planning
  • Comprehensive Planning

Teachers can use to facilitate instructional planning:

  • Proportion of students on a path to success (proficiency or higher)
  • Proportion of students at risk for reaching proficient or advanced levels of performance
  • Plan for instructional strategies and classroom practices based on students’ needs

F2F Overview 2013

for more information6
For More Information

TWO Virtual

Learning Modules

Student Projection Report

Projection Summary Report

F2F Overview 2013

questions2
Questions

F2F Overview 2013

slide106

LOOKING AHEAD

LOOKING BACK

F2F Overview 2013

pa s intent for pvaas
PA’s Intent for PVAAS
  • Measure of Growth to Add to Existing Achievement Measures
  • Indicator of Likelihood of Future Student Performance to Add to Current Achievement Data
  • Another Tool for Continuous Improvement
  • Not to Be Used in Isolation

F2F Overview 2013

pvaas resources

PVAAS Resources

F2F Overview 2013

pvaas virtual learning modules
PVAASVirtual Learning Modules
  • Online, interactive resources
  • Provides real-time professional development
  • Designed in sections – pick & choose based on your needs
  • Various types of VLMs
    • Concept & foundations of measuring growth
    • Understanding and using reports
    • Integrating reports for decision-making

F2F Overview 2013

new virtual learning modules
New! Virtual Learning Modules
  • Scatterplots
  • District & School Search
  • Concept of Growth: Introduction to Measuring Academic Growth
  • Introduction to High Achievement & Growth
  • Custom Diagnostic Report
  • Student Search & Custom Student Report
  • Using PVAAS for Scheduling Students
  • PVAAS Misconceptions

F2F Overview 2013

pvaas help menus
PVAAS Help Menus

F2F Overview 2013

pvaas data packets
PVAAS Data Packets

F2F Overview 2013

pvaas data packets1
PVAAS Data Packets
  • Packet choices include:
    • comprehensive planning
    • school level planning
    • grade and course level planning
    • district diagnostic
    • school projection summaries
    • student projections

F2F Overview 2013

additional pvaas supports
Additional PVAAS Supports

F2F Overview 2013

slide115

Additional PVAAS Supports

http://www.education.state.pa.us/pvaas

F2F Overview 2013

slide116
Questions?PVAAS Materials or Statewide Implementationpdepvaas@iu13.org717-606-1911PVAAS Report Web Sitehttps://pvaas.sas.com

F2F Overview 2013

pennsylvania value added assessment system pvaas

PENNSYLVANIAValue Added Assessment System______________________________________________________________________PVAAS

www.pde.state.pa.us

333 Market Street Harrisburg, PA 17126