Determining informative student growth on the florida assessments for instruction in reading
Download
1 / 27

Determining Informative Student Growth on the Florida Assessments for Instruction in Reading - PowerPoint PPT Presentation


  • 109 Views
  • Uploaded on

Determining Informative Student Growth on the Florida Assessments for Instruction in Reading. Yaacov Petscher, Ph.D. Director of Research Florida Center for Reading Research. Common Questions. How much growth should occur? What score type should be used for growth?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Determining Informative Student Growth on the Florida Assessments for Instruction in Reading' - teague


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Determining informative student growth on the florida assessments for instruction in reading

Determining Informative Student Growth on the Florida Assessments for Instruction in Reading

Yaacov Petscher, Ph.D.

Director of Research

Florida Center for Reading Research


Common questions
Common Questions Assessments for Instruction in Reading

  • How much growth should occur?

  • What score type should be used for growth?

  • Why are students’ PRS scores in Grade 1 decreasing?

  • What do the Reading Comprehension and Word Analysis Ability scores really tell us?


Things to discuss
Things to Discuss Assessments for Instruction in Reading

  • Review Goals and FAIR Psychometrics

  • Review of FAIR & Score Types

    • How they were derived

    • 2010-2011 changes to FAIR

  • Growth in FAIR subtests

  • Comparison of Growth


Goals in assessment k2
Goals in Assessment – K2 Assessments for Instruction in Reading

  • What do we want to maximize in a screen?

    • Correct classification or base rates

  • Focus on correct classification

    • Cost = higher false positives/false negatives

  • Focus on predictive power

    • Cost = Lower sensitivity


Reliability of broad screen
Reliability of Broad Screen Assessments for Instruction in Reading

*The Letter Sounds task was more reliable than Letter Names at AP 1; however, due to the restricted range for high risk, a policy decision was made to use the Letter Name task in order to better capture the floor of the distribution. Because the Broad Screen in G2 is a timed task, precision estimates are not reported; however, test-retest reliability was strong (.79-.84).


Predictive validity of broad screen
Predictive Validity of Broad Screen Assessments for Instruction in Reading


Goals of assessment 3 12
Goals of Assessment – 3-12 Assessments for Instruction in Reading

  • What do we want to maximize?

    • Reliable estimate of student ability

  • Computer adaptive test (CAT)

    • Allows for individual test creation

    • Limits form effects

    • Difficult to teach to the test

  • Starting students in the CAT


Fair fcat relationship
FAIR-FCAT Relationship Assessments for Instruction in Reading


Fair k 2 broad screen
FAIR K-2 Broad Screen Assessments for Instruction in Reading

  • Each grade has different tasks

    • Kindergarten

    • Grade 1

    • Grade 2

  • Probability of Reading Success (PRS)

    • What does this mean?

    • How can it be used?


“The Probability of Reading Success (PRS) score predicts the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.The PRS can be used descriptively to compare class, school, or district level performance from one AP to the next. “


State median prs k 2
State Median PRS K-2 the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.


Analyzing student progress
Analyzing Student Progress the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

  • Make descriptive comparisons

    • <85% PRS

      • Was there a change in PRS?

        • Yes – YAY!

          • Did they shift zones?

        • No – Look at TDI information and examine progress

    • >=85% PRS

      • Did the student remain the “green zone”?

      • Grade 1 question…


Previous grade 1 prs
Previous Grade 1 PRS the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.


New grade 1 prs
New Grade 1 PRS the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.


RC Screen the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

Helps us identify students who may not be able to meet the grade level literacy standards at the end of the year as assessed by the FCAT without additional targeted literacy instruction. 

Mazes

Helps us determine whether a student has more fundamental problems in the area of text reading efficiency and low level reading comprehension. 

Word Analysis

Helps us learn more about a student's fundamental literacy skills--particularly those required to decode unfamiliar words and read accurately. 

Purpose of Each 3-12 Assessment


Fair 3 12 score types
FAIR 3-12 Score Types the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.


Percentiles
Percentiles the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

  • Raw score transformation that indicates the rank of the student compared to others of the same grade

    • Does not denote mastery (criteron-referencing) but relative performance


Standard scores
Standard Scores the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

  • Standardized scores are derived from raw scores to compare one student’s performance on a test to the mean of all other students at that grade

  • Mean = 100, SD = 15, Range = 55-145


Ability scores
Ability Scores the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

  • Similar to standard scores!

    ......but different!

  • Mean = 500, SD = 100, Range = 200-800


Why we use ability scores
Why we use Ability Scores the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.


Reading the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

Comprehension

Mazes

Word Analysis

AP Score

PM score

AP Score

PM score

AP Score

PM score

student

lexile

score

student

lexile

score

Percentile

rank

WAAS

Percentile

rank

Adj. Maze

SS

FSP

SS

RCAS

%ile & SS


Why not fsp
Why not FSP? the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

  • FSP includes previous FCAT

  • Differential calculation of FSP

  • Student may gain in FAIR Reading but not change FSP


Gain score analyses
Gain Score Analyses the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

  • Simple difference scores

  • Only students who were in all APs

  • Only within the testing window

  • Only for consistent grade students


Analyzing student progress1
Analyzing Student Progress the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

  • Unlike K-2 we have Ability Scores

    • Determine if score type is AP or PM score

  • AP score

    • <85% FSP

      • Was there a change in FSP?

    • >=85% FSP

      • Did the student remain the “green zone”?

  • PM score

    • Examine AS in light of state results

    • Did ability score increase for the student?


Changes to rcas
Changes to RCAS the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

  • New passages and linking

    • Range of scores for RCAS & Lexile changing

  • RCAS 2009-2010

    • 200-800

  • RCAS 2010-2011

    • 150-1000

  • Uncapped Lexile 2009-2010

    • 220L – 1735L

  • Uncapped Lexile 2010-2011

    • 225L – 2105L


Next steps
Next Steps the student’s percent chance of being at or above grade level by the end of the year based on the performance for that assessment period (AP) and time of year. The 40th percentile on the SESAT (K) or SAT-10 (grade 1 and 2) is the cut point for grade level performance.

  • Analyzing specific growth targets

  • Is there merit in knowing the gain scores?

  • Working with JRF to provide guidelines to districts/schools


ad