Developing an Early Diagnostic Assessment for Reading Skills: The Research Basis for the
This presentation is the property of its rightful owner.
Sponsored Links
1 / 37

David J. Francis Texas Institute for Measurement, Evaluation, & Statistics PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on
  • Presentation posted in: General

Developing an Early Diagnostic Assessment for Reading Skills: The Research Basis for the Texas Primary Reading Inventory. David J. Francis Texas Institute for Measurement, Evaluation, & Statistics Department of Psychology University of Houston. UT-Houston HSC Barbara R. Foorman

Download Presentation

David J. Francis Texas Institute for Measurement, Evaluation, & Statistics

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


David j francis texas institute for measurement evaluation statistics

Developing an Early Diagnostic Assessment for Reading Skills: The Research Basis for the Texas Primary Reading Inventory

David J. Francis

Texas Institute for Measurement, Evaluation, & Statistics

Department of Psychology

University of Houston


Collaborators in initial development of tpri center for academic and reading skills

UT-Houston HSC

Barbara R. Foorman

Jack M. Fletcher

Angeliki Mousaki

Univ. of Houston

Chris Schatschneider

Coleen D. Carlson

Kimberly J. Wristers

Dung-tsa Chen

Collaborators in Initial Development of TPRICenter for Academic and Reading Skills


Overview

Overview

  • Background on Reading and Reading Problems

  • Overview of TPRI

  • Development of TPRI Screening Components

  • Reliability and Validity of Screening Components

  • Conclusions and Future Directions


What we know about reading reading acquisition and reading problems

What we know about reading, reading acquisition, and reading problems

Reading is

  • the process of extracting meaning from print

  • a developmental process

  • dependent on language

  • a complex behavior that is acquired

    • playing the piano vs learning one’s native language


What we know about reading reading acquisition and reading problems1

What we know about reading, reading acquisition, and reading problems

  • Reading outcomes vary across children.

    • some children learn almost magically (about 5%)

    • others learn easily no matter what method of instruction is used (about 20%-30% )

    • about 30% of children struggle to learn to read (equal for boys and girls)


What we know about reading reading acquisition and reading problems2

What we know about reading, reading acquisition, and reading problems

  • Most reading problems occur at the level of the single word

    • reading is characterized by slow and labored decoding

    • comprehension suffers due to inefficiency of decoding

  • Phonemic awareness is an important skill that facilitates the development of word recognition skills


Importance of early assessment and intervention for reading problems

Importance of Early Assessment and Intervention for Reading Problems

  • Reading problems are associated with poor educational and social outcomes

  • Children do not simply outgrow reading problems.

  • Early intervention is clearly effective (Torgesen, 1997; Vellutino et al., 1996)

  • Not all children respond equally well to all interventions.

  • Combining early identification with targeted intervention, the effectiveness of early interventions may be enhanced


Reading in texas

Reading in Texas

  • State Curriculum Standards (TEKS)

  • State-wide Accountability System

    • TAAS (TEKS based CRT beginning in G3)

    • Academic Performance Indicators

  • 1997 - Diagnostic Reading Instruments Bill

    • Link to TEKS

    • Teacher administered to individual students

    • Identify reading problems in K-2  Inform instruction

    • Inform instruction  All children read by Grade 3

    • NOT part of State-wide Accountability System


Overview of the tpri

Overview of the TPRI

  • Teacher administered

  • Kindergarten to Grade 2

  • At each grade, test comprised of two major sections:

    • Screening (identify children NOT at risk)

    • Inventory (inform instruction)


Kindergarten tpri

Kindergarten TPRI

  • Administered middle and end of year

  • Screening

    • Letter Names & Sounds

    • Phonemic Awareness (Blending Onsets & Rimes)

  • Inventory

    • Book & Print Awareness

    • Phonemic Awareness (Rhyming, Blending Word Parts, Blending Phonemes, Detecting Initial Sounds, Detecting Final Sounds)

    • Graphophonemic Knowledge (Letter Name Identification, Letter to Sound Linking)

    • Listening Comprehension


First grade tpri

First Grade TPRI

  • Screening (beginning and end of year)

    • Letter Names & Sounds (beginning of year only)

    • Phonemic Awareness

      • Blending Word Parts (Beginning of year only)

      • Blending Phonemes (End of year only)

    • Word Reading (beginning and end of year)

  • Inventory

    • Phonemic Awareness

    • Graphophonemic Knowledge

    • Word Reading (Placement on comprehension stories)

    • Reading Comprehension


Second grade tpri

Second Grade TPRI

  • Screening (beginning of year only)

    • Word Reading

  • Inventory

    • Graphophonemic Knowledge

    • Word Reading (Placement on comprehension stories)

    • Reading Comprehension


Empirical data behind tpri design of ears study

Empirical Data Behind TPRIDesign of EARS Study


Sample demographics

Sample Demographics

  • Total N=945 across the 5 cohorts

  • 3 schools

  • All children in all K, 1, and 2 invited. (Random sample of those consenting - 80+%)

  • Free lunch participation ranged from 13% to 30%

  • Boys and girls were equally represented

  • Caucasian (54%), African American (18%), Hispanic (15%), Asian (12%)

  • SES - LC (9%), WC (43%), MC (48%)


Measures assessed four times per year

Measures Assessed Four Times per Year


End of year outcomes in grades 1 and 2

End of Year Outcomes in Grades 1 and 2

  • Woodcock-Johnson Reading (LW, WA, PC)

  • KTEA Spelling

  • Gray Oral Reading Test

  • Formal Reading Inventory

  • WISC

  • WJ Broad Reading Score was used to form criterion outcome in Grades 1 (1.4) and in Grade 2 ( 2.4), which correspond to the 18th and 35th percentiles, respectively


Longitudinal sample sizes in ears study

Longitudinal Sample Sizes in EARS Study


Steps involved in development of each screen

Steps Involved in Development of Each Screen

1)Scale Predictors Using IRT where possible and Estimate Ability for all subjects

2)Determine Criterion of Interest

3)Predict Criterion from Latent Ability Estimate

4)Manipulate Cut-Point to Achieve Desired False-Negative Rate

5)Determine Latent Ability at the Cut-Point (4)

6)Select Items to Discriminate at Latent Ability Determined in (5)


Irt vs classical test theory

IRT vs. Classical Test Theory

  • Classical test theory defines true score as persons expected (i.e., average) score on the test

    • score is dependent on the items used in making up the test. If the test is made more difficult, a person’s “abilty” goes down

  • The IRT model is a latent trait, or strong true score model

    • ability is independent of the items used in the test

    • because we know the fundamental relation between item responses and ability, we can estimate ability on this constant metric, even though the items in the test may change


Advantages of irt utilized in the tpri

Advantages of IRT utilized in the TPRI

  • Focus the selection of screening items at the cut-point on the ability dimension

  • Keep screening short and still afford accurate discrimination around the cut-point

  • Develop new screening and inventory items through appropriate linking studies

  • Develop word lists linked to stories for placement of students on comprehension stories


Definitions of decisions

Definitions of Decisions


Steps involved in development of each screen1

Steps Involved in Development of Each Screen

1)Scale Predictors Using IRT where possible and Estimate Ability for all subjects

2)Determine Criterion of Interest

3)Predict Criterion from Latent Ability Estimate

4)Manipulate Cut-Point to Achieve Desired False-Negative Rate

5)Determine Latent Ability at the Cut-Point (4)

6)Select Items to Discriminate at Latent Ability Determined in (5)


Steps involved in development of each screen2

Steps Involved in Development of Each Screen

1)Scale Predictors Using IRT where possible and Estimate Ability for all subjects

2)Determine Criterion of Interest

3)Predict Criterion from Latent Ability Estimate

4)Manipulate Cut-Point to Achieve Desired False-Negative Rate

5)Determine Latent Ability at the Cut-Point (4)

6)Select Items to Discriminate at Latent Ability Determined in (5)


End of grade 1 basic reading using december kindergarten decision rules

End of Grade 1 Basic Reading Using December Kindergarten Decision Rules


End of grade 1 basic reading using april kindergarten decision rules

End of Grade 1 Basic Reading Using April Kindergarten Decision Rules


End of grade 1 reading using october grade 1 word reading and phonological awareness

End of Grade 1 Reading Using October Grade 1 Word Reading and Phonological Awareness


End of grade 2 using end of grade 1 word reading

End of Grade 2 using End of Grade 1 Word Reading


End of grade 2 using end of grade 1 word reading and phonological awareness

End of Grade 2 using End of Grade 1 Word Reading and Phonological Awareness


Beginning of grade 2 to end of grade 2

Beginning of Grade 2 to End of Grade 2


Tpri field study

TPRI Field Study

  • Investigate teacher use

  • Examine sources of variability in test performance (Content, administrator, time)

  • Data Collected on 4 forms

    • End of Kindergarten

    • Beginning of First Gradea

    • End of First Grade

    • Beginning of Second Gradeb

      aparticipants were at the end of Kindergarten

      bparticipants were at the end of Grade 1


Design of tpri field study

Design of TPRI Field Study


Field study estimates of reliability for screens

Field Study Estimates of Reliability for Screens


Conclusions

Conclusions

  • The TPRI is currently in use in over 80% of Texas Public Elementary schools

  • Feedback from teachers has been widely positive

  • Teachers have asked for more training

  • Current emphasizing development of interventtion strategies and developing more effective means for disseminating this information to teachers

  • The screening device has proven helpful in optimizing assessment time for teachers


Conclusions cont

Conclusions (cont.)

  • Opportunities for Research

    • linking to outcome assessments used in the accountability system

    • evaluation of instructional decision making and intervention strategies

    • improved decision making through computer aided administration and student profiling

    • development of comparable instruments for literacy instruction in Spanish (Tejas Lee)


  • Login