slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Oregon Reading First PowerPoint Presentation
Download Presentation
Oregon Reading First

Loading in 2 Seconds...

play fullscreen
1 / 55

Oregon Reading First - PowerPoint PPT Presentation


  • 230 Views
  • Uploaded on

Oregon Reading First. Review of Supplemental and Intervention Programs Carrie Thomas Beck, Ph.D. Edward J. Kame’enui, Ph.D. Acknowledgments. Oregon Department of Education Institute for the Development of Educational Achievement, College of Education, University of Oregon

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Oregon Reading First' - ostinmannual


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Oregon Reading First

Review of Supplemental and Intervention Programs

Carrie Thomas Beck, Ph.D.

Edward J. Kame’enui, Ph.D.

acknowledgments
Acknowledgments
  • Oregon Department of Education
  • Institute for the Development of Educational Achievement, College of Education, University of Oregon
  • U.S. Department of Education, Office of Special Education Programs
  • Oregon Reading First Supplemental and Intervention Curriculum Review Panel
content development
Content Development

Content developed by:

Carrie Thomas Beck, Ph. D.

Coordinator, Oregon Reading First Center

University of Oregon

Edward J. Kame’enui, Ph. D.

Professor, College of Education

University of Oregon

Additional support:

Patrick Kennedy-Paine

Katie Tate

University of Oregon

copyright
Copyright
  • All materials are copy written and should not be reproduced or used without expressed permission of Dr. Edward J. Kame’enui or Dr. Deborah C. Simmons. Selected slides were reproduced from other sources and original references cited.
ibr foundational features translating research into practice

Schoolwide: Each & All

Results Focused

All Readers by 3rd Grade

Prevention Oriented

Scientifically Based

IBR Foundational Features: Translating Research into Practice
ibr guiding questions
IBR Guiding Questions
  • Goals: What outcomes do we want for our students in our state, district, and schools?
  • Knowledge: What do we know and what guidance can we gain from scientifically based reading research?
  • Progress Monitoring Assessment: How are we doing? What is our current level of performance as a school? As a grade? As a class? As an individual student?
  • Outcome Assessment: How far do we need to go to reach our goals and outcomes?
  • Core Instruction: What are the critical components that need to be in place to reach our goals?
  • Differentiated Instruction: What more do we need to do and what instructional adjustments need to be made?

Today’s Focus

objective of reading first
Objective of Reading First
  • (1) “To provide assistance to state educational agencies and local educational agencies in establishing reading programs for students in kindergarten through grade 3 that are based on scientifically based reading research to ensure that every student can read at grade level or above not later than the end of grade 3.”
  • NCLB, 2001, Part B, Sec. 1201
understanding the purpose of different programs

Core

Reading Program

Meeting the needs for most

Understanding the Purpose of Different Programs

Programs are tools that are implemented by teachers to ensure that children learn enough on time.

Classifying Reading Programs:

What is the purpose of the program?

1. Core

2. Supplemental

Core

Intervention

3. Intervention

Supplemental

Supplemental

Intervention

Reading Program

Reading Program

Supporting the Core

Meeting the needs for each

(Vaughn et al. 2001)

core program
Core Program

A core program is the “base” reading program designed to provide instruction on the essential areas of reading for the majority of students schoolwide. In general, the core program should enable 80% or more of students to attain schoolwide reading goals.

A Core Instructional Program of Validated Efficacy Adopted and Implemented Schoolwide

evaluating core programs identifying gaps
Evaluating Core Programs: Identifying Gaps

One size does not fit all—

Period!

We may need to supplement or modify, but we must do it judiciously.

supplemental programs
Supplemental Programs:
  • Support and extend the critical elements of a core reading program.
  • Provide additional instruction in one or two areas (e.g., phonological awareness, fluency) and
  • Provide more instruction or practice in particular area(s) of need.
intervention programs
Intervention Programs:
  • Designed for children who demonstrate reading difficulty and are performing below grade level.
  • Provide more explicit, systematic instruction to accelerate learning and bring the learner to grade-level performance.
  • Focus on more than one area (e.g., phonics, fluency, and comprehension).
  • May focus exclusively on one essential reading area.
  • Allow teachers to meet the needs of individual students who are struggling in their classrooms.
  • Specialized, intense, and typically delivered in small group settings.
understanding the purpose of different programs13

Core

Reading Program

Meeting the needs for most

Understanding the Purpose of Different Programs

Programs are tools that are implemented by teachers to ensure that children learn enough on time.

Classifying Reading Programs:

What is the purpose of the program?

1. Core

2. Supplemental

Core

Intervention

3. Intervention

Supplemental

Supplemental

Intervention

Reading Program

Reading Program

Supporting the Core

Meeting the needs for each

(Vaughn et al. 2001)

identifying gaps examine the core review
Identifying Gaps: Examine the Core Review

Summary of Houghton Mifflin: Possible Gaps Are Areas <75%

identifying gaps examine the core review15
Identifying Gaps: Examine the Core Review

Summary of Open Court: Possible Gaps Are Areas <75%

call for supplemental and intervention programs
Call for Supplemental and Intervention Programs
  • Announcement posted on Oregon Reading First and WOATRA’s websites inviting publishers to submit supplemental and intervention programs for review. (August 15, 2003 deadline for submissions.)
  • Second call for supplemental and intervention programs in October of 2003 with a December 19th deadline for submissions.
supplemental and intervention programs to review
Supplemental and Intervention Programs to Review
  • 42 publishers submitted 230 programs in first call.
  • In October, 2003, regional coordinators and mentor coaches were asked to identify high-priority programs from the list of 230 programs and were given the opportunity to nominate programs that were not on the list.
  • Any program that received 2 or more votes was put on a priority list.
  • 106 high-priority programs were identified.
more programs to review
More Programs to Review
  • 44 programs written in by coaches and regional coordinators.
  • 13 additional programs were identified from Reading First Centers in other states
  • Approximately 60 programs received through the second call.
  • With first call programs, a total of over 300 programs to review!!
curriculum review panel for supplemental and intervention programs
Curriculum Review Panel for Supplemental and Intervention Programs
  • 66 members:
    • 35 Reading First Mentor Coaches
    • 4 Reading First Regional Coordinators
    • 5 staff members from the Western Regional Reading First Technical Assistance Center
    • 3 faculty and staff from the University of Oregon
    • 6 research assistants from the University of Oregon
    • 7 researchers from the Pacific Institute for Research
    • 4 teachers
    • 2 district curriculum specialists
oregon reading first curriculum review panel members
Oregon Reading First Curriculum Review Panel Members
  • Karen Aquinas RF Mentor Coach, Chiloquin Elementary
  • Julie Bainbridge RF Mentor Coach, May Roberts Elementary
  • Doris Baker Research Assistant, University of Oregon
  • Scott Baker, Ph.D. Researcher, Pacific Institute for Research
  • Leanne Bettesworth Research Assistant, University of Oregon
  • Judy Chesnut RF Mentor Coach, Grove and Freewater Elementaries
  • Linda Carnine, Ph.D. Consultant, Western Regional RF Technical Assistance Center
  • Ben Clarke, Ph.D. Researcher, Pacific Institute for Research
  • Debbie Connolly RF Mentor Coach, Howard Elementary
  • Lisa Cook Teacher (Title I), Cherry Park Elementary
  • Annette Culbertson RF Mentor Coach, Beach Elementary
  • Carol Dissen RF Mentor Coach, Mooberry Elementary
  • Hank Fien RF Regional Coordinator
  • Toni Fisher RF Mentor Coach, William Walker Elementary
  • Toni Gula RF Mentor Coach, Lincoln Park Elementary
  • Barbara Gunn, Ph.D. Researcher, Pacific Institute for Research
  • Sheila Hallinan RF Mentor Coach, Scott Elementary
  • Mariko Harris Teacher (ELL), Mill Park Elementary
  • David Howe Tech. Assistance Coordinator, Western Regional RF Tech. Assist. Center
  • Kathy Howe, Ph.D. Deputy Director, Western Regional RF Tech. Assistance Center
oregon reading first curriculum review panel members cont
Oregon Reading First Curriculum Review Panel Members (cont.)
  • Susan Josephson RF Mentor Coach, Woodlawn Elementary
  • Edward Kame’enui, Ph.D. Professor, University of Oregon
  • Ruth Kaminski, Ph.D. Researcher, Pacific Institute for Research
  • Rachell Katz RF Regional Coordinator
  • Meera Kreitzer RF Mentor Coach, Washington Elementary
  • Katherine Kurdock RF Mentor Coach, Aloha Park Elementary
  • Jim Lehmkuhler RF Mentor Coach, Myrtle Creek Elementary
  • Barbara Low RF Mentor Coach, Oak Grove Elementary
  • Kristen MacConnell Research Assistant, University of Oregon
  • Jerrie Matuszak RF Mentor Coach, Sunrise Elementary
  • Laurene McClintock RF Mentor Coach, Mill Park Elementary
  • Sarah McDonagh, Ph.D. Tech. Assist. Coord, Western Regional RF Tech Assist Center
  • Stacie Moncrief RF Mentor Coach, Ventura Park Elementary
  • Shawna Moran RF Mentor Coach, Hayesville Elementary
  • Linda Myers RF Mentor Coach, Hayesville Elementary
  • Pat Nash RF Mentor Coach, Lincoln Elementary
  • Kristin Orton Research Assistant, University of Oregon
  • Janet Otterstedt Research Assistant, Pacific Institute for Research
  • Mary Peake RF Mentor Coach, Humboldt Elementary
  • Maria Randle RF Mentor Coach, Westside Elementary
oregon reading first curriculum review panel members cont26
Oregon Reading First Curriculum Review Panel Members (cont.)
  • Michael Rebar, Ph.D. Researcher, Pacific Institute for Research
  • Connie Robertson RF Mentor Coach, David Hill Elementary
  • Donna Rodgers RF Mentor Coach, Warm Springs Elementary
  • Katie Rotz Research Assistant, University of Oregon
  • Robin Rovers Teacher (ERC), Lincoln Park Elementary
  • Kathy Ryan RF Mentor Coach, Cherry Park Elementary
  • Amanda Sanford Research Assistant, University of Oregon
  • Mary Sharp RF Mentor Coach, Tri City Elementary
  • Nicole Sherman Brewer RF Regional Coordinator
  • Mandy Shoemaker RF Mentor Coach, W.L. Henry Elementary
  • Jerry Silbert Tech. Assistance Provider, Western. Regional RF Tech. Assist. Center
  • Deborah Simmons, Ph.D. Professor, University of Oregon
  • Sylvia Smith, Ph.D. Researcher, Pacific Institute for Research
  • Kimberly Skach RF Mentor Coach, West Powellhurst Elementary
  • Rea Snyder RF Mentor Coach, Jackson Elementary
  • Maryann Stalnaker TOSA, Portland Public Schools
  • Annie Tabshy RF Mentor Coach, Boise-Eliot Elementary
  • Linda Taylor RF Mentor Coach, Jefferson Elementary
  • Carrie Thomas Beck, Ph.D. Coordinator, Oregon Reading First Center
  • Kathy Tosti TOSA, Portland Public Schools
oregon reading first curriculum review panel members cont27
Oregon Reading First Curriculum Review Panel Members (cont.)
  • Gretchen Van Hoet RF Mentor Coach, Whitcomb Elementary
  • Jim Walt RF Mentor Coach, Alameda Elementary School
  • Gloria Weinberg RF Mentor Coach, Lent Elementary
  • Angela Whalen, Ph.D. Instructor & School Psych Practicum Coord., U of O
  • Deborah White Oregon Teacher of the Year, Lyle Elementary School
  • Cynthia Wood RF Mentor Coach, Whitman Elementary School
a consumer s guide to evaluating supplemental and intervention reading programs grades k 3
A Consumer’s Guide to Evaluating Supplemental and Intervention Reading Programs Grades K-3
  • ORGANIZATION:
    • by essential component
    • items generally ordered from K to 3
    • appropriate grades marked with an X
    • two columns for documentation
      • Initial Instruction
      • Additional Evidence
    • summary and comment pages following each essential component
breakdown by essential component
Breakdown by Essential Component:
  • Phonemic Awareness - 10 items
  • Phonics:
    • Decoding - 13 items
    • Irregular Words - 6 items
  • Fluency - 12 items
  • Vocabulary - 10 items
  • Comprehension - 10 items
scoring criteria

= Program consistently meets/exceeds criterion

  • = Program partially meets/exceeds criterion
  • = Program does not satisfy the criterion
Scoring Criteria:

Use the following criteria to score each item:

When evaluating individual elements, slash ( / ) the respective circle that represents your rating (e.g., ).

scoring guidelines
Scoring Guidelines
  • Identify essential components and grade levels that the program targets.
  • Complete only those items that evaluate the targeted components and grades.
  • For multi-grade programs that instruct students across grade levels (e.g., a K-1 program), mark one program rating per relevant item.
  • For grade-specific programs, review each grade level separately and assign a distinct score for each grade for each item.
  • Complete additional items for intervention programs.
items for analysis of k 3 reading intervention programs
Items for Analysis of K-3 Reading Intervention Programs
  • INSTRUCTIONAL DESIGN AND EMPHASIS (7 Items)
  • INSTRUCTIONAL GROUPING (3 Items)
  • INSTRUCTIONAL ASSESSMENT (4 Items)
program assignment
Program Assignment
  • Initially, CRP members were randomly assigned to programs to review. Each member was assigned 4 programs, starting with the high priority programs.
  • After a couple weeks of reviewing, all incomplete programs were combined into one pool. As reviewers finished a program, the next high priority program was selected from the pool.
  • The following were taken into consideration for program assignment:
    • availability of reviewer
    • length of program
    • physical size of program
    • technology needs
    • location of reviewer
  • CRP members were not permitted to review any program for which they are an author, consultant, or advisor. (All reviewers signed a statement of disclosure.)
program assignment cont
Program Assignment (cont.)
  • In most cases, each reviewer was assigned to evaluate all levels of a program (e.g., K, 1, 2, and 3).
  • Each reviewer was not assigned more than 2 programs from the same publisher.
  • To date, each member has reviewed from 1-14 programs based on reviewer availability and length of programs.
  • Time to complete a program varied greatly. (e.g., In a group of 10 reviewers who each completed 1 program at the beginning of December, completion time ranged from 3.5 to 25 hours.)
review procedures
Review Procedures
  • Each program was reviewed by 2 independent reviewers. (Reviewers were assigned code numbers to ensure a blind review.)
  • Review work was completed at the Oregon Reading First Center or materials were checked out to complete the review off site.
  • Initial review work was completed between October, 2003 and January, 2004.
  • 53 programs completed to date.
review procedures reliability between reviewers
Review Procedures: Reliability Between Reviewers
  • Upon completion of a program, scores between the two initial reviewers were compared.
  • Each rating was assigned a point value:
  • = 2 points
  • = 1 point
  • = 0 points
  • Items that were off by one were averaged (e.g. full circle and partial circle = 1.5)
  • For items that were off by more than one (e.g. full circle and empty circle), a third reviewer was asked to reconcile the items.
review procedures reliability between reviewers43
Review Procedures: Reliability Between Reviewers
  • Third reviewers assigned to “reconcile” did not conduct the initial review of the program.
  • Third reviewers examined the documented evidence from the 2 initial reviewers and the program materials to determine which of the two scores best represented the selected item as the final score.
summarizing results
Summarizing Results
  • Result of the review process is one averaged/ rectified score for each item for a program.
  • Final Report includes a completed Consumer’s Guide for each program (item by item).
  • Final Report summarizes results by program and by essential components.
  • NOTE: Intervention items have not been analyzed yet.
final report
Final Report

Oregon Reading First

Review of Supplemental and Intervention Programs

1. Table of Contents

2. List of Completed Programs

3. Consumer’s Guide for Each Completed Program (alphabetically by program name)

4. Summary By Big Idea (Essential Component)

• Total percentages by grade for PA, phonics, and fluency

• Item analyses for each big idea

program summary phonemic awareness example
Program Summary: Phonemic Awareness Example
  • Kindergarten/ First Grade Phonemic Awareness Instruction
  • Tally the number of elements with each rating.

11 + 2_3_ + _3_

(2 pts) (1.5 pts) (1 pt) (.5 pts) (0 pts)

  • Total Points/Total Possible Points 7/20 = 35%
percentage summaries for vocabulary and comprehension
Percentage Summaries for Vocabulary and Comprehension?
  • Oregon Reading First Center will NOT report percentage scores for vocabulary and comprehension.
  • Lack of convergence in the NRP report on vocabulary and reading comprehension (i.e., NRP not able to conduct a meta-analysis because of limited studies) makes it very difficult to argue convincingly and to justify a quantification using percentage scores that ostensibly results in a rank order of program design and quality.
oregon reading first guidelines for purchasing supplemental and intervention programs
Oregon Reading First:Guidelines for Purchasing Supplemental and Intervention Programs
  • Vocabulary and Comprehension Programs:
    • Schools are required to critically examine the results for each of these big ideas and determine program quality based on a close scrutiny of the evaluation on an item-by-item basis.
    • Schools must take into consideration (a) their core program and (b) student performance in making the decision on which vocabulary and comprehension programs to purchase.
  • Phonemic Awareness, Phonics, and Fluency Programs:
    • Those programs that scored at least 75% of total points will be approved for purchasing.
and the review goes on
And the Review Goes On . . .
  • CRP members will continue to review programs over the next several months.
  • Results of individual programs will be released as reviews are completed.
  • Many programs are near completion and will be released in the upcoming weeks.
  • Programs on the high priority list (n = 106) will be reviewed first.
and the review goes on55
And the Review Goes On . . .
  • Carrie will email coaches each time a new program review is completed.
  • The results will be posted on the Oregon Reading First Website

(http://oregonreadingfirst.uoregon.edu)