OR Reading First: Review of Comprehensive Programs

1. OR Reading First: Review of Comprehensive Programs

2. Objective of Reading First (1) “To provide assistance to State educational agencies and local educational agencies in establishing reading programs for students in kindergarten through grade 3 that are based on scientifically based reading research to ensure that every studentcan read at grade level or above not later than the end of grade 3.” • NCLB, 2001, Part B, Sec. 1201.

3. Why Focus on Reading Programs Now • Unprecedented convergence about what children need to be successful readers • National syntheses provide scientific evidence on which to base practice • Much classroom practice is shaped by textbooks • State standards are embracing the science • Publishers respond to the marketplace and need • Window of opportunity to align what we know, what we use, and how we teach to attain critical results

4. Comprehensive Reading Programs • Purpose: • to provide sufficient instruction in the core components of reading (enough of the “right stuff” in a systematic design) • to provide instruction that enables the majority of students to meet or exceed grade-level standards on all the key Reading First elements • to serve as the primary reading program for a school within and between grades (K-3)

5. Why Adopt A Comprehensive Reading Program? • Increases continuity, coherence, and community of effort within and between grades (all teachers are aware and working toward the same goals) • Creates more “buying power” regarding professional development • Affords greater differentiation of instruction for children (can share children within and between grades) • Reading instruction is rocket science (Moats, 1999) and expecting teachers to construct and instruct is unreasonable and too important to leave to chance.

6. Oregon Reading First Curriculum Review • Focus: Comprehensive reading programs • Purpose: To determine alignment with SBRR & provide consumers guidance • Curriculum Review Panel • Tool/Criteria: Consumer’s Guide (Simmons & Kame’enui) • Review Process

7. Curriculum Review Panel: Selection of Members • Former State Superintendent of Public Instruction, Stan Bunn, invited 25 educators across the state to serve on the Curriculum Review Panel. • 5 additional members from the U of O, Eugene Research Institute, and Oregon Research Institute and 3 members from ODE were identified in Oregon’s Reading First grant.

8. Curriculum Review Panel: 17 final members • 2 district administrators • 2 district curriculum specialists • 1 classroom teacher • 4 university faculty from three state universities (U of O, Pacific, Univ. of Portland) • 2 doctoral students from the U of O • 4 researchers from ERI • 1 researcher from ORI • 1 educational specialist from ODE

9. Oregon Reading First Curriculum Review Panel Members • Julie Anderson Educational Specialist/English Language Arts, ODE • Scott Baker, Ph.D Researcher, Eugene Research Institute • Lynette Doht Reading Specialist, Portland Public Schools • Hank Fien Research Assistant, University of Oregon • Barbara Gunn, Ph.D Researcher, Oregon Research Institute • Arlene Hett, Ph.D Director of Teacher Education, University of Portland • Sara Johnson Principal, Henry L. Slater Elementary School • Edward J. Kame’enui , Ph.D Professor, University of Oregon • Kristen MacConnell Research Assistant, University of Oregon • Anita McClain, Ph. D Professor, Pacific University • Janet Otterstedt Research Assistant, Eugene Research Institute • Sandra Pellens, Ph.D Director of Instruction, Molalla River SD 35 • Michael Rebar, Ph.D Researcher, Eugene Research Instiitute • Deborah C. Simmons, Ph.D Professor, University of Oregon • Sylvia Smith, Ph.D Researcher, Eugene Research Institute • Carrie Thomas-Beck, Ph.D Curriculum Specialist, Springfield SD 19 • Deborah White Teacher of the Year, Lyle Elementary School

10. TOOL: Consumer’s Guide to Evaluating a Core Reading Program • Developers: Drs.Deborah C. Simmons and Edward J. Kame’enui, University of Oregon • Why Developed: To assist states, districts and schools in the selection of research-based tools • When Developed: As part of National Center to Improve the Tools of Educators’ scope of work (1990-2000) • Purpose: To document and quantify the design and delivery features of core reading programs.

11. Consumer’s Guide: Organization • Programs Evaluated by Grade • Within Grade by Essential Component: • Phonemic Awareness • Phonics • Fluency • Vocabulary • Comprehension

12. Essential Components by Grade

13. Consumer’s Guide:Organization (cont.) • For each Essential Component: • High Priority Items • Discretionary Items • Overarching Design Items for Each Grade

14. Consumer’s Guide:Sample Items • KINDERGARTEN • Phonemic Awareness • High Priority Items: • #1 Progresses from the easier phonemic awareness activities to more difficult (e.g., isolation, blending, segmentation, and manipulation) (ss) • #2 Teaches skills explicitly and systematically (w) • #4 Integrates letter-sound correspondence instruction to phonological awareness (w) • #5 Focuses on segmentation or the combination of blending and segmenting for greatest transfer (ss)

15. Consumer’s Guide:Sample Items • KINDERGARTEN • Phonemic Awareness • Discretionary Items: • #1 Focuses beginning instruction on the phonemic level of phonological units with short words (two to three phonemes; e.g., at, mud, run) • #3 Focuses first on the initial sound (sat), then on the final sound (sat), and lastly on the medial sound (sat) in words.

16. Consumer’s Guide:Sample Items • KINDERGARTEN • Design Features • 1. Coordinates and integrates phonemic awareness and phonics instruction and student materials. • 2. Provides ample practice on high-priority skills. • 3. Provides explicit and systematic instruction. • 4. Includes systematic and cumulative review of high priority skills. • 5. Demonstrates and builds relationships between fundamental skills leading to higher order skills.

17. Scoring Criteria Use the following criteria to score each item: • = Program consistently meets/exceeds criterion • = Program partially meets/exceeds criterion • = Program does not satisfy the criterion When evaluating individual elements, slash ( / ) the respective circle that represents your rating (e.g., ).

18. Analysis of High Priority Items

19. Within a Sequence of Lessons Analysis (W)

20. Scope and Sequence Analysis (SS)

21. Skills Trace of Letter-Sound Review (ST) - K

22. Review Process • Announcement posted on Oregon Reading First and WOATRA’s websites inviting publishers to submit core reading programs for review. (February 14, 2003 deadline for submissions)

23. Call for Comprehensive Programs Comprehensive Programs: • (a) include materials for all grades K-3, • (b) comprehensively address the “five essential components” of the Reading first legislation in scientifically based beginning reading instruction: phonemic awareness, phonics, reading fluency, vocabulary, and reading comprehension.

24. Comprehensive Programs Reviewed: • 15 programs submitted / 9 reviewed as comprehensive programs: • Harcourt • Houghton-Mifflin • Macmillan/McGraw-Hill • Open Court • Reading Mastery • Rigby • Scott Foresman • Success For All • Wright Group

25. Review Process: Program Assignment • OR CRP members provided with 6 hours of training by OR RF staff (February 21, 2003) • CRP members randomly assigned to programs to review. • Each reviewer assigned to either a K/1 or 2/3 grade. No reviewer evaluated an entire program K-3. • Each member reviewed 1 to 4 programs based on availability. • CRP members were not permitted to review any program for which they were an author, consultant, or advisor. (Reviewers signed statements of disclosure.) • Initial review completed March to May, 2003.

26. Review Process: • Each section of a program (K/1 or 2/3) was reviewed by 2 independent reviewers. • The same two reviewers were never paired more than once. • Thus, each program was reviewed by 4 different members of the review panel. • Members spent from 8 to 30 hours to complete a program assignment (K/1 or 2/3) • All review work was completed at the OR Reading First Center.

27. Review Process:Reliability Between Reviewers • Upon completion of the review, scores between the two reviewers were compared. • Each rating was assigned a point value: • = 2 points • = 1 point • = 0 points • Items that were off by one were averaged (e.g. full circle and partial circle = 1.5) • For items that were off by more than one (e.g. full circle and empty circle), a third reviewer was asked to reconcile the items.

28. Review Process:Third Reviewer • Third reviewers assigned to “reconcile” did not conduct the initial review of the program. • Third reviewers examined the documented evidence from the 2 initial reviewers and the program materials to determine which of the two scores best represented the selected item as the final score.

29. Review Process:Summarizing Results • Result of the review process is one averaged/ rectified score for each item for a program. • Final Report includes a completed Consumer’s Guide for each program (item by item). • Final Report summarizes results by program, by essential components, and by grade. • For each program, results are summarized by high priority, discretionary, and design items.

30. Final Report:Sample Consumer’s Guide

31. Final Report:Sample Consumer’s Guide (cont.) • Kindergarten Phonemic Awareness Instruction - High Priority • Tally the number of elements with each rating. • 22 + 1 __ + __ • (2 pts) (1.5 pts) (1 pt) (.5 pts) (0 pts) • Total Points/Total Possible Points 8/10 = 80%

32. Sample: Summary of Kindergarten Ratings

33. Next Steps • Final report distributed to districts, schools, and publishers for guidance in program selection at IBR I in June 2003. • Final report discussed in detail on Day 3 of IBR I. • Programs will be on display throughout the four days of IBR I for school staff to examine. • Schools will need to allot their professional development funds to pay a team of educators to examine the programs more closely during the last week of June.

34. Next Steps (cont.) • Schools will need to notify ODE of their program selection by early July to secure approval then order materials. • Schools will select those programs that were reviewed favorably by Oregon’s RF Curriculum Review Panel. • Schools will then need to arrange for professional development on their new program early in the fall.