Joshua Wilson Yan Wei Michael Faggella-Luby, PhD

1. Adolescent Literacy Assessment: Observing and Coaching Evidence-Based Instruction in Discipline-Specific ClassroomsSpotlight on Assessment, Evaluation, and Research Joshua Wilson Yan Wei Michael Faggella-Luby, PhD 3/26/2013

2. Overview • Setting the Stage • Content & Pedagogy of Adolescent Literacy • Research on Observational Assessments • The Observation Tool • Suggestions and Lessons Learned

3. I. Setting the Stage

4. Who is “at-risk”? Students who are successful during their 9th grade year are 3.5 times more likely to graduate • One F decreases likelihood of graduating from 83% to 60% • 2 Fs decreases likelihood to 44% • 3 Fs decreases likelihood to 31% Allensworth, E., & Easton, J. (2005). The on-track indicator as a predictor of high school graduation. Chicago: Consortium of Chicago School Research

6. Here in CT • “At least 72 percent of [students] attending community colleges require remedial or developmental math or English; for the Connecticut state universities — Central, Western, Southern and Eastern — the figure is • … 65 percent.” Megan, Kathleen (10/27/2010). “Many state high school graduates attending public colleges unprepared.”Hartford Courant.

7. What impacts student outcomes? PROPORTION OF VARIANCE IN STUDENT GAIN SCORES-- READING, MATH (Gr.4,8,10)-- EXPLAINED BY LEVEL--PROSPECTS STUDY Classroom Instruction! STUDENTS 28% R 19% M CLASS 60% READING 52-72% MATH SCHOOLS 12% R 10-30% M ROWAN, ET AL., “. . .PROSPECTS. . .”TEACHERS COLLEGE RECORD( 2005). Slide From R. Elmore

8. What do observation tools show about typical practice? or How can we explain these outcomes?

9. Schumaker et al., 2002

10. 1. Lecture/read 2. Give directions 3. Listening 4. Ask question 5. Monitor 6. Model 7. Verbal rehearsal 8. Simple enhancer 9. Advance organizer 10. Role Play 11. Content Enhancement (complex) 12. Elaborated Feedback 13. Write on board 14. Describe skill/strategy 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Schumaker et al., 2002

11. II. Content & Pedagogy of Adolescent Literacy

12. SRBI/RTI Framework Tier 3: Specialized, Individualized Intervention for Students at High Risk 5% 15% Tier 1: Comprehensive & Coordinated Instruction for All Students Tier 2: Supplemental Instruction for Students at Some Risk 80% of Students

13. Tiered System of Instruction Alterable Components of Instruction • Content • Components of Reading and Writing • Content-specific instructional methods • Programs/Materials • Pedagogy • Explicitness • Intensity and Pacing • Interactiveness • Grouping • Interventionist Expertise

14. Critical Content:What to Teach • Background Knowledge • Knowledge of Text Structure • Cognitive Strategies • Motivation • Word Study • Writing References Biancarosa & Snow, 2004 Ehren, 2005 Faggella-Luby & Deshler, 2008 Gersten et al., 2001 Graham & Hebert, 2010 McCabe, 2009 Torgesen et al., 2007

15. 1. Background Knowledge Example: Social Studies Classroom • Vocabulary • Democracy, communism, filibuster, gridlock • Academic or Signal Language • Passage-specific vocabulary • Concepts • Representative government vs. communism • Free-market economy vs, socialism • World Knowledge • Travel • Experience with an adult voting • Reading/Listening/Viewing political advertisements or debates CCSS: R1-R3; L4-L6

16. 2. Knowledge of Text/Discourse Structures • Narrative/Expository organizational patterns • Student awareness • Strategic use Examples • Narrative Text Structure • Sequential • In medias res • Expository Text Structure (Essay) • Persuasive • Comparison-Contrast • Description • Cause-Effect • Sequential • Problem-Solution • Lab Report • Journal Article • Policy CCSS: R4-R6

17. 3. Cognitive Strategies • Goal-specific • Packaging • Monitoring and repair Examples • Goal-Specific Strategies • questioning • summarization • prediction • inferencing • Packages of Strategies • peer assisted learning (e.g., CWPT) • Reciprocal Teaching • Self-Monitoring/Repair • Collaborative Strategic Reading CCSS: R7-9; R10

18. 4. Motivation • Engagement • Self-efficacy • Task persistence Examples • Engagement • Provide rationale for learning/academic tasks • Co-Construct meaning • Use first-glance influence • Self-efficacy • Students set goals, develop a plan, self-monitor, and reflect • Timely teacher feedback • Engineer success • Task Persistence • Balance of challenging and independent material • Student choice of reading material CCSS: R10

19. 5. Word Study • Multi-syllabic words • Difficult often content specific syntax • Unique semantic constructions • Fluency Examples • Multi-Syllabic Words • Onomatopoeia • Stoichiometry • Pythagoras/Pythagorean • Difficult Often Content Specific Syntax • Word problem in math • Elizabethan English • Legislative language • Unique Semantic Constructions • ‘Republican’ in Federalist papers vs. present day • Use semantic map line labels • Pre-teach key terms and explanations • Fluency • Reading Shakespeare for prosody CCSS: L4, RF3-RF4

20. 6. Writing • Writing Fluency skills • Spelling • Sentence construction • Word choice • Writing Process • Strategies to compose genre-specific text Examples • Writing Fluency Skills • Teach spelling and word-study skills together • Use text models to illustrate sentence fluency and word choice • Writing Process • Planning and Revision • Summarization • Genre-specific Composition • Link strategies for identifying and utilizing text-structure with composition strategies CCSS: L1-L2; W4-W6; W1-W3

21. K.L.P.s - Content(key learning points) • What do I think is most important about what we’ve covered? • How can I use this information?

22. How Teachers Teach Struggling Readers A significant difference between struggling readers and their typically achieving peers is not what they are taught about literacy, but how they are taught!

23. Instructional Core Daily review Introduce lesson objectives Present new content Guided practice Independent Practice Evaluation Formative Summative Priority Instruction Small groups* Strategy Cueing and Questioning Advance Organizer Distributed Practice Corrective and elaborative feedback Pedagogy That Builds Literacy Skills References: Faggella-Luby& Deshler, 2008; Rosenshine,1995; Swanson, 1999; Swanson & Hoskyn, 2001

24. Instructional Core – Teach Intentionally • Introduce Lesson Objectives • Explain how the strategy will improve learning • Clarify expectations for learning and independent use • Present New Content • Share important strategy components or steps • Share the sequence of strategy steps as necessary • Guided Practice • Present a verbal think-aloud of how to use strategy • Identify obstacles and explain how to overcome them • Complete at least one successful application of the strategy

25. Instructional Core – Teach Intentionally (cont.) • Independent Practice • Prompt student involvement to get them actively thinking about the strategy • Practice with appropriate examples to allow for successful student practice • Create opportunities for teacher questioning and student self-questioning • Evaluation • Formative: determine specific instructional needs prior to instruction, frequently check for understanding • Summative: design summative assessments to assess student mastery of learning objectives

26. Gradual Release and Intensity

27. K.L.P.s - Pedagogy(key learning points) • What do I think is most important about what we’ve covered? • How can I use this information?

28. III. Research on Observational Assessments

29. Recent Research on Observational Tools in Secondary Classrooms An electronic search was conducted for studies published between 1990 to 2012 from the ERIC, PsycINFO and PQDT. To be included in this summary, a study was required to meet the following criteria: • An observation study of literacy instruction in ELA classes or in discipline-specific classrooms. • Pertaining to secondary level (9th-12th grade). • Conducting planning and/or observational tools in tier I instruction. • Designed to observe teaching behaviors and/or teaching instructions in the study.

30. Recent Research on Observational Tools in Secondary Classrooms Summarized research from the following elements: • Audience • Person observed • Intended purpose: improve instruction; motivate teachers; make principals accessible to students and teachers; • Observation categories • Method for use: length of observation, tools to observe, recommended use • Observation tools

31. Observation Tools for Teaching Behaviors in Secondary Classrooms

32. Classroom Observation Sheet

33. Observation Tools for Teaching Behaviors in Secondary Classrooms

34. ALWP Template

35. Observation Tools for Teaching Behaviors in Secondary Classroom

36. Checklist for Integrated Reading/Language Arts Instruction

37. Observation Tools for Teaching Behaviors in Secondary Classroom

38. The Writing Observation Framework

39. In Summary Reasons to share: • Observe literacy practices in four instructional settings: • co-teaching classroom, discipline-specific classroom, reading/language arts program, and writing classroom. • Address the components of evidence-based literacy instruction in tier I settings, including content of instruction and teaching pedagogy. • Demonstrate different types of observational formats (checklists, questions, reflections).

40. Strengths and Limitations All Together Strengths: • Increase communication and collaboration among teachers, peers , and administrators through providing frequent and timely feedback at pre/post-observational conferences. • Teachers gaining new ideas and perspectives about teaching from colleagues. • Provide guidelines for developing school-wide literacy instruction(e.g., Vogt, 1991). Limitations: • Possible bias relating to the observer's own beliefs about teaching. • Many elements to observe at once. • Limited observer reliability, variable observation times (short to long). • No studies embedding CCSS into instructional observation.

41. IV. The Observation Tool Putting Adolescent Literacy Content & Pedagogy Together

42. The Observational Tool • Audience: school administrators, researchers, teachers, and/or peers • Person observed: a content-area teacher • Settings: secondary classroom in tier I instruction • The intended purpose: coaching, planning, and professional feedback • Length of the observation: not specified

43. The Tool Page 1

44. The Tool Page 2

45. Implementing the Tool • Before: • Review the protocol outlines and clarify operational definitions of each items before observation. • Clearly orient the observational goals, share lesson plan, and confirm the observation and meeting time with the teacher. • During: • Observer visits the classroom for a specified length of time (longer and more frequent are best). • Sit in a location to minimize student distraction. • Complete Observation Tool by recording Observed Data & Observer Comments • Give to teacher at end to complete Teacher Reflection prior to follow-up

46. Implementing the Tool • After: • Teacher completes reflection • Conduct a follow-up discussion between teacher and observer after observation • Share summary, feedback on positive comments, and areas of concern. • Based on reflections, co-construct action-oriented goals and future action plans jointly.

47. “We can have 30 years of research on instruction, but if it doesn’t get implemented by teachers, it’s not going to make a big difference.What we want to know is what’s the most efficient way to translate those ideas into practice in the classroom.”–Jim Knight, KUCRL

48. Coaching &Professional Development The Observation Tool measures the behavior, but how do we change the behavior? • Use Tool to identify key areas of professional development • Less top down and more practice driven • Use Partnership Learning approach to PD

49. Teacher Planning Questions - Content • Background Knowledge • Have I determined what relevant prior knowledge is necessary for understanding the academic task? • Have I analyzed the academic tasks for any vocabulary that may be unfamiliar to my students? • Text Structure • Have I selected readings that represent clear examples of a variety of text structures? • Cognitive Strategies • Have I determined what strategies my students need to help them accomplish the academic task? • Have I selected a small number of powerful strategies for students to learn to mastery levels?

50. Teacher Planning Questions - Content • Motivation • Am I prepared to answer the ‘So What?’ question? • What types of in-class support will I provide to promote student reading persistence? • Word Study • Does my lesson focus on common and/or difficult root words and affixes? • Have I chosen texts for fluency practice that support content area learning? • Writing • How can my instruction emphasize specific writing skills during reading tasks? • How can I use text models to instruct genre-specific composing?