Will that work for us interpreting research from the memphis striving readers project msrp
This presentation is the property of its rightful owner.
Sponsored Links
1 / 88

Will That Work for Us? Interpreting Research from The Memphis Striving Readers Project (MSRP) PowerPoint PPT Presentation


  • 77 Views
  • Uploaded on
  • Presentation posted in: General

Will That Work for Us? Interpreting Research from The Memphis Striving Readers Project (MSRP). Presented by Ric Potts, MCS; J. Helen Perkins, U of M; Elizabeth Heeren, MCS; Rorie Harris, MCS; and Jill Feldman, RBS 2008 International Reading Association Research Conference Atlanta, GA.

Download Presentation

Will That Work for Us? Interpreting Research from The Memphis Striving Readers Project (MSRP)

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Will that work for us interpreting research from the memphis striving readers project msrp

Will That Work for Us? Interpreting Research from The Memphis Striving Readers Project (MSRP)

Presented by

Ric Potts, MCS; J. Helen Perkins, U of M; Elizabeth Heeren, MCS; Rorie Harris, MCS; and Jill Feldman, RBS

2008 International Reading Association Research Conference

Atlanta, GA


Session overview

Session Overview

  • Introduction to the Striving Reader’s grant

  • Overview of Memphis SR research design

  • Year One Impact Analyses

  • Collection of implementation fidelity data

    • implications for practitioners and researchers

  • Planned (Ongoing) Analyses

  • Q & A /Group Discussion


Introduction memphis striving readers project msrp

Introduction: Memphis Striving Readers Project (MSRP)

Ric Potts, PI – MSRP

Memphis City Public Schools


Will that work for us interpreting research from the memphis striving readers project msrp

Memphis-The City

  • The City of Memphis has a population of 642,251.

  • 63.1% African American

  • 31.3% Caucasian

  • 4.1% Hispanic


And one elvis

And one Elvis


Will that work for us interpreting research from the memphis striving readers project msrp

Approximately 70 percent of adolescents

struggle to read. The young people enrolled in

middle and high school who lack the broad

literacy skills to comprehend and learn advanced

academic subjects will suffer serious social,

emotional, and economic consequences.

Reading at Risk: The State Response to the Crisis in Adolescent Literacy, Oct. 2005


Urban child institute the state of children in memphis and shelby county 2006

Urban Child InstituteThe State of Children in Memphis and Shelby County2006

“Under-educated children have no future.”


Urban child institute the state of children in memphis and shelby county 20061

Urban Child InstituteThe State of Children in Memphis and Shelby County2006

  • by U.S. standards roughly 75 percent of students in Tennessee fail to meet national grade appropriate standards, and Memphis is at the bottom in Tennessee. . . . Memphis is one of theleast-educated cities in America.


Motivation behind memphis striving readers project

Motivation behindMemphis Striving Readers Project

  • Memphis is among the poorest and least-educated cities in the US

    • 30.1% of all children live in poverty

    • 24.3% of adults have less than a HS education

    • 36.7% have HS diploma or equivalent

    • 30.5% have Assoc. or some college

    • 8.5% have at least a BA

  • MCS is 21st largest K12 district in US >116,000 students

    • Over 95% of MCS’ 196 schools are Title I schools

    • 71% of MCS students qualify for free/reduced price lunch

    • MCS students are 87% AA; 9% White; 4% “other”

    • In 85% of MCS schools, 33% of students change schools during year

    • In 2003-04, the system-wide graduation rate was 61 percent

    • 71% of students in grades 6-8 scored below the 50th percentile on TCAP (Reading/Language Arts)


Striving readers a federal response

Striving Readers – A Federal Response

  • In 2005, the Department of Education called for proposals for the Striving Readers grant.

  • In March, 2006, Memphis was one of eight sites awarded the grant.


Memphis striving reader program targeted schools

Memphis Striving Reader Program Targeted Schools


The whole school intervention memphis content literacy academy mcla

The Whole School Intervention: Memphis Content Literacy Academy (MCLA)

Overview presented by

J. Helen Perkins, SR Co-PI

University of Memphis


A change model

A Change Model


Will that work for us interpreting research from the memphis striving readers project msrp

A Capacity-Building Model for

Teacher Development

(Cooter & Cooter, 2003)

Expertise

& Ability

to Coach Others

Refined and

Expanded

Capacity

Practice with Coaching

Deeper Learning with

Limited Capacity

First Exposure

No Knowledge

Emphasis: “Deep Training”

(180 hours over two years) …


Will that work for us interpreting research from the memphis striving readers project msrp

Memphis Content Literacy AcademyInfusing Simultaneously Across Core Subject Areas Scientifically-based Reading Research (SBRR) Strategies in…

Vocabulary

Reading Comprehension

Reading Fluency


Benefits to teacher laureates

Benefits to Teacher – “Laureates”…

  • Advanced Training (180 hours) on scientifically-based reading instruction (SBRR) for urban children

  • A Master Teacher “Coach” to Assist (30 hours) with Implementing New Strategies (in their own classrooms!)

  • Twelve (12) Graduate Semester Hours of Credit from University of Memphis (FREE) (applicable to an advanced degree)

  • Can Seek “Highly Qualified” Endorsement in Reading

  • Books and Materials (FREE)

  • Successin Helping Children Achieve “AYP”

  • Principal Support


Mcla year 1 selected strategies

Fluency

Choral Reading

Paired reading

Guided, repeated,

oral reading (pairs)

MCLA Year 1: Selected Strategies


Will that work for us interpreting research from the memphis striving readers project msrp

  • Comprehension

  • Question Generation

  • Three- Level Retelling

    • Oral

    • Graphic Organizor

    • Written

  • Comprehension monitoring

  • Expository Text Patterns

  • Multiple Strategies


Will that work for us interpreting research from the memphis striving readers project msrp

  • Vocabulary Development

  • Pre-instruction of vocabulary

  • Repeated, multiple exposures

  • Semantic Maps


Classroom organizational tools strategies year 1

CREDE Standards

Whole class v. collaborative small group

Reading Next Elements

Classroom Organizational Tools & Strategies: Year 1

  • Use of leveled materials

  • (e.g., National Geographic)


Crede formatting of professional development training

CREDE Formatting of Professional Development Training

http://crede.berkeley.edu/standards/standards.html


Will that work for us interpreting research from the memphis striving readers project msrp

Classroom Action Plans (CAPs)Spring 2008Science, Social Studies, & ELAYour task is to develop a series of class lessons where you teach academic vocabulary in a unit of your choice.You must have at least one vocabulary learning strategy/activity that occurs:1. BEFORE students read the assigned text,2. DURING the reading assignment, and3. AFTER the reading assignment


Mcla classroom model

MCLA Classroom Model

  • Gradual release of responsibility

    (teacher modeling, guided practice, independent practice, independent use)

  • Integration of 12 literacy strategies

    (vocabulary, fluency & comprehension)

  • Development of Classroom Action Plans (CAPs)

    (content area lesson plans for strategy implementation including procedures for

    student assessment)

  • On-site support provided by coaches

  • Use of Curriculum Resource Center (CRC) materials


The principals fellowship

The Principals’ Fellowship

  • Literacy Leadership Practices

  • Real World Problem Solving

  • Create “Literacy Materials Centers”

  • Early Identification w/ Intense/Focused

    Remediation

  • Research-Informed Decision Making

  • Involve Families

  • Needs-Based Scheduling

  • Matching the Most Successful Teachers

    with “Critical Condition” Kids


Will that work for us interpreting research from the memphis striving readers project msrp

READ 180, Our Targeted Intervention

Overview provided by

Elizabeth Heeren, SR Grant Coordinator

Memphis City Schools


Program components

Program Components

Support materials for differentiated instruction in small group rotation

Tools for student placement and assessment

Student workbooks for Independent Practice in small and whole group rotations


Key elements of read 180

Key Elements of READ 180

  • Fidelity of Implementation

  • 90 minute classes

  • Certified teachers (LA or Reading)

  • District Instructional Support

  • District Technological Support

  • Scholastic training (site-based and on-line)


R180 correlations to reading next recommendations for adolescent literacy

R180 Correlations to Reading Next Recommendations for Adolescent Literacy

  • Direct, explicit comprehension instruction

  • Motivation and self-directed learning

  • Strategic tutoring

  • Differentiated texts (levels and topics)

  • Technology component

  • Ongoing formative assessment

  • Extended time for literacy

  • Professional development (long-term and on-going)


Memphis implementation

Memphis Implementation

  • We have 8 schools in the Striving Readers Grant, with up to 120 randomly selected R180 students at each school.

  • Students receive R180 instruction for 2 years.

  • Each student placed in R180 falls in the lowest quartile of TCAP (Reading score).

  • Each student in R180 is paired with a similar student from the lowest quartile who does not receive the treatment (for impact comparison).


Msrp research design

MSRP Research Design

Overview presented by

Jill Feldman, SR Research Director

Research for Better Schools


Overall msrp goals

Overall MSRP Goals

To determine:

  • The effects of MCLA on core subject teachers’ knowledge and use of SBRR

    2.The separate and combined effects of MCLA and Read 180 on students’ reading achievement levels, especially students who are identified as struggling readers

    3.The separate and combined effects of MCLA and Read 180 on students’ achievement in core subjects, especially students who are identified as struggling readers


Will that work for us interpreting research from the memphis striving readers project msrp

MCLA Program Logic Model

Outputs

Long-term Outcomes

Short–termOutcomes

Funding, staff, curriculum resource center, facilities, incentives, research materials

Principals

45 hours of Principal Fellowship participation

100% of principals incorporate plan for using MCLA strategies in SIP

100% attendance of key MCLA events

80% of principals report actively supporting the program

100% of MCLA schools have allocated space for the CRC

Teachers

90 of hours of MCLA training/yr for 2 years (180 hours)

Engage in weekly coaching sessions or as needed to meet teachers’ differentiated needs

8 CAP “cycles” completed each year for two years

100% of teachers complete performance measures identifying supplemental resources available/those necessary to support content area instruction

Students

50% of students attend 4 classes taught daily by teachers participating in MCLA

Students learn to use 7 of 8 MCLA CAP strategies

Principals

Awareness of and interest in staff implementation of MCLA concepts and strategies

Increased advocacy for school-wide use of MCLA strategies

Teachers

Increased knowledge about MCLA strategies

Improved preparedness to use research-based literacy strategies to teach core academic content

Increased use of direct, explicit instruction to teach research-based comprehension, fluency, and vocabulary strategies in content area classes

Integrated use of multiple MCLA strategies to support ongoing development of content-related instructional units

Students

Increased familiarity with and use of MCLA strategies when engaging with text

Increased internalization of

literacy strategies

Increased confidence engaging with content related texts

Increased interest in school/learning

Principals

Improved school climate

School-wide plans include focus on content literacy

Improved instructional leadership

Teachers

Increased effectiveness supporting students’ content literacy development

Continued collaboration among community of teachers to develop and implement CAPs

Students

Improved reading achievement and content literacy:

10% increase in students scoring proficient in Reading/LA and other subject areas of TCAP

mean increase of five NCEs on ITBS

Increased performance on gateway and EOC exams

Principals

Attend 45-hour sessions/yr (2 yrs)

Participate in motivational, recruitment and celebratory events

Discuss MCLA at faculty meetings

Conduct walkthrough observations

Provide opptys for teacher collab

Allocate space for CRC materials

Teachers

Attend 30 weekly 3-hour MCLA training sessions/yr (2 years)

Develop and implement 8 CAPs per year in collab content-area groups

Meet with coaches for feedback to improve impl of MCLA strategies

Learn to use of leveled texts to support SR content literacy needs

Students

Learn to use MCLA strategies to read/react to content related text (

Higher Quality Teaching

Higher Student Achievement


Study design and analytic approach mcla

Study Design MCLA:

Evaluate teacher and student outcomes

experimental design

randomly assigning schools

(to treatment and control conditions)

Teacher outcomes include

preparedness

frequency of literacy strategy use

Analytic Approach MCLA:

Two-level HLM

spring ITBS and TCAP scores as a function of teacher and school variables

Study Design and Analytic Approach: MCLA


Analytic decisions

Analytic Decisions

  • Missing Data

    • students missing pretest score(s) deleted from impact analysis on relevant measure(s)

    • teachers missing pretest score deleted from impact analysis on measure

  • Covariates

    • include all student- and school-level covariates in the model

    • run the model

    • eliminate the school covariate with the lowest significance level

      (highest p-value) not less than 0.2

    • repeat steps 2 and 3 until the remaining covariates had p-values less than 0.2

    • repeat steps 2-4 for the student covariates


Mcla random assignment of schools

MCLA: Random Assignment of Schools


Demographic characteristics of year 1 mcla student sample

Demographic Characteristics of Year 1 MCLA Student Sample


Will that work for us interpreting research from the memphis striving readers project msrp

Baseline Comparisons of Students in MCLA Treatment and Control Schools


Selected characteristics of the year 1 teacher sample for mcla impact analyses

Selected Characteristics of the Year 1 Teacher Sample for MCLA Impact Analyses


All variables included in mcla impact analytical models for year 1

All Variables Included in MCLA Impact Analytical Models for Year 1


Read 180 logic model

READ 180 Logic Model


R180 study design analytic approach

Study Design:

Evaluate student outcomes using RCT based on random assignment of students to conditions across schools

Student outcome measures:

reading achievement (ITBS)

core content areas (TCAP)

Analytic Approach:

Cross-sectional ITT analyses of reading and core content area achievement

Two-level models using spring ITBS and TCAP scores as a function of student and school variables

R180 Study Design Analytic Approach


Read 180 enrolled students

READ 180: Enrolled Students


Will that work for us interpreting research from the memphis striving readers project msrp

Variables Included in READ 180 Impact Analytic Models (Year One):Dependent and Independent


Will that work for us interpreting research from the memphis striving readers project msrp

Variables Included in READ 180 Impact Analytic Models (Year One): Covariates


Year one impact

Year One Impact


Will that work for us interpreting research from the memphis striving readers project msrp

Comparison of Teachers in MCLA Treatment and Control Schools on Year-End Indices for Preparedness and Frequency of Use


Will that work for us interpreting research from the memphis striving readers project msrp

MCLA Impacts on Students (Year One)


Will that work for us interpreting research from the memphis striving readers project msrp

READ 180 Impacts on Students (Year 1)


Collection of data about implementation fidelity

Collection of Data about Implementation Fidelity


Implications for researchers and practitioners

Implications for Researchers and Practitioners

What are our purposes for collecting implementation data?

  • To provide other districts with information about outcomes they might expect when implementing similar interventions with their struggling readers*

  • To set the context for understanding student outcomes

    *Requires MCS to place the needs of the field above local concerns


Reasons to collect double data

Reasons to Collect “Double Data”

R180 evaluation is intended to test effects of a

replicable intervention in the real-world:

  • Without the support of external evaluators

  • In ways that emulate what districts will need to do to:

    • monitor implementation

    • obtain process feedback


Reasons to collect double data1

Reasons to Collect “Double Data”

Collecting data about MCLA and R180 fidelity

  • helps researchers explain patterns of impact findings

  • can be useful in identifying predictors of outcomes


What is the role of the researcher

What Is the Role of the Researcher?

  • RBS collects data about:

    • Impact (MCLA & R180)

    • Implementation fidelity

      • To better understand impact or lack thereof

        (MCLA & R180)

      • To support development of MCLA (only)

    • Counterfactual

      • To compare effects to what would have happened in SR schools in the absence of MSRP


What is the role of mcs

What is the Role of MCS?

  • Implement R180 & MCLA

  • Monitor the implementation process

    • Ensure implementation is “on model”

    • Refine service delivery based on formative data


Defining implementation fidelity mcla

Defining Implementation Fidelity: MCLA

Innovation Configuration Mapping


Mcla implementation framework

MCLA Implementation Framework

  • Developing an Innovation Configuration (IC) Map

    (Hall & Hord, 2006)

    • Operationally defines levels of implementation fidelity among clusters of “key active ingredients”

    • Iterative process involving key stakeholders

      • Development team (University of Memphis)

      • Grantee (Memphis City Public Schools)

      • Researchers (Research for Better Schools)


Mcla roles responsibilities

MCLA: Roles & Responsibilities

MCS Administrators:

  • Participate in Principal’s Fellowship

  • Support recruitment and retention efforts

  • Link MCLA w/School Improvement Plan

  • Observe MCLA teachers

    (once/marking period)

  • Allocate space for CRC materials

  • Protect/respect role of coach

    Developer:

  • Design MCLA curricula

    (for teachers & principals)

  • Facilitate writing team activities

  • Meet weekly with instructors (& coaches)

  • Disseminate research about adolescent SR


Mcla training

MCLA Training

Provided by the Developer:

  • 3-hour weekly principal meetings

    (fall;Year 1)

  • 3-hour weekly teacher training sessions per content area

    (180 hours over 2 years)*

  • PD for coaches in

    Mentorship; Urban education; Adolescent lit

    Provided by MCS (coaches):

  • On-site observation of CAPs

  • Model/co-teach strategies

  • Feedback

  • Supplemental resources

    *has included coaches since spring 2007


Will that work for us interpreting research from the memphis striving readers project msrp

MCLA Innovation Configuration Map Framework


Instrument development

Instrument Development

With the IC map guiding development, the following

measures were designed to collect data about MCLA

implementation:

  • Surveys

    • Teacher knowledge about & preparedness to use MCLA strategies

    • Teacher demographic characteristics

    • Teachers’ MCLA Feedback

  • Interviews

    • Principals, coaches, development team, and MCS administrators

  • Teacher Focus Group Discussions


Operationally defining components job definition

Operationally defining components:“Job Definition”


Aligning the ic map and instrument development job definition teacher survey

Aligning the IC Map and Instrument Development: “Job Definition” – Teacher Survey


Job definition principal interviews

“Job Definition” - Principal Interviews


Will that work for us interpreting research from the memphis striving readers project msrp

MCLA Innovation Configuration Map Framework


Where the rubber hits the runway

Where the rubber hits the “runway”…

MCLA Classroom Implementation


Operationally defining components implementation of lesson plans

Operationally defining components: Implementation of Lesson Plans


Implementation of lesson plans collecting classroom observation data

Implementation of lesson plans:Collecting classroom observation data


Implementation of lesson plans collecting classroom observation data1

Implementation of lesson plans:Collecting classroom observation data


Mcla implementation barriers

MCLA: Implementation Barriers

Barriers:

  • Limited development/planning time

  • Need for coaches with disciplinary content knowledge

  • Challenges in establishing a critical mass of enrolled teachers at each school

  • CRC materials not received until spring 2007

  • Pressure to focus on TCAP test preparation (spring)

  • Difficulty maintaining principal attendance at weekly meetings


Mcla planned implementation changes

MCLA: Planned Implementation Changes

Changes:

  • Adoption of CREDE (UC-Berkeley) JPA instructional model

  • Reduction in the number of CAPs required of teachers

  • Shortened class schedule/more intensive work with coaches

  • Inclusion of special education teachers among those eligible to enroll

  • Restructured Principal Fellowship

    (includes other school leaders; meets monthly)


Defining implementation fidelity r180

Defining Implementation Fidelity: R180

Rorie Harris

Memphis City Public Schools


Findings related to implementation

Findings Related to Implementation

  • Scheduling

    • Scheduling 90 minute blocks in schools using the Middle School concept is difficult. Teams of core content teachers traditionally have 55 minute classes.

    • Interruptions to the 90 minute block occur.

  • Special Education Students

    • READ 180 will only suffice as a SPED student’s intervention if the teacher is SPED-certified.


Findings related to implementation1

Findings Related to Implementation

  • Use of Technology

    • Technology issues can negatively affect instructional time.

  • Parents & Students

    • Some parents do not want their children in Reading Intervention classes. They feel like this is a “label.”

    • Classroom management issues impact instruction.

    • Student mobility affects the scope and sequence of reading instruction.


Findings related to implementation2

Findings Related to Implementation

  • School Administration

    • Without administrator “buy-in” to the importance of smaller classes and protection of the 90 minute block, fidelity is not supported.

  • Read 180 Teachers

    • It is challenging to encourage ALL teachers to engage in on-line professional development and/or to attend network meetings.

    • Teacher turn-over brings out the need for repeated initial training and reduces the development of teacher leaders.


Indicators of read 180 implementation

Indicators of Read 180 Implementation

  • Scholastic identifies several key program aspects

    • Teacher Training/Professional Development

    • Computer Hardware/Software Use

    • Use of Read 180 Materials

    • Group Rotation

    • Class Size

    • Classroom Environment

    • Student Engagement


Sources of implementation data

Sources of Implementation Data

  • Classroom observations during the school year (Fall & Spring)

  • Read 180 program databases (SAM)

  • Enrollment and course-related data from district databases

  • Surveys administered to students (Fall & Spring) and teachers (Spring)

  • Information collected during professional development programs


Mcs data linked to implementation indicators

MCS Data Linked to Implementation Indicators


Mcs data linked to implementation indicators1

MCS Data Linked to Implementation Indicators


Overview of year one conclusions

Overview of Year One Conclusions

Jill Feldman, RBS


Brief conclusions discussion

(Brief) Conclusions & Discussion

READ 180: No significant Year One student impact

  • Late startup

  • (Most) students will receive two years of intervention

    Planned Future Analyses:

  • Three-level analyses planned to examine whether teacher characteristics exert a moderating effect on student outcomes

  • Exploratory analyses of relationships between amount of

    READ 180 instruction and effects on student outcomes


Brief conclusions discussion1

(Brief) Conclusions & Discussion

MCLA:

  • Significant (moderate) impact on teachers’ frequency and preparedness to use MCLA strategies

  • No significant impact on students’ achievement in reading or core content areas

    Discuss:

  • Subjectivity of measure (“Hawthorne Effect”)

  • Teacher findings support program logic model

  • Explore relationship between impact and participation in PD


Next steps

Next Steps…


Planned exploratory analyses

Planned Exploratory Analyses

  • Re-run HLM impact analyses to test effects of teacher variables on outcomes

    • Preparedness and use of MCLA strategies

    • Age

    • Experience as teacher (& years at MCS)

    • PD in year prior to MCLA


Planned ongoing analyses

Planned/ongoing analyses

  • Individual student’s growth over time

  • Rerun HLM with student-level variables

    • # MCLA teachers

    • Student’s school attendance

  • ITS analyses

    • Using TCAP Spring 2003 & 2004 scores

  • Correlating R180 data with TCAP & ITBS

    • for possible use as covariates in HLM


Now it s your turn

Now It’s Your Turn

  • Ask the panel

  • Share your experiences

    • Triumphs

    • Tribulations


Thank you for joining us

Thank you for joining us!

For additional information contact:

[email protected]


  • Login