Will that work for us interpreting research from the memphis striving readers project msrp
Download
1 / 88

Will That Work for Us? Interpreting Research from The Memphis Striving Readers Project (MSRP) - PowerPoint PPT Presentation


  • 119 Views
  • Uploaded on

Will That Work for Us? Interpreting Research from The Memphis Striving Readers Project (MSRP). Presented by Ric Potts, MCS; J. Helen Perkins, U of M; Elizabeth Heeren, MCS; Rorie Harris, MCS; and Jill Feldman, RBS 2008 International Reading Association Research Conference Atlanta, GA.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Will That Work for Us? Interpreting Research from The Memphis Striving Readers Project (MSRP)' - roth-gilmore


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Will that work for us interpreting research from the memphis striving readers project msrp

Will That Work for Us? Interpreting Research from The Memphis Striving Readers Project (MSRP)

Presented by

Ric Potts, MCS; J. Helen Perkins, U of M; Elizabeth Heeren, MCS; Rorie Harris, MCS; and Jill Feldman, RBS

2008 International Reading Association Research Conference

Atlanta, GA


Session overview
Session Overview

  • Introduction to the Striving Reader’s grant

  • Overview of Memphis SR research design

  • Year One Impact Analyses

  • Collection of implementation fidelity data

    • implications for practitioners and researchers

  • Planned (Ongoing) Analyses

  • Q & A /Group Discussion


Introduction memphis striving readers project msrp

Introduction: Memphis Striving Readers Project (MSRP)

Ric Potts, PI – MSRP

Memphis City Public Schools


Memphis-The City

  • The City of Memphis has a population of 642,251.

  • 63.1% African American

  • 31.3% Caucasian

  • 4.1% Hispanic



Approximately 70 percent of adolescents

struggle to read. The young people enrolled in

middle and high school who lack the broad

literacy skills to comprehend and learn advanced

academic subjects will suffer serious social,

emotional, and economic consequences.

Reading at Risk: The State Response to the Crisis in Adolescent Literacy, Oct. 2005


Urban child institute the state of children in memphis and shelby county 2006
Urban Child InstituteThe State of Children in Memphis and Shelby County2006

“Under-educated children have no future.”


Urban child institute the state of children in memphis and shelby county 20061
Urban Child InstituteThe State of Children in Memphis and Shelby County2006

  • by U.S. standards roughly 75 percent of students in Tennessee fail to meet national grade appropriate standards, and Memphis is at the bottom in Tennessee. . . . Memphis is one of theleast-educated cities in America.


Motivation behind memphis striving readers project
Motivation behindMemphis Striving Readers Project

  • Memphis is among the poorest and least-educated cities in the US

    • 30.1% of all children live in poverty

    • 24.3% of adults have less than a HS education

    • 36.7% have HS diploma or equivalent

    • 30.5% have Assoc. or some college

    • 8.5% have at least a BA

  • MCS is 21st largest K12 district in US >116,000 students

    • Over 95% of MCS’ 196 schools are Title I schools

    • 71% of MCS students qualify for free/reduced price lunch

    • MCS students are 87% AA; 9% White; 4% “other”

    • In 85% of MCS schools, 33% of students change schools during year

    • In 2003-04, the system-wide graduation rate was 61 percent

    • 71% of students in grades 6-8 scored below the 50th percentile on TCAP (Reading/Language Arts)


Striving readers a federal response
Striving Readers – A Federal Response

  • In 2005, the Department of Education called for proposals for the Striving Readers grant.

  • In March, 2006, Memphis was one of eight sites awarded the grant.



The whole school intervention memphis content literacy academy mcla

The Whole School Intervention: Memphis Content Literacy Academy (MCLA)

Overview presented by

J. Helen Perkins, SR Co-PI

University of Memphis


A change model
A Change Model Academy (MCLA)


A Capacity-Building Model for Academy (MCLA)

Teacher Development

(Cooter & Cooter, 2003)

Expertise

& Ability

to Coach Others

Refined and

Expanded

Capacity

Practice with Coaching

Deeper Learning with

Limited Capacity

First Exposure

No Knowledge

Emphasis: “Deep Training”

(180 hours over two years) …


Memphis Content Literacy Academy Academy (MCLA)Infusing Simultaneously Across Core Subject Areas Scientifically-based Reading Research (SBRR) Strategies in…

Vocabulary

Reading Comprehension

Reading Fluency


Benefits to teacher laureates
Benefits to Teacher – “Laureates” Academy (MCLA)…

  • Advanced Training (180 hours) on scientifically-based reading instruction (SBRR) for urban children

  • A Master Teacher “Coach” to Assist (30 hours) with Implementing New Strategies (in their own classrooms!)

  • Twelve (12) Graduate Semester Hours of Credit from University of Memphis (FREE) (applicable to an advanced degree)

  • Can Seek “Highly Qualified” Endorsement in Reading

  • Books and Materials (FREE)

  • Successin Helping Children Achieve “AYP”

  • Principal Support


Mcla year 1 selected strategies

Fluency Academy (MCLA)

Choral Reading

Paired reading

Guided, repeated,

oral reading (pairs)

MCLA Year 1: Selected Strategies


  • Comprehension Academy (MCLA)

  • Question Generation

  • Three- Level Retelling

    • Oral

    • Graphic Organizor

    • Written

  • Comprehension monitoring

  • Expository Text Patterns

  • Multiple Strategies


  • Vocabulary Development Academy (MCLA)

  • Pre-instruction of vocabulary

  • Repeated, multiple exposures

  • Semantic Maps


Classroom organizational tools strategies year 1

CREDE Standards Academy (MCLA)

Whole class v. collaborative small group

Reading Next Elements

Classroom Organizational Tools & Strategies: Year 1

  • Use of leveled materials

  • (e.g., National Geographic)


Crede formatting of professional development training
CREDE Formatting of Academy (MCLA)Professional Development Training

http://crede.berkeley.edu/standards/standards.html


Classroom Action Plans (CAPs) Academy (MCLA)Spring 2008Science, Social Studies, & ELAYour task is to develop a series of class lessons where you teach academic vocabulary in a unit of your choice.You must have at least one vocabulary learning strategy/activity that occurs:1. BEFORE students read the assigned text,2. DURING the reading assignment, and3. AFTER the reading assignment


Mcla classroom model
MCLA Classroom Model Academy (MCLA)

  • Gradual release of responsibility

    (teacher modeling, guided practice, independent practice, independent use)

  • Integration of 12 literacy strategies

    (vocabulary, fluency & comprehension)

  • Development of Classroom Action Plans (CAPs)

    (content area lesson plans for strategy implementation including procedures for

    student assessment)

  • On-site support provided by coaches

  • Use of Curriculum Resource Center (CRC) materials


The principals fellowship
The Principals’ Fellowship Academy (MCLA)

  • Literacy Leadership Practices

  • Real World Problem Solving

  • Create “Literacy Materials Centers”

  • Early Identification w/ Intense/Focused

    Remediation

  • Research-Informed Decision Making

  • Involve Families

  • Needs-Based Scheduling

  • Matching the Most Successful Teachers

    with “Critical Condition” Kids


READ 180, Our Targeted Intervention Academy (MCLA)

Overview provided by

Elizabeth Heeren, SR Grant Coordinator

Memphis City Schools


Program components
Program Components Academy (MCLA)

Support materials for differentiated instruction in small group rotation

Tools for student placement and assessment

Student workbooks for Independent Practice in small and whole group rotations


Key elements of read 180
Key Elements of READ 180 Academy (MCLA)

  • Fidelity of Implementation

  • 90 minute classes

  • Certified teachers (LA or Reading)

  • District Instructional Support

  • District Technological Support

  • Scholastic training (site-based and on-line)


R180 correlations to reading next recommendations for adolescent literacy
R180 Correlations to Academy (MCLA)Reading Next Recommendations for Adolescent Literacy

  • Direct, explicit comprehension instruction

  • Motivation and self-directed learning

  • Strategic tutoring

  • Differentiated texts (levels and topics)

  • Technology component

  • Ongoing formative assessment

  • Extended time for literacy

  • Professional development (long-term and on-going)


Memphis implementation
Memphis Implementation Academy (MCLA)

  • We have 8 schools in the Striving Readers Grant, with up to 120 randomly selected R180 students at each school.

  • Students receive R180 instruction for 2 years.

  • Each student placed in R180 falls in the lowest quartile of TCAP (Reading score).

  • Each student in R180 is paired with a similar student from the lowest quartile who does not receive the treatment (for impact comparison).


Msrp research design

MSRP Research Design Academy (MCLA)

Overview presented by

Jill Feldman, SR Research Director

Research for Better Schools


Overall msrp goals
Overall MSRP Goals Academy (MCLA)

To determine:

  • The effects of MCLA on core subject teachers’ knowledge and use of SBRR

    2. The separate and combined effects of MCLA and Read 180 on students’ reading achievement levels, especially students who are identified as struggling readers

    3. The separate and combined effects of MCLA and Read 180 on students’ achievement in core subjects, especially students who are identified as struggling readers


MCLA Program Logic Model Academy (MCLA)

Outputs

Long-term Outcomes

Short–termOutcomes

Funding, staff, curriculum resource center, facilities, incentives, research materials

Principals

45 hours of Principal Fellowship participation

100% of principals incorporate plan for using MCLA strategies in SIP

100% attendance of key MCLA events

80% of principals report actively supporting the program

100% of MCLA schools have allocated space for the CRC

Teachers

90 of hours of MCLA training/yr for 2 years (180 hours)

Engage in weekly coaching sessions or as needed to meet teachers’ differentiated needs

8 CAP “cycles” completed each year for two years

100% of teachers complete performance measures identifying supplemental resources available/those necessary to support content area instruction

Students

50% of students attend 4 classes taught daily by teachers participating in MCLA

Students learn to use 7 of 8 MCLA CAP strategies

Principals

Awareness of and interest in staff implementation of MCLA concepts and strategies

Increased advocacy for school-wide use of MCLA strategies

Teachers

Increased knowledge about MCLA strategies

Improved preparedness to use research-based literacy strategies to teach core academic content

Increased use of direct, explicit instruction to teach research-based comprehension, fluency, and vocabulary strategies in content area classes

Integrated use of multiple MCLA strategies to support ongoing development of content-related instructional units

Students

Increased familiarity with and use of MCLA strategies when engaging with text

Increased internalization of

literacy strategies

Increased confidence engaging with content related texts

Increased interest in school/learning

Principals

Improved school climate

School-wide plans include focus on content literacy

Improved instructional leadership

Teachers

Increased effectiveness supporting students’ content literacy development

Continued collaboration among community of teachers to develop and implement CAPs

Students

Improved reading achievement and content literacy:

10% increase in students scoring proficient in Reading/LA and other subject areas of TCAP

mean increase of five NCEs on ITBS

Increased performance on gateway and EOC exams

Principals

Attend 45-hour sessions/yr (2 yrs)

Participate in motivational, recruitment and celebratory events

Discuss MCLA at faculty meetings

Conduct walkthrough observations

Provide opptys for teacher collab

Allocate space for CRC materials

Teachers

Attend 30 weekly 3-hour MCLA training sessions/yr (2 years)

Develop and implement 8 CAPs per year in collab content-area groups

Meet with coaches for feedback to improve impl of MCLA strategies

Learn to use of leveled texts to support SR content literacy needs

Students

Learn to use MCLA strategies to read/react to content related text (

Higher Quality Teaching

Higher Student Achievement


Study design and analytic approach mcla

Study Design MCLA: Academy (MCLA)

Evaluate teacher and student outcomes

experimental design

randomly assigning schools

(to treatment and control conditions)

Teacher outcomes include

preparedness

frequency of literacy strategy use

Analytic Approach MCLA:

Two-level HLM

spring ITBS and TCAP scores as a function of teacher and school variables

Study Design and Analytic Approach: MCLA


Analytic decisions
Analytic Decisions Academy (MCLA)

  • Missing Data

    • students missing pretest score(s) deleted from impact analysis on relevant measure(s)

    • teachers missing pretest score deleted from impact analysis on measure

  • Covariates

    • include all student- and school-level covariates in the model

    • run the model

    • eliminate the school covariate with the lowest significance level

      (highest p-value) not less than 0.2

    • repeat steps 2 and 3 until the remaining covariates had p-values less than 0.2

    • repeat steps 2-4 for the student covariates







Read 180 logic model
READ 180 Year 1 Logic Model


R180 study design analytic approach

Study Design: Year 1

Evaluate student outcomes using RCT based on random assignment of students to conditions across schools

Student outcome measures:

reading achievement (ITBS)

core content areas (TCAP)

Analytic Approach:

Cross-sectional ITT analyses of reading and core content area achievement

Two-level models using spring ITBS and TCAP scores as a function of student and school variables

R180 Study Design Analytic Approach






Comparison of Teachers in MCLA Treatment and Control Schools on Year-End Indices for Preparedness and Frequency of Use


MCLA Impacts on Students (Year One) on Year-End Indices for Preparedness and Frequency of Use


READ 180 Impacts on Students (Year 1) on Year-End Indices for Preparedness and Frequency of Use


Collection of data about implementation fidelity
Collection of Data about Implementation Fidelity on Year-End Indices for Preparedness and Frequency of Use


Implications for researchers and practitioners
Implications for Researchers and Practitioners on Year-End Indices for Preparedness and Frequency of Use

What are our purposes for collecting implementation data?

  • To provide other districts with information about outcomes they might expect when implementing similar interventions with their struggling readers*

  • To set the context for understanding student outcomes

    *Requires MCS to place the needs of the field above local concerns


Reasons to collect double data
Reasons to Collect “Double Data” on Year-End Indices for Preparedness and Frequency of Use

R180 evaluation is intended to test effects of a

replicable intervention in the real-world:

  • Without the support of external evaluators

  • In ways that emulate what districts will need to do to:

    • monitor implementation

    • obtain process feedback


Reasons to collect double data1
Reasons to Collect “Double Data” on Year-End Indices for Preparedness and Frequency of Use

Collecting data about MCLA and R180 fidelity

  • helps researchers explain patterns of impact findings

  • can be useful in identifying predictors of outcomes


What is the role of the researcher
What Is the Role of the Researcher? on Year-End Indices for Preparedness and Frequency of Use

  • RBS collects data about:

    • Impact (MCLA & R180)

    • Implementation fidelity

      • To better understand impact or lack thereof

        (MCLA & R180)

      • To support development of MCLA (only)

    • Counterfactual

      • To compare effects to what would have happened in SR schools in the absence of MSRP


What is the role of mcs
What is the Role of MCS? on Year-End Indices for Preparedness and Frequency of Use

  • Implement R180 & MCLA

  • Monitor the implementation process

    • Ensure implementation is “on model”

    • Refine service delivery based on formative data


Defining implementation fidelity mcla

Defining Implementation Fidelity: MCLA on Year-End Indices for Preparedness and Frequency of Use

Innovation Configuration Mapping


Mcla implementation framework
MCLA Implementation Framework on Year-End Indices for Preparedness and Frequency of Use

  • Developing an Innovation Configuration (IC) Map

    (Hall & Hord, 2006)

    • Operationally defines levels of implementation fidelity among clusters of “key active ingredients”

    • Iterative process involving key stakeholders

      • Development team (University of Memphis)

      • Grantee (Memphis City Public Schools)

      • Researchers (Research for Better Schools)


Mcla roles responsibilities
MCLA: on Year-End Indices for Preparedness and Frequency of UseRoles & Responsibilities

MCS Administrators:

  • Participate in Principal’s Fellowship

  • Support recruitment and retention efforts

  • Link MCLA w/School Improvement Plan

  • Observe MCLA teachers

    (once/marking period)

  • Allocate space for CRC materials

  • Protect/respect role of coach

    Developer:

  • Design MCLA curricula

    (for teachers & principals)

  • Facilitate writing team activities

  • Meet weekly with instructors (& coaches)

  • Disseminate research about adolescent SR


Mcla training
MCLA Training on Year-End Indices for Preparedness and Frequency of Use

Provided by the Developer:

  • 3-hour weekly principal meetings

    (fall;Year 1)

  • 3-hour weekly teacher training sessions per content area

    (180 hours over 2 years)*

  • PD for coaches in

    Mentorship; Urban education; Adolescent lit

    Provided by MCS (coaches):

  • On-site observation of CAPs

  • Model/co-teach strategies

  • Feedback

  • Supplemental resources

    *has included coaches since spring 2007


MCLA Innovation Configuration Map Framework on Year-End Indices for Preparedness and Frequency of Use


Instrument development
Instrument Development on Year-End Indices for Preparedness and Frequency of Use

With the IC map guiding development, the following

measures were designed to collect data about MCLA

implementation:

  • Surveys

    • Teacher knowledge about & preparedness to use MCLA strategies

    • Teacher demographic characteristics

    • Teachers’ MCLA Feedback

  • Interviews

    • Principals, coaches, development team, and MCS administrators

  • Teacher Focus Group Discussions


Operationally defining components job definition
Operationally defining components: on Year-End Indices for Preparedness and Frequency of Use“Job Definition”


Aligning the ic map and instrument development job definition teacher survey
Aligning the IC Map and Instrument Development: on Year-End Indices for Preparedness and Frequency of Use“Job Definition” – Teacher Survey


Job definition principal interviews
“Job Definition” - Principal Interviews on Year-End Indices for Preparedness and Frequency of Use


MCLA Innovation Configuration Map Framework on Year-End Indices for Preparedness and Frequency of Use


Where the rubber hits the runway

Where the rubber hits the “runway”… on Year-End Indices for Preparedness and Frequency of Use

MCLA Classroom Implementation



Implementation of lesson plans collecting classroom observation data
Implementation of lesson plans: PlansCollecting classroom observation data


Implementation of lesson plans collecting classroom observation data1
Implementation of lesson plans: PlansCollecting classroom observation data


Mcla implementation barriers
MCLA: Implementation Barriers Plans

Barriers:

  • Limited development/planning time

  • Need for coaches with disciplinary content knowledge

  • Challenges in establishing a critical mass of enrolled teachers at each school

  • CRC materials not received until spring 2007

  • Pressure to focus on TCAP test preparation (spring)

  • Difficulty maintaining principal attendance at weekly meetings


Mcla planned implementation changes
MCLA: Planned Implementation Changes Plans

Changes:

  • Adoption of CREDE (UC-Berkeley) JPA instructional model

  • Reduction in the number of CAPs required of teachers

  • Shortened class schedule/more intensive work with coaches

  • Inclusion of special education teachers among those eligible to enroll

  • Restructured Principal Fellowship

    (includes other school leaders; meets monthly)


Defining implementation fidelity r180

Defining Implementation Fidelity: R180 Plans

Rorie Harris

Memphis City Public Schools


Findings related to implementation
Findings Related to Implementation Plans

  • Scheduling

    • Scheduling 90 minute blocks in schools using the Middle School concept is difficult. Teams of core content teachers traditionally have 55 minute classes.

    • Interruptions to the 90 minute block occur.

  • Special Education Students

    • READ 180 will only suffice as a SPED student’s intervention if the teacher is SPED-certified.


Findings related to implementation1
Findings Related to Implementation Plans

  • Use of Technology

    • Technology issues can negatively affect instructional time.

  • Parents & Students

    • Some parents do not want their children in Reading Intervention classes. They feel like this is a “label.”

    • Classroom management issues impact instruction.

    • Student mobility affects the scope and sequence of reading instruction.


Findings related to implementation2
Findings Related to Implementation Plans

  • School Administration

    • Without administrator “buy-in” to the importance of smaller classes and protection of the 90 minute block, fidelity is not supported.

  • Read 180 Teachers

    • It is challenging to encourage ALL teachers to engage in on-line professional development and/or to attend network meetings.

    • Teacher turn-over brings out the need for repeated initial training and reduces the development of teacher leaders.


Indicators of read 180 implementation
Indicators of Read 180 Implementation Plans

  • Scholastic identifies several key program aspects

    • Teacher Training/Professional Development

    • Computer Hardware/Software Use

    • Use of Read 180 Materials

    • Group Rotation

    • Class Size

    • Classroom Environment

    • Student Engagement


Sources of implementation data
Sources of Implementation Data Plans

  • Classroom observations during the school year (Fall & Spring)

  • Read 180 program databases (SAM)

  • Enrollment and course-related data from district databases

  • Surveys administered to students (Fall & Spring) and teachers (Spring)

  • Information collected during professional development programs




Overview of year one conclusions

Overview of Year One Conclusions Plans

Jill Feldman, RBS


Brief conclusions discussion
(Brief) Conclusions & Discussion Plans

READ 180: No significant Year One student impact

  • Late startup

  • (Most) students will receive two years of intervention

    Planned Future Analyses:

  • Three-level analyses planned to examine whether teacher characteristics exert a moderating effect on student outcomes

  • Exploratory analyses of relationships between amount of

    READ 180 instruction and effects on student outcomes


Brief conclusions discussion1
(Brief) Conclusions & Discussion Plans

MCLA:

  • Significant (moderate) impact on teachers’ frequency and preparedness to use MCLA strategies

  • No significant impact on students’ achievement in reading or core content areas

    Discuss:

  • Subjectivity of measure (“Hawthorne Effect”)

  • Teacher findings support program logic model

  • Explore relationship between impact and participation in PD



Planned exploratory analyses
Planned Exploratory Analyses Plans

  • Re-run HLM impact analyses to test effects of teacher variables on outcomes

    • Preparedness and use of MCLA strategies

    • Age

    • Experience as teacher (& years at MCS)

    • PD in year prior to MCLA


Planned ongoing analyses
Planned/ongoing analyses Plans

  • Individual student’s growth over time

  • Rerun HLM with student-level variables

    • # MCLA teachers

    • Student’s school attendance

  • ITS analyses

    • Using TCAP Spring 2003 & 2004 scores

  • Correlating R180 data with TCAP & ITBS

    • for possible use as covariates in HLM


Now it s your turn
Now It’s Your Turn Plans

  • Ask the panel

  • Share your experiences

    • Triumphs

    • Tribulations


Thank you for joining us

Thank you for joining us! Plans

For additional information contact:

[email protected]


ad