slide1
Download
Skip this Video
Download Presentation
Research and Policy Support Group

Loading in 2 Seconds...

play fullscreen
1 / 23

Research and Policy Support Group - PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on

Evaluating the NYC Core Knowledge Language Arts Pilot: Summary of Kindergarten and Grade 1 Results HIGHLIGHTS. Research and Policy Support Group. Summary of Kindergarten Results 2008-2009 School Year. Research and Policy Support Group.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Research and Policy Support Group' - bianca-mullins


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Evaluating the NYC Core Knowledge Language Arts Pilot: Summary of Kindergarten and Grade 1 Results

HIGHLIGHTS

Research and Policy Support Group

slide2

Summary of Kindergarten Results

2008-2009 School Year

Research and Policy Support Group

methodology a multi method longitudinal research design
Methodology: A multi-method, longitudinal research design

Focus of the Evaluation

Hypothesis:Kindergarteners taught with the Core Knowledge Language Arts (CKLA) Program will gain reading competencies at a faster rate than their peers.

  • YEAR 1
  • Literacy Assessments (at 10 CKLA schools & 10 comparison schools, 4 of which are Reading First schools):
    • Pre- and post-test of literacy skills
    • Additional tests of literacy skills at end of each year
    • Periodic assessments throughout the year (DIBELS)
  • Teacher and Administrator Surveys (at 10 CKLA schools):
    • Assesses satisfaction with and impact of CKR
  • Case studies (at 3 CKLA schools):
    • Classroom observations, administrator & teacher interviews
similar demographics at ckla and comparison schools
Similar Demographics at CKLA and Comparison Schools

Percent of Students

(Number of Students for School Size)

CKLA Schools (N = 584)

Comparison Schools

(N = 307)*

* N = the number of students for whom both fall and spring data were available.

Note: These and other data were used to select comparison schools (data as of 2007-08 school year).

slide5
Different Literacy Domains: Greater Gains & Higher Spring Scores for CKLA Students than Comparison Students in All Literacy Domains

W-J Brief Reading

5

slide6

5X Greater Literacy Gains for CKLA Students than Students at Demographically Similar Comparison Schools

Average Fall-Spring Gain

in Scale Score Points

Woodcock-Johnson (Brief Reading Test)

CKLA Students

Significantly Higher

p < .001

CKLA Comparison

slide7

6X Greater Literacy Gains for CKLA Students than Students at Reading First Comparison Schools

Average Fall-Spring Gain

in Scale Score Points

Woodcock-Johnson (Brief Reading Test)

CKR Students

Significantly Higher

p < .001

(N = 314)*

(N = 103)*

* Data presented represent the four comparison schools that are Reading First schools and the four CKR schools who were statistically comparable matches.

7

slide8

At All Achievement Levels, Greater Literacy Gains for CKLA Students than Students at Comparison & Reading First Schools

Average Fall-Spring Gain

in Scale Score Points

Woodcock-Johnson (Brief Reading Test)

CKR & All Comparison Schools

CKR & Reading First Schools

CKR Schools

CKR Schools

Comparison Schools

Reading First Schools

8

summary of findings
Summary of Findings
    • By nearly all measures, Core Knowledge Language Arts (CKLA) students made significantly greater gains in early literacy than peer students.
      • Compared to peers, kindergarteners taught with the CKLA program made more progress in all areas of reading tested: spelling, phonemic awareness, decoding, and comprehension.
      • These findings hold for two types of peer comparisons: 1) demographically similar schools and 2) demographically similar schools using Reading First (another intensive, focused literacy program).
  • Surveys and case studies indicate overall high levels of administrator and teacher satisfaction with the CKLA Pilot, while also offering guidance for year 2 implementation and evaluation work.
      • Administrators would recommend program to others; teachers rate CKLA more favorably than other programs.
      • Administrators report change in teacher practice: more data-driven instruction & teacher collaboration.
      • Teachers and administrators feel more positively about the Skills Strand than the Listening and Learning Strand, particularly regarding student engagement.
      • Teachers may need more support addressing needs of struggling readers with CKLA & managing time to complete lessons.
slide10

Summary of Grade 1 Results

2009-2010 School Year

Research and Policy Support Group

methodology a multi method longitudinal research design1
Methodology: A multi-method, longitudinal research design

Focus of the Evaluation

Hypothesis:First Graders taught with the Core Knowledge Language Arts (CKLA) Program will gain reading competencies and content knowledge (science and social studies skills) at a faster rate than their peers.

  • YEAR 2
  • Assessments (at 10 CKLA schools & 10 comparison schools, 3* of which are Reading First schools):
    • Pre- and post-test of literacy skills
    • Additional tests of literacy skills at end of each year
    • Tests of science and social studies skills at end of each year
  • Teacher and Administrator Surveys (at 10 CKLA schools):
    • Assesses satisfaction with and impact of CKLA
  • Case studies (at 5 CKLA schools):
    • Classroom observations, administrator & teacher interviews

* Last year there were 4 Reading First comparison schools. This year there were 3 Reading First comparison schools: one comparison school used Reading First in Year 1 but not in Year 2.

slide12

Overall, CKLA and comparison schools had similar demographic profiles. However, the differences between groups, especially % of Black/ Hispanic students, was significantly larger than last year.

As a result, all analyses control for student demographics.

Percent of Students

(Number of Students for School Size)

CKLA Students (N = 781)

Comparison Students

(N = 343)*

  • N = the number of students for whom both fall and spring data were available.
  • A random sample of half of the students in the comparison schools were selected for testing. All students in CKR schools were tested.

Note: Data from 08-09 was used to select comparison schools (data presented here are from the 2009-10 school year).

slide13

CKLA students displayed greater gains & higher Spring scores in nearly all literacy and content knowledge domains relative to comparison students

Note: All analyses control for student demographic characteristics.

13

slide14
CKLA students displayed nearly 2x greater literacy gains relative to students at demographically similar comparison schoolsThe magnitude of the group difference was smaller than last year (in Year 1, CKLA students’ gains were 5X greater than their peers).

Average Fall-Spring Gain

in Scale Score Points

Woodcock-Johnson (Brief Reading Test)

CKLA Students

Significantly Higher

p < .0001

(N = 758)

(N = 342)

slide15

CKLA students displayed 6x greater literacy gains relative to students at Reading First comparison schools

The magnitude of the group difference was smaller than last year (in Year 1, CKLA students’ gained 14.3 pts while Reading First schools declined 4.1 pts).

Average Fall-Spring Gain

in Scale Score Points

Woodcock-Johnson (Brief Reading Test)

CKLA Students

Significantly Higher

p < .0001

(N = 233)*

(N = 100)*

* Data presented represent the three comparison schools that are Reading First schools and the three CKR schools who were statistically comparable matches.

15

slide16

Measuring fidelity of implementation to the CKLA curriculum

  • Examining implementation fidelity allows us to better determine:
    • 1) whether achievement gains can be attributable to the CKR program and
    • 2) which components of the CKR program teachers are struggling to implement and those they are implementing successfully
  • We took several different approaches to measuring fidelity in the CKR Pilot schools:
    • Site visits with classroom observations
      • Site visits were conducted at 5 randomly selected Pilot schools
      • 15 classrooms were observed
    • Interviews with administrators and teachers
      • Interviews were conducted with 13 teachers and 7 administrators
slide21

Fidelity of implementation and teacher effectiveness made a difference:

Three Site Visit Schools Did Not Significantly Gain or Declined in Performance from Fall to Spring:

  • One of these schools was the low fidelity school described previously.
  • One of these schools had a teacher new to teaching first grade who had been in and out of the classroom, and the relationship between the principal and this teacher was strained.
  • All schools spent less than the required 60 minutes on the Listening and Learning Strand and the Skills Strand.
  • One of these schools also had a very strong focus on writing, which would occasionally cut into their other strand time.

Significantly

Positive Gain

Non-significant

Gain

Significantly

Negative Gain

slide22

Executive Summary – Achievement Gains

  • By nearly all measures, Core Knowledge Language Arts (CKLA) students made significantly greater gains in Year 2 than their peers
    • Overall Achievement
      • When compared to peers (both peers at demographically similar schools and peers at Reading First schools), 1st graders taught with the CKLA program made more progress in reading and social studies.
        • Spring science scores were significantly greater for students in pilot schools than comparisons schools, but not significantly different from students in Reading First comparison schools.
    • High vs. Low Performers
      • High and low performing CK students made larger gains than comparison students
    • Year 2 vs. Year 1 of the Program.
      • Both students with only one year of CKLA instruction and those with two years of CKLA instruction made greater gains than their peers at comparison schools.
slide23

Executive Summary – Surveys & Case Studies

  • Surveys and case studies indicate overall high levels of administrator and teacher satisfaction with the CKLA Program and high levels of fidelity in program implementation.
  • While survey respondents had favorable reactions to CKR, they indicated that they felt CKLA rated similarly to other programs they had used.
  • Surveys and case studies revealed that teachers are struggling with how to differentiate instruction with CKR, in particular for special populations.
  • Case study schools’ fidelity to the program aligned with their achievement gains: schools with low fidelity demonstrated less gains
ad