Washington state laser highlights from recent evaluation studies
This presentation is the property of its rightful owner.
Sponsored Links
1 / 48

Washington State LASER Highlights From Recent Evaluation Studies PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on
  • Presentation posted in: General

Washington State LASER Highlights From Recent Evaluation Studies. Dave Weaver RMC Research Corporation 111 SW Columbia, Suite 1200 Portland, Oregon 97201. Agenda For This Segment. Impact on Student Achievement Studies Student-Level analysis of 2006 Science WASL results

Download Presentation

Washington State LASER Highlights From Recent Evaluation Studies

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Washington state laser highlights from recent evaluation studies

Washington State LASERHighlights From Recent Evaluation Studies

Dave WeaverRMC Research Corporation111 SW Columbia, Suite 1200Portland, Oregon 97201

RMC Research Corp.


Agenda for this segment

Agenda For This Segment

  • Impact on Student Achievement Studies

    • Student-Level analysis of 2006 Science WASL results

    • School-Level analysis of 2007 Science WASL results

    • West Valley School District

  • Implementation Study

    • Study of Highly Successful Schools

  • Evaluation activities for 2007-2008

RMC Research Corp.


Student level analysis of 2006 science wasl results

Student-Level Analysis of 2006 Science WASL Results

Spring & Summer 2007

RMC Research Corp.


Student level analysis of grade 5 wasl data for 2006

Student-Level Analysis of Grade 5 WASL Data for 2006

  • Factors considered

    • Student demographics (i.e., socioeconomics, race/ethnicity)

    • Grade 5 science WASL results for 2006

    • Grade 4 math and Reading WASL results for 2005

    • The number of LASER PD hours the student’s Grade 5 teacher received each year and prior to the administration of the 2006 WASL

    • The number of LASER PD hours the student’s Grade 4 teacher received each year and prior to the administration of the 2005 WASL.

RMC Research Corp.


2 years of teacher influence

2-Years of Teacher Influence

2006 Student-Level Science Data for Grade 5

PD Hours of Student’s Grade 5 Teacher in 2006

Linked By Teacher

Linked By Student

2005 Student-Level Math Data for Grade 4

PD Hours of Student’s Grade 4 Teacher in 2005

Linked By Teacher

RMC Research Corp.


Grade 5 regression analysis results

Grade 5 Regression Analysis Results

Dependent Variable: 2006 Grade 5 Science Scale Score

RMC Research Corp.


Factors influencing 2006 grade 5 science wasl results

Factors Influencing 2006 Grade 5 Science WASL Results

32,228 Students

860 Schools

1325 Classes

R2 = .645

RMC Research Corp.


Grade 8 regression analysis results

Grade 8 Regression Analysis Results

Dependent Variable: 2006 Grade 8 Science Scale Score

RMC Research Corp.


Factors influencing 2006 grade 8 science wasl results

Factors Influencing 2006 Grade 8 Science WASL Results

7,432 Students

R2 = .634

RMC Research Corp.


Finding 1

Finding 1

  • After controlling for demographics and entry skill levels of students, the results reveal that the current year of LASER professional development is a significant positive predictor of scores on the science WASL results.

  • If all things remain equal, 25 hours of professional development translates into a 1-point gain in the science scale score.

RMC Research Corp.


Finding 2

Finding 2

  • The professional development provided during the year that the teacher had the student in class had the greatest impact on student performance.

  • The accumulated effect of previous years of professional development diminished predictive value.

RMC Research Corp.


Finding 3

Finding 3

  • The current year of professional development of the 4th and 5th grade teachers is a significant positive predictor of student subscale scores for:

    • Inquiry in Science (.039)

    • Open-ended responses (.029)

    • Applications of Science (.020)

RMC Research Corp.


Finding 4

Finding 4

  • Factors beyond the control of LASER (entry skills, demographics, poverty) account for 90% of knowable variance.

  • Changes in any of these other factors can easily overshadow the positive effect of LASER professional development.

RMC Research Corp.


School level analysis of 2007 science wasl results

School-Level Analysis of 2007 Science WASL Results

Fall 2007

RMC Research Corp.


What we did

What We Did

  • Data Sources

    • School Level WASL Data from 2007

    • LASER Professional Development Database

    • School Demographic Data for 2006-07

  • Regression Analysis

    • Independent Variable:

      • Total PD hours of core teachers in a school for 2 years prior to 2007 WASL per FTE

    • Dependent Variable:

      • Percent of students in the school that met the Grade 5 science standard on the 2007 WASL

    • Control Variables:

      • Percent of students who qualify for free or reduced price lunch

      • Percent of students in the school that met the Grade 4 math standard on the 2006 WASL

      • Students per classroom teacher

RMC Research Corp.


Grade 5 regression analysis results1

Grade 5 Regression Analysis Results

R2 = .617

RMC Research Corp.


Findings

Findings

  • The number of hours of LASER professional development per FTE over a 2 year period prior to the WASL is a significant predictor of student achievement on the Grade 5 science WASL above and beyond what can be explained by the number of students who qualify for free or reduced price lunch, class size, or student skills.

  • This finding was not evident at Grade 8.

RMC Research Corp.


2 year pd hours per fte vs student achievement

2-Year PD Hours Per FTE vs. Student Achievement

RMC Research Corp.


Quartile grouping of schools

Quartile Grouping of Schools

RMC Research Corp.


Washington state laser highlights from recent evaluation studies

Total School 2-Year PD Hours Per FTE Quartile Ranking vs. Percent of Students Who Met The Grade 5 Science WASL in 2007

Note: Means adjusted for Percent of students on free or reduced price lunch, class size, and previous year mathematics achievement.

RMC Research Corp.


Distribution of study schools by laser alliance

Distribution of Study Schools By LASER Alliance

RMC Research Corp.


Analysis of west valley school district data

Analysis of West Valley School District Data

Fall 2005

RMC Research Corp.


West valley school district study

West Valley School District study

  • Peter Finch, Assistant Superintendent

  • Pilot test involving phased implementation of inquiry-based instructional materials

  • Noticed an important relationship between the number of modules students experienced and Grade 5 science WASL scores.

RMC Research Corp.


Science wasl wvsd grade 5 students meeting standard students receiving laser science instruction

Science WASL – WVSD Grade 5 % students meeting standardStudents receiving LASER Science instruction

State Average

RMC Research Corp.


Regression analysis of 2004 grade 5 science wasl results

Regression Analysis of 2004 Grade 5 Science WASL Results

RMC Research Corp.


Regression analysis of 2005 grade 5 science wasl results

Regression Analysis of 2005 Grade 5 Science WASL Results

RMC Research Corp.


Conclusion

Conclusion

The number of science modules that students experience has a significant positive impact on their science WASL achievement above and beyond what can be explained by demographic factors.(FRL, Special Education, & %White)

RMC Research Corp.


Study of highly successful schools

Study of Highly Successful Schools

Winter & Spring 2007

RMC Research Corp.


Purpose of study

Purpose of Study

  • To identify schools that have demonstrated a consistent increases in the percentage of students who met the science standard on the Washington State Assessment of Student Learning (WASL).

  • Interview the staff of those schools to determine to what they attribute their success.

RMC Research Corp.


Criteria for selecting of schools

Criteria for Selecting of Schools

  • Focus on Grade 5

    • Very few schools could demonstrate consistent improvement at Grade 8 and 10.

  • At least 10 Grade 5 students complete the science WASL in 2004, 2005, and 2006.

  • An increase of at least 6% or more students meeting the standard each year.

  • An increase of at least 17.5% or more students meet the science standard across the 2 time intervals.

  • Started with at least 20% of the students meeting the science standard in 2004.

RMC Research Corp.


Description of selected schools

Description of Selected Schools

  • 40 schools selected for interview

  • 18 (45%) were in districts served by a LASER Alliance.

  • 31 (78%) were in districts that participated in at least 1 LASER Strategic Planning Institute

  • 10 (25%) are in districts that participate in one of the state MSP projects

  • 4 (10%) are participating in the NCOSP

  • 1 school did not participate in the interviews

RMC Research Corp.


Interviews

Interviews

  • During December 2006 and January 2007.

  • Conducted By:

    • RMC Research staff

    • Graduate students from the science education department of Portland State University

  • Using structure interview protocol

    • Developed with input from LASER project leadership

    • Training provided

  • With person most knowledgeable about success

    • Initial contact with Principal and sometimes other school staff

RMC Research Corp.


Two tiered approach

Two Tiered Approach

  • Open ended initial question

    • Summarized increases in WASL scores.

    • What is your school doing to improve science teaching and learning? To what do you attribute this success?

  • Followed by questions about the contribution of specific aspects such as

    • Professional development

    • Instructional materials

    • Assessments

    • LASER services

RMC Research Corp.


Finding 1 instructional materials

Finding 1– Instructional Materials

  • All schools used inquiry-based instructional materials – Mostly FOSS

  • 49% (19) of the schools indicated that recent changes to the science instructional materials attributed to their success

  • Some customized or locally developed kits

  • Heavily supplemented

  • Non-LASER Alliance schools use a wider variety of supplementary materials and approaches.

RMC Research Corp.


Finding 2 wasl preparation

Finding 2 – WASL Preparation

  • Initially, 54% (21) of the schools indicated that they engage students in some form of preparation for the WASL.

  • Upon questioning, 95% (37) indicated that they engage students in some form of preparation for the WASL.

RMC Research Corp.


Finding 2 wasl preparation1

Finding 2 – WASL Preparation

  • Most use WASL release items

    • Prepare students before WASL administration

    • Develop WASL-like practice assessment during the year

  • Explicit emphasis on

    • WASL “format”

    • WASL vocabulary

RMC Research Corp.


Finding 3 professional development

Finding 3 – Professional Development

  • 82% (32) of the schools indicated that professional development contributed to improving student WASL scores

  • PD in Non-LASER Alliance schools dominated by kit and WASL training

  • PD in LASER Alliance schools is more varies

    • Journaling/Notebooks

    • Leadership

    • Science Content

    • Inquiry

    • Coaching, Mentoring, & Consultant

    • EALRs and GLEs

    • More Organized School-Based PD

RMC Research Corp.


Finding 4 science advocate

Finding 4 – Science Advocate

  • 77% (30) of schools interviewed identified 1 or 2 people who were advocates for science education.

  • Passionate about science – Made things happen

  • Roles included:

    • Science specialist (16)

    • Teacher leaders (9)

    • Strong teacher or team (6)

    • Community member (3)

    • Administrator (2)

RMC Research Corp.


Finding 5 ealrs gle

Finding 5 – EALRs & GLE

  • 51% (20) of the schools attributed the success of their students to increased emphasis on and alignment of curriculum to the EALR’s and GLEs.

    • 11 are Non-LASER Alliance schools

    • 9 are LASER Alliance schools

RMC Research Corp.


Finding 6 instructional time

Finding 6 – Instructional Time

  • 38% (15) of the schools attributed their success at least partially to increased science instructional time

    • 8 are Non-LASER Alliance schools

    • 7 are LASER Alliance schools

RMC Research Corp.


Recommendations from schools

Recommendations From Schools

  • Interviewees were asked:

    • What 2 recommendations would you give to the staff of another school working to improve science teaching and learning that would improve student performance on the science WASL?

  • Responses varied considerably

RMC Research Corp.


Top 6 recommendations

Top 6 Recommendations

  • Use released items & PCAs to develop skills needed to be successful on the WASL (18).

  • Provide adequate professional development (13) on materials and WASL.

  • Increase science instruction time (10).

  • Engage students in hands-on, inquiry-based science activities (9).

  • Provide teacher collaboration time (7).

  • Align with EALRs and GLEs (6).

RMC Research Corp.


Conclusions

Conclusions

  • LASER actively promotes most of the key findings from this study

    • Inquiry-based instructional materials

    • Preparing students for the science WASL

    • Professional development

    • Science advocacy (leadership)

    • Alignment of curriculum to EALR’s & GLEs

    • Increased science instructional time

RMC Research Corp.


Evaluation plans for 2007 08

Evaluation Plans for 2007-08

RMC Research Corp.


Sentinel site visits

Sentinel Site Visits

  • Identify of approximately 30 schools in 2007–08 and 30 in 2008–09

  • Schools With Significant LASER Participation

  • Two Groups of 15 schools each

    • Demonstrated significant positive gains

    • Demonstrated little, no, or negative gains

  • Each site visit: 1 ½ to 2 days

RMC Research Corp.


Site visit data collection

Site Visit Data Collection

  • Interviews with principals and/or science teacher leader

  • Interviews with teachers

  • Observations of at least 3 lessons where inquiry–based instructional materials are used

    • Based on STAR Protocol

  • Rubrics for rating level of implementation of inquiry-based science

  • Analyze results for significant relationships between level of implementation to student achievement

RMC Research Corp.


Assessment of student content knowledge

Assessment of Student Content Knowledge

  • Obtain permission to use Bellevue School District assessments for Grades 6 & 7.

  • Identify teacher participants

    • 25 first time teachers & 25 veteran teachers

  • Data collection for each student

    • Pre and post student scores

    • Student demographics

    • Special program participation

      • SPED, FRL, ELL, Migrant, etc.

  • Gift certificate compensation to teachers

RMC Research Corp.


Questions

Questions !

Dave Weaver

RMC Research Corporation111 SW Columbia, Suite 1200Portland, Oregon 97201

(800) 788-1887

[email protected]

RMC Research Corp.


  • Login