Comparing the Effects of Two Versions of Professional Development
This presentation is the property of its rightful owner.
Sponsored Links
1 / 99

Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on
  • Presentation posted in: General

Comparing the Effects of Two Versions of Professional Development on Science Curriculum Implementation and Scaling-Up. Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting Montreal, Canada. Introduction and Overview.

Download Presentation

Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Session 43 030 april 13 2005 american educational research association annual meeting

Comparing the Effects of Two Versions of Professional Development on Science Curriculum Implementation and Scaling-Up

Session 43.030

April 13, 2005

American Educational Research Association Annual Meeting

Montreal, Canada


Introduction and overview

Introduction and Overview

Paul R. BrandonCurriculum Research & Development GroupUniversity of Hawai‘i at Mānoa


Project purpose

Project Purpose

  • A randomized study of the effects of variations in professional development (PD) on

    • program implementation

    • student achievement

    • scale-up

  • First phase (ending 2/28/06) to prepare for five-year second phase (if funded)


Session 43 030 april 13 2005 american educational research association annual meeting

FAST

  • The project compares two versions of teachers’ PD for Foundational Approaches in Science Teaching (FAST).

    • An award-winning middle-school inquiry-based science program

    • Shown to positively affect achievement and other student outcomes


Phase i project tasks

Phase-I Project Tasks

  • Prepare the 5-day PD institute with electronic resource and on-line course

  • Prepare instruments for the second phase:

    • Teacher questionnaire

    • Teacher log

    • Observation procedures

    • Student assessments


Focus on span of pd

Focus on Span of PD

  • Traditionally, FAST PD is delivered in a 10-day institute.

  • We will compare it with an alternative 5-day institute followed by an online course, with an electronic multimedia resource.


Phase ii hypothesis

Phase-II Hypothesis

The 5-day inquiry-based science training institute, followed by an on-line university-credit course with computer-based multimedia, will have more favorable outcomes than a 10-day institute without the follow-up university course.


Examining the effects of pd on

Examining the effects of PD on:

  • levels of classroom implementation

  • intensiveness and extensiveness of long-term use of inquiry-based science (i.e., scale-up)

  • levels of student achievement


Experimental design

Experimental Design

  • Randomized cluster sample, with schools (N = 80) as clusters

  • One teacher per school

  • Even if attrition is 25%, statistical power will be .80.


Merit of the study

Merit of the Study

  • Addresses at least five deficits in the literature:

    • Few studies on effects of PD time span

    • Few studies about using technology in PD

    • Few randomized studies

    • Few studies of the effects of PD on student learning

    • Few, if any studies, of inquiry science PD


Order of presentations

Order of Presentations

  • Gray, Nguyen, & Speitel: Description of PD

  • Brandon: Log and questionnaire development and questionnaire validation

  • Taum: Observation guide development

  • Ayala: Student assessment development

  • Lawton: Comparison of early effects of the two versions of PD


Introduction and overview1

Introduction and Overview

Paul R. BrandonCurriculum Research & Development GroupUniversity of Hawai‘i at Mānoa


Developing and implementing an alternative version of fast professional development

Developing and Implementing an Alternative Version of FAST Professional Development

Mary E. Gray

ThanhTruc T. Nguyen

Thomas W. Speitel

University of Hawai‛i at Mānoa


Session 43 030 april 13 2005 american educational research association annual meeting

FAST

  • Foundational Approaches in Science Teaching

  • Inquiry-based, middle school science

  • Physical, biological, and earth sciences

  • Exemplary program (USDOE, 2001)


Fast professional development

FAST Professional Development

  • All FAST teachers must participate in PD

    • Certification required to purchase science curricula

  • Content

  • Inquiry

  • Standards

  • Assessment

  • Classroom organization and management

  • Safety


The challenge

Year Round Schedules

The Challenge

To develop and implement an alternative version of FAST I professional development

Teacher Time

Resources

Standards

Licensure


Considerations

Considerations

  • Current needs of science teachers

  • Future possibilities (emerging technologies)

  • Unique challenges (the dynamic nature of inquiry)


Review of literature

Review of Literature


Review of literature professional development

Review of Literature Professional Development

  • Strong subject area content (Kennedy, 1999)

  • Focus on higher-order teaching strategies (Porter, Garet, Desimone, Yoon, & Birman, 2000)

    • Collaborative, Same subject-grade-school, Active learning, Consistent with teacher goals


Review of literature professional development1

Review of Literature Professional Development

  • Quantity linked to improvements (Radford, 1998; Supovitz, Mayer, & Kahle, 2000; Fishman, Marx, Best, & Revital, 2003)

    • science content knowledge, process skills, and attitudes

  • Quantity linked to standards based teaching (Supovitz & Turner,2000)

  • Problems with researching inquiry-based teaching and student learning (Fishman et al., 2003)


Review of literature on line learning

Review of Literature On-line Learning

  • 3 million adult learners on-line (Waitts & Lewis, 2003)

  • Convenience and increased flexibility (Hülsmann,1999)

  • Evidence of implementation of various pedagogical approaches(Schlager & Schank, 1997)


Review of literature on line learning1

Review of Literature On-line Learning

  • Quantifiable learning outcomes were not significantly linked to technology adoption (Jones and Paolucci,1998).

  • Focus on motivation, skills and knowledge, self-directed learning, interactive competence, and technology skills(Kabilan, 2004)

  • Increased efficacy and self-perception in teachers(Huai, Braden, White, & Elliot, 2003)


Fast pro

FASTPro


How are the traditional pd and fastpro different

Two-weeks in duration, face-to-face

One-week in duration, face-to-face

How are the traditional PD and FASTPro different?

  • Conduct 43 investigations

  • Conduct 19 investigations

  • Extensive modeling and practicing time

  • Electronic resource

  • WebCT-based learning community


Inquiry

Inquiry

Teaching and learning science through inquiry is a new experience for many teachers and requires a significant change in attitude and behavior.


Fast pro addressing inquiry

FASTPro – Addressing Inquiry

  • Strategies (Loucks-Horsley, Hewson, Love, & Stiles,1998)

    • Examples

      • Immersion in inquiry into science

      • Coaching and mentoring

      • Technology for professional learning


Fast pro addressing inquiry1

FASTPro – Addressing Inquiry

  • Change (Hord, Rutherford, Huling-Austin, & Hall,1987; Guskey, 2000)

    • takes time and persistence

      • awkwardness and frustration expected

    • as teacher’s progress through a change process, their needs for support and assistance change

  • Optimal Mix (Guskey, 2000)

    • A combination of teacher, program and change agent that will help create a positive relationship for PD to be effective


Fast pro1

FASTPro


Faststart

FASTStart

  • One week face-to-face institute

  • Teachers conduct nearly half of actual student investigations

  • Use inquiry methodology including modeling, discussion, assessment, and practice


Faster

Observe interactions of students and teachers

View and reflect upon investigations not experienced

See detailed step-by-step procedure suggestions

Movies and animations

Web interface

Quicktime and Flash plugins

DVD-ROM medium

FASTeR


Examples from fast er

Examples from FASTeR


Examples from fast er1

Examples from FASTeR

  • Slideshow 9, Density and the Cartesian Diver

  • Slideshow 22, Identifying Unknown Substances


Fastforward

FASTForward


Formative findings

Formative Findings

  • Suggest that teachers were able to implement successfully and share teaching practice

  • Added utility, focused and meaningful to teachers’ own environment


Future considerations

Infuse technological aspects into the FASTStart face-to-face experience

Address credit equivalencies to enable more robust activity requirements

Expand to include FAQ’s, indexing, and rich descriptions

Expand to include a Website as well as DVD-ROM

Automate some computer assisted instruction and feedback mechanisms

Future considerations


Reflections

Reflections…

computer breakdowns

scheduling

Months of editing

kids fascination with cameras


Reflections1

Reflections…

  • Teamwork and flexibility

  • Coordination of equipment and resources


Future considerations1

Future Considerations

  • Building capacity

  • Application to other CRDG quality educational programs


Mahalo

Mahalo.


Developing and implementing an alternative version of fast professional development1

Developing and Implementing an Alternative Version of FAST Professional Development

Mary E. Gray

ThanhTruc T. Nguyen

Thomas W. Speitel

University of Hawai‛i at Mānoa


Session 43 030 april 13 2005 american educational research association annual meeting

Instrument Development for a Study Comparing Two Versions of Inquiry Science Professional Development

Paul R. Brandon

Alice K. H. Taum

University of Hawai‘i at Mānoa


Identifying constructs

Identifying Constructs

  • on teaching science with inquiry methods

    • Reviewed FAST and other inquiry science documents and worked closely with inquiry science experts.

  • on the context within which inquiry-based science is taught.

    • Reviewed 55 books and key articles on curriculum indicators and school effectiveness.


Developing the teacher log

Developing the Teacher Log

  • Purpose: To identify the extent to which teachers implement the key features of inquiry-based science.

  • Reviewed the recent literature on logs.

  • Kept the instrument short to avoid overburdening teachers (21 items).

  • Is completed immediately after finishing each student science investigation.


Focus of the teacher log

Focus of the Teacher Log

  • Instrument addresses topics such as:

    • students’ questioning behaviors.

    • the teacher’s use of questioning strategies.

    • the teacher’s circulation about the classroom.

    • teacher-led discussions about variations in the data.


Developing the teacher questionnaire

Developing the Teacher Questionnaire

  • Purposes:

    • To obtain information about implementation that is most appropriately collected once a year.

    • To collect information on implementation of investigations not covered on logs.


Focus of the questionnaire

Focus of the Questionnaire

  • 150 items address topics such as:

    • implementation of key features.

    • the investigations taught during the entire year.

    • the adequacy of materials, equipment, etc.

    • teacher demographics.

    • teacher attitudes toward science.

    • teacher participation in science activities outside the classroom


Types of validation studies to date

Types of Validation Studies to Date

  • Two types of questionnaire-data validation studies conducted to date:

    • analyses addressing the relationship between the responses on the log and the questionnaire (“concurrent” validity)

    • an analysis examining the theoretical model underlying our study (construct validity)


Relationship between log and questionnaire responses

Relationship Between Log and Questionnaire Responses

  • Compared results on total scores for five items that the two instruments had in common

  • Correlation = .54; effect size = .50


Examining the theoretical model

Examining the Theoretical Model

  • Second set of analyses examined whether the expected relationships were found among some questionnaire variables.

  • We regressed a variable measuring program implementation on five independent variables measuring:

    • teachers’ ongoing learning about science.

    • the resources available for teaching science.

    • the number of science courses taken.

    • the number of years have taught K–12 science

    • teachers’ attitudes toward teaching science


Validity argument

Validity Argument

  • Teachers’ ongoing learning about science (including learning that occurred in recent PD) should predict implementation more than

    • science education background.

    • years of experience teaching science.

    • attitudes toward teaching science.

    • classroom resources available.


Results of second analysis

Results of Second Analysis

  • Teacher learning about science outside the classroom:

  • beta = .19(significant at the .05 level)

  • Teacher attitudes toward teaching science:

  • beta = .17 (significant)

  • Classroom and school resources for teaching science:

  • beta = .04(not significant)

  • Number of university science courses taken:

  • beta = -.02 (not significant)

  • Number of years the teacher has taught K–12 science: beta = -.02 (not significant)


Session 43 030 april 13 2005 american educational research association annual meeting

Instrument Development for a Study Comparing Two Versions of Inquiry Science Professional Development

Paul R. Brandon

Alice K. H. Taum

University of Hawai‘i at Mānoa


Coding teachers in science classrooms using the inquiry science observation guide

Coding Teachers in Science Classrooms using the Inquiry Science Observation Guide

Alice K. H. Taum

Curriculum Research & Development GroupUniversity of Hawai‘i at Mānoa


Inquiry science observation guide

Inquiry Science Observation Guide

  • Introduction and Overview

    • FAST, SCUP Project, Purpose and Development of ISOG

  • General coding guidelines

  • Descriptions for each of the six Activities

  • Definitions of Activity details

  • Code Sheet

  • Recording Sheet

  • Reconciling Recording Sheet


Development of the code sheet

Development of the Code Sheet

  • 1.5 year process

  • Collaborative efforts between researchers at the University of Hawai‘i at Mānoa, Stanford University and Sonoma State University; FAST teachers; curriculum developers, and coders

  • 36+ revisions


Goal of the code sheet

Goal of the Code Sheet

  • Identify strings of activities/teacher behaviors using the details in columns A, B, C and D.

    • questioning strategies used

    • level of engagement between teacher and students

    • creating a profile of teachers implementation of FAST in the classroom through science inquiry


Design of the code sheet

Design of the Code Sheet

  • 6 activities.

  • 66 possible activity details to plug into the 6 activities.

  • Multiple variations between an activity and the activity details which describe an activity string of teachers pedagogical practices.


Session 43 030 april 13 2005 american educational research association annual meeting

1. Teacher directs student(s)

_____A_______ to

_____B_____ _____C_____.


Session 43 030 april 13 2005 american educational research association annual meeting

1. Teacher directs student(s) _____A_____ to _____B___________C_____.

1. Teacher directs student(s) individually (1A1) to discuss (1B2) procedures (1C3).


Inquiry science observation recording sheet

Inquiry Science Observation Recording Sheet

  • Notes the start time of when the “broader” and “otheractivities” a teacher is engaged in begins

  • Allows for comments to facilitate the reconciliation process

  • Designed to record multiple activities occurring at the same time


Broader and other activities

Broader and Other Activities

  • The broader activity captures the larger activity occurring:

    Through interactive questioning (2A2b), the teacher introduces or provides an overview of (2B1) science investigation (2C4).

  • The other activities capture the details occurring within the broader activity:

    Teacher questions students through clarifying (3A1a) questioning and responds to student comment (3B1) by repeating (3C2) the comment and probing further (3C6).


Reconciling codes

Reconciling Codes

  • Two coders are paired to compare their independent codings, identifying any differences between them.

  • Differences are then discussed and each coder provides a rationale for his or her selected code.

  • When necessary, a review of the DVD is conducted by the paired coders.

  • Coding differences are discussed until consensus is reached between coders.


Inquiry science observation reconciling code sheet

Inquiry Science Observation Reconciling Code Sheet


Challenges

Challenges

  • greatest challenges:

    • establishing universal definitions for terms and phrases

      • What constitutes a “procedure”?

      • What is the difference between a science “term” and a science “concept”?

    • standardizing the recording of activity strings using the Recording Sheet

      • broader verses other activities

    • focusing on the teacher, rather than on the students


What next

What Next?

  • To date we have:

    • videotaped 15 teachers;

    • 140 videotape cassettes have been digitized to DVDs;

    • 116 have been quality-checked for audio clarity and teacher visibility; and

    • 8 coders are trained and ready to begin coding!


Coding teachers in science classrooms using the inquiry science observation guide1

Coding Teachers in Science Classrooms using the Inquiry Science Observation Guide

Alice K. H. Taum

Curriculum Research & Development GroupUniversity of Hawai‘i at Mānoa


Developing student outcome measures frameworks

Developing Student Outcome Measures Frameworks

Carlos C. Ayala

Sonoma State University


Carlos ayala

Scaling Up:

Student Measures

The Buck Stops Here

Carlos Ayala


Session 43 030 april 13 2005 american educational research association annual meeting

Let’s Talk Charge

Develop student assessment instruments in order to tease out differences between professional development models.

2 Session Pre Test

3 Session Post Test Suite


Where we expect differences

Where we expect differences

  • Student’s content knowledge

  • Student’s science inquiry skills

  • Student’s views of the nature of science

  • Student’s efficacy toward science

  • Student’s motivation to learn science


Final assessment suite

Final Assessment Suite

  • Pre-test

    • 30 item multiple-choice/ short answer test (α= .86)

    • Attitudinal Survey

  • Post-test

    • 30 item multiple-choice/ short answer test

    • Attitudinal survey

    • Mānoa River performance assessment


Session 43 030 april 13 2005 american educational research association annual meeting

Knowledge Type Framework

Type of

KnowledgeDefinition Examples Prompt

DeclarativeKnowing thatConcepts & facts “What is percolation?”

ProceduralKnowing howActions, steps, “How do measure how much & procedures water soil will hold?”

SchematicKnowing whyPrinciples & “How does the water cycle mental models work?”


Content validity

Content Validity

  • Reviewed curriculum and created content matrices

  • Curriculum developers parsed content down

  • Linked matrices to materials

  • Created test linking items to matrices

  • Piloted assessments

  • Talk alouds


Student science inquiry

Student Science Inquiry

  • As students become more engaged in the FAST curriculum and the teacher more fully implements the curriculum, students will be more proficient at Science Inquiry.

    • Design and conduct a scientific investigation

    • Use appropriate tools to gather, analyze and interpret data

    • Develop descriptions, explanations predictions and models using evidence

  • Targets based on

    • Duschl’s Transformations in Three Domains

    • Pottenger’s Deductive Explanatory Inquiry

  • NRC. (1996). National Science Education Standards . Washington D.C.: National Academy of the Sciences.


Student science inquiry1

Transformation 1

Data to Evidence: Deciding if the data are evidence, irrelevant and/or problematic.

Reformulation: From explanations to new questions: Deciding what next questions to ask and what new data are needed.

Transformation 2: Evidence to patterns or models decisions about selecting tools for identifying patterns or models

Transformation 3:

Patterns and models to explanations. Deciding how the patterns or models lead to explanations.

Student Science Inquiry


Nature of science

Nature of Science

  • As students become more engaged in the FAST curriculum and the teacher more fully implements the curriculum, students will understand that

    • anyone can be a scientist.

    • science knowledge is useful.

    • science knowledge builds over time.

    • science is creative.

Lederman, Abd-El-Khalick, Bell and Schwartz (2002) Views of Nature of Science Questionnaire; Toward Valid and Meaningful Assessments of Learner’ Conceptions of the Nature of Science.


Self efficacy

Self Efficacy

  • As students become more engaged in the FAST curriculum, the greater control they will feel towards science, science investigations and science knowledge.

    • I can make accurate measurements during a science investigation.

    • I can make appropriate predictions about what will happen during a science investigation.

Britner, S. and Pajares F., (2001) Self-Efficacy Beliefs, Motivation, Race and Gender in Middle School Science.


Session 43 030 april 13 2005 american educational research association annual meeting

Motivation

  • As students become more engaged in the FAST curriculum, patterns of motivation may change.

  • Dweck Groups (goals and epistemic beliefs)

    • Mastery Orientation

    • Ego Orientation

    • Helpless Orientation

Haydel, A., & Roser, R. (2001). On the links between students' motivation patterns and their perceptions of, beliefs about and performance on different types of science achievement, Multidimensional Approach to Achievement


Next steps

Next Steps

  • 760 Tests and Surveys completed

  • Collect post test data

  • Run analyses

  • Provide results to group


Developing student outcome measures frameworks1

Developing Student Outcome Measures Frameworks

Carlos C. Ayala

Sonoma State University


Session 43 030 april 13 2005 american educational research association annual meeting

The Differential Effects of Two Versions of Professional Development on Teachers’ Self-Efficacy to Implement Inquiry-Based Science

Brian Lawton

University of Hawai‘i at Mānoa


Purposes

Purposes

  • Effects of both versions of PD institutes on teacher self-efficacy to implement program

  • Differences between the two versions

  • Factors associated with self-efficacy change


Data collection

Data Collection

  • Qualitative and quantitative methods used

  • Self-efficacy scale

    • Developed to facilitate focus group discussion

  • Focus groups

    • Conducted immediately following the institutes

    • Provide in-depth information about changes in self-efficacy

  • Five stages of inquiry science

    • Introducing new science investigations

    • Facilitating valid experimental design

    • Facilitating the investigation process

    • Constructing meaning

    • Linking knowledge to new situations

  • Seven teachers from the 10-day and seven teachers from the 5-day participated in the study


  • Data collection cont

    Data Collection (cont.)

    Self-Efficacy Scale

    1.STAGE:Introducing new science investigations - my ability to introduce students to new science investigations by reviewing and tying in previous work.

    Now

    Low High

    ability ability

    Before

    Low High

    ability ability


    Data analysis

    Data Analysis

    • Only presenting focus group results

    • Analyzed for changes in self-efficacy

      • Two categories:

        • Statements implying an increase in self-efficacy

        • Statements implying no change in self-efficacy

    • Categories analyzed to identify the influencing factors

    • The statements and factors were quantified


    Results influencing factors

    Results, Influencing Factors

    • Overall seven factors were identified as influencing self-efficacy.

      1 Factors identified as most influential in increasing self-efficacy

      2 Factor identified as most associated with no change in self-efficacy


    Results changes in self efficacy

    Results: Changes in Self-Efficacy


    Discussion

    Discussion

    • Group increases in self-efficacy

    • Changes in self-efficacy between the groups

    • Groups perceived gains in content and pedagogical knowledge

    • Importance of 5-day follow-on support


    Further study

    Further Study

    • A study comparing the groups after the 5-day group has received all their training.


    Session 43 030 april 13 2005 american educational research association annual meeting

    Thanks!


    Session 43 030 april 13 2005 american educational research association annual meeting

    The Differential Effects of Two Versions of Professional Development on Teachers’ Self-Efficacy to Implement Inquiry-Based Science

    Brian Lawton

    University of Hawai‘i at Mānoa


    Session 43 030 april 13 2005 american educational research association annual meeting

    Comparing the Effects of Two Versions of Professional Development on Science Curriculum Implementation and Scaling-Up

    Session 43.030

    April 13, 2005

    American Educational Research Association Annual Meeting

    Montreal, Canada


    Questions and comments

    Paul Brandon - Overview

    Mary Gray and Truc Nguyen - Description of PD

    Paul Brandon - Log and questionnaire development and questionnaire validation.

    Alice Taum –Observation guide development

    Carlos Ayala - Student assessment development

    Brian Lawton - Comparison of early effects of the two versions of PD

    Paper, Presentation and Contact Info available at:

    http://hisii.hawaii.edu/SCUP/research/

    Questions and Comments


  • Login