Slide1 l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 64

Studying Fidelity of Implementation (FOI): How FOI influenced SCALE-uP’s Theory of Action for Middle School Science Curriculum Materials PowerPoint PPT Presentation


  • 143 Views
  • Uploaded on
  • Presentation posted in: General

Studying Fidelity of Implementation (FOI): How FOI influenced SCALE-uP’s Theory of Action for Middle School Science Curriculum Materials. *SCALE-uP = Scaling up Curriculum for Achievement Learning and Equity Project, a partnership between George Washington University

Download Presentation

Studying Fidelity of Implementation (FOI): How FOI influenced SCALE-uP’s Theory of Action for Middle School Science Curriculum Materials

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Slide1 l.jpg

Studying Fidelity of Implementation (FOI): How FOI influenced SCALE-uP’s Theory of Action for Middle School Science Curriculum Materials

*SCALE-uP = Scaling up Curriculum for Achievement Learning and Equity Project, a partnership between George Washington University

and Montgomery County Public Schools**

Sharon Lynch, PI

Co-PIs: Curtis Pyke, Joel Kuipers, Michael Szesze** & Bonnie Hansen-Grafton**

http://www.gwu.edu/~scale-up/

Prepared for Researchers Without Borders Webinar, May 26, 2010


Foi research group l.jpg

FOI Research Group

  • Carol O’Donnell

  • Suzanne Merchlinski & MCPS evaluation staff

  • Bonnie Hansen-Grafton

  • Joelle Lastica

  • Vasuki Rethinam

  • Bill Watson

  • Rob Ochsendorf

  • Liz Hatchuel

  • Annie Hansen

    With special thanks to MCPS middle school science teachers who participated in this study


Disclaimer l.jpg

Disclaimer

  • Indebted to the Interagency Educational Research Initiative (IERI) administered by the NSF, for research funding for SCALE-uP (7 year research program).

  • I am currently working at NSF as a Program Director in EHR/DRL ( return to my position as a professor at GWU in September).

  • However, the ideas and opinions discussed here are entirely my own and in no way represent the those of NSF.


Scale up foi webinar cautionary tale w happy ending l.jpg

SCALE-uP/FOI Webinar:Cautionary Tale w/ Happy Ending

  • Background of SCALE-uP and initial Theory of Action

  • Year 0 Pilot Study: Curriculum modifications are tricky business!

  • Year 1+: Comparison groups are (incredibly) handy in developing FOI instruments and understanding the study context

  • Year 2 & 3: Comparison group and FOI evidence are crucial for credible evidence of effectiveness

  • Year 4: Putting it all together: How the Theory of Action was changed by FOI evidence: FOI as “process” and “structure” constructs for both teacher and student


Background for scale up and initial theory of action l.jpg

Background for SCALE-uP and Initial Theory of Action

  • In 1990’s, AAAS Project 2061 developed a Curriculum Analysis to identify curriculum materials likely to help students learn a target idea (benchmark/standard).

  • Curriculum Analysis relied on experts’ judgment of written curriculum materials.

  • Two parts:

    --Focused, accurate, coherent content on a standard/benchmark

    --Instructional strategies contained in written curriculum materials


Project 2061 instructional strategies for curriculum materials l.jpg

Project 2061 Instructional Strategies for curriculum materials

  • Convey sense of purpose

  • Address student ideas and misconceptions

  • Promote engagement with relevant phenomena

  • Developing, using scientific ideas

  • Encourage student thinking

  • Encourage assessment of progress

  • Creating positive learning environment: curiosity, all students

AAAS. Project 2061.


Background for scale up l.jpg

Background for SCALE-uP

  • Project 2061 Curriculum Analysis had located only 2 acceptable curriculum units in middle school science.

  • Units had been field-tested with small numbers of students (no comparison groups).

  • Note. More mathematics curriculum materials had acceptable ratings and were field-tested and studied and scaled.


Background for scale up and research questions l.jpg

Background for SCALE-uP and Research Questions

If science curriculum materials having Project 2061 attributes were studied in a series of large (N = ~ 2000) quasi-experiments using carefully matched comparison groups:

  • Would they be effective?

  • Would they be equitable?

  • Would there be a relationship between fidelity of implementation to a unit and student outcomes?

  • Could the materials be scaled-up in this large school district?

  • How did they function in classroom (video-ethnography)?


Scale up s interventions 3 science units with coherent focused on target ideas l.jpg

SCALE-uP’s Interventions: 3 Science Units with Coherent Focused on Target Ideas

  • State of Michigan’s Chemistry That Applies (CTA) focuses on conservation of matter. 8th graders, unit ~ 6 weeks long.

  • GEMS Lawrence Hall of Science Real Reasons for the Season (Seasons) focuses on the reasons for the Earth’s seasons. 7th graders, unit, ~ 3 weeks.

  • ARIES Harvard Smithsonian Motion and Forces (M&F) focuses on portions of Newton’s Laws. 6th graders, unit ~ 6 weeks long.


Slide10 l.jpg

Curriculum Analysis: Instructional Strategies

● =Excellent, ◕=Very Good, ◒=Satisfactory, ◔=Fair ○=Poor


Slide11 l.jpg

● =Excellent, ◕=Very Good, ◒=Satisfactory, ◔=Fair ○=Poor


Scale up s outcome measures l.jpg

SCALE-uP’s Outcome Measures

  • Curriculum-independent measure for each unit focusing on the unit’s target idea.

  • Assessments had good psychometric properties and were developed using a Project 2061 assessment system.

  • Multiple choice and constructed response items designed to be maximally accessible to students of varied language skills.


Questions l.jpg

Questions?


Background for scale up and initial theory of action c 2001 l.jpg

Background for SCALE-uP and Initial Theory of Action, c. 2001

Curriculum units highly rated on Curriculum Analysis could be effective overall because:

each focused coherently on one big idea/standard/ benchmark

each had a carefully planned sequence of activities, and

each contained identified instructional strategies leading students to construct understanding of one target idea/benchmark/standard.

Big Question: Would they be equitable?

Assumption: “Business as usual” comparison classrooms would be less focused, rely more on textbooks and worksheets, and provide less time for guided inquiry and lab work.


Typical theory of action l.jpg

“Typical” Theory of Action

Teacher

Curriculum

Materials

Student

Outcomes


Fidelity of implementation mowbray holter teague bybee 2003 l.jpg

Fidelity of implementationMowbray, Holter, Teague & Bybee, 2003

  • “Fidelity of implementation is the extent to which the delivery of an intervention adheres to the original program theory behind its development; it confirms that the implementation of the independent variable in outcome research occurred as planned…

  • …(and involves) the dynamic nature of fidelity criteria, appropriate validation and statistical analysis methods, the inclusion of structure and process criteria in fidelity assessment and the role of program theory in deciding on the balance between adaptation versus exact replication of model programs.”


Scale up theory of action c 2001 l.jpg

SCALE-uP Theory of Action c. 2001

Teacher FOI

Instructional

Strategies

Teacher

Curriculum

Materials

Student

Outcomes


Scale up theory of action c 200118 l.jpg

SCALE-uP Theory of Actionc. 2001

  • Teachers would need to locate and enact the instructional strategies embedded in the curriculum unit (identified via the Project 2061 Curriculum Analysis).

  • Doing this well would be to “implement with fidelity”.

  • Hypothesis: The better the implementation of the unit’s embedded instructional strategies in a classroom, the higher the student outcomes.

  • All SCALE-uP needed to do to create a classroom observation instrument that captured teachers’ implementation of the strategies.

  • This would become Instructional Strategies Classroom Observation Protocol (more on this later).

  • Simple!


Questions19 l.jpg

Questions?

On to Fidelity of Implementation (FOI)!


Year 0 pilot study results chemistry that applies cta with modifications encouraged l.jpg

Year 0 (Pilot Study) Results: Chemistry That Applies (CTA) with Modifications Encouraged

  • CTA’s results showed effect sizes = .52.

  • Results disaggregated for subgroups of students showed that no students disadvantaged by CTA.

  • Teachers were asked to modify CTA for diverse learners and record modifications.

  • Virtually none did! This raised questions about whether to modify CTA in future. Teacher beliefs about the unit were mixed.

  • Co-PI Szesze wanted to be sure units were unambiguously effective, or not.

  • Decision to “implement with fidelity” in the future studies .

  • Teachers and researchers drew up fidelity guidelines together.


Lesson learned modifying curriculum unit is tricky business l.jpg

Lesson Learned :Modifying Curriculum Unit is Tricky Business!

  • If an intervention does not have solid evidence of efficacy/effectiveness, then modifications muddy the water; CTA and the other units were “unproven” interventions.

  • Therefore, their critical components could not be “known”, but assumed.

  • In retrospect, this was a good decision for SCALE-uP; capturing modifications while trying to establish if a unit “worked” would not be credible in this school district context.


Slide22 l.jpg

Iterative Process of Identifying Critical Attributes, Measuring Attributes, and Looking for Relationships between FOI and Outcomes

Identify Critical

Attributes of

Intervention

FOI

measure

Reconsider

Critical

Attributes

Outcomes

Mowbray, C., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24(3),315-340.


Questions23 l.jpg

Questions?

On to Year 1

Replication of CTA

Building the first FOI Instrument


Yr 1 replication of cta and develop iscop foi process l.jpg

Yr. 1 Replication of CTA and Develop ISCOP/FOI Process

  • Given Theory of Action (the more teachers implemented Project 2061 instructional strategies embedded in curriculum units, the higher student outcomes), we began to develop an instrument that could capture fidelity to identified Project 2061 instructional strategies.

  • A “generic” instrument for all 3 units studied, but the units had been carefully vetted and had much in common.

  • ISCOP (Instructional Strategies Classroom Observation Instrument), the first FOI measure was born, and developed over the next 4 years.


Effect sizes cta year 1 l.jpg

Effect Sizes: CTA (Year 1)


Years 1 2 3 4 test iscop in treatment and comparison classrooms l.jpg

Years 1, 2, 3, 4: Test ISCOP in Treatment and Comparison Classrooms

  • Was the ISCOP capturing fidelity to instructional strategies in a way that discriminated between Treatment and Comparison classrooms?

  • ISCOP did not discriminate very well, so it was refined and refined and refined.

  • Was ISCOP a “bad” instrument? Or were Treatment and Comparison classrooms similar for Instructional Strategies?

  • If Treatment and Comparison classrooms were similar in strategies, should the Theory of Action be revisited?


Lesson learned comparison classrooms invaluable as counterfactuals l.jpg

Lesson Learned: Comparison Classrooms Invaluable as Counterfactuals

  • ISCOP data suggested that more complexity than assumed in the Theory of Action.

  • Measuring FOI relying solely with ISCOP might not answer FOI research question if comparison classroom data were taken into consideration.

  • Generic measures of instructional strategies/FOI process are notoriously hard to develop—valid and reliable?


Questions29 l.jpg

Questions?

On to Years 2 and 3 with a new unit, Real Reasons for the Seasons

(Seasons)


Year 2 and 3 results comparison group outscores seasons group how to account for this l.jpg

Year 2 and 3 Results: Comparison group outscores Seasons group: How to account for this?

  • Seasons studied in 7th grade classrooms (N ~ 2000 students and ~ 40 classrooms).

  • Year 2 and 3 results: Comparison group has higher outcome than Seasons on curriculum independent measure (ES = -.36, -.18, respectively).


Comparison classrooms invaluable as counterfactuals l.jpg

Comparison Classrooms Invaluable as Counterfactuals

  • Comparison classroom teachers surveyed and interviewed.

  • Data showed that Comparison classrooms were:

    -Non-traditional

    -Variety of curriculum materials used, including other inquiry units

    -Focused on the target idea

    -Equal Duration of Treatment and Comparison units

    -Comparison and Treatment teacher characteristics similar.


Seasons replication in year 3 focus on foi l.jpg

Seasons Replication in Year 3: Focus on FOI

  • Use ISCOP in Treatment and Comparison classrooms.

  • Develop a new Lesson Flow FOI process measure that gauged Teacher, Student-Group or Individual Centeredness of classrooms based on hunch that Student-Group Centeredness was important to student construction of science ideas in groups.

  • Videotaped a Seasons and Comparison classroom.

  • Interviewed and surveyed Seasons and Comparison Teachers again.


Lesson flow classroom observation instrument l.jpg

Lesson Flow Classroom Observation Instrument


Effect sizes seasons year 3 overall es 18 l.jpg

Effect Sizes: Seasons (Year 3)Overall ES = -.18


Year 3 foi results l.jpg

Year 3 FOI Results

  • Only 3 items on ISCOP showed significant differences between Seasons and Comparison classrooms, and 2 of 3 were observed more often in Comparison classrooms.

  • Lesson Flow: Seasons classrooms were more teacher-centered than Comparison:

    Teacher-Centeredness

    Seasons = 71 % of time

    Comparison =58 % of time

  • Video-data backed this up.

  • Emerging Conjecture: Students need time in to work and talk in groups to develop their ideas and Seasons allowed less time for this.


Scale up theory of action c 2006 l.jpg

SCALE-uP Theory of Action c. 2006

Teacher FOI

Instructional

Strategies

Teacher

Curriculum

Materials

Student

Outcomes

Student FOI

Student Group

Centeredness


Changing theory of action and approach to foi c 2006 l.jpg

Changing Theory of Action and Approach to FOI c. 2006

  • Focusing FOI entirely on teacher in a guided inquiry unit may be simplistic.

  • Student agency implicated in FOI constructs.

  • But Lesson Flow is controlled by the teacher who provides students with the time/space to work in groups; students can choose to do the science work, or not.

  • Lesson Flow is not Time On Task.


Seasons unit and foi the research re visited l.jpg

Seasons Unit and FOI:The Research Re-visited

  • Seasons was designed to be a supplementary unit.

  • Developers seemed not to understand the implications of this when agreement was made to study effectiveness/FOI.

  • SCALE-uP researchers did not appreciate the differences in philosophy of Seasons with research design.

  • Seasons was a bad match for this study.

  • Revisions made to Seasons based upon feedback from MCPS teachers.


Questions39 l.jpg

Questions?

On to Year 4 and Motion and Forces

(M&F)

Note this is the third curriculum unit studied,

not to be confused with the first two, and was implemented in 6th grade classrooms.


M f results in years 2 and 3 underwhelming l.jpg

M&F Results in Years 2 and 3:Underwhelming

  • Year 2 ES = +.10

  • Year 3 ES = -.06

  • FOI had not been emphasized, so there was little information to explain lackluster outcomes.

  • However, we learned that students had not been issued M&F student Journals in Years 2 and 3. Was this an important, overlooked FOI issue?


Year 4 replication of m f focus on foi l.jpg

Year 4 Replication of M&F: Focus on FOI

  • Replicated quasi-experiment in 10 new schools (N ~ 2000 students).

  • Teachers asked to focus on FOI.

  • Wonderful teachers who understood the study goal: study the impact of M&F.


Foi in year 4 with m f l.jpg

FOI in Year 4 with M&F

FOI included:

  • ISCOP (Process FOI)

  • Lesson Flow (Process FOI)

  • Adherence to unit (ACOP), a new instrument that measured close adherence to M&F’s structure (Structure FOI)

  • Teacher Interviews/Surveys

  • Teacher Logs

  • Student Journal Entries, # of responses to Journal (Structure FOI)

  • Student survey about self-reporting use of instructional strategies (Process FOI)


Overall results for m f in year 4 l.jpg

Overall Results for M&F in Year 4

  • Student Level results using traditional ANOVA, ES = .23

  • Classroom Level results using HLM,

    ES = .56 (Rethinam, Lynch, & Pyke, 2008)


Effect sizes for subgroups of students m f year 4 l.jpg

Effect Sizes for subgroups of students: M&F (Year 4)


Iscop strategies means and correlations with outcomes for m f l.jpg

ISCOP Strategies: Means and Correlations with Outcomes for M&F


Iscop l.jpg

ISCOP

  • This “generic” instrument for measuring FOI instructional strategies did not obviously distinguish between M&F and Comparison classrooms.

  • But some items were correlated with student outcomes for M&F classrooms.

  • O’Donnell dissertation elegantly teased out which instructional strategies seemed to matter for higher student outcomes.

  • ISCOP needs more work on validity and reliability--OR SCALE-uP Theory of Change needs refinement—Project 2061 Curriculum Analysis needs refinement?


Lesson flow for instruction for units on motion and force l.jpg

Lesson Flow for Instruction for units on Motion and Force

M&F

Comparison


Lesson flow l.jpg

Lesson Flow

  • M&F classrooms provided more time for students to work in groups and individually than Comparison classrooms.

  • Students construct meaning in groups, consistent with notion of community of practice and situated cognition.


Adhering to m f lesson components acop l.jpg

Adhering to M&F Lesson Components (ACOP)

  • M&F teachers adhered to M&F unit > 80% of time.

  • ACOP did not predict outcomes because the range was narrow; this was a good thing for this study because high fidelity in this measure of “structure” is credible and strong—good face validity for FOI, and highly reliable—the unit was well-implemented.

  • Teachers adhered to the unit for this study, to ascertain impact of M&F.

  • Thank you teachers.


Results teacher foi structure l.jpg

Results: Teacher FOI Structure


Student journals l.jpg

Student Journals

  • M&F students completed their journal responses ~ 80% of time.

  • Rates of journal question completion predicted classroom outcomes.


Unpublished year 4 hlm results for m f iscop acop lesson flow student journals l.jpg

Unpublished Year 4 HLM Results for M&F: ISCOP, ACOP, Lesson Flow, Student Journals

  • HLM analysis found one classroom-level factor that predicted student outcomes:

    --Amount of Student-Group-Centeredness (Lesson Flow)

  • and one student level factor

    --Student Journal Completion

  • Thanks to Dr. Jaewa Choi of GWU for this data analysis.


Scale up theory of action c 2010 l.jpg

SCALE-uP Theory of Action c. 2010

Teacher

Instructional

Strategies

Teacher

Adherence

to Lesson

Structure

ACOP

ISCOP

Structure

FOI

Process

FOI

Teacher

Curriculum

Materials

Student

Outcomes

Structure

FOI

Student

Adherence

to Lessons

Student Group

Centered

Instruction

Process

FOI

Lesson Flow

Student Journal


Scale up theory of action c 201054 l.jpg

SCALE-uP Theory of Action c. 2010

Teacher

Instructional

Strategies

Teacher

Adherence

to Lesson

Structure

ACOP

ISCOP

Teacher

Curriculum

Materials

Student

Outcomes

Student

Adherence

to Lessons

Student Group

Centered

Instruction

Lesson Flow

Student Journal


Scale up theory of action c 201055 l.jpg

SCALE-uP Theory of Action c. 2010

Teacher

Instructional

Strategies

Teacher

Adherence

to Lesson

Structure

ACOP

ISCOP

Teacher

Curriculum

Materials

Student

Outcomes

Students

Student

Adherence

to Lessons

Student Group

Centered

Instruction

Lesson Flow

Student Journal


Scale up theory of action c 201056 l.jpg

SCALE-uP Theory of Action c. 2010

Teacher

Instructional

Strategies

ISCOP

ACOP

Teacher

Curriculum

Materials

Teacher

Adherence

to Lesson

Structure

Student

Outcomes

Students

Student

Adherence

to Lessons

Student Group

Centered

Instruction

Lesson Flow

Student Journal


Questions57 l.jpg

Questions?


Summary findings l.jpg

Summary Findings

  • Score Card: CTA and M&F seemed to be effective and equitable in this school system, Seasons did not seem to be effective.

  • Theory of Action: Changed from looking at one relatively “generic” measure of teacher process fidelity to multiple measures of FOI adding student FOI and measures of FOI structure, and developing a better Theory of Action.

  • Lesson Flow and Student Journal response best predicted student outcomes; ISCOP analysis illuminated most potent instructional strategies for M&F.


Foi lessons learned l.jpg

FOI Lessons Learned

  • Studying curriculum adaptations for an “unproven” curriculum is a slippery slope because critical ingredients are unknown.

  • Study of FOI in Comparison groups provides important counterfactual.

  • This study shows potential for both teacher and student FOI, as well as process and structural approaches.

  • Study of FOI should reveal more about critical components of an intervention if measures of each component shows a positive relationship with student outcomes.


The end l.jpg

The End

Thanks to Researchers Without Borders for sponsoring this Webinar.


References l.jpg

References

  • American Association for the Advancement of Science (AAAS). (2003). Project 2061 middle grades science textbooks: A Benchmarks-based evaluation. Retrieved June 1, 2004, from http://www.project2061.org/tools/textbook/mgsci/index.htm.

  • Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research Theory and Practice, 18(2), 237-256.

  • Harvard-Smithsonian Center for Astrophysics. (2001). ARIES: Exploring motion and forces: Speed, acceleration, and friction. Watertown, MA: Charlesbridge Publishing

  • Kesidou, S., & Roseman, J.E. (2002). How well do middle school science programs measure up? Findings from Project 2061’s curriculum review. Journal of Research in Science Teaching, 39(6), p. 522-549

  • Lastica, J.R., & O'Donnell, C.L. (2007, April). Considering the role of fidelity of implementation in science education research: Fidelity as teacher and student adherence to structure. In C. O'Donnell (Chair), Analyzing the relationship between Fidelity of Implementation (FOI) and student outcomes in a quasi-experiment. Symposium conducted at the Annual Meeting of the American Educational Research Association, Chicago, IL.Lawrence Hall of Science. (2000).GEMS:The real reasons for seasons—Sun-Earth connections. Berkeley: The Regents of the University of California.

  • Lynch, S. (2000). Equity and science education reform. Mahwah, NJ: Lawrence Erlbaum and Associates.

  • Lynch, S., Kuipers, J.C., Pyke, C., & Szesze, M. (2005). Examining the effects of a highly rated science curriculum unit on diverse students: Results from a planning grant. Journal of Research in Science Teaching, 42(8), 912-946.


Slide62 l.jpg

Lynch, S. & O’Donnell, C. (2005, April). The evolving definition, measurement, and conceptualization of fidelity of implementation in scale-up of highly rated science curriculum units in diverse middle schools. In S. Lynch (Chair), The role of fidelity of implementation in quasi-experimental and experimental research designs: Applications in four studies of innovative science curriculum materials and diverse student populations. Symposium conducted at the Annual Meeting of the American Educational Researchers Association, Montreal, Canada.

Lynch, S., O'Donnell, C., Hatchuel, E., & Rethinam, V. (2007, April). A model predicting student outcomes in middle school science classrooms implementing a highly-rated science curriculum unit. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, New Orleans, LA.

Lynch, S., O'Donnell, C., Hatchuel, E., Rethinam, V., Merchlinsky, S., & Watson, W. (2006, April). What’s up with the Comparison group?: How large quasi-experimental study of highly rated science curriculum units came to grips with unexpected results. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA.

Lynch, S., Taymans, J. Watson, W., Ochsendorf, R., Pyke, C. & Szesze, M. (2007). Effectiveness of a highly-rated science curriculum unit for students with disabilities in general education classrooms. Exceptional Children, 73(2), 202-223.

Merchlinsky, S. & Hansen-Grafton, B. (2007, April). Considering the role of Fidelity of Implementation (FOI) in science education research: Evaluation and science specialists' role in collecting FOI data in a large school district. In C.L.

Michigan Department of Education. (1993). Chemistry That Applies. The State of Michigan.

Mowbray, C., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24(3),315-340.

O’Donnell, C. L. (2007). Fidelity of implementation to instructional strategies as a moderator of curriculum unit effectiveness in a large-scale middle school science experiment. Dissertation Abstracts International, 68(08). (UMI No. AAT 3276564)


Slide63 l.jpg

O'Donnell, C.L., Lynch, S., Lastica, J., & Merchlinsky, S. (2007, April). Analyzing the relationship between Fidelity of Implementation (FOI) and student outcomes in a quasi-experiment. Symposium conducted at the Annual Meeting of the American Educational Research Association, Chicago, IL.

O'Donnell, C.L., Lynch, S., Watson, W., & Rethinam, V. (2007, April). Teacher and student Fidelity of Implementation (FOI) to process: Quality of delivery and student responsiveness and relationships to classroom achievement. In C.L. O'Donnell (Chair), Analyzing the relationship between Fidelity of Implementation (FOI) and student outcomes in a quasi-experiment. Symposium conducted at the Annual Meeting of the American Educational Research Association, Chicago, IL.

O'Donnell, C. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78(1), 33-84.

O'Donnell, C., & Lynch, S. (2008, March). Fidelity of implementation to instructional strategies as a moderator of science curriculum unit effectiveness. Paper presented at the 2008 annual meeting of the American Educational Research Association, New York, NY.

Rethinam, V., Pyke, C., & Lynch, S. (2008). Using multilevel analyses to study the effectiveness of science curriculum materials. Evaluation and Research in Education, 21(1), 18-42


Slide64 l.jpg

Rethinam, V., Pyke, C., & Lynch, S. (in preparation.). Using Multilevel Analyses to Study Individual and Classroom Factors in Science Curriculum Effectiveness.

Songer, N. B., & Gotwals, A. W. (2005, April). Fidelity of implementation in three sequential curricular units. In S. Lynch (Chair), “Fidelity of implementation” in implementation and scale-up research designs: Applications from four studies of innovative science curriculum materials and diverse populations. Symposium conducted at the meeting of the Annual Meeting of the American Educational Research Association. Montreal, Canada.

Stern, L. & Ahlgren, A. (2002). Analysis of students’ assessments in middle school curriculum materials: Aiming precisely at benchmarks and standards. Journal of Research in Science Teaching, 39, 889-910.

Watson, W., Lynch, S., Rethinam, V., & O’Donnell, C. (2006, April). Development of an instrument to measure student responsiveness to implementation of science curriculum materials. Paper given at the annual meeting of the National Association for Research in Science Teaching, April, 2006 , San Francisco.

Further acknowledgments of important contributions to this work on fidelity of implementation made by Rob Ochsendorf, Aiyita Ruiz-Primo, Doug Clement, Okhee Lee, Bruce Ward, Carolyn Walton, Theron Blakeslee, Andy Anderson, Phyllis Blumenfeld.


  • Login