lessons learned about random assignment evaluation implementation in educational settings l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Lessons Learned About Random Assignment Evaluation Implementation in Educational Settings PowerPoint Presentation
Download Presentation
Lessons Learned About Random Assignment Evaluation Implementation in Educational Settings

Loading in 2 Seconds...

play fullscreen
1 / 10

Lessons Learned About Random Assignment Evaluation Implementation in Educational Settings - PowerPoint PPT Presentation


  • 145 Views
  • Uploaded on

Lessons Learned About Random Assignment Evaluation Implementation in Educational Settings. SREE Conference March 4, 2010 Raquel Sanchez and Fannie Tseng. Introduction. Hands-on overview of our experiences implementing random assignment evaluations in the classroom.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Lessons Learned About Random Assignment Evaluation Implementation in Educational Settings


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
lessons learned about random assignment evaluation implementation in educational settings

Lessons Learned About Random Assignment Evaluation Implementation in Educational Settings

SREE Conference

March 4, 2010

Raquel Sanchez and Fannie Tseng

introduction
Introduction
  • Hands-on overview of our experiences implementing random assignment evaluations in the classroom.
  • Extending list of lessons discussed in past literature.
  • Brief description of the random assignment evaluations upon which our experiences were drawn.
  • Discuss difficulties with implementing random assignment in classroom settings.
  • Discussion of lessons learned
past literature on random assignment implementation
Past Literature on Random Assignment Implementation
  • Gueron (2002)
  • Ritter and Holley (2008)
  • Raudenbush (2005)
  • Burghardt and Jackson (2007)
overview of our random assignment evaluation studies
Overview of Our Random Assignment Evaluation Studies
  • Two school-level random assignment studies of the effectiveness of professional development programs that focus on developing the reading comprehension skills of English language learners (ELLs)
  • One center-level random assignment study of a professional development program targeting caregivers of children ages 0-3
  • One student-level random assignment study of a curriculum that combines explicit and implicit approaches to instruction in increasing the literacy skills of adult ESL students
challenges to implementing rcts in educational settings
Challenges to Implementing RCTs in Educational Settings
  • Threats to integrity of random assignment
    • Crossovers and contamination
  • Recruiting and obtaining buy-in
  • Dilution of intervention effectiveness
    • Lack of teacher buy-in
    • Effect of crossovers and contamination
  • Documenting treatment dosage
  • Conflicting interventions on the ground
  • Local conditions and circumstances
lessons learned
Lessons Learned
  • Perform in-person recruiting visits at all levels of school administration to maximize buy-in
  • Foster good communication between school staff, program developers and research team
  • Follow-up data collection requires persistence, patience and adequate funding
    • Keep in touch with assessment data administrators, even in off-months of study
    • If possible, retain local research staff
  • Use conservative statistical power calculations to factor in potential implementation challenges
lessons learned 2
Lessons Learned (2)
  • Throughout the course of the study, establish a separate identity from the program you are evaluating
    • Hand out separate evaluation study materials with research organization’s logo
    • Gift cards help
  • Be proactive about developing plans for documenting treatment dosage
    • If possible, use more than one source of data
  • Importance of qualitative studies to accompany impact studies
contact us
Contact Us
  • Raquel Sanchez, Ph.D.raquel@bpacal.com
  • Fannie Tseng, Ph.D.fannie@bpacal.com

Berkeley Policy Associates

440 Grand Ave., Suite 500

Oakland, CA 94610-5085

Ph: 510-465-7884

Fax: 510-465-7885

www.berkeleypolicyassociates.com