1 / 24

Conducting Rigorous Evaluations of Interventions That (May) Improve Student Learning: A Researcher/Teacher Reflects

Conducting Rigorous Evaluations of Interventions That (May) Improve Student Learning: A Researcher/Teacher Reflects. Jon R. Star Michigan State University Harvard University (as of July 2007). About me. Former middle and high school mathematics teacher

maxine
Download Presentation

Conducting Rigorous Evaluations of Interventions That (May) Improve Student Learning: A Researcher/Teacher Reflects

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Conducting Rigorous Evaluations of Interventions That (May) Improve Student Learning:A Researcher/Teacher Reflects Jon R. Star Michigan State University Harvard University (as of July 2007)

  2. About me • Former middle and high school mathematics teacher • PhD in Educational Psychology, with emphasis on middle/high school students’ learning of mathematics • Two grants from US Dept. of Ed (IES) exploring interventions designed to improve students’ learning of mathematics

  3. Why I was asked to speak today • Perhaps useful for me to share my experiences as a researcher who designs and conducts rigorous (experimental) evaluations of interventions designed to improve students’ learning of mathematics • What are the challenges that I face in my work that I feel are endemic to the evaluation work that many of you are trying to do?

  4. Example 1 Example 1 Example 2 p.1 p.1 Example 2 p.2 Research examples (1) • The benefits of comparison for learning mathematics • Better to solve problems by viewing worked examples sequentially or side by side?

  5. Then If ? ? ? = Proportion Problem Schemata ? ? ? ? Ratio Problem Schemata Research examples (2) • Learning strategies for solving ratio and proportion problems using schema • Is using schema is better than not using schema?

  6. Two types of challenges • Tension between: • my role as a researcher, and • my heart and soul as a teacher • Balancing between: • the need to design the intervention to be very ‘clean’ to enable an experimental study, and • the need to provide experiences for students that are appreciative of good (and sometimes ‘messy’) instructional practices

  7. Researcher vs.teacher

  8. Flow of intervention • Let intervention run its course • Give help in standardized way • Don’t deviate from the instructional protocol or script • If students don’t ‘get it’, spend extra time with them • Focus on mastery of the material • Spend extra time with struggling students

  9. = Scorecard Flow of intervention

  10. Assessing learning • Materials need to discriminate and have room for learning • OK with pretest scores of 10% • OK with posttest scores of 50% • Materials should be challenging but students should be able to succeed • ‘Failing’ tests is disheartening • Average posttest grade of a B?

  11. = Flow of intervention Scorecard Assessing learning

  12. Relations with schools • Keep professional distance from schools and teachers • Don’t want to ‘spill the beans’ so that we can use the school in future work • Don’t tell if study didn’t work • Establish deep and long-lasting relationships with schools, teachers, and students • Researchers shouldn’t zip in/out • Go back and visit; share results

  13. = Flow of intervention Assessing learning Scorecard Relations with schools

  14. Ethics of study design • Experimental design tells us the most about the intervention • Control group is necessary • There may be “winner” and “loser” conditions • All conditions should lead to student learning • Design should allow non-treatment students to experience treatment eventually (if successful)

  15. = Flow of intervention Assessing learning Relations with schools Scorecard Ethics of study design

  16. Two types of challenges • Tension between: • my role as a researcher, and • my heart and soul as a teacher • Balancing between: • the need to design the intervention to be very ‘clean’ to enable an experimental study, and • the need to provide experiences for students that are appreciative of good (and sometimes ‘messy’) instructional practices

  17. Clean vs.messy

  18. Instructional format • Students working alone is an easier design • Logistically • Statistically • Students working with partners is often better for learning • Also can help teachers get more comfortable with this instructional format

  19. = Scorecard Instructional format

  20. Instructional delivery • Researchers providing instruction is easier: • no PD • consistency across schools • Letting students’ regular teachers do the teaching is better: • we are unfamiliar with norms • we don’t know students’ names

  21. = Instructional format Scorecard Instructional delivery

  22. Relations with teachers • Better if teachers don’t know too much about the intervention • might bias their interactions with students • The best way to have a long term impact on students is by working with teachers on instructional improvement

  23. = Instructional format Instructional delivery Scorecard Relations with teachers

  24. Thanks! Jon Star jonstar@msu.edu www.msu.edu/~jonstar

More Related