1 / 29

GATEWAY INITIATIVE

GATEWAY INITIATIVE . Hillsborough Community College Fall 2007 Preliminary Results A Formative Evaluation. Preliminary Formative Analysis Fall 2007 Results. To provide feedback to participating faculty as quickly as possible ( Got Class, p. 40).

dulcina-varian
Download Presentation

GATEWAY INITIATIVE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GATEWAY INITIATIVE Hillsborough Community College Fall 2007 Preliminary Results A Formative Evaluation

  2. Preliminary Formative Analysis Fall 2007 Results • To provide feedback to participating faculty as quickly as possible (Got Class, p. 40). • To answer initial questions at the pilot stage: • Are things working the way we expected? • What do the initial outcomes suggest? • Are there any surprises? • What can we learn that will improve the process?

  3. FALL 2007 • INTERVENTIONS - TWO PILOT COURSES: • MAC-1105 (College Algebra) • Cumulative Homework • In-class Group Review • PSY-2012 (General Psychology) • Team-Teaching • Use of Clickers (Electronic Student Response System) in the classroom • BASELINE DATA – THREE COURSES: • ACG-2021 (Financial Accounting) • CGS-1000 (Intro. To Computers & Technology) • ECO-2021 (Principles of Microeconomics)

  4. Data Collection Strategy • Pretest and Posttest • Questions addressed identified learning outcomes • Three scores for each student: • Pretest Score • Posttest Score • Gain Score (Posttest Score – Pretest Score) • Students’ Grades in the Course were Not Affected • Surveys of Student Characteristics • Descriptive and Demographic Information • Students’ Perceptions

  5. RESULTS OF ANALYSES • ANOVA/ANCOVA • Compared Gain Scores for Experimental and Control Groups in each intervention. • No Significant Differences between Experimental and Control Groups • MAC-1105 • PSY-2012

  6. PRETEST/POSTTEST COMPARISONSMAC-1105

  7. PRETEST/POSTTEST COMPARISONSPSY-2012

  8. SURPRISE!!! • Distributions of Gain Scores revealed some “negative gains.” • GAIN SCORE = (Posttest Score – Pretest Score) • These students scored higher on the pretest than on the posttest.

  9. Things were not working the way we expected. • Negative Gain Scores • Observed in both Control and Experimental sections of the pilot courses • (MAC-1105, PSY-2012). • Observed in all three of the courses collecting baseline data. • (ACG-2021, CGS-1000, ECO-2023)

  10. MAC-1105

  11. PSY-2012

  12. ACG-2021

  13. CGS-1000

  14. ECO-2023

  15. What Went Wrong? • Because it did not affect their final grades, students were not taking the posttest seriously. • Tired of being tested • Ready for Winter Break • No incentive to do their best • Marking answers at random (all 1s, all 5s, patterns) • Leaving answers blank

  16. What Was Learned Scheirer (1994) recommends using formative evaluation in a pilot situation to collect information on the feasibility of activities and their acceptance by recipients. • Posttest – Not Feasible, Not Accepted • Negative gain scores – evidence that the posttest is not accurately assessing students’ KSAs. • Not a valid measure. • No conclusions can be based on these results. • A valid measure must be found for use in the future. Scheirer, M. A. (1994). Designing and using process evaluation. In J. S. Wholey, H. Hatry and K. Newcomer (Eds.), Handbook of Practical Program Evaluation (pp. 40-68). San Francisco : Josey Bass.

  17. How Will We Improve the Process? • Lead Faculty Acted Quickly and Decisively • MAC-1105 • Posttest questions imbedded in the final exam • PSY-2012 • Performance on posttest worth extra credit • Implemented in Time for Spring Posttests

  18. EXPECTED RESULTS • Tying the posttest to the final grade appears to have corrected the problem. • Lead faculty grading SP08 posttests report seeing a difference. • We are expecting meaningful results from the revised posttest procedure as we conclude our first year of the study.

  19. ABOUT OUR STUDENTS

  20. STUDENTS’ TERM CREDIT LOAD

  21. STUDENTS RECEIVING SOME FORM OF FINANCIAL AID

  22. WORK HOURS per WEEK (self-report)

  23. “What were your biggest obstacles in learning (this subject)?”

  24. “What additional resources would have helped you to be more successful ( in this class)?”

  25. Other Resources Identified

  26. “Which of the following helped you the most to succeed in (this class)?”

  27. PERCEIVED HELPFULNESS OF INTERVENTIONS EXPERIMENTAL SECTIONS ONLY MAC-1105 • Group Test Review (n=150) • Helped the Most – 44.67% • Other Resources that Helped – 48.67% • Cumulative Homework (n=98) • Helped the Most – 47.96% • Other Resources that Helped – 60.20%

  28. PERCEIVED HELPFULNESS OF INTERVENTIONS EXPERIMENTAL SECTIONS ONLY Psy-2012 • Clicker Technology (n= 80) • Helped the Most – 55.0% • Other Resources that Helped – 58.75% • Team-Teaching (n=58 ) • Helped the Most – 24.14 % • Other Resources that Helped – 20.69%

  29. LOOKING FORWARD • Students’ feedback suggests that these interventions ought to be effective. • Spring semester’s results should provide a more realistic picture of their impact on students’ learning outcomes.

More Related