1 / 26

The Five Step Program Improvement Process

The Five Step Program Improvement Process. STE P FOUR Cincinnati Pilot Site Training February 28 th , 2011. Create an Evaluation Plan. Why Evaluate?. Make sure your improvement strategy works somewhere before you attempt to apply it everywhere. Types of Evaluations.

vina
Download Presentation

The Five Step Program Improvement Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Five Step Program Improvement Process

  2. STEP FOURCincinnati Pilot Site TrainingFebruary 28th, 2011 Create an Evaluation Plan

  3. Why Evaluate? Make sure your improvement strategy works somewhere before you attempt to apply it everywhere

  4. Types of Evaluations • Formative Evaluation • Process measures • How well was the program implemented • “The cook tasting the soup” • Summative Evaluation • Outcome measures • Did the program produce the desired results • “The guests taste the soup”

  5. Set Realistic Goals • Broad statements of what you want to accomplish • The main single aim • Not strictly measurable or tangible • Long term • The purpose toward which an endeavor is directed

  6. Write SMART Objectives • Evaluation objectives must be • Specific • Measurable • Achievable • Relevant • Time-limited

  7. Formative (Process) Evaluation What we all know and do well

  8. Select Process Measures • Attendance • Response rate • Satisfaction of implementers with process • Satisfaction of participants with process • On time accomplishment of tasks • Quality evaluation of implementation services • More

  9. Process Objectives • 75% of invitees will respond to the invitation by the RSVP date • 60% of the invitees will attend the event • 50% of the participants will bring at least one parent or guardian to the event • 75% of the participants will score their satisfaction with the event as a 4 (very good) or 5 (excellent) on the exit evaluation instrument

  10. Process Measurement Tools • Invitation/Outreach records • Registration/Response records • Attendance records • Participant demographics • Evaluation instrument • Planning timeline • More

  11. Develop a Process Evaluation Plan

  12. Summative (Outcome) Evaluation The ultimate impact of our efforts RESULTS

  13. Select Outcome Measures • Short-term • Increase in awareness • Increase in knowledge • Change in attitude • Change in behavior • Long-term • Enrollment • Achievement • Completion • Graduation • Transition

  14. Short-term Objectives • 80% of participants in Agri-women of Today workshop will increase their knowledge of farm business practices as measured in pre-and post-workshop survey. • After participating in Focus Your Future event 50% of the participating students will show a one point (on a 5 point likert scale) increase in their willingness to consider enrolling in the pre-engineering program at the high school.

  15. Short-term Measurement Tools • Pre-post attitude assessment • Pre-post knowledge assessment • Evaluation instrument • Post activity survey

  16. Select Outcome Measures • Short-term • Increase in awareness • Increase in knowledge • Change in attitude • Change in behavior • Long-term • Enrollment • Achievement • Completion • Graduation • Transition

  17. Long-term Objectives • From FY09 to FY11, female enrollment in Ag 010101 will increase by 5 percentage points as measured by Overton Banner Data Collection System • The participation rate of girls in the Gateway to Technology course at the middle school will increase by 5 percentage points the year following the implementation of the Focus Your Future event as measured by Perkins reported enrollment data

  18. Long Term Measurement Tools Typically maintained by administration or institutional research office and reported to a funder, accrediting agency or system agency • Student level demographic data • Attendance records • Achievement records • Graduation records • Perkins core indicator data

  19. Develop a Outcome Evaluation Plan

  20. Write a goal and objective for your selected strategy Step 4 Worksheet

  21. Assessment Resources • STEM Equity Pipeline Webinars • May 21, 2008 Assessing Effectiveness: Do Your Program Activities Make a Difference? • June 16, 2008 Building Effective Program Assessments • Archived at www.stemequitypipeline.org • Assessing Women and Men in Engineering - www.aweonline.org

  22. Summary • Set realistic goals • Measure both process and outcomes • Develop short term and long term objectives • Check for any required data • Determine types of data needed— quantitative, qualitative, or both? • Decide on data collection instrument(s) • Develop an Evaluation Plan

  23. The Five Step Program Improvement Process

  24. Questions? Katherine Weber State Facilitator STEM Equity Pipeline Project www.stemequitypipeline.org kweber@napequity.org

More Related