1 / 27

Program Evaluation How to Effectively Evaluate Your Program

Program Evaluation How to Effectively Evaluate Your Program. Carol Pilcher Department of Entomology Iowa State University. Logic Model University of Wisconsin-Extension. Purpose of Logic Model. Visual display to connect inputs, outputs and outcomes “Helps people get on the same page”

norman-garrison
Download Presentation

Program Evaluation How to Effectively Evaluate Your Program

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Program EvaluationHow to Effectively Evaluate Your Program Carol Pilcher Department of Entomology Iowa State University

  2. Logic ModelUniversity of Wisconsin-Extension

  3. Purpose of Logic Model • Visual display to connect inputs, outputs and outcomes “Helps people get on the same page” “Describes a program and why it works” “Roadmap to help you know where you are going”

  4. Inputs Outputs Outcomes-Impact What the Short Term Results Are What the Medium Term Results Are What We Invest What We Do Who We Reach What the Ultimate Impact Is Priorities Situation Assumptions External Factors Logic Model Adapted From University of Wisconsin-Extension

  5. Situation • What is the problem or issue? • Background information, grant proposal • Why is this a problem? • Background information, grant proposal • Who has a stake in the problem? • Identify stakeholders

  6. Priorities • Determine the focus of your program • Establish priorities based on: • Resources • Experience • Situation • Objectives and Timeline, grant proposal

  7. Inputs Outputs Outcomes-Impact What the Short Term Results Are What the Medium Term Results Are What We Invest What We Do Who We Reach What the Ultimate Impact Is PRIORITIES SITUATION Assumptions External Factors Logic Model

  8. What resources will you invest? Staff Number of staff and hours available Time Hours reported Money Allocated funds Inputs

  9. Inputs Outputs Outcomes-Impact What the Short Term Results Are What the Medium Term Results Are What We Invest What We Do Who We Reach What the Ultimate Impact Is PRIORITIES SITUATION Assumptions External Factors Logic Model

  10. Outputs-Activities • What will you do? • Conduct workshops, meetings • Deliver services • Develop products, resources

  11. Outputs-Activities • Examples Thirty-nine food safety programs were held including hand-washing demonstrations and food safety during the RAGBRAI intra-state cycling trek Starting with a pilot of 4 districts during the 2001-2002 school year, the School Integrated Pest Management Program has trained more than 30 districts statewide.

  12. Output-Participation • Who will you reach? • Participants • Clients • Agencies • Decision makers • Customers

  13. Output-Participation • Examples • 5,982 individuals were involved in parent education programs • Attendance at Crop Advantage Meetings • 2002—933 • 2003—720 • 2004—1735

  14. Output-Reactions • Reactions about course materials • Reactions about program

  15. Output-Reactions • Example Of the participants that completed the survey (n=120), 98% indicated that the information presented was useful for their farm operation

  16. Inputs Outputs Outcomes-Impact What the Short Term Results Are What the Medium Term Results Are What We Invest What We Do Who We Reach What the Ultimate Impact Is PRIORITIES SITUATION Assumptions External Factors Logic Model

  17. Outcomes-ImpactsShort Term Results • Amount of learning • How much was learned during the activity? • KASA • Knowledge, Attitudes, Skills, Awareness

  18. Outcomes-ImpactsKASA • Example Of those who returned evaluation surveys, 78% reported moderate to significantly improved understanding of the factors affecting groundwater and surface water contamination of pesticides.

  19. Outcomes-ImpactsMedium Term Results • Changes in actions • Behavioral changes • Changes in practices • Changes in decision making

  20. Outcomes-ImpactsChange in Actions • Example • Year One • As a result of today’s training, will you use a drift-reduction nozzle in your sprayer to assist in reducing drift • Adopted Prior To Training, • Plan to Adopt, • Choose Not to Adopt, • Does Not Apply • Year Two • As a result of last year’s program, do you use a drift-reduction nozzle in your sprayer to assist in reducing drift • I Adopted • I Did Not Adopt • Does Not Apply

  21. Outcomes-ImpactsChange in Actions • Examples As a result of the program, 50% of the participants said they would use drift reduction nozzles in their sprayers to reduce drift. This post-training evaluation examined if participants had indeed successfully implemented new pesticide safety activities. According to the respondents, 85% of the participants said that they now use drift reduction nozzles in their sprayers. Ninety-eight percent of the food safety program participants reported adopting one of more of the recommended food handling practices.

  22. Outcomes-ImpactsUltimate Impacts • Changes in conditions • Social/Health • Environment • Economic

  23. Outcomes-ImpactsChanges in economic conditions • Example Approximately, 4.3 million acres were managed with the IPM practices of improved disease resistant varieties, use of safe and effective fungicides as recommended by a forecasting model, and use of the wheat midge risk map and midge scouting. A total of $105.7 million was gained by producers using these IPM practices in 2003.

  24. Outcomes-ImpactsUltimate Impacts

  25. Inputs Outputs Outcomes-Impact What the Short Term Results Are What the Medium Term Results Are What We Invest What We Do Who We Reach What the Ultimate Impact Is PRIORITIES SITUATION Assumptions External Factors Logic Model

  26. Evaluation Answers the “so what” question. What difference does your program make for the participants? Determines the worth of a program.

More Related