Evaluation based program development
This presentation is the property of its rightful owner.
Sponsored Links
1 / 35

Evaluation Based Program Development PowerPoint PPT Presentation


  • 58 Views
  • Uploaded on
  • Presentation posted in: General

Evaluation Based Program Development. This material was developed in partnership with Rutgers's University and Lydia B. Blalock, Youth Development Specialist, New Brunswick, New Jersey. Why Evaluate?. I dunno I think I knew why–once They make me do it Department chair

Download Presentation

Evaluation Based Program Development

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Evaluation based program development

Evaluation Based Program Development

This material was developed in partnership with Rutgers's University and

Lydia B. Blalock, Youth Development Specialist, New Brunswick, New Jersey


Evaluation based program development

Why Evaluate?

  • I dunno

  • I think I knew why–once

  • They make me do it

    • Department chair

    • Funding organization

    • My mom

  • I don’t

  • Demonstrate success

  • Understand failure

  • Provide feedback for program improvement

    • Process

    • Outcome

  • Nip problems in the bud


Process objectives

Process Objectives

If I:

  • Develop and presentseminar on EBPD

  • Develop and distribute appropriate handouts

  • Engage participants in group activities

  • Provide participants an opportunityto practice

  • Encouragequestions

    THEN


Outcome objectives

Outcome Objectives

Participants will:

  • Program Development

    • Understand how to clearly define issue program will address.

    • Use program theory to plan activities.

    • Write clear outcome and process objectives.

    • Use EBPD Model to plan programs.

  • Program Evaluation

    • Develop skills to evaluate programs.

    • Select appropriate evaluation methods and designs.

    • Select criteria to evaluate objectives.

    • Use EBPD Model to plan evaluation.


But first is it a p rogram or a p rogram

BUT FIRST: Is it a Program or a program

  • Stakeholder leadership

  • Faculty leadership

  • Multi-disciplinary

  • Sustainable

  • Academic department

  • Student component

  • Research, teaching, outreach


The steps of evaluation based program development

The Steps of Evaluation-Based Program Development


Step 1 define the problem

Step 1: Define the Problem

  • Describe Current Situation

    • How does the world look today?

  • Why a Problem

    • Clearly articulate why a problem

    • Ask why until get to root of problem

    • Symptom vs. Cause

  • Identify Stakeholders

    • Population: Who is affected? How many?

    • Stakeholder Leadership: interest in solving problem?

    • Potential Collaborators: Already working on problem?

  • How: Evidence? Learn more? References & Facts

    • Learn all you can about the issue/problem –


Evaluation based program development

  • Set Priorities

    • Factors that influence priorities

      • Difference between actual and target states

      • Degree of difficulty in addressing needs

      • Cost of implementing solutions

      • Political and other factors

    • Derive priorities from several data sources

    • Priorities without resources

  • Select Solution Strategies to Meet Needs

    • Consider alternative solutions for each need

    • Evaluate alternatives

    • Select solutions to implement

    • Scholarship Opportunities?


Talk it over

TALK IT OVER

Fill Out Table 1 with your group.


Step 2 develop outcome objectives

Step 2: Develop Outcome Objectives

  • Describe Ideal Situation

    • How would world look w/o problem (perfect world)?

  • Define Target Audience/s

    • Who/what can we change (youth, families, policies, etc.)?

    • What do we know about culture/s of audience?

  • Identify Outcome Objectives

    • What changes can I influence in my target audience/s?

  • Identify Resources

    • What do we need?

    • What do we have?


Step 2 develop outcome objectives1

Step 2. Develop Outcome Objectives

Ï


Writing measurable objectives

Writing Measurable Objectives

  • Specific

    • Will increase knowledge vs. Will know 3 sources of Vitamin C

  • Time delineated

    • At the end of the program, in 6 weeks, etc.

  • Measurable

    • 75%, all, etc.

  • Example

    • Participants will increase knowledge of nutrition.

    • 75% of program participants will know 3 sources of Vitamin C 6 weeks after the program.


Table 2 clarifying your projected impact

Table 2: Clarifying Your Projected Impact


Talk it over1

Talk it Over

Fill out Table 2 with your group.

#4: Brainstorm Resources


Step 3 plan program

Step 3. Plan Program

Use “Program Theory” to develop process objectives! IF:THEN


Program theory the key to un lock ing change

Program Theory: The Key  to Unlocking  Change


Output the process of an outcome wha

Output the Process of an Outcome …wha…?

Outcome Objectives: the changes you will influence in your targetaudience (Learning, Action, Condition)

Process Objectives: the events, programs, activities you will conduct, deliver, develop, train,provide, access, facilitate to reach your outcome objectives

Program Outputs: your deliverables

Outcome & Process Objective Activity


Talk it over2

Talk it Over

Fill out table 3 with your group


So what

SO WHAT?

Evaluation

Friend to Groucho Marx: “Life is difficult!” 

Marx to Friend: “Compared to what?”


Why we don t evaluate

Why We Don’t Evaluate

  • Too hard

  • Not qualified

  • Costs too much

  • Takes too much time

  • Don’t think about it “in time”

  • No one ever reads evaluation reports

  • Everyone “knows” it’s a good program


Step 4 program evaluation

Step 4: Program Evaluation

Select Evaluation Design

Select Data Collection Method


Evaluation based program development

One-Shot

Data collected following program,

Retrospective Pretest

Data collected following program; participants recall behavior prior to and following program.

One Group Pretest-Posttest

- Data collected prior to

and following program

Time Series

Data collected prior to, during, after program

Evaluation Design 101

  • Pretest-Posttest Control-Group

    • Data collected from 2 different groups prior toandfollowing program.

    • Treatment group: program

    • Control group: no program

  • Post-Only Control-Group

    • Data from 2 different groups collected following program only


Evaluation design 101

Evaluation Design 101


Data collection methods

Data Collection Methods

  • Face to Face Surveys

    • Respondent completes survey in presence of evaluator

    • Evaluator may record responses and is able to answer questions

  • Self Administered Questionnaires

    • Respondent given survey to complete on his/her own.

    • Respondent’s responsibility to return survey.

  • Personal Interviews

    • Respondent asked pre-determined questions.

    • Evaluator records responses, can ask respondent to clarify answers.

  • Focus Groups

    • Several respondents meet with evaluator.

    • Evaluator facilitates discussion around pre-determined questions.

    • Discussion recorded.


Learned anything let s check

Learned Anything? Let’s Check!

Surprise Exam!!!


Learning action condition objectives

Learning, Action & Condition Objectives


Think it over

Think it Over

Fill out Table 4 with your group.

Learning Indicators

Action Indicators

Condition Indicators


Cultural aspects of ebpd

Cultural Aspects of EBPD

  • What is culture?

    • “Behaviors and beliefs characteristic of a specific social, ethnic, or age group.”

    • We are a product of our culture

    • Culture influences (unconsciously) what we think of people from other cultures.

    • Recognize cultural biases towards participants and strive to eliminate them.

  • Be aware of how differences between cultures may impact EBPD

  • Neutralize cultural biases and prevent EBPD design flaws:

    • Know your audience (Step 2 – Develop Outcome Objects)

    • Develop cross-cultural efficacy!


Culturally sensitive design

Culturally Sensitive Design


Reporting scholarship

Reporting & Scholarship


Process objectives1

Process Objectives

If I, as facilitator:

  • Develop and presentseminar on EBPD

  • Develop and distribute appropriate handouts

  • Engage participants in group activities

  • Provide participants an opportunityto practice

  • Encouragequestions

    THEN


Outcome objectives1

Outcome Objectives

Participants will:

  • Program Development

    • Understand how to clearly define issue program will address.

    • Use program theory to plan activities.

    • Write clear outcome and process objectives.

    • Use EBPD Model to plan programs.

  • Program Evaluation

    • Develop skills to evaluate programs.

    • Select appropriate evaluation methods and designs.

    • Select criteria to evaluate objectives.

    • Use EBPD Model to plan evaluation.


Condition changes

Condition Changes

We’ll report stunning impacts from well planned programs

And the folks in Pullman will love us the most

And people will dump bucket loads of money into our programs

And the world will be a better place!


  • Login