note to evaluator
Download
Skip this Video
Download Presentation
Note to evaluator…

Loading in 2 Seconds...

play fullscreen
1 / 32

Note to evaluator… - PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on

Note to evaluator…. The overall purpose of this presentation is to guide evaluators through the completion of steps 4 to 6 of the UFE checklist. The main goal is to suggest a process that can help UFE evaluators facilitate the formulation of key evaluation questions.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Note to evaluator…' - megan-douglas


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
note to evaluator

Note to evaluator…

The overall purpose of this presentation is to guide evaluators through the completion of steps 4 to 6 of the UFE checklist. The main goal is to suggest a process that can help UFE evaluators facilitate the formulation of key evaluation questions.

Please adapt this presentation to the context of the project that you are evaluating and to your facilitation style.

slide2
Facilitating UFE step-by-step:

a process guide for evaluators

Module 2: Steps 4-6 of UFE checklist

Joaquín Navas & Ricardo Ramírez

December, 2009

meeting s objectives

Meeting’s objectives

To review the most relevant topics of previous meeting(s) – comments on distributed report(s).

To formulate the first draft of the key evaluation questions.

agenda

Agenda

Summary and brief discussion on the previous meeting and/or report.

Group reflection on some of the discussed topics.

Break.

Formulation of key evaluation questions – first draft.

what have we achieved so far

What have we achieved so far?

Common understanding among participants on basic UFE principles.

Stakeholder identification within the overall project.

Role definition within the evaluation process.

Identification of possible challenges of the evaluation process.

review basic ufe principles

Review – Basic UFE principles

UFE is a PROCESS for helping primary intended users select the most appropriate content, model, methods, theory, and uses for their particular situation.

UFE is a COLLABORATIVE APPROACH that seeks to generate learning.

Evaluation should be JUDGED by its utility and USE in the real world.

Evaluation plan needs to be part of the INITIAL DESIGN of project.

The evaluator’s role is to COLLABORATE with those engaged in the design of the evaluation process.

group reflection available resources

Group reflection – Available resources…

In the previous meeting you identified x, y, z, as the most limiting resources or factors of the evaluation process. Have you identified any other limiting resources/factors since that meeting?

¿Do you still think that the available resources are enough (or not enough) to carry out the evaluation process?

group reflection stakeholders 1 4

Group reflection – Stakeholders (1/4)…

From the stakeholder groups that you identified in the previous meeting(s), what group(s) do you think you represent as primary intended users of the evaluation?

group reflection stakeholders 3 4

Group reflection – stakeholders (3/4)…

What key stakeholder groups are not represented by the primary intended user?

Would there be any implications on the use of the evaluation as a result of not having these stakeholder groups represented by the primary intended users?

What would those implications be?

group reflection stakeholders 4 4

Group reflection – Stakeholders (4/4)…

Can you think of any political factors within the project that could affect the use of the evaluation?

group reflection previous experience 1 3

Group reflection – previous experience (1/3)

In previous meeting(s) you made the following comments regarding your previous evaluation experiences: X, Y, Z. What did you learn from those experiences?

group reflection previous experience 2 3

Group reflection – previous experience (2/3)

Suppose that at the beginning of this project you had the required resources and total freedom for implementing or not implementing a formal evaluation plan. What factors would have discouraged you about implementing the evaluation plan? Why?

group reflection previous experience 3 3

Group reflection – previous experience (3/3)

What would have motivated you to implement the evaluation plan? Why?

group reflection evaluation utility 1 3

Group reflection – evaluation utility (1/3)

Do you think that this evaluation process can contribute to program improvement? How?

reflexi n evaluation utility 2 3

Reflexión – evaluation utility (2/3)…

Do you think that this evaluation process can contibute to making major decisions? How?

group reflection evaluation utility 3 3

Group reflection – evaluation utility (3/3)

Do you think that this evaluation process can contribute to the generation of knowledge? How?

group reflection critical dates

Group reflection – critical dates…

What are your expectations regarding the completion date of this evaluation?

Can you think of any milestone dates that could be critical or useful to have in mind for major decision-making throughout the project subject of this evaluation?

the trajectory of change

The trajectory of change…

CONTROL

&

PREDICTION

INPUT / RESOURCES

ACTIVITIES

OUPUTS

OUTCOMES

IMPACT / RESULTS

?

the evaluation s purpose

The evaluation’s purpose

According to Patton (2008) evaluation can be oriented towards different purposes based on the findings’ primary intended USES.

menu of intended uses 1 2

MENU OF INTENDED USES (1/2)

Evaluation Purposes

Primary intended uses

Typical primary users

Overall summativejudgment

“To provide data for judging the overall value of a program and deciding whether it is worth continuing with it or not” (p. 114).

Those charged with making major decisions: funders, directors, other adopters of model, etc.

Program administrators, staff, those involved in the day-to-day management.

“To provide data for program improvement” (p. 116).

Formative improvement & learning

Program designers, planners, modelers, theorists, scholars, policy-makers.

“To look across findings from different programs to identify patterns of effectiveness” (p. 131).

Knowledge generating

monitoring
MENU OF ITENDED USES (2/2)

Purposes

Typical primary users

Intended primary uses

Organization / Program Development

“To provide data for adapting interventions to emergent conditions” (p. 137).

Social innovators, those involved in bringing about major systems change in dynamic environments.

Monitoring

“To provide data for describing and explaining achievements” (p. 121).

Those with administrative and funding authority, responsible for resource use.

Accountability

Program managers responsible for internal accountability and information system management (Adaptaded from Patton p. 139 – Ch. .4).

“To provide information about key areas that require managerial attention” (Pg 126).

what makes good keqs adapted from dart 2007

What makes good KEQs? (adapted from Dart, 2007)

Specific enough to be useful in guiding you through the evaluation

Broad enough to be broken down - are not the same as a question in a survey

Data (qualitative/quantitative) can be brought to bear on the KEQ

KEQs are open questions (can’t answer yes or no!)

Have meaning for those developing the plan

Lead to useful, credible, evaluation

There aren’t too many of them (2-4 is enough).

categories of key evaluation questions

Categories of key evaluation questions

INPUT / RESOURCES

IMPACT

OUTCOMES

APPROACH / MODEL

PROCESS

QUALITY

COST- EFFECTIVENESS

linking keq to the project objectives

Linking KEQ to the project objectives

List the specific project objectives here…

references

References

Dart, J. 2007. “Key evaluation questions”. Presentation at the Evaluation in Practice Workshop. Kualal Lumpur, December. http://evaluationinpractice.files.wordpress.com/2008/01/keyquestionschoices.pdf

Patton, M.Q. (2008) Utilization focused evaluation, 4th Edition. Sage.

ad