slide1
Download
Skip this Video
Download Presentation
MODULE II

Loading in 2 Seconds...

play fullscreen
1 / 69

MODULE II - PowerPoint PPT Presentation


  • 78 Views
  • Uploaded on

MODULE II. Monitoring and Evaluating Gender-Based Violence Prevention and Mitigation Programs Anastasia J. Gage Melissa Dunn. Learning Objectives. By the end of this session, participants will be able to: differentiate between monitoring and evaluation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' MODULE II' - margarita-vargas


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

MODULE II

Monitoring and Evaluating Gender-Based Violence Prevention and Mitigation ProgramsAnastasia J. GageMelissa Dunn

learning objectives
Learning Objectives
  • By the end of this session, participants will be able to:
    • differentiate between monitoring and evaluation
    • write goals and smart objectives for gender-based violence (GBV) programs
    • design a logic model for a GBV program
    • identify criteria for indicator selection and information sources for GBV indicators
    • discuss factors to consider when choosing an evaluation design
at least 4 different categories of gbv interventions
At Least 4 Different Categories of GBV Interventions
  • Community mobilization
  • Behavior change communication
  • Service delivery
  • Laws and public policy
m e challenges of gbv programs 1
M&E Challenges of GBV Programs (1)
  • Hard to separate contribution of multiple strategies
  • Lack of investment in rigorous evaluation
    • limited evidence of effectiveness
  • Non-standardized GBV definitions and indicators
  • Enhancing comparability of data
  • Climate of urgency around results but behavior change is long-term
m e challenges of gbv programs 2
M&E Challenges of GBV Programs (2)
  • Sensitive nature of the information and hidden forms of violence (homicide, trafficking, female genital mutilation)
  • Changing violence levels hard to measure and interpret
  • Ethical and methodological issues
monitoring versus evaluation
Monitoring versus Evaluation

MONITORING =

  • Tracking changes in program performance over time

EVALUATION =

  • Assessing whether objectives have been met
  • Assessing extent to which program is responsible for observed changes
illustration of program monitoring
Illustration of Program Monitoring

PROGRAM INDICATOR

TIME 

Program

start

Program

end

illustration of program impact
Illustration of Program Impact

With program

CHANGE IN

PROGRAM OUTCOME

Without program

Program

impact

TIME 

Program

start

Program

end

activity 1 is it monitoring or evaluation
Activity 1: Is It Monitoring or Evaluation?
  • The Ministry of Women’s Affairs wants to know if programs carried out in Province A are reducing the prevalence of intimate partner violence (IPV).
  • USAID wants to know how many villages in Region B have been reached with anti-GBV messages by your program this year.
  • A country director is interested in finding out if the care provided to rape victims in public clinics meets national standards of quality.
purpose of monitoring and evaluation
Purpose of Monitoring and Evaluation

The purpose of monitoring and evaluation is to measure program effectiveness.

monitoring and evaluation questions
Monitoring and Evaluation Questions
  • Were resources made available to the program in the quantities and at the times specified by the program plan?
  • Were the program activities carried out as planned?
  • Which program activities were more effective and which were less effective?
  • Did the expected changes occur? How much change occurred?
  • Can improved health outcomes be attributed to program efforts?
  • Did the target population benefit from the program and at what cost?
slide12

M&E Across Program Life Cycle

ASSESSMENT

What is the nature of the (health) problem?

1

EVALUATION

How Do I know that the strategy is working? How do I judge if the intervention is making a difference?

STRATEGIC PLANNING

What primary objectives should my program pursue to address this problem?

2

5

4

3

3

IMPLEMENTATION/MONITORING

How do I know the activities are being implemented as designed? How much does implementation vary from site to site? How can be program become more efficient or effective?

DESIGN

What strategy, interventions and approaches should my program use to achieve these priorities?

activity 1 discussion key elements of m e plan
Activity 1 Discussion: Key Elements of M&E Plan
  • Do participants’ programs have M&E plans?
  • What are the components of the plans? What sections do the plans include?

Distribute Handout 1: Template for M&E Plan

  • What is missing from their programs’ M&E plans?
  • What additional components do their programs’ M&E plans include?
activity 2 identifying the problem
Activity 2: Identifying the Problem

Organize into 4 groups representing the different categories of GBV intervention:

  • community mobilization
  • behavior change
  • service delivery
  • laws and public policy

Review Handout 2: Gender-Based Violence Case Study

  • What is the problem from your group’s perspective?
  • What are 2 or 3 barriers your GBV intervention group might face when trying to help women like Marie?
program goals
Program Goals

A goal is a broad statement of a desired, long-term outcome of a program. Goals represent general big picture statements of desired results.

Examples:

  • To improve the lives of women who experience gender-based violence
  • To decrease injury and mortality due to gender-based violence
  • To eliminate gender-based violence
program objectives
Program Objectives

Objectives are statements of desired, specific, realisticand measurable program results

  • “SMART” acronym:
    • Specific: identifies concrete events or actions that will take place
    • Measurable: quantifies the amount of resources, activity, or change to be expended and achieved
    • Appropriate: logically relates to the overall problem statement and desired effects of the program
    • Realistic: provides a realistic dimension that can be achieved with the available resources and plans for implementation
    • Time-bound: specifies a time within which the objective will be achieved

Source: GAP 2003

activity 3 is it a goal or an objective
Activity 3: Is it a Goal or an Objective?
  • To reduce GBV
  • To increase the % of men and women in beneficiary population who believe that violence is not an acceptable way of dealing with conflict from 40% in 2002 to 80% by 2007
  • To increase the number of domestic violence clients recommended for counseling who utilize counseling services by 30%
activity 4 defining program goals and objectives
Activity 4: Defining Program Goals and Objectives
  • Return to small groups from previous activity
  • Develop one goal and two objectives for a program that will address problems identified in Handout 2 (GBV Case Study)
  • After 15 minutes, a member of each group will share the program goal and objectives
developing program strategies

Developing Program Strategies

Role of a Conceptual Framework

conceptual frameworks
Conceptual Frameworks

Diagram that identifies and illustrates the relationships between all relevant

systemic, organizational, individual, or other salient factors that may influence

program/project operation and the successful achievement of program or project

goals.

M&E purpose:

  • To show where program fits into wider context
  • To clarify assumptions about causal relationships
  • To show how program components will operate to influence outcomes
  • To guide identification of indicators
  • To guide impact analysis (causal pathways)
illustrative conceptual model for intimate partner violence ipv
Illustrative Conceptual Model for Intimate Partner Violence (IPV)

Individual perpetrator

Relationship

Society

Community

Society

  • norms
  • laws
  • notions of masculinity
  • gender roles
  • being male
  • witnessing marital violence as child
  • being abused as child
  • absent or rejecting father
  • alcohol use
  • poverty
  • unemployment
  • family isolation
  • community acceptance of violence
  • marital conflict
  • male control of wealth
  • male control of decision making in family

Source: Heise 1998

activity 5 developing program activities
Activity 5: Developing Program Activities
  • Return to small groups from previous activity
  • Develop 3 program activities for the goal and objectives that the group had formulated earlier to address the problems identified in the GBV case study
  • In 15 minutes, a member of each group will share the program strategies and activities
what is a logic model
What is a Logic Model?

Logic models are diagrams connecting program inputs to processes, outputs, outcome, and impact as they relate to a specific problem or situation. Logic models show what resources your program will need to accomplish its goals, what your program will do, and what it hopes to achieve, emphasizing the links between all these things.

logic model components
Logic Model Components
  • Input – Resources used in a program (e.g., money, staff, curricula, and materials)
  • Process – Activities conducted by program to accomplish its objectives (e.g., outreach, advocacy, training, BCC, etc.)
  • Output – Direct results of program activities (e.g., people trained, materials distributed, couples counseled, etc.)
  • Outcome – Program results at population level (e.g., knowledge, attitudes, behavior, coverage)
  • Impact – Long-term change in morbidity, mortality, and fertility
activity 6 identifying logic model components
Activity 6: Identifying Logic Model Components
  • Distribute Handout 5
  • Each of the scenarios in the handout corresponds to one of the five components in the logic model: Input, process, output, outcome, and impact
  • As a group, decide which component the scenario illustrates
handout 6 illustrative logic model for provider training program
Handout 6: Illustrative Logic Model for Provider Training Program

INPUT

Human and financial resources to develop training materials & implement training program

  • PROCESS
  • Develop GBV clinical training curriculum
  • Conduct training-of-trainers workshops
  • Conduct GBV training for providers
  • OUTPUT
  • Providers trained in GBV
  • Improved provider attitudes toward GBV
  • Increased provider ability to identify, counsel, care for, and refer GBV victims
  • OUTCOME
  • Increased awareness of GBV as a health issue
  • Increased disclosure of GBV
  • Increased knowledge and utilization of GBV services

IMPACT

Improved health and safety of GBV victims

role of logic model
Role of Logic Model

Program planning

  • Helps one think through program strategy – where you are and where you want to be

Program management

  • Helps one track and monitor operations to better manage results

Communication

  • Shows stakeholders at a glance what the program is doing and what it expects to achieve
role of logic model1
Role of Logic Model

Consensus building

  • Promotes common understanding about what program is, how it works, and what it is trying to achieve

Fundraising

  • Can structure and streamline grant writing

Monitoring and evaluation

  • Provides thorough understanding of what resources a program has to work with, what it is doing, and what it hopes to achieve.
activity 7 developing a logic model
Activity 7: Developing a Logic Model
  • Return to small groups from previous activities
  • Develop a logic model for two of the activities of your program
  • After 20 minutes, a member of each group will share the logic model
results framework
Results Framework

Diagram that identifies steps or levels of results and illustrates the causal relationships linking all levels of a program’s objectives

Purposes

    • Provides a clarified focus on the causal relationships that connect incremental achievement of results to the comprehensive program impact
    • Clarifies project/program mechanics and factors’ relationships that suggest ways and means of objectively measuring the achievement of desired ends

Other terms used

  • Strategic framework
slide34

Illustrative Results Frameworks

SO1: IncreasedUtilization of Reproductive Health Services

IR2 Expansion of high-quality health services for GBV victims in the public and private sectors

IR1 Strengthened sustainability of GBV programs

IR1.1 Improved policy environment for GBV prevention and response

IR2.1 Increased availability of GBV screening and referral

IR2.2 Improved provider attitude towards GBV

IR1.2 Strengthened NGO advocacy for GBV prevention

IR2.3 Improved provider competence in GBV screening, referral, care, & counseling

activity 8 develop monitoring and evaluation questions
Activity 8: Develop Monitoring and Evaluation Questions
  • Return to your previous group activities where you developed your program goals and objectives and logic model
  • Come up with at least 1 monitoring question and 1 evaluation question
  • After 10 minutes, a member of the group will share the M&E questions
slide37

What is an Indicator?

An indicator is a specific, observable, and measurable characteristic that can be used to show changes or progress a program is making toward achieving a specific outcome.

characteristics of good indicators
Characteristics of Good Indicators
  • Valid: accurate measure of a behavior, practice or task
  • Reliable: consistently measurable in the same way by different observers
  • Precise: operationally defined in clear terms
  • Measurable: quantifiable using available tools and methods
  • Timely: provides a measurement at time intervals relevant and appropriate in terms of program goals and activities
  • Programmatically important: linked to the program or to achieving the program objectives that are needed for public health impact
characteristics of good indicators cont
Characteristics of Good Indicators (cont.)

Indicators do not specify a particular level of achievement — the words “improved,” “increased,” or “decreased” do not belong in an indicator.

common indicator metrics
Common Indicator Metrics

Counts

  • no. of legal and service organizations for violence against women and girls (VAW/G) survivors
  • no. women and children using VAW/G social welfare services

Calculations: percentages, rates, ratios

  • % of health units with at least one service provider trained to care for and refer survivors
  • % of women aged 15-49 who have ever experienced physical violence from an intimate partner

Index, composite measures

  • gender-equitable men (GEM) scale
  • sexual relationship power scale

Thresholds

presence, absence; pre-determined level or standard

common challenges in indicator selection
Common Challenges in Indicator Selection
  • Choosing an indicator that program activities cannot affect
  • Choosing an indicator that is too vague
  • Indicators that do not currently exist and cannot realistically be collected
  • Selecting an indicator that does not accurately represent the desired outcome
  • Too many indicators
how many indicators are enough
How Many Indicators Are Enough?
  • At least one or two indicators per key activity or result (ideally, from different data sources)
  • At least one indicator for every core activity (e.g., training, BCC)
  • No more than 8-10 indicators per area of significant program focus
  • Use a mix of data collection strategies/source
factors to consider when selecting indicators
Factors to Consider When Selecting Indicators
  • Logic/link to framework
  • Programmatic needs/information for decision making
  • Data availability
  • Resources
  • External requirements (government, donor, headquarters)
  • Standardized indicators (if available)
activity 9 selecting indicators
Activity 9: Selecting Indicators
  • Return to small groups from previous activities where you developed goals, objectives, activities, and M&E questions
  • Select 3 indicators that your group might use to measure progress towards your program goals and objectives
  • Into which logic model component does each indicator fall?
  • After 15 minutes, a member of each group will share the indicators
activity 10 selecting indicators cont d
Activity 10: Selecting Indicators Cont’d

As a class, look at the indicators that the groups selected in the previous activity

Discuss the following:

  • To what extent are the indicators logically linked to the group’s M&E framework?
  • Do the proposed indicators share the characteristics of good indicators?
  • How will the indicators be used for program decision-making and for what decisions?
  • Are data available to measure the indicators?
  • Can the indicators be realistically collected given available resources?
  • Are there government or donor requirements for measuring the proposed indicators?
where to find standardized indicators for gbv programs
Where to Find Standardized Indicators for GBV Programs

Bloom, Shelah S. Violence Against Women and Girls: A Compendium of Monitoring and Evaluation Indicators (MS-08-30). Chapel Hill, NC: MEASURE Evaluation, Carolina Population Center, University of North Carolina and Chapel Hill; 2008.

http://www.cpc.unc.edu/measure/publications

United Nations Division for the Advancement of Women, United Nations Economic Commission for Europe, United Nations Statistical Division. Indicators to Measure Violence against Women. Report of the Expert Group Meeting, 8 to 10 October 2007. Geneva: United Nations Human Rights Council; 2008.

Report of the special rapporteur on violence against women, its causes and consequences, by Yakin Ertürk. Indicators on violence against women and state response. Advance edited version 28 January 2008.

illustrative indicators community mobilization
Illustrative Indicators: Community Mobilization
  • Proportion of people who would assist a woman being beaten by her husband or partner
  • Proportion of people who say that wife beating is an acceptable way for husbands to discipline their wives
  • Proportion of people who agree that rape can take place between a man and woman who are married
  • Proportion of youth-serving organizations that include training for beneficiaries on sexual and physical VAW/G
illustrative indicators behavior change communication
Illustrative Indicators: Behavior Change Communication
  • Proportion of people who have been exposed to VAW/G prevention messages
  • Proportion of girls who say that they would be willing to report any experience of unwanted sexual activity
  • Proportion of girls that feel able to say no to sexual activity
  • Proportion of individuals who know any of the legal sanctions for VAW/G
illustrative indicators service delivery
Illustrative Indicators: Service Delivery
  • Proportion of health units that have documented and adopted a protocol for the clinical management of VAW/G survivors
  • Proportion of health units with at least one service provider trained to care for and refer VAW/G survivors
  • Proportion of women who were asked about physical and sexual violence during a visit to the health unit
  • Proportion of rape survivors who received comprehensive care
illustrative indicators law and public policy
Illustrative Indicators: Law and Public Policy
  • Proportion of law enforcement units following a nationally established protocol for VAW/G complaints
  • No. of law enforcement professionals trained to respond to incidents of VAW/G according to an established protocol
  • No. of VAW/G complaints reported to the police
  • Proportion of VAW/G cases that were investigated by the police
measuring indicators
Measuring Indicators
  • Indicator 1:No. of service providers trained (in the past year or other period) to identify, refer, and care for VAW/G survivors
  • “providers” include any clinician providing direct clinical services to clients at public health facilities
  • “trained” refers to attending every day of a five-day training course on VAW/G
measuring indicators1
Measuring Indicators
  • Indicator 2: % of women who were asked about physical and sexual violence during a visit to a health unit
  • Numerator: Number of women who were asked, during the course of their service provision at the health unit, about any violence that had ever occurred, either physical or sexual, in the geographic area of study (nation, province, state, community)
  • Denominator:
    • If record review, total number of women’s records that were reviewed at the health unit
    • If exit interview, total number of women interviewed
activity 11 operational definition of indicators
Activity 11: Operational Definition of Indicators

Distribute Handout 8

Organize into 4 groups

Write an operational definition for one of the following indicators:

  • intimate partner physical violence
  • appropriate care for rape survivors
  • support-seeking behavior of GBV victims
  • gender-norm related attitudes that reinforce VAW/G

After 15 minutes, a member of each group will share the operational definitions

measuring composite indicators
Measuring Composite Indicators
  • Example: How to measure “attitudes toward GBV”?
  • What question do you ask?
  • How do you know any one question is a good representation of general attitude?
  • How do you scale each question (agree/disagree; strongly disagree to strongly agree; yes/no)?
  • How do you combine answers to the questions to come up with one indicator?
  • What value or weight do you assign to any one question on GBV attitude in the survey?
example of measuring attitudes toward wife beating
Example of Measuring Attitudes Toward Wife Beating

Sometimes a man is annoyed or angered by things that his wife does. In your opinion, is a man justified in hitting or beating his wife in the following situations:

  • she goes out without telling him
  • she neglects the children
  • she argues with him
  • she refuses to have sex with him
  • she burns the food
activity 12 specifying indicator metrics
Activity 12: Specifying Indicator Metrics
  • Return to small groups from previous activities
  • Look at the indicators that your group has selected to measure progress towards your goals and objectives
  • Define the metrics for each indicator
indicator reference sheets
Indicator Reference Sheets

Distribute Handout 9

setting indicator targets useful information sources
Setting Indicator Targets: Useful Information Sources
  • past trends
  • client expectations
  • donor expectations
  • expert opinion
  • research findings
  • what has been accomplished elsewhere
activity 13 setting indicator targets based on past trends
Activity 13: Setting Indicator Targets Based on Past Trends

One of the objectives of the Ministry of Women’s Affairs in Country X is to reduce the prevalence of intimate partner violence. The 2000 DHS showed that 30% of ever-married women aged 15-49 years in the country had ever experienced intimate partner emotional, sexual, or physical violence. A comparable DHS conducted in 2005-2006 showed that the prevalence of intimate partner violence (emotional, physical, or sexual) was 25% among ever-married women aged 15-49. Based on these trends, what would be a realistic national target for this indicator in 2010?

types of information sources
Types of Information Sources

Quantitative data

  • useful for tracking trends accurately and highlighting differences

Qualitative data

  • useful for understanding the context in which the trends/differences occur and to interpret quantitative data accurately (e.g., focus groups, in-depth interviews)
quantitative information sources
Quantitative Information Sources

Population-based surveys

  • DHS (domestic violence module), VAW/G survey,

facility surveys

  • Client exit interviews; provider surveys; clinic observations

Health service statistics

Program statistics

Crime statistics (of poor quality in many LDCs)

Behavioral surveillance systems

triangulating data sources soul city
Triangulating Data Sources: Soul City

Community

6. Cost – Effectiveness Study

Individual

1. National Survey

2. Sentinel Site Studies

3. (National) Qualitative Impact Assessment

Society

4. NNVAW Partnership Study

5. Media Monitoring

and Analysis

activity 14 identifying information sources for indicators
Activity 14: Identifying Information Sources for Indicators
  • Return to small groups from previous activities
  • Look at the indicators that your group has selected to measure progress towards your program goals and objectives
  • For each indicator, identify the potential source of data needed for its calculation
  • What challenges might you face in using these potential information sources?
  • After 10 minutes, a member of the group will present
guiding questions for choosing an evaluation design
Guiding Questions for Choosing an Evaluation Design
  • What question does your program need to answer?
  • What do you want to measure (indicators)?
  • How sure to you want to be? (VERY SURE = GOOD )
  • What is the cost of making a mistake (low, medium, high)?
  • When do you need the results? (FAST)
  • How much are you willing to pay? (CHEAP)
  • Has the program already started?
choosing an evaluation design
Choosing an Evaluation Design
  • GOOD, FAST, CHEAP:
  • PICK 2!
  • HIRE SOMEONE TO HELP YOU OUT!
  • NO EVALUATION DESIGN IS IMMUNE TO PROBLEMS!
slide67

When monitoring and evaluating GBV programs, the confidentiality, privacy, safety, and well-being of GBV survivors must be top priority at all times.

references
References
  • Bloom, SS. Violence Against Women and Girls: A Compendium of Monitoring and Evaluation Indicators (MS-08-30). Chapel Hill, NC: MEASURE Evaluation, Carolina Population Center, the University of North Carolina and Chapel Hill; 2008.
  • Bott S, Guedes A, Claramunt MC, Guezmes A. Improving the Health Sector Response to Gender-Based Violence: A Resource Manual for Health Care Professionals in Developing Countries. New York: International Planned Parenthood Federation, Western Hemisphere Region; 2004.
  • Interagency Gender Working Group. Addressing Gender-Based Violence from the Reproductive Health/HIV Sector: A Literature Review. Washington: The Population Technical Assistance Project (POPTECH); 2004.
references continued
References (continued)
  • Soul City 4. Impact Evaluation; Violence Against Women, Vols. I and II. 2001.
  • World Health Organization (WHO). WHO Ethical and Safety Recommendations for Researching, Documenting and Monitoring Sexual Violence in Emergencies. Geneva: World Health Organization; 2007.

http://www.who.int/hac/network/interagency/news/ethical_and_safety_recommendations/en/index.html

ad