Targeted Evaluation of Five Programs
1 / 23

Targeted Evaluation of Five Programs Supporting Orphans and Vulnerable Children: Background and Methods - PowerPoint PPT Presentation

  • Uploaded on

Targeted Evaluation of Five Programs Supporting Orphans and Vulnerable Children: Background and Methods. Florence Nyangara, PhD MEASURE Evaluation/Futures Group Dissemination Meeting, September 3 rd , 2009 Washington, DC. The number of OVC and their corresponding programs increasing.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Targeted Evaluation of Five Programs Supporting Orphans and Vulnerable Children: Background and Methods' - philena

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

Targeted Evaluation of Five Programs Supporting Orphans and Vulnerable Children:Background and Methods

Florence Nyangara, PhD

MEASURE Evaluation/Futures Group

Dissemination Meeting, September 3rd, 2009

Washington, DC

The number of ovc and their corresponding programs increasing l.jpg
The number of OVC and their corresponding programs increasing


  • SSA - 12 million orphans (0-17 yrs), 2003

  • 2010 - over 18 million orphans (O)

  • Other millions are made vulnerable – HIV/AIDS, dire poverty, war, etc (V)

  • Response – increased attention to the plight of OVC (funds, programs)

Sub-Saharan Africa’s population of children orphaned by AIDS increasing

Children on the Brink, 2004

Response to the ovc crisis l.jpg
Response to the OVC Crisis increasing

OVC programs – emergency response to areas most HIV-affected

Strategies used were based on existing cultural support systems, conventional wisdom, and lessons learned from other program areas;

Support community-based responses (capacity & resources)

Household/family support (capacity & resources)

Direct support to families & OVC (access to essential services)

Gap – lack of evidence to guide OVC programs

Call for evidence based programming

2006 - USAID funded MEASURE evaluation to conduct targeted evaluations to fill this evidence gap


Evaluation goals l.jpg
Evaluation Goals increasing

  • Find out “what works” in terms of

    • intervention models and program components

    • cost effectiveness, and

    • outcomes (benefits) for OVC and their caregivers in resource poor settings

  • Provide evidence to guide program decisions such as;

    • Scaling-up of best practices (models, strategies), and

    • Modify & improve interventions - to make them effective

Research preparation activities l.jpg
Research Preparation Activities increasing

Funds were made available – USAID/PEPFAR/PHE (4) and USAID/Tanzania mission funded (1) program evaluation.

Research team formed - MEASURE Evaluation

Extensive literature reviews (early 2006)

Consultation meetings with stakeholders

Identified OVC programs to be evaluated

Research protocol developed

Ethical approvals obtained – US, Kenya, and Tanzania

Identified local research partners (PSRI – KE; AXIOS - TZ)

Programs evaluated l.jpg
Programs Evaluated increasing

Evaluated Five programs : 2 in Kenya & 3 in Tanzania

They have different intervention models with varied combinations of child, family/household, and/or community centered approaches (multi-faceted).

*** Although, the approaches vary, the goal for all of these programs is to improve the well-being of OVC and their families.

Programs evaluated7 l.jpg
Programs Evaluated increasing

Kenya (2)

Kilifi OVC Project,Catholic Relief Services (CRS)

Operating in Kilifi District for two years

Community Based HIV/AIDS Prevention, Care and Support Program (COPHIA), Pathfinder & Integrated AIDS Program (IAP)

Operating in Thika District for 4 years

Tanzania (3)

CARE Tumaini Project, Allamano, CARE, FHI (Allamano)

Operating in Iringa Region for five years

Mama Mkubwa & Kids Club, Salvation Army (TSA)

Operating in Mbeya Region for 2-years

Jali Watoto, Pact/SAWAKA (Jali Watoto) – Field funded

Operating in Karagwe, Kagera Region for four years

Key research questions l.jpg
Key Research Questions increasing

Impact of indirect support:

How do efforts targeted at the structural systems surrounding children– household and community– affect:

Children well-being

Caregivers well-being

Community attitudes and support of OVC & families?

Impact of direct support on child outcomes

What is the impact of educational, health, legal support, and other direct services on child & families?

Methods l.jpg
Methods increasing

Case studies (2006 and 2007)

Site visits, interviews, program document review

Provide understanding of program strategies, components, goals, and expected outcomes

Document lessons learned from implementation

Case Studies available

Program expenditures (2006)

Expense data collected and social costs estimated

Quantified the costs corresponding to specific intervention (e.g. food supplementation, psychosocial service, educational support)

Methods11 l.jpg
Methods increasing

Outcome evaluation (2007 and 2008)

Post-test study design with intervention & comparison groups

Exposed Vs. Non-exposed

Surveyed children age 8-14 “or 7-15” & their caregivers

Up to 2 children per household

Four questionnaires were applied in each household:

Q1: Household Questionnaires

Q2: Parent/Guardian/Caregiver Questionnaire

Q3: Parent/Guardian/Caregiver Regarding Child Questionnaire

Q4: Child (age 8-14 “or 7-15”) Questionnaire

Post test study design used l.jpg
Post-test Study Design Used increasing

The Groups are Not Randomly Assigned


X O1



** Jali Watoto – Study compared intact groups of

intervention versus comparison

Principles guiding questionnaires development l.jpg
Principles Guiding Questionnaires Development increasing

Capture multiple measures for each domain

Use existing standardized scales where possible (PSS, SES)

Intervention Exposure questions to be specific to each program

To facilitate comparison across countries and program models, same survey instruments were used except intervention modules

Multi-faceted programs necessitated sufficient questions across multiple domains

Multiple perspectives on child well-being (child and caregiver)

Measures of caregiver, household & community well-being

Outcomes examined l.jpg
Outcomes Examined increasing

Psychosocial well-being – multiple measures – standard scales used where appropriate (child and caregivers)

Education – enrollment & attendance (child)

Health – self-reported health status and access to health services (child and caregiver)

HIV-prevention – HIV-knowledge (child) & HIV-testing (caregiver)

Legal protection – birth registration, alternate caregiver

Community support – stigma and in-kind support (child & caregiver)

Study strengths and limitations l.jpg
Study Strengths and Limitations increasing


Yielded immediate data on program effects

Results can be used to improve current programs

Ethical - not withholding services for experiment sake


Post-test design – no baseline data - impossible to make conclusions concerning change in outcomes resulting from program exposure

Selection bias - self-selection to participate and those who did not -makes it difficult to conclude with certainty that the interventions are responsible for the observed differences

Analyses plan l.jpg
Analyses Plan increasing

Who are the OVC/MVC program beneficiaries?

Effects of community level interventions i.e.

Community care and support meetings/sensitization

Effects of household or caregiver level interventions i.e.

Community volunteer or Health Worker home visits

Caregivers participation in OVC care seminars

Effects of child level interventions i.e.

Kids clubs

Basic needs support (e.g., education, health, legal)

Statistical analyses l.jpg
Statistical Analyses increasing

Descriptive analysis (Univariate)

Bivariate analysis (ANOVA and Chi-square)

Multivariate (logistics, and linear regression)

Control variable: non-program factors e.g. socio-demographic

Child Level – Age, sex, orphan status, relationship to caregiver, and number of different homes the child had lived in the past year.

Caregiver level – Age, sex, marital status, education, illness, SES, and # children

Description of ovc program beneficiaries l.jpg
Description of OVC Program Beneficiaries increasing

  • Who is enrolled in OVC programs (MVC Profile):-

    • Although, these programs targeted geographic areas most affected by HIV/AIDS, MVC were identified and assisted regardless of specific causes of vulnerability

    • Majority of children enrolled in OVC programs are vulnerable in several fronts & not just orphanhood

Ovc profile l.jpg
OVC Profile increasing

  • Orphans (over 66% across programs)

  • Living in food insecure HH (over 80% across programs)

  • Poorest households (< 2assets) – over 40%

  • Living with chronically ill primary caregiver (over 20%)

  • Living with caregiver aged 50+ (about 23%)

  • Lived in two or more households in past year (14%)

Thank you l.jpg
Thank YOU! increasing

Key Findings are presented next….

Slide23 l.jpg

MEASURE Evaluation is funded by the U.S. Agency for increasing

International Development through Cooperative Agreement

GHA-A-00-08-00003-00 and is implemented by the Carolina

Population Center at the University of North Carolina at

Chapel Hill, in partnership with Futures Group International,

ICF Macro, John Snow, Inc., Management Sciences for

Health, and Tulane University. The views expressed in this

presentation do not necessarily reflect the views of USAID or the United States government.

Visit us online at