Measuring success for mHealth
1 / 7

Measuring success for mHealth Lessons from monitoring and evaluation of Vodafone Foundation UN Foundation s mHealth p - PowerPoint PPT Presentation

  • Uploaded on

Measuring success for mHealth Lessons from monitoring and evaluation of Vodafone Foundation & UN Foundation’s mHealth program in Africa. 28 October 2009 Andrew Stern – Partner, Dalberg Global Development Advisors. Background and summary.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Measuring success for mHealth Lessons from monitoring and evaluation of Vodafone Foundation UN Foundation s mHealth p' - guri

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

Measuring success for mHealthLessons from monitoring and evaluation of Vodafone Foundation & UN Foundation’smHealth program in Africa

28 October 2009

Andrew Stern – Partner, Dalberg Global Development Advisors

Background and summary l.jpg
Background and summary

  • Dalberg has been working with UNF and Vodafone Foundation for two years to understand the impact of its mHealth program, with an in-depth focus on the programs in Senegal, Ethiopia, and Kenya

  • The approach used a ‘Theory of Change’ approach to analyze and monitor activities, outputs, and outcomes that lead to impact

  • Focusing on activities, outputs, and outcomes for monitoring & evaluation enables:

    • Identification of critical constraints to achieving impact and therefore an opportunity to develop mitigation approaches;

    • Focus on cost efficiency to yield greater program effectiveness;

    • Demonstration of relative effectiveness of a program across geographies

  • Comparing across countries, major success drivers have been identified including:

    • The quality of data-driven analytical and decision-making processes to make use of the data being collected;

    • The fact that narrow but deep pilots often work better for demonstration; and,

    • How critical it is to ensure complementary inputs, such as transportation costs, to ensure collection of data

The unf vf partnership l.jpg
The UNF-VF partnership

Since 2005, the Partnership has worked with DataDyne and WHO to develop and roll out EpiSurveyor

EpiSurveyor is now active in 19 countries

Objective: Bring the latest technology solutions to UN field work, using sustainable, digital tools to save lives

Solution: EpiSurveyor enables development of customizable surveys for cell phones and PDAs

Partners: DataDyne, World Health Organization, Ministries of Health, UNF-VF

Duration: 2005-2010

Theory of change l.jpg
Theory of change


~3-5 years

Saving and improving lives


~6 months

  • Improved health systems and monitoring

  • More effective campaigns (vaccinations, etc.)

  • More effective outbreak response


~3 months

Health data collection

Analysis and information sharing

Decision-making and action to meet needs





~3 months

Collection management and supervision


Software Development


Logistical support

Customizable survey development

Partnership activities


Indicators and performance management l.jpg
Indicators and performance management




~3-5 years

Reduce lives lost and improve lives

10. Disability adjusted life years (DALYs) prevented


~6 months

Country health systems with more, better, and faster data for decisions and action

9. Percent of survey questions showing positive improvements from prior month

8. Percent of decision-making meetings that discuss the data

7. Records analyzed as % of records collected

6. Records collected as a % of visits conducted

5. Visits conducted as % of potential visits in program

4. Visits covered by program as % of all potential visits

3. % of Ministry of Health divisions using PDA


~3 months

Provide data collection method for as many health applications and health facilities as possible


~3 months

Do more with same funding, or less

1. Cost per survey record collected


Slide6 l.jpg
Focusing on measuring outputs and outcome allows for the identification and mitigation of critical challenges that constrain impact



  • Improved health systems and monitoring

  • More effective campaigns (vaccinations, etc.)

  • More effective outbreak response





Health system supervisory data collection

Analysis and info sharing

Decision-making and action to address needs

  • Records may not be compiled

  • Few records may be collected

  • Some records may be lost

  • May have insufficient analytical tools to do trend and other analyses

  • Analytical needs at various levels may not be met

  • May not have clear use of data by decision makers

  • May not have clear tracking of decisions made with data


Country example senegal yielded a tangible example of impact l.jpg
Country example: Senegal yielded a tangible example of impact

Case Study: Partogram Usage in Senegal

  • Context

  • In Senegal, there is evidence that the process is working: data is collected, analyzed and then used as an input into decision making.

  • Given that foundation, the team identified a specific instance where the deployment of the PDA and integrated survey can be reasonably linked to an improved health outcome.

  • Story

  • Initial PDA collected data showed low usage of the partogram, a simple form used by midwives to monitor the status of a delivery [at left].

  • Delivering babies without using the partogram increases the chances of complications.

  • Seeing this gap in his district, the Pikine District Chief ordered a tripling of production of the forms and ensured they were distributed to midwives.

  • Between March and August 2008, data collected with the PDA showed an increase in partogram usage in the pilot districts by 14 percentage points, even though usage remained unchanged outside those districts.

    Just one example

  • The partogram case study is one example of estimated health impact.

  • Two-thirds of the 82 questions tracked by EpiSurveyor showed improvement and increased by 10% or more.

  • Additional case studies may be feasible based on previous data and continued support for data collection in the country.