1 / 44

Outline

Impact 2 An innovative tool for measuring the impact of reproductive health programmes Overview and Training. Outline. Background What’s new and different How it works Types of results Using the model-- quick demo and activity Using results in your work. Why use Impact 2?.

dahlia
Download Presentation

Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Impact 2An innovative tool for measuring the impact of reproductive health programmesOverview and Training

  2. Outline • Background • What’s new and different • How it works • Types of results • Using the model-- quick demo and activity • Using results in your work

  3. Why use Impact 2?

  4. Why we need models Challenge: difficult (or impossible!) to measure outcomes & goals – so use Impact 2 instead! Output Outcome & Goals

  5. What it does Estimates wider impacts of family planning services Helps service providers, donors & governments to communicate: Demonstrate value for money Compare FP with other health interventions Set realistic programme goals Monitor progress over time Motivate staff

  6. Impact 2 – what’s new? 2009 Impact Estimator 1.2 2010 REACH Calculator 1.2 CYPs  Births averted Maternal deaths averted $s saved etc. FP services  FP users Market share CPR contribution 2012 Impact 2

  7. What’s new? • Impacts are no longer based on CYPs- instead run off service provision data by method/year • We can now estimate annual impacts (ie maternal deaths averted in 2010) • Different ways to see impacts– not just 1 result! • You can now set goals for reducing maternal deaths and unsafe abortions nationally. 

  8. service provision to impact CPR contribution , service/product provision Estimated users m arket share, maternal deaths averted , etc. How it works Past and/or planned future Impact Impact goal to services provision

  9. Background Information • Excel-based • Transparent & simple to use for non-experts • Open-source • All tools & training available on MSI website late March 2012 • Pre-loaded with best data from UN, WHO, DHS etc for all developing countries • (Very little data entry required) • Expert collaboration & review • Presented at Global Family Planning Conference, Senegal 2011 • At least 8 organisations (inc DFID) using older Impact 1.2 • Peer review of methodology: DFID, UNFPA, Guttmacher, LSHTM, Population Council, Futures Institute, Futures Group, PSI, IPAS, IPPF, EngenderHalth

  10. Family planning services/products provided Count backwards Long acting & Permanent methods Short-term methods Apply continuation and mortality rates to past LAPM clients Products provided Units needed per year CPR contribution Estimated Family Planning Users Market share Pregnancies averted Miscarriages and stillbirths averted Births averted Abortions averted Child deaths averted (Due to improved birth spacing) Maternal deaths averted Unsafe abortions averted Costs saved to healthcare system (Direct treatment costs: ANC, delivery, PAC, pregnancy and birth complications) DALYs averted (maternal morbidity & mortality, child mortality)

  11. Post-abortion care (PAC) services provided Market share (PAC provided/abortions nationally) Maternal deaths averted (unsafe abortions x unsafe abortion mortality) DALYs averted (years lost due to maternal mortality; and unsafe abortion related maternal morbidity)

  12. Safe abortion services provided Market share (safe abortions provided/abortions nationally) Unsafe abortions averted (safe abortions x probability unsafe abortion per safe abortion) Maternal deaths averted (unsafe abortions x unsafe abortion mortality) DALYs averted (years lost due to maternal mortality; and unsafe abortion related maternal morbidity) Direct costs saved to healthcare system (cost of PAC for women needing treatment)

  13. Limitations • Data availability • Infrequent • Often regional rather than national • Reliant on assumptions: LAPM discontinuation rates, mortality rates, pregnancy rates, method failure rates, etc. • Some rates/coefficients held constant over time due to limited data (i.e. unmet need, abortion rates) • Emphasise that all results are only estimates and don’t measure real life

  14. Types of results

  15. Timeframe • Annual: snapshot of impact that happen in a given year • impact from services provided in that year • impact from LAPM clients who received services in past years, but, are estimated to still be using the method • Service lifespan: impacts that will happen over multiple future years, from services provided in a given year • Similar to CYP-based impacts from old model • Number of future years depends on mix of LAPMs

  16. Timeframe example Annual impact in 2008 = impacts from these users 2009 2008 2009 2010 2010 2011 2007 2008 2009 2006 2005 2007 2008 2006 2007 Service lifespan impacts in 2006 = impacts to women who received services in 2006 over multiple years

  17. Historic services? • Annual impacts can be looked at: • including historic services:includes services provided prior to the start year of your trend • without historic services:does not include services provided prior to the start year of your trend Trends from 2007 - 2010 2012 2011 2010 2007 2008 2010 2008 2009 2011 2010 2009 2009 2006 2005 2007 2009 2008 2008 2006 2007 Include = full picture Without = isolate impact of project

  18. What to count (1) • Total: counts the full impact of your programme • Includes impacts to all clients, regardless of their previous use • Incremental: counts impacts for which your organisation is responsible • or: impacts that would not exist without us, unless other providers increased their service levels • Excludes impacts to clients who were “taken” from another provider

  19. Total v incremental (1) Not using any modern method Your clinic Other provider 14 7 Number of clients doubled…

  20. Total v incremental (2) Not using any modern method Your clinic Other provider Total users = 14 Incremental users = 9

  21. What to count (2) Before: 41% market share After: 73% market share Despite large increase in market share, almost no change to the “market” 17 women using FP 19 women using FP

  22. What to count (3) • Growing the market = increasing CPR • But. . . • Don’t forget about population growth! Must reach more and more women just to maintain CPR. . .

  23. What to count (4) • Total and incremental impacts do not take any of this into account. • These estimates include impacts that were already being averted, because women were already using FP • You could increase your total/incremental impact without having any impact at the national level • National contribution (annual only): counts impacts which contribute to a national reduction in burden • Only includes impacts to women increasing CPR

  24. National contribution Had no women been using FP = more unintended pregnancies = more unsafe abortions Unsafe abortions continuing to be averted Already being averted “national contribution”– offset population growth, then reach adopters Actual number of unsafe abortions 2010 2011

  25. Increase CPR Maintain CPR contribution Maintain user numbers Increasing CPR • Maintain number • maintain percent (due to population growth- adopters only) • increase CPR (adopters only) Accounting for historic: Including- baseline = year before trend starts Without- baseline = 0

  26. Key assumptions Several key assumptions for national contributions: • All other providers at least maintain their contributions • If not, your increase will offset declines from other provides • Your client profile is accurate • There are no unexpected changes at national level • i.e. sudden increase in MMR

  27. Client profile • Adopters: women who were not using a modern FP method before coming to your programme • Continuers: women who were already using a modern method, and received the method from your programme • Provider changers: women who were already using a modern method, and received the method from another provider MSI uses exit interviews to get this information

  28. Which timeframe? • Annual: useful to give a ‘snapshot’ of your impact– works best to look an organisation as whole, or entire regions or country programmes • Service lifespan: shows all the impacts that will come from an investment– works best for donors and to look at impacts of particular projects

  29. Which type of impact? • Total: most simple (and largest!) result, gives overall picture of the impact of our work; does not require client profile data When possible, in addition to total you may wish to show: • Incremental: gives overall picture of the impact of your work, with some consideration for who we are reaching (i.e. excludes impacts taken from other providers) • National reduction: shows meaningful result about your role in changing the national picture

  30. When in doubt… • Use totalservice lifespan impacts • Easiest • Closest to old model • If you have client profile data– can get more ‘meaningful’ estimates (i.e. incremental, national reduction)– eventually moving in this direction, but, not there yet. . .

  31. Remember! • Always use the word “estimated”– these are modelled, and not measures of real life • For service lifespan results, always use the future tense (“will be averted”)– these impacts are spread over multiple future years

  32. Impact 2 demo and activity

  33. Tip: Don’t forget to enable macros!

  34. Tip: you can edit/change any yellow box!

  35. Tip: print this out for a quick reference of data and sources

  36. Tip: You must enter service data by method

  37. Tip: a low default is set for the client profile, change if you have this data Tip: only need to enter your costs to see “cost per xxx”

  38. Tip: all graphs and tables can be cut and paste into presentations, reports, etc.

  39. Exploring Impact 2 Can you create the following? • Graph of annual (including historic services) maternal deaths and child deaths averted in Tanzania from 2005 to 2010. • Table of life-span demographic impacts (pregnancies, births, and abortions averted) in Myanmar from 2007 to 2011 • Bar chart of the number of using an implant or IUD (including historic services) from Marie Stopes Cambodia in from 2001 to 2011 • Pie chart of modern market share (including historic services) in Pakistan in 2010

  40. Using results

  41. Using results in your work • Proposals to donors • Tracking progress over time • Setting realistic goals • Communicating with Governments and donors • Decision making– compare different situations

  42. Helpful handouts

  43. More information • Download Impact 2: http://www.mariestopes.org/Resources/Tools.aspx • Additional trainings being developed on: • How to do goal setting • Using Impact 2 for proposal writing • Using Impact 2 for planning • How to generate multi-country results • Other needs? Please get in touch with MSI’s Impact Analysis Team: research@mariestopes.org

More Related