Implementing USAID Poverty Assessment Tools

1. Implementing USAID Poverty Assessment ToolsMaterials Developed by The IRIS Center at the University of Maryland

2. What is Poverty?How have you measured it?

3. Methodology: Identifying the Very Poor • Very poor households have non-standard and highly variable sources of income → poverty must be measured using expenditure data. • Expenditure surveys are too costly and time-consuming to conduct on all beneficiaries → short-cut tools should be developed and tested. • Can measure at household level, divide by number of household members for approximation • IRIS testing methodology evaluates both the accuracy and the practicality of shortcut poverty assessment tools.

4. What is a Poverty Assessment ‘Tool’? • Includes: • Sets of poverty indicators and coefficients in poverty level calcualtion • Integration into program implementation: who implements the tool on whom and when • Data entry and analysis: MIS or other data collection system/template • Training materials for users

5. Background: Legislation • US Congress: half of all USAID microenterprise funding must reach the “very poor” • Microenterprise: Mincrofinance, Business Development Services (+ Enabling Environment) • “Very poor”: • Bottom 50% below a national poverty line OR • Under PPP$1/day: international poverty line • USAID must develop and certify at least two tools to measure this level of outreach

6. Background (cont.) • USAID-certified tools are objective, absolute, accurate, and low-cost • Tools were tested for practicality before certification

7. Accuracy Tests

8. Balancing Errors: Accuracy in the Aggregate • Attention should be paid to the errors of classification: classifying the very poor as not very poor (undercoverage) or the not very poor as very poor (leakage). • Measuring accuracy at the aggregate level can allow a “cancelling out” of errors which does not occur at the individual household level. • See handout “PAT Errors Example” and the “Note on Assessment and Improvement of Tool Accuracy” for more details.

9. Tests of Accuracy • Testing indicators for their ability to act as proxies for poverty • Tests of accuracy completed in Bangladesh, Uganda, Kazakhstan, and Peru • Sampling: nationally representative sample of 800 randomly selected households • Full reports available at http://www.povertytools.org

10. Design of Tests in Four Countries Two-step process obtains data on: - poverty indicators from a Composite Survey Module (compiled from existing indicators and literature), and - Benchmark per-capita-expenditures from an adapted LSMS Consumption Expenditure Module.

11. Analysis of Eight LSMS Data Sets Objective: • Assess robustness of results from main study over larger number of countries, using methodology and set of indicators as similar as possible to 4 field countries. 8 LSMS data sets: • Africa: Ghana, Madagascar • Asia: India, Vietnam • Eastern Europe and FSU: Albania, Tajikistan • Latin America and Caribbean: Guatemala, Jamaica

12. From Accuracy to Practicality: Developing Tools

13. Developing Poverty Assessment Tools • Preliminary accuracy testing across 12 countries yielded 110 indicators that were best at predicting poverty. • These 110 indicators were divided into 6 prototype questionnaires. • Each questionnaire tested in at least 3 countries and 2 regions of the world.

14. Objectives of the Practicality Tests • “Test-drive” data collection methodology • Test indicators for applicability, difficulty • Use this information to create final tools that balance accuracy and practicality

15. Practicality Criteria • Low risk of misreporting or manipulation • Cost for implementation and for client • Ease of data collection and analysis

16. Participants in the Practicality Testing • Prototype tools tested by 17 microenterprise practitioners in 14 countries. • Three data collection methodologies were tested: household interview, intake, and ongoing monitoring. • Feedback provided via reporting from implementing organizations and in-country debriefs.

17. Discerning Practicality Lessons • IRIS gathered all information from the participating practitioners about the six questionnaires and three methods of implementation. • Analysis of this data yielded lessons on which indicators and data collection practices were impractical. • Lessons applied to create new tools for 12 countries.

18. Certified USAID Tools

19. Characteristics of Certified Tools • Tools are country specific. • Poverty indicators may vary substantially between tools, as they were selected for accuracy in each country individually. • Indicators were selected according to data from the national level, not regional. Regional data is more accurate but very costly to collect. Some indicators may be impractical in some regions of a country, but on the overarching national level, the indicator proved to be a good predictor of poverty. • Tools were created via a process of trade-offs between accuracy and practicality. Similar to the difference of using an atomic clock or a sundial to measure time.

20. Characteristics of Certified Tools (cont.) • Tools are accurate at predicting the aggregate levels of very poor households (by balancing errors). • Tools are also practical to implement by taking field experience into account. • Tools were developed in an iterative and collaborative process between USAID, tool designers, and the broader microenterprise community. • Tools are for public use.

21. What the final certified tools will DO • Measure aggregate number of clients above or below the legislative poverty line

22. What the tools are NOT designed to do • Measure complex nature of poverty • Measure multiple types of poverty • Measure relative poverty • Target clients for inclusion in the program (results are only known in the aggregate) • Measureimpactor movement out of poverty for individual clients