Measuring and monitoring poverty

1. Measuring and monitoring poverty Angus Deaton Princeton University

2. Main issues Designing surveys Timely and consistent information Estimating poverty Monitoring for everyone Project evaluation How to learn from projects & programs Fostering debate Access and use of data

3. Programme of surveys Need to be regularized: more than once every 5 or ten years Commitment to monitoring living standards, more than just poverty A system of surveys, regular core, but different topics for each Irregular surveys change every time, so comparability is impossible Gradual evolution of survey design allows comparisons

4. Progress in recent years Uganda has something close to a regular monitoring system in place Ethiopia, Kenya, and Tanzania less far along All have experience with household consumption surveys All have had more than one DHS: useful for health, especially children All have had experience with PPAs Range of other surveys: labor force, census, welfare monitoring survey, etc.

5. Comparison with India Indian National Sample Survey runs every year Every 5 years, large scale expenditure survey Every year, smaller sample, with expenditure questions, but main survey topic is something else Other complementary surveys, such as DHS In recent years, open access (for fee) to researchers and commentators Lively public debate on poverty issues Feedback from researchers to NSS Improvements in survey design Some hiccups & mis-steps

6. Key properties Each survey covers a whole year Otherwise seasonality spoils comparability Multistage stratified survey, with �regions� as strata Possible to estimate poverty at regional level Sample size is set by need to obtain accurate poverty estimates at state level Used to make transfers from center to state Much smaller sample sizes needed for good national poverty estimates

7. Key design issues Many things affect the expenditure totals, and the poverty estimates Number of consumption items Traditionally several hundred: recent positive experience with one page Reporting periods Switching from 30 days to 7 days for food & tobacco cut measured poverty by half Longer reporting periods for infrequently bought items reduced both mean & variance Diaries versus recall period No Indian experience: less known Doesn�t work very well, even in US

8. Consistency Getting these things right is less important than consistency Timeliness is more important than perfection If the poverty rate is 0.4, say, 100 households will give a standard error of 0.045, 1000 a s.e. of 0.015 For change, more difficult. For one point reduction per year, ss 1000, need 5 years before we are sure Could be reduced by using panels, but other problems, see later Argument for infrequent surveys: doesn�t work, Indian example of the reforms in the early 1990s Shorter, cheaper questionnaires have high payoff in terms of sustainability

9. What sort of surveys? Participatory poverty assessments Traditional expenditure surveys Demographic and Health Surveys Much recent use Focus on health: vital poverty issue in its own right, e.g. child nutritional status Helpful in monitoring consequences of HIV/AIDS: e.g. orphanhood Some limited economic information, mostly about assets, which has been widely used Perhaps the economic component could be extended: short simple consumption questions

10. PPAs Have taught us much Voices of the poor: identification of issues PPA techniques have been incorporated into survey practice Village census Facilities Less useful for monitoring over time �Everyone� is poor Adaptation to an unknown extent Self-reports not acceptable for general level of living But �ladder� questions about economic status are informative & cheap Make greater use in standard expenditure surveys

11. Expenditure surveys Monitoring surveys Expenditure questionnaire need not be long But coverage cannot change too often or too quickly Ownership of durable goods Link with DHS Household roster, and basic demographic information Need to be spread throughout the year Permanent capacity works best Staff training & consistency Can be expanded at lower frequency

12. Panel data? In an ideal world! Reduction in variance of measured changes But there are problems: Finding households again can be hard Not always clear whether the same Sometimes hard to link households in the data Attrition can be large Increasingly unrepresentative Attrition and no new groups Dynamics of poverty & income doesn�t work because measurement error Vulnerability is a nice concept, but hard to measure I am very skeptical

13. Poverty maps Combine census with survey data to generate predictions of poverty for small areas Problem is that census information does not predict poverty very well Poor fit More important, it misses prices & returns These are often the most important part of poverty reduction Predictions have larger standard errors than are calculated Indexes of human & physical assets, not of poverty

14. Poverty estimates Given expenditure distributions, how do we calculate poverty? Poverty line or lines Prices: over time and space Equivalence scales But survey data are for everyone Average living standards Regional patterns Not just poverty Need the support of population at large if surveys are to be sustainable

15. Poverty lines Calorie methods are fine, as a start But dangers of separate calorie based lines in different places Overstate poverty for more sedentary populations Updating over time is usually done by price indexes, not calorie recalculation Poverty lines must be politically & socially acceptable My own preference would be to ask people Update over time, and spatially, using price indexes Similar if calorie line is only calculated once

16. Prices and price indexes Often quite difficult: CPI biased towards large cities Difficult to institute local price monitoring & relevance doubtful Unit values from households in expenditure survey Several in the region Direct measurement of quantities, esp food, for households Possibly the best indexes, even if imperfect

17. Equivalence scales Many countries use per capita expenditure: $1-a-day e.g. Obviously a bias here: large households look too poor, small ones too rich But we are aware of it, and not a big problem Equivalence scales OK if transparent Assumption, not estimation There are no �experts� on equivalence scales This is a political & social issue, not a scientific one

18. Challenges to surveys Surveys need to be done carefully, professionally & with adequate training Refusals & substitutions have to be carefully monitored Consistency with national accounts? NAS far from perfectly accurate Coverage is different: e.g. FISIM Use of old rates & ratios will overstate growth in the National Accounts Many countries have NAS expenditures growing faster than survey expenditures

19. Evaluating projects Overall poverty monitoring Also need to monitor how projects and programs work Learning what works & what doesn�t Many methods of program (impact) evaluation E.g. econometric & statistical Performance (accounting) checks Only rigorous method is randomized evaluation

20. Randomized evaluations Like RCTs in drug trials Experimental and control groups Randomly selected Difference, if significant, must be treatment No other way of controlling for unobservable characteristics that affectt the outcome Not for all cases: e.g. macro policy But can be applied much more widely Good examples of social programs in LA, especially Progresa in Mexico Work in Kenya on education & health Stop us moving from one fad to another

21. Creating & managing debate Poverty monitoring works only if people care Poverty estimates are headline news Controversy about policy and data Trustworthy data Insulation of stats office from govt. Panels with experts, civil society, etc Feedback from users to collectors Causes some problems, but more if not Data should not be isolated from political debate Data should be in the public domain Public use files Used for research, checking, teaching, training Indian example: nothing bad happened, many good things happened