Health Metrics to Inform Health Policy Christopher Murray MD, D.Phil Director IHME Professor of Global Health Universi

1. Health Metrics to Inform Health PolicyChristopher Murray MD, D.PhilDirector IHMEProfessor of Global Health University of Washington

2. Outline Metrics for Population Health Decision-Making Scientific Challenges in Quantifying Population Health Problems Some Global Health Applications of Metrics What are the Right Interventions and Service Delivery Platforms? How Much of Potential Health Gain is Delivered? Challenges Ahead

3. Population Health Decision-Making Decision-making for population health is most often undertaken by national or local health authorities rather than individual providers and clients. Population health decision-making expands the number of dimensions beyond patient outcomes to include: the responsibility to respond to epidemics such as the recent swine flu, understanding and responding to drivers of adverse health trends or disparities, being an effective steward to complex and often pluralistic health systems. Large premium on metrics that can provide valid, reliable and comparable insights into health problems, their systemic solutions and costs. Metrics need to be comprehensible, comparable, and credible.

4. At Least Five Key Analytics • Quantifying the levels, trends and disparities in population health and attributing these to diseases, injuries and risk factors. • Systematically assessing the costs, efficacy and likely effectiveness of specific interventions and service delivery platforms. • Quantification of the effective coverage of key interventions – or how much health gain is actually being delivered. • Tracking public, private and international resources for health and the distribution across households of where these resources come from. • Putting resources, health gain and outcomes together to assess health care performance.

5. Outline Metrics for Population Health Decision-Making Scientific Challenges in Quantifying Population Health Problems Some Global Health Applications of Metrics What are the Right Interventions and Service Delivery Platforms? How Much of Potential Health Gain is Delivered? Challenges Ahead

6. Some Challenges for Population Health Quantification Relating the levels, trends and disparities in health outcomes to key causes for which interventions are available. Extensive work in public health, epidemiology, health statistics, demography and other disciplines provide a strong basis for quantification of health problems. Challenges remain: widespread pervasive differential item functioning for measures of functional health status, limited comparability of cause of death certification, missing data on exposures to some key risks particularly in population sub-groups, multiple risk factor interactions, and two competing concepts of causal attribution in health statistics and epidemiology (categorical and counterfactual).

7. Illustrating Metrics Challenges with the Example of US Disparities: Male life expectancy in US counties, 1997-2001: Huge Variation Highest life expectancy County State e(0) Grand, Clear Creek CO 80.2 Summit, Park, Jackson, Eagle, and Gilpin Summit & Morgan UT 79.4 Montgomery MD 79.3 Lowest life expectancy County State e(0) Marlboro SC 65.1 Baltimore City MD 63.8 Jackson, SD 62.0 Washabaugh, Mellette, Bennett, Todd, Shannon Ezzati et al PLoS Medicine 2008

8. Life Expectancy is Declining for Women in Some Counties, 1983-2003 Female Male Statistically Significant Decline in E(0) Ezzati et al PLoS Medicine 2008

9. Life Expectancy at Birth in the Eight Americas Females Males To facilitate analysis of mortality disparities by disease and injury by age and sex, US divided into 8 Americas based on race and location Murray et al. PLoS Medicine 2006 9

10. Male Causes of Death in the Eight Americas Compared to Japan, UK, Russia and West Africa Murray et al. PLoS Medicine 2006 10

11. US Deaths Under Age 70 Attributable to Key Risks Danai et al 2009 PLOS Medicine

12. Understanding US Disparities: Data and Technical Limitations We do not have all the data to analyze attributable mortality by risks for the 8 Americas or any other race/ethnicity group in specific communities. Extending the analysis of disparities both levels and trends to encompass non-fatal health outcomes is severely affected by problems of comparability of the available self-reported instruments. Even cause of death certification varies by race/ethnicity and geography in a high-income country like the US. Because of limitations, cannot easily identify the set of interventions that will address widening race/ethnicity/place disparities in the US.

13. Non-Fatal Health Outcomes: A Bigger Challenge at the Population Level Critical to extend quantification of population health problems by cause to include non-fatal health outcomes and not just mortality. Comparable quantification overtime and across subgroups of non-fatal health outcomes limited by inter-temporal and cross-cultural differential item functioning. Similar problem of widespread and correlated DIF across multiple items plagues attempts to compute metrics of functional health status across countries at the population level. Challenges of DIF extend beyond general health items to domain and symptom-specific measures.

14. Self-Rated General Health Item from Four US National Surveys Surveys All four surveys use exactly the same wording for the item. Salomon et al AJE 2009

15. Cultural Confounding of Psychosis Screening Item, Evidence from the World Health Survey Results for 50+ Countries A feeling that your thoughts were being directly interfered or controlled by another person, or your mind was being taken over by strange forces?

16. Active Research on Solutions for Correlated DIF: Anchoring Vignettes and Direct Measurements Anchoring vignettes is a method where respondents also report on self-reported domain-specific functioning for a standardized case – CHOPIT model developed and non-parametric models for analyzing this type of data (King et al APSR 2003). Anchoring vignettes have been used in health-related areas such as Kapteyn et al. American Economic Review 2007 analysis of work disability in the Netherlands and the USA. An alternative is to study correlated DIF using performance tests for domains where meaningful comparable performance tests can be implemented.

17. Outline Metrics for Population Health Decision-Making Scientific Challenges in Quantifying Population Health Problems Some Global Health Applications of Metrics What are the Right Interventions and Service Delivery Platforms? How Much of Potential Health Gain is Delivered? Challenges Ahead

18. Summary Measures Population Health For quantification of population health to feed into high level policy debate, summary measures of population health are a useful tool. Two broad families in use: health expectancies and health gap measures. The most widely used summary measure of population health is DALYs. For example, the World Health Organization on a regular basis publishes updates by disease, injury and risk factor. Many countries also produce national burden of disease estimates using DALYs. Global Burden of Disease Study is a collaborative of 800+ researchers coordinated by IHME, Johns Hopkins University, Harvard University and the World Health Organization. A complete revision is due end of 2010 for 240+ causes for 21 regions.

19. Global Burden of Disease, 2004

20. Leading Causes of the Global Burden of Disease 2004, Percent of Global DALYs

21. Attributable Disease Burden of 20 Risk Factors

22. Outline Metrics for Population Health Decision-Making Scientific Challenges in Quantifying Population Health Problems Some Global Health Applications of Metrics What are the Right Interventions and Service Delivery Platforms? How Much of Potential Health Gain is Delivered? Challenges Ahead

23. Population Health Investment Choices National or local authorities investment choices often relate to: • categorical non-personal interventions such as media campaigns, tobacco sale restrictions, food inspection etc. • personal interventions that require special action such as immunization programs • investment in service delivery organizations or quality enhancement for clinics, hospitals, local public health services – service delivery platforms • human resources for health development. These choices especially 3 and 4 often require information beyond the traditional cost-effectiveness of interventions.

24. Some Further Challenges: Operator Dependent Idiosyncratic Results, WHO CHOICE vs DCP2 Two major international efforts to systematize the cost-effectiveness of interventions for low and middle-income countries illustrate further challenges. The correlation coefficient between them is low. Why? Weak costing information, different approaches to disease modeling, and huge variation in the estimation of effectiveness from systematic reviews of the efficacy data.

25. Ex Ante vs Ex Post CEA The validity of predicting effectiveness for use in CEA could in principle be assessed by comparing across a large set of interventions the ex ante assessment of CE and the ex post realization of CE in similar units. Like publication bias plots, we would expect the points to be scattered around the 45 degree line. Very few comparisons of ex ante and ex post CEA. At this point, difficult to assess how well this work is doing overall. Predicting effectiveness is highly idiosyncratic and depends particularly in low and middle-income countries dramatically on provider attributes as well as household demand and adherence.

26. Agenda for Rigorous Assessment of Costs and Consequences of Policies Assessing the cost and consequence of policy options whether it is the adoption of a new drug, vaccine or diagnostic or a change in how health workers are compensated is a challenge of predictive validity. Can we predict the cost, who pays and who benefits from the proposed change. Developments in the science of weather, climate, geology and other complex systems should be captured in trying to predict complex system changes in healthcare.

27. Outline Metrics for Population Health Decision-Making Scientific Challenges in Quantifying Population Health Problems Some Global Health Applications of Metrics What are the Right Interventions and Service Delivery Platforms? How Much of Potential Health Gain is Delivered? Challenges Ahead

28. What is Effective Coverage? First introduced as part of the World Health Organization’s (WHO) Framework for Health Systems Performance Assessment • Key intermediary pathway • Direct measure of the output of health systems

29. What is Effective Coverage? Fraction of the potential health gain of an intervention that is being delivered to a population Measure: the population in need of the intervention (N); utilization of the intervention amongst the population in need (U); and the quality of the intervention (Q), measured as the fraction of potential health gain realized, being delivered in those who receive the intervention For individual i, intervention j: ECij = QijUij | Nij=1 Fraction of potential health gain (Q) : e.g. if the optimal reduction in an individual’s blood pressure is 20mmHg and the individual has a reduction of 10mmHg, then the fraction of potential health is 50% Shengelia et al SSM 2004

30. Global Trends in DTP3 Coverage Based on a Systematic Assessment of Survey and Administrative Data Country Reported UNICEF/WHO Systematic Assessment Lim et al Lancet 2008

31. Annual change in DTP3 coverage, 2000 to 2006

32. Outline Metrics for Population Health Decision-Making Scientific Challenges in Quantifying Population Health Problems Some Global Health Applications of Metrics What are the Right Interventions and Service Delivery Platforms? How Much of Potential Health Gain is Delivered? Challenges Ahead

33. Science of Metrics and Privacy Many advances in improved health metrics require the analysis of large datasets. The information content of these surveys and administrative datasets is dramatically enhanced through record linkage. Concerns over privacy, regulation and legislation on health information may make the analysis of these datasets harder to undertake. For example, during the Bush administration, even data on mortality by cause at the county level was not released to the research community. Drive towards more transparent and accountable healthcare systems is counterbalancing privacy concerns.

34. Values and Science in Health Metrics Use of health metrics in decision-making ends up requiring a range of social values to balance competing concerns and benefits. The science of measurement is inextricably linked with various value choices embedded in for example the importance attached to a child death versus and adult death. We should seek make the scientific foundations of measurement as strong as possible, ensure that measurements are replicable by different scientists and make all value choices transparent. Value choices, however, should not be a reason to not engage in rigorous measurement of health inputs, outputs and outcomes.