Data Tools for MCH Professionals: Introduction to Local Data Sources and Analytic Considerations

1. Data Tools for MCH Professionals:Introduction to Local Data Sources and Analytic Considerations Michael D. Kogan, PhD Director, Office of Data and Program Development US Dept of Health and Human Services Health Resources and Services Administration Maternal and Child Health Bureau Laurin Kasehagen Robinson, PhD Senior MCH Epidemiologist CDC Assignee to CityMatCH Adjunct Assistant Professor in Pediatrics University of Nebraska Medical Center

2. Workshop Overview Role of local health departments Importance of local data Evidence-based public health Introduction to basic epidemiologic concepts Introduction to local data sources and overview of the reference guides What�s available How to use it Advantages and limitations of these data sources Hands-on case studies I and presentations Break Hands-on case studies II and presentations Discussion What was most useful? What was missing?

3. Role of Local Health Departments Local health departments play a key role in the provision of public health services to both rural and urban communities are the closest source for information on and assistance with public health issues and concerns in a community Serve 3 core functions

4. Core Function #1 Assess community problems, needs, and resources, through Health needs assessments Data Surveillance

5. Core Function #2 Provide leadership in organizing strategies to address health problems, through Programs designed to meet community needs

6. Core Function #3 Assure that direct services necessary for meeting local public health goals are available to all community residents, through Community health services, including Screenings Education Prevention Outreach

7. Why is local data important? Essence of the importance of local level data summarized by Shah, Whitman & Silva in �Variations in the Health Conditions of 6 Chicago Community Areas: A Case for Local-Level Data� �Variations in health measures identified at the local level shed light on the limitations of the existing city data often used in establishing public health policies and monitoring population health. . . . [Such] data are essential in identifying communities most at risk of poor health outcomes, exploring the determinants of such variations in health, and ultimately guiding community health programs and policies.�

8. Potential Limitations of / for Local Data Often limited to jurisdictions with populations of at least 100,000 Why? Issues of small numbers, accuracy and confidentiality Sometimes limited because of relatively rare events E.g., maternal mortality, autism, teen pregnancies, unintentional injuries The data may not be current Denominators may be based on the 2000 Census City / County / MSA population may be based on 2000 Census Data may not be collected at the household or city or county level Utility of instruments that gather data at micropolitan / local levels � generalizability, small numbers / few events, ability to analyze data for subsets of population (e.g., by age group, race / ethnicity, or some other characteristic) Nevertheless, it is the local level data that is needed to further reduce / have an impact on disparities in health, wellnessUtility of instruments that gather data at micropolitan / local levels � generalizability, small numbers / few events, ability to analyze data for subsets of population (e.g., by age group, race / ethnicity, or some other characteristic) Nevertheless, it is the local level data that is needed to further reduce / have an impact on disparities in health, wellness

9. Evidence-Based Public Health: Gathering and Using the Best Evidence for Local Data

10. Evidence-Based Medicine Health care practices based on review of current best evidence on the effectiveness of a test, drug, surgery or other medical practice Collect and analyze all of the research studies conducted on a particular intervention Evidence is then graded Best evidence based on findings from clinical trials and meta-analysis Weakest evidence based on case reports

11. Definition of Evidence-Based Public Health �EBPH is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of communities and populations in the domain of health protection, disease prevention, health maintenance and improvement.� Jenicek (1997) Key terms in this definition of EBPH are� EXPLICIT: Use defined methods JUDICIOUS: Use judgment BEST EVIDENCE: Sort through the literature and identify what is useful Key terms in this definition of EBPH are� EXPLICIT: Use defined methods JUDICIOUS: Use judgment BEST EVIDENCE: Sort through the literature and identify what is useful

12. Differences between Public Health and Medicine

13. So what is �best evidence�?

14. Best Evidence Makes sense (it�s relevant) Unbiased Available Statistically significant Significant to public health Leads to correct decisions

15. Evidence

16. Steps of Evidence-Based Public Health Develop an initial statement of the issue Search the scientific literature and organize information Quantify the issue using sources of existing data Develop and prioritize program options; implement interventions Evaluate the program or policy These are the steps of Evidence-Based Public Health (EBPH) as described by Brownson et al. These are the steps of Evidence-Based Public Health (EBPH) as described by Brownson et al.

17. Different Sources of Evidence in Public Health: The Information Continuum

18. So why isn�t evidence-based decision-making used more often?

19. How are Decisions Often Made? Decisions on policies and programs are often made based on: Personal experience What we learned in formal training What we heard at a conference What a funding agency required / suggested What others are doing

20. Evidence and Public Health Decision Making Good news Strong evidence on the effect of many policies / programs aimed to improve public health, like immunizations or smoking cessation Major efforts underway to assess the body of evidence for wide range of public health interventions, like the Cochrane Collaborative or the AMCHP Best Practices program

23. What Works to Improve the Public�s Health? Bad news Many public health professionals are unaware of this evidence Some who are aware don�t use it Many existing disease control programs have interventions with insufficient evidence �while others use interventions with strong evidence of effectiveness Lack of use of effective interventions can adversely affect fulfilling mission and getting public support

24. Evidence-Based Maternal and Child Health True or false? For women who are experiencing problems with their pregnancy, bed rest is effective in preventing preterm labor.

25. Evidence-Based Maternal and Child Health FALSE! Obstetric practices for which there is little evidence of effectiveness in preventing or treating preterm labor include bed rest. (Goldenberg, Obstetrics and Gynecology, 2002)

26. The True Story of the 3 Local MCH Departments and Governor Wolf�s Office

27. Once� �the office of Governor Wolf called up the first local MCH department and wanted to know the preterm birth rate for 2006 and 2007. The local data staff ran to the computer and quickly calculated the number of preterm births divided by the number of normal gestational age births. And proudly showed it to the Governor. And it was also the wrong denominator to begin with!And it was also the wrong denominator to begin with!

28. �That�s not a rate, that�s a ratio!!!� thundered Governor Wolf (who had a doctorate in epidemiology). And he huffed and he puffed and he blew away 25% of their funding.

29. So, the office of Governor Wolf called up the second local MCH department and wanted to know the preterm birth rate for 2006 and 2007. The local data staff ran to the computer and quickly calculated the number of preterm births divided by the total number of births. And proudly showed it to the Governor.

30. �Great,� said the Gov, �is it the same in 2006 and 2007?� �Oh, we�re not sure of the year� said the second local MCH staff. �Then it�s not a rate, it�s a proportion!!!� thundered Governor Wolf. And he huffed and he puffed and he blew away 35.8% of their funding.

31. And then, Governor Wolf called up the third local MCH department and wanted to know the preterm birth rate for 2006 and 2007. The local data staff ran to the computer and quickly calculated the number of preterm births divided by the total number of births for each year. And proudly showed them to the Governor.

32. �Great,� said the Gov, �is it the same in 2006 and 2007?� �No, it was 12.8 per 100 live births in 2006, and 10.2 per 100 live births in 2007; a significant decline� said the third local MCH department staff. �Excellent!!!� cried Governor Wolf.

33. And he wiped out their funding altogether because of an immediate state budget crisis.

34. Was Governor Wolf correct? Or, would any of the local health department responses suffice? (or, was the Governor just throwing around his epidemiologic weight)

36. Measures of Disease Frequency

37. Counts Simplest, most frequently performed quantitative measure in epidemiology Refer to the number of cases of disease, injury, events, or other health phenomenon being studied Examples No. of pregnant women who were screened for Hepatitis B during a prenatal care visit No. of women who initiated breastfeeding in the U.S. in 2007 No. of newborns screened for genetic, metabolic, hormonal and/or functional conditions within 24-48 hours of birth

38. Why isn�t enumeration sufficient? Can�t / Don�t always detect ALL events Census Sample How would you know whether the counts Represent events that are big, small, a problem, important? Represent phenomena common or unique to a population? Change over time? Are similar or different between 2 different populations?

39. Frequency Measures � Ratio, Proportion, Rate Characterize part of a distribution Can be used to compare one part of a distribution to another part of a distribution Contrast to measures of central tendency that provide single values that summarize entire distributions of data (e.g., mean, median, mode) Fractions are made up of two parts: Numerator is the upper portion of the fraction; Denominator is the lower portion of the fraction. Fractions are made up of two parts: Numerator is the upper portion of the fraction; Denominator is the lower portion of the fraction.

40. What is a ratio? A fraction in which the numerator is NOT part of the denominator Numerator and denominator need not be related Limits -- 8 to 8 Result is often expressed as the �x�:1 E.g., male-to-female ratio no. of controls to no. of cases no. of LBW births to no. of violent crimes in a neighborhood

41. How to Calculate a Ratio Ratio = Number or rate of events, items, persons, etc. in one group Number or rate of events, items, persons, etc. in another group Example: Sex ratio � male live births to female live births = 2,118,982 / 2,019,367 = 1.049:1 (or 1,049 male live births per 1,000 female live births)

42. What is a proportion? Compares a part to the whole The numerator is ALWAYS part of the denominator Type of ratio, �x/y� May be expressed as a decimal, a fraction, or a percentage Limits � 0 to 1 In epidemiology, tells us the fraction of the population that�s affected E.g., proportion of children in a school vaccinated against measles proportion of women in PRAMS who initiated breastfeeding % of women who initiated PNC in the 1st trimester

43. How to Calculate a Proportion Proportion = Number of persons or events with a particular characteristic Total number of persons or events of which the numerator is a subset

44. What is a rate? A ratio that consists of a numerator and a denominator in which TIME forms a part of the denominator Measures the frequency with which an event occurs in a defined population over a specified period of time

45. Properties and Uses of Rates Useful for putting disease frequency in the perspective of the size of the population Can be used to compare among different groups of persons with potentially different sized populations (i.e., rate is a measure of risk) Limits � 0 to 8 Can be expressed in any form that is convenient (e.g., per 1000, per 100,000, etc.)

46. How to Calculate a Rate Rate = No. of persons or events in a given time period No. of persons or events in a reference population (at mid-point of year or time period)

47. Are percentages ratios? Proportions? And/or Rates? Yes, Ratio � e.g., number of mothers in one group (e.g., 1st trimester) over the number of mothers in another group (e.g., all who had late or no PNC) Yes, Proportion � e.g., the ratio of mothers in one group who are a subset of the other group Perhaps, Rate � when percentages are a ratio that consists of a numerator and a denominator in which TIME forms a part of the denominator

48. Incidence Refers to the occurrence of new cases of disease, injury, attribute or events in a population over a specified period of time Is a proportion, rate Fundamental tool for exploring the etiology and causality of disease because new events provide estimates of risk of developing disease Several types of incidence measures Incidence proportion Attack rates Incidence rate

49. How to Calculate Incidence Proportion (Risk) Incidence Number of NEW cases of disease, injury, events, or deaths Proportion = during a specified period of time _______________________________________________ Population at start of the specified period of time

50. Uses of Incidence Data Determining the extent of a disease or health problem in a community Helping to determine etiology of disease because an estimate of risk of developing disease can be calculated Identifying changes in disease over time Comparing incidence rates in populations that differ in exposure � permits estimation of effects of exposure to a hypothesized factor of interest

51. Prevalence Refers to the number of persons in a population with a specified disease, injury or attribute or event at a specified point in time or over a specified period of time Is a proportion, rate Point prevalence Measured at a particular point in time Period prevalence Measured over an interval of time

52. How to Calculate Prevalence Total number of persons with [NEW + PREEXISTING cases of Prevalence disease] OR [attribute of interest] during a specified period of time of Disease / = _________________________________________________________ an Attribute Population during the same specified period of time

53. Properties of Prevalence Data Prevalence and incidence are frequently confused . . . Prevalence refers to the proportion of persons who have a condition at or during a specific period of time Incidence refers to the proportion or rate of persons who develop a condition during a particular period of time

54. Uses of Prevalence Data Provides an indication of the extent of a health problem and may have implications for the scope of health services needed Useful for Describing the health burden of a population Estimating frequency of an exposure Allocating health resources BUT, NOT for determining etiology

55. Measures of Association Quantify the relationship between exposure and disease among two groups of people within the same population or two different populations of people Exposure is used loosely to mean inherent characteristics, biologic characteristics, acquired characteristics, activities, social or environmental conditions, etc. Includes Relative risk (risk ratio) Rate ratio Odds ratio Proportionate mortality ratio

56. Relative Risk / Risk Ratio (RR) Compares the risk of a health event among one group with the risk among another group The two groups are typically differentiated by demographic features or exposure to a suspected risk factor Measure of association for cohort studies When RR = 1, same risk among the two groups RR > 1, increased risk for the group in the numerator (usually the exposed group) RR < 1, decreased risk for the group in the numerator (in some instances the exposure might be a protective factor)

57. Relative Risk of Hashimoto�s Thyroiditis

58. Rate Ratio Compares the incidence rates, person-time rates, or mortality rates of two groups The two groups are typically differentiated by demographic features or exposure to a suspected risk factor When Rate ratio = 1, equal rates in the two groups Rate ratio > 1, increased risk for the group in the numerator (usually the exposed group) Rate ratio < 1, decreased risk for the group in the numerator (could indicate that the exposure is a protective factor)

59. Male:Female Rate Ratio of Syphillis

60. Odds Ratio (OR) Quantifies the relationship between an exposure with two categories and health outcome Sometimes called the cross-product ratio Measure of choice in case-control studies Often, the size of the population from which the cases are identified is not known; thus, risks, rates, risk ratios, and rate ratios cannot be calculated Odds ratios approximate risk ratios (relative risks), particularly when the disease or outcome is rare When Odds ratio = 1, equal rates in the two groups Odds ratio > 1, increased risk for the exposed group Odds ratio < 1, decreased risk for the unexposed group

61. Odds Ratios of Self-Reported Severity of Asthma Symptoms

62. Measures of Natality

63. Measures of Morbidity

64. Measures of Mortality A pregnancy-associated death is the death of any woman, from any cause, while pregnant or within 1 calendar year of termination of pregnancy, regardless of the duration and the site of pregnancy. A pregnancy-related death is a pregnancy-associated death resulting from 1) complications of the pregnancy itself, 2) the chain of events initiated by the pregnancy that led to death, or 3) aggravation of an unrelated condition by the physiologic or pharmacologic effects of the pregnancy that subsequently caused death. See CDC�s �Structure of Pregnancy-Related Mortality�A pregnancy-associated death is the death of any woman, from any cause, while pregnant or within 1 calendar year of termination of pregnancy, regardless of the duration and the site of pregnancy. A pregnancy-related death is a pregnancy-associated death resulting from 1) complications of the pregnancy itself, 2) the chain of events initiated by the pregnancy that led to death, or 3) aggravation of an unrelated condition by the physiologic or pharmacologic effects of the pregnancy that subsequently caused death. See CDC�s �Structure of Pregnancy-Related Mortality�

65. Measures of Public Health Impact Used to place the association between an exposure and an outcome into a meaningful public health context Reflect the burden that an exposure contributes to the frequency of disease in a population Contrasts with measures of association, which quantify the relationships between exposures and diseases and provide insight to causal relationships Includes Attributable proportion Efficacy Effectiveness Efficacy NE Effectiveness Efficacy = indicates that the therapeutic effect of a given intervention (e.g. intake of a medicine, an operation, or a public health measure) is acceptable. 'Acceptable' in that context refers to a consensus that it is at least as good as other available interventions to which it will have ideally been compared to in a clinical trial. In strict epidemiological language, 'efficacy' refers to the impact of an intervention in a clinical trial, differing from 'effectiveness' which refers to the impact in real world situations. Effectiveness = ability to cause the expected or intended effect or result. The word effective is sometimes used in a quantitative way, "being very or not much effective". However it does not inform on the direction (positive or negative) and the comparison to a standard of the given effect. Efficacy, on the other hand, is the ability to produce a desired amount of the desired effect, or success in achieving a given goal. Contrary to efficiency, the focus of efficacy is the achievement as such, not the resources spent in achieving the desired effect. Therefore, what is effective is not necessarily efficacious, and what is efficacious is not necessarily efficient. Efficacy NE Effectiveness Efficacy = indicates that the therapeutic effect of a given intervention (e.g. intake of a medicine, an operation, or a public health measure) is acceptable. 'Acceptable' in that context refers to a consensus that it is at least as good as other available interventions to which it will have ideally been compared to in a clinical trial. In strict epidemiological language, 'efficacy' refers to the impact of an intervention in a clinical trial, differing from 'effectiveness' which refers to the impact in real world situations. Effectiveness = ability to cause the expected or intended effect or result. The word effective is sometimes used in a quantitative way, "being very or not much effective". However it does not inform on the direction (positive or negative) and the comparison to a standard of the given effect. Efficacy, on the other hand, is the ability to produce a desired amount of the desired effect, or success in achieving a given goal. Contrary to efficiency, the focus of efficacy is the achievement as such, not the resources spent in achieving the desired effect. Therefore, what is effective is not necessarily efficacious, and what is efficacious is not necessarily efficient.

66. Measures of Spread Standard deviation Conveys how widely or tightly the observations are distributed from the center point or values Measure of spread used most commonly with the mean Usually calculated only when the data are more or less normally distributed Dark blue is less than one standard deviation from the mean. For the normal distribution, this accounts for about 68% of the set (dark blue) while two standard deviations from the mean (medium and dark blue) account for about 95% and three standard deviations (light, medium, and dark blue) account for about 99.7%. Dark blue is less than one standard deviation from the mean. For the normal distribution, this accounts for about 68% of the set (dark blue) while two standard deviations from the mean (medium and dark blue) account for about 95% and three standard deviations (light, medium, and dark blue) account for about 99.7%.

67. Standard Error of the Mean Refers to the variability that could be expected in the means of repeated samples taken from the same population Assumes sample comes from a large population Sample of interest is just one of an infinite number of possible samples The mean is just one of an infinite number of sample means Standard error quantifies the variation observed in the sample means Primary use of standard error is in calculating confidence intervals around the mean SE = std dev vn

68. Confidence Intervals Common method for indicating a measurement�s precision Narrow interval = high precision Wide interval = low precision Represents the range of values consistent with the data from a study . . . Simply a guide to the variability in a study Confidence intervals can be calculated for some, but not all, epidemiologic measures . . . Regardless of measure, the interpretation is the same Means Geometric means Proportions Risk ratios Odds ratios

69. Some Methods to Compare Differences between Groups Rate ratios Used to compare rates for 2 populations Simply the ratio of 2 rates Note: the multiplier must be the same for both rates Relative percent difference (RPD) Another method for comparing differences between 2 groups using prevalence

70. Prevalence of Diabetes and Relative Percent Difference RPD between the rate of diabetes in Hispanics and non-Hispanic white women Mention confidence intervalsMention confidence intervals

71. Assessing Trends Trend = long-term movement in an ordered series Can be used to assess the overall pattern of change of an indicator, geographic areas, time periods, populations Can be influenced by small numbers, changes in how data collected / defined Can minimize effect by �smoothing� data via 3-year moving averages or data transformation (natural log scale) Also can be used loosely to refer to an association which is consistent between 2 sets of data or strata, but not necessarily statistically significant Can test for difference between trends using a chi-square test for trend or regression analysesCan test for difference between trends using a chi-square test for trend or regression analyses

72. How to Judge / Evaluate Data Sources Timeliness Geographic specificity Specificity of demographic data Data consistency and standardization Availability over time Ability to identify individuals / events Adequate sample size Sample validity Primary data collection potential

73. Caveats Caveat . . . unique data sources Not necessarily an abundance for local data, but may be packaged or presented in different ways Some states try to ensure that data are available at county level A number of websites that catalog or compile links to data sources, e.g., California � UCSF Family Health Outcomes Project -- http://familymedicine.medschool.ucsf.edu/fhop/htm/ca_mcah/index.htm Texas � UT School of Public Health -- http://www.sph.uth.tmc.edu/charting/

74. Next Steps in this Workshop What you have in hard copy and on disk Source descriptions Source quick reference guide Case studies Case studies �cheat sheet� Copy of this presentation Let�s take a look and GET STARTED!

75. Acknowledgments Belovich-Faust and Ligi. Role of the Local Health Dept., Bethlehem, PA Health Dept. Shah, Whitman & Silva. Variations in the Health Conditions of 6 Chicago Community Areas: A Case for Local-Level Data. Am J Public Health 96(8): 1485-91 (2006). Jenicek. Epidemiology, Evidence-Based Medicine, and Evidence-Based Public Health. J Epidemiol 7:187-97 (1997). Brownson, et al. Evidence-Based Decision-Making in Public Health. J Public Health Manag Prac 5:86-87 (1999). Goldenberg. The Management of Preterm Labor. Obstetrics and Gynecology 100(5 Pt 1):1020-37 (2002). Lewis. Moneyball, 2003.

76. Contact Information & Copies of Workshop Training Materials Laurin Kasehagen Robinson, PhD, MA CityMatCH Senior MCH Epidemiologist / CDC Assignee to CityMatCH Adjunct Asst Professor in Pediatrics University of Nebraska Medical Center, Department of Pediatrics 982170 Nebraska Medical Center Omaha, NE 68198-2170 402-561-7523 lkasehagen@unmc.edu Michael D. Kogan, PhD HRSA/MCHB Director, Office of Data and Program Development 5600 Fishers Lane, Room 18-41 Rockville, MD 20857 301-443-3145 mkogan@hrsa.gov