Epidemiology Workshop II Oak Ridge Reservation Health Effects Subcommittee December 3, 2001

1. Epidemiology Workshop IIOak Ridge Reservation Health Effects SubcommitteeDecember 3, 2001 Evaluating an Epidemiologic Study Lucy Peipins, PhD Sherri Berger, MSPH Division of Health Studies

2. How Science Works Every epidemiologic study and every analysis is built on a body of knowledge and is done to expand that body of knowledge Is there a relationship between a particular exposure and a disease? Epi Study Animal studies Case Reports Epidemiologic study Epidemiologic study Toxicologic Models

3. Evaluating an Epidemiologic Study • What is the purpose of scientific publication? • To present a new finding. Results are shared and debated with other scientists and the public • To demonstrate validity of study results by presenting the study methods, design, analysis and interpretation. • This provides the opportunity for others to judge the quality of the study

4. Evaluating an Epidemiologic Study • Like all scientists, epidemiologists have criteria to judge the scientific quality and validity of a study • Has the author presented a good case for his or her hypothesis?

5. Evaluating an Epidemiologic Study • Each journal judges the scientific content and validity of an article by a peer review process • Reviewers typically do not know the identity of the author • Reviewers make recommendations on whether the article should be published

6. Evaluating an Epidemiologic Study • To illustrate these criteria, we will use the article ‘Cancer Mortality near Oak Ridge, Tennessee’ by J. Mangano published in the International Journal of Health Services • The criteria we outline can be applied to any study.

7. Evaluating an Epidemiologic Study • The first thing we need to find out is… Why was this study done? or, another way of saying this is… What hypothesis is the author testing?

8. Evaluating an Epidemiologic Study • The main purpose was to examine change in cancer mortality in 94 counties within 100 miles surrounding Oak Ridge, Tennessee, in 1950-1952 and in 1987-1989, and compare this with mortality rate change in the U.S. and the Southeast • Exposure of interest is radiation from Oak Ridge weapons production which began in the early 1940s

9. Evaluating an Epidemiologic Study Specific Hypotheses: • Increase in all cancer mortality ‘near’ Oak Ridge should exceed the national and regional areas • Within Oak Ridge area, increase in cancer mortality rate should be greatest in rural areas • Increase in cancer mortality should be greatest ‘near’ the weapons plant • Cancer mortality should be greatest in mountainous areas • Cancer mortality increases should be greater downwind of weapons plant

10. Evaluating an Epidemiologic Study • What type of study was this? Descriptive study. This type of study describes health outcomes alone. It answers the “who, what, when and where” of a disease in a group of people.

11. Evaluating an Epidemiologic Study Analytic study.This type of study answers “why” a population or group of people get a particular disease.

12. Evaluating an Epidemiologic Study • This is primarily descriptive. Exposure is based on county of residence Hypotheses based on “where”-- counties nearest to weapons plant, counties downwind, mountainous counties, rural counties It is also analytic because the comparison groups--Southeast or U.S.

13. Evaluating an Epidemiologic Study • Keeping in mind that epidemiology is the study of groups of people or populations, who was the study group?

14. Evaluating an Epidemiologic Study • White population living less than 100 miles (in 94 counties) from Oak Ridge between 1950-1952 and 1987-1989 • Comparison population was the white population of the Southeast and the U.S. in those time periods

15. Evaluating an Epidemiologic Study • What was the exposure of interest?

16. Evaluating an Epidemiologic Study • Several types of exposure were mentioned: • radiation, or chronic, low-level ingested radiation, • emissions from nuclear plant into both air and water, and radiation from consumption of milk and vegetables were also mentioned

17. Evaluating an Epidemiologic Study • How was exposure defined and measured?

18. Evaluating an Epidemiologic Study • Exposure was not measured. • Exposure was defined as residence in a county near weapons plant, residence in mountainous counties, residence in downwind counties and residence in rural counties.

19. Evaluating an Epidemiologic Study • Let’s review the hierarchy of exposure assessment from our first workshop and see how to characterize Mangano’s exposure classification

20. Measuring Exposure • We can measure exposures or their surrogates along the entire path from emissions to body burden Types of DataApproximation to Actual Exposure _________________________________________________________________________ 1) Residence in a defined geographical Poorest x area (e.g. county) of a site | 2) Residence in a geographical area in | x proximity to to a site where exposure is assumed | x 3) Distance or duration of residence | 4) Distance and duration of residence | 5) Quantified surrogate of exposure (e.g. estimate | of drinking-water use | 6) Quantified area measurements in vicinity of the | residence | 7) Quantified personal measurement Best Baker D, et al. Environmental Epidemiology: A Textbook on Study Methods and Public Health Applications, 1999

21. Evaluating an Epidemiologic Study • What are some limitations or criticisms you might have about how exposure was measured?

22. Evaluating an Epidemiologic Study • Residence in county is a poor measure of exposure. • All persons living in the 94 counties are assigned the same exposure • Persons who lived in the county at the time of death may not have been long-term residents or persons who were long-term residents may have retired, moved and died in another county or state • All persons in the rest of the southeast or U.S. are assigned no exposure

23. Evaluating an Epidemiologic Study • Proximity to site is poorly defined in this study. • Why is Roane county not considered to be near the site? • Why are the comparison 12 counties within 40 miles of the site and not further away? • Different pathways of exposure (water and air pathways) may exert a stronger influence than vicinity alone

24. Evaluating an Epidemiologic Study • Mountainous vs. non-mountainous counties • Rainfall in mountain areas is higher that valleys in general, but there is a great deal of variation between mountainous counties and between mountains and valleys in each county. Category of ‘mountainous’ is too general.

25. Evaluating an Epidemiologic Study • No rationale (or references) for the assumption of urban/rural differences in cancer mortality. • Why select Georgia, NC, SC and Tennessee and not include Kentucky and Virginia? • No references on why rural population might be a lower risk of cancer than urban populations. • Rural areas are not free of risks -- pesticides and herbicides, smoking, and others

26. Evaluating an Epidemiologic Study • Downwind counties • We have more information from dose reconstructions that contradicts some of the hypothesized wind patterns

27. Evaluating an Epidemiologic Study • What was the outcome of interest? • Deaths from cancer among whites in 94 counties between 1950-1952 and 1987-1989 compared with cancer deaths in U.S. and Southeast.

28. Evaluating an Epidemiologic Study • How was the health outcome measured in this study? • Cancer mortality data by counties were obtained from the National Cancer Institute • Based upon death certificates

29. Evaluating an Epidemiologic Study • What are some limitations or criticisms you might have about how the outcome was chosen and measured?

30. Evaluating an Epidemiologic Study • All cancers together rather than specific cancers were considered. • Remember, cancer is not one disease but over 100 different diseases arising from 40 sites in the body • There are many causes of these cancers-not all can be linked to environmental causes.

31. Evaluating an Epidemiologic Study • There is considerable inaccuracy in death certificate data (25% error) • Accuracy may vary by region-depending on who codes the certificates • Accuracy may vary by time. Improvements in diagnosis or coding may have led to an increase in cancer rates over time

32. Evaluating an Epidemiologic Study • Selection bias - could there have been bias(distortion or error) in how study subjects got selected into the study? • Mangano used already existing data so subjects were not selected.

33. Evaluating an Epidemiologic Study • Information bias - could there have been bias (distortion or error) in how information was collected from the study group and the comparison group or groups? • Accuracy of cause of death certification may vary by region of the country

34. Evaluating an Epidemiologic Study • Could the results have been due to confounding? • Confounding is the possibility that the relationship we see between exposure and disease could have been due, totally or in part, to other factors or differences between the study group and comparison group.

35. Evaluating an Epidemiologic Study • Potential Confounders • Smoking, diet, physical activity, and occupational exposures might have differed between the study counties and the rest of the Southeast or U.S. • Author assumes little or no migration. Yet we know there was a large influx of people into the Oak Ridge area in the 1940s and a subsequent declined in population. Younger people have a lower risk of cancer.

36. Evaluating an Epidemiologic Study • Life expectancy may differ by region over time. As life expectancy increased in the southeast, deaths from cancer might increase since it is primarily a disease of older people • Other sources of environmental contamination may have varied by region • Potential increase in pollution from utility and other plants in the area over time

37. Evaluating an Epidemiologic Study • Did the author measure any confounders? No. • No information on confounders is found on death certificates

38. Evaluating an Epidemiologic Study • Describe the methods used to measure the relationship between exposure and outcome. • Age-adjusted mortality rates (#deaths/population) for selected counties were calculated for 1950-1952 and 1987-1989 and compared to Southeast counties and U.S.

39. Evaluating an Epidemiologic Study • Statistical Analysis • Very little information on statistical methods including why 1950 standard was used • Why choose only 1950-52 and 1987-89? Why not rates by decade or calculate average annual increases? • Why not calculate rates for different age groups • Why not calculate rates for different cancers

40. Evaluating an Epidemiologic Study • What are the major results of the study. Author states that: • The cancer mortality rate increase in the Oak Ridge area (94 counties) exceeded the national and regional increases (30.5% vs. 5.1%) • Rural areas experienced the greatest increase

41. Evaluating an Epidemiologic Study • Counties downwind and in those in N.W. wind direction had greatest increase • Anderson county had greater increase than selected counties < 40 miles from plant • Most (but not all) mountainous areas had greatest increase

42. Evaluating an Epidemiologic Study • Results were positive for all hypotheses, however, • Has the author justified his conclusions?

43. Evaluating an Epidemiologic Study • How strong was the association between exposure and outcome? • Large differences were seen between study counties and the U.S. • Not so strong for Southeast

44. Evaluating an Epidemiologic Study • Has this relationship been seen by other researchers conducting other types of studies in different populations? • Several studies have looked at residence near nuclear facilities. Worker studies have also looked at radiation and cancer.

45. Evaluating an Epidemiologic Study • Is there a biologically plausible explanation between exposure and disease? Yes. • Ionizing radiation is known to cause cancer. However, radiation levels could not be measured --county of residence was the surrogate for ionizing radiation

46. Evaluating an Epidemiologic Study • Does the risk of disease increase with increasing exposure (dose-response). • Not demonstrated. No doses calculated, therefore no dose-response

47. Evaluating an Epidemiologic Study • Has exposure occurred before disease? • Unknown for an unknown number of people due to moving into and out of study counties

48. Evaluating an Epidemiologic Study • Authors conclusion: • Analysis of change in cancer mortality suggests a link between radiation contamination and increased risk to populations living near nuclear sites

49. Evaluating an Epidemiologic Study • Has the author convinced us of his conclusions? • Did this study advance our knowledge about the relationship between exposure and disease?

50. Evaluating an Epidemiologic Study • Limitations of study • No exposure measure – Limitations in use of residence in county surrogate for radiation exposure • Limitations to use of death certificate data and mortality rather than incidence of cancer • Lack of control for potential confounders • Focus on all cancers rather than those most likely to be related to exposure