Methods to analyze real world databases and registries

1. Methods toanalyze real worlddatabases and registries Hilal Maradit Kremers, MD MSc Mayo Clinic, Rochester, MN Clinical Research Methodology Course NYU-Hospital for Joint Diseases December 11, 2008

2. Disclosure Research funding from • National Institutes of Health (RA) • Amgen (psoriasis) • Pfizer (pulmonary arterial hypertension)

3. Outline • Terminology • Clinical trials versus observational studies and registries • Types of observational studies in rheumatic diseases • Descriptive epidemiology (incidence, prevalence) • Disease definitions (i.e. classification criteria) • Examining outcomes (including effectiveness of therapy) and risk factors (environmental, genetic) • Tips when interpreting results

4. Terminology “Real-world databases” & registries Observational studies =

5. Terminology of related observational research disciplines Health Services Research Epidemiology Economics Clinical Epidemiology Health Economics Outcomes research Pharmaco- epidemiology

6. Terminology: Clinical medicine versus epidemiology CLINICAL MEDICINE • Natural history of the disease • Signs and symptoms • Diagnosis (how and when) • Current clinical practice • Clinical literature • Drug-induced illnesses EPIDEMIOLOGY • Distribution and determinants of diseases in populations • Study design • Data collection • Measurement • Analyses • Interpretation • Critical review

7. Clinical trials versus observational studies and registries

8. Clinical trials versus observational studies and registries CLINICAL TRIAL Exposure - Disease Exposure + Exposure - COHORT / REGISTRY Disease Exposure + Disease - CASE- CONTROL Exposure Disease +

9. Why do we need registries • Limitations of pre-marketing trials • Unresolved issues from pre-marketing studies • New signals or inconsistent signals from post-marketing surveillance • Evolving concerns about safety • Establishing risk-benefit margins • Learn about use, Rx decisions, compliance and other physician/patient behaviors • To evaluate a risk management program

10. Clinical trial vs observational studies/registries – four “toos” • Too few • Too brief • Too simple • Too median-aged

11. Implications of four “toos” • Relative effectiveness unknown • Effectiveness in comparison to alternative therapies • Surrogate vs. clinical endpoints • Bone mineral density, blood pressure, lipid levels, tumor size, joint counts vs radiographic damage • Infrequent adverse events • Long latency adverse events • DES & adenocarcinoma of vagina • Special populations • Women, children, elderly, multiple comorbidities • Drug use in clinical practice

12. What is a registry? • Definition of a registry • An organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by particular disease, condition or exposure, and that serves a predetermined scientific, clinical, or policy purpose(s). • Different types of registries • Disease registry • Product registry • Health services registry • Pregnancy registries Registries for Evaluating Patient Outcomes. AHRQ Publication No. 07-EHC001. May 2007.

13. Purpose of a registry • Describe the natural history of disease • Determine clinical effectiveness or cost effectiveness of health care products, drugs and services • Measure or monitor safety and harm • Measure quality of care

14. Registry types • Disease registry • Patients who have the same diagnosis • e.g. all RA or SLE patients or rheumatic diseases • Product registry • Patients who have been exposed to biopharmaceutical products or medical devices • Health services registry • Patients who have had a common procedure, clinical encounter or hospitalization (TKA-THA registries)

15. Registries useful when: • Outcome is relatively common, well-defined and ascertainable & serious • Extensive drug exposure • Appropriate reference group • Data on relevant covariates ascertainable • Minimal channeling (preferential prescribing of a new drug to patients at a higher risk) • Minimal confounding by indication • Onset latency <2-3 years • Required drug exposure <2-3 years • Not an urgent drug safety crisis

16. Registries may not be useful when: • Outcome: poorly-defined, difficult to validate outcomes (depression, psychosis) • Exposure • Rare drug exposure • Intermittent exposure • OTC drugs, herbals • Significant confounding by indication • Antidepressants and suicides • Inhaled beta-agonists and asthma death • Certain settings • Specialty clinics, in-hospital drug use

17. Consequences of not doing registries or observational studies • Arguing over case reports • Lack of data on real benefit-risk balance • Less effective and usually biased decision-making • Possibly false conclusions • Law suits

18. Types of observational studies in rheumatology

19. Drug exposed patients Case reports Case series Registries Other Ecological studies Exposed vs. unexposed Cross-sectional Prospective cohort Case-control Observational study designs

20. Ecological studies – time series • When drug is predominant cause of the disease • Changes in outcomes following an abrupt change in drug exposure, as result of a policy or regulatory change, publications, media coverage • Reported Cases of Reye's Syndrome in Relation to the Timing of Public Announcements Belay et al. NEJM 1999; 340:1377

21. Ecological studies – time seriesSecular trends in NSAID use and colorectal cancer incidence Lamont: Cancer J 2008:14(4):276-277

22. Ecological studies – time seriesRofecoxib-celecoxib and myocardial infarction Brownstein et al. PLoS ONE. 2007:2(9):e840.

23. Summary: ecological studies Limitations • Complexity of disease causation • Confounding by the “ecological fallacy” Advantages • Cost ↓, time ↓, using routinely collected data • New hypotheses about the causes of a disease and new potential risk factors (e.g. air pollution) • Provides estimates of causal effects that are not attenuated by measurement error • Some risk factors for disease operate at the population level (i.e. SES status)

24. Studies on descriptive epidemiology of rheumatic diseases Incidence Prevalence Mortality

25. Diseased (RA) N=9 Prevalence:Proportion of individuals in a defined population who have a particular disease at a given point in time Population on 1/1/2005 N=100 Prevalence = 9/100 Prevalence = Incidence of disease x Duration of disease Diseased individuals

26. Incidence: Proportion of new cases of a disease or health-related condition in a population-at-risk over a specified period of time Population on 1/1/2005 N=100 1 year f-up Exclude prevalent cases leaving N=91 at risk Incidence=2 cases/91 person-years New-onset disease during 1 yr f-up Diseased individuals on 1/1/2005 deceased

27. Incidence of RA in Olmsted County, MN (1955-2005) Gabriel et al. A&R 2008: 58(9):S453

28. Incidence of PSA by age and sex (1970-2000) 20 15 Male 10 Incidence rate (per 100,000) 5 Female 0 20 30 40 50 60 70 80 Age Wilson et al. AC&R 2009: in press.

29. Incidence study requires keeping track of both the numerator & denominator! Population on 1/1/2005 N=100 1 yr 1 yr • Residents who die or move out of the city • New residents (i.e. new folks who move into the city) • All new-onset disease while living in the city • Possible in few locations in the world

30. Mortality analyses • RA: 124 studies in 84 unique cohorts1 • 15 key points in interpretation1 • Incident vs prevalent cases • Population-based vs clinic-based • SMR • Cause-specific mortality2 • 3 time dimensions in mortality analyses3 • Duration of RA • Timing of onset of RA relative to death • Calendar time 1 Sokka et al. Clinical Exp Rheum 2008;26(Suppl. 51): S35-S61 2 Aviña-Zubieta et al. A&R 2008; 59:1690-1697 3 Ward. A&R 2008; 59: 1687-1689

31. Mortality in incidence cohorts < prevalence cohorts 1 Sokka et al. Clinical Exp Rheum 2008; 26 (Suppl. 51): S-35-S-61 2 Aviña-Zubieta et al. A&R 2008; 59:1690-1697

32. Mild disease Referral bias: Population-based vs clinic-based cohorts Reality in the population N=100 What the GP sees N=92 What the rheumatologist sees! N= 40

33. SMR • Observed deaths ÷ expected deaths • Strongly influenced by choice of data to calculate expected deaths • Age and gender specific • Time period • Complete follow-up • Example: • RA cohort assembled between 1970-1990 with follow-up until 2000 • Expected mortality derived from US mortality rates between 1970-1990

34. 5 5 4 4 3 3 Mortality Rate (per 100 py) Mortality Rate (per 100 py) 2 2 1 1 0 0 1970 1980 1990 2000 1970 1980 1990 2000 Calendar Year Calendar Year Trends in RA Mortality vs. Expected* Males Females RA RA Expected Expected Gonzalez A, et al. Arthritis Rheum 2007;56(11):3583-587

35. Observed: expected mortality in RA Expected (non-RA) Observed (RA) Survival (%) P<0.001 0 5 10 15 20 Years after RA incidence Gabriel et al. A&R 2003; 48:54-58

36. Time: disease duration and CV mortality in RA Maradit Kremers A&R 2005; 52: 722-732

37. Summary: incidence, prevalence and mortality Consider • Underlying data source • Population-based or not • Incident vs prevalent cases • Methodology • Case ascertainment • Completeness of follow-up • Comparison data!

38. Disease definitions and classification criteria in rheumatic diseases Developed using observational study methodologies

39. 100 80 2 or more 60 3 or more Cumulative incidence, % 40 4 or more 20 5 or more 0 0 5 10 15 20 25 Years since RA incidence Dynamic nature of rheumatic diseases • 25% who initially met RA criteria still had evidence of RA 3-5 years later O’Sullivan et al. Ann Intern Med 1972; 76: 573-7. Mikkelsen et al. A&R 1969; 12: 87-91. Lichtenstein et al. J Rheumatol 1991; 18: 989-93. Icen et al. J Rheumatol 2008.

40. Typical vs desired methodology for classification criteria TYPICAL DESIRED Observe disease evolution Patients with established disease Patients with new-onset disease Compare characteristics Compare characteristics Patients with other established rheumatic diseases Patients with other new-onset rheumatic diseases Observe disease evolution

41. Examining outcomes and risk factors in rheumatic diseases Cohort Studies (outcomes) Registries (outcomes) Case-control studies (risk factors)

42. Types of Cohort Studies • Designated by the timing of data collection in the investigator’s time: • Prospective • Retrospective (historical) • Mixed • Mayo studies: retrospective • Registries: prospective

43. Investigator begins study Investigator begins study Investigator begins study Types of Cohort Studies Selection of Cohort Prospective (concurrent) Study Retrospective non-concurrent Study Mixed (P+R) Study All designs feasible either as ad hoc registries or in automated database studies.

44. Cohort study: design options • Prospective vs. retrospective • Entry into cohort: closed or open • Timing of exposure: new users or not • Source of un-exposed cohort • Internal • External • drug exposed subjects only, registries

45. Cohort Study: Steps • Cohort identification • Define subjects & follow-up period • Risk factor/drug exposure measurement throughout follow-up • Outcome (disease) ascertainment • Confounder measurements (throughout follow-up) • Analysis

46. Step 1 - Cohort identification • Trade-off between external and internal validity • Retrospective vs. prospective • Consider feasibility and costs • Follow-up • Tracking of drug changes over time • Losses to follow-up, esp. if likely to be differential (different for drug users and non-users)

47. Step 2 – Risk factor/Drug exposure measurement • New versus old users • Ability to account confounders before drug started • Ability to quantify outcomes early after starting the drug (compliance, early drop-offs due to intolerance) • Incomplete drug exposure • E.g. One time measurement of DMARD use and mortality • Drug exposure metric • Ever/never, dose (average, cumulative), duration • Reference group • Non-users, past users, users of other drugs • Misclassification of episodic use

48. Step 2 - Timing: patterns of drug use Antibiotic NSAIDs DMARDs

49. Step 2 - Drug exposure measurement methods • Interviews • Face-to-face, phone or self-administered • Excellent to capture current use but not for past use or changing drug use over time • Loss of memory – cognitively intact subjects & regularly used drugs • Biological testing • Blood or urine • Excellent to capture current use but not for past use • Non-differential (unless disease affects the assay) • Pharmacy or claims records • Medical records

50. Step 2 - Pharmacy or claims records for drug exposure • Drugs obtained by prescription • Drug details available • Accurate & complete for both past and current drug exposure • Temporal tracking possible • Limitation  compliance • Prescription filled and drug taking • Validation studies are necessary