Epi235 epi methods in hsr
1 / 47

EPI235: Epi Methods in HSR - PowerPoint PPT Presentation

  • Uploaded on

EPI235: Epi Methods in HSR. April 12, 2007 L4 Evaluating Health Services using administrative data 3: Advanced Topics on Risk Adjustment and Sensitivity Analysis (Dr. Schneeweiss)

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' EPI235: Epi Methods in HSR' - alanna

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Epi235 epi methods in hsr
EPI235: Epi Methods in HSR

  • April 12, 2007 L4

  • Evaluating Health Services using administrative data 3: Advanced Topics on Risk Adjustment and Sensitivity Analysis (Dr. Schneeweiss)

  • Risk adjustment in studies using administrative databases is limited to observed confounders. Dr. Schneeweiss will illustrate theory and practice of assessing the sensitivity of epidemiologic risk estimates towards unobserved confounding. An interactive Excel program will be used for illustration.

  • Background reading:

  • Walker AM: Observation and inference, Chapter 9. Epidemiology Resources, Newton Lower Falls, 1991.

  • Schneeweiss S, Glynn RJ, Tsai EH, Avorn J, Solomon DH. Adjusting for unmeasured confounders in pharmacoepidemiologic claims data using external information: The example of COX2 inhibitiors and myocardial infarction. Epidemiology 2005;16:17-24.

Unmeasured residual confounding
Unmeasured (residual) Confounding

  • Confounding factors that are not measured are hard to adjust for in observational analyses

  • If unadjusted they lead to residual confounding

Unmeasured residual confounding1





Drug exposure



Unmeasured (residual) Confounding:

[smoking,healthy lifestyle, etc.]

Unmeasured confounding in claims data
Unmeasured Confounding in Claims Data

  • Database studies are criticized for their inability to measure clinical and life-style parameters that are potential confounders in many pharmacoepi studies

    • OTC drug use

    • BMI

    • Clinical parameters: Lab values, blood pressure, X-ray

    • Physical functioning, ADL (activities of daily living)

    • Cognitive status

Strategies to discuss residual confounding
Strategies to Discuss Residual Confounding

  • Qualitative discussions of potential biases

  • Sensitivity analysis

    • SA is often seen as the ‘last line of defense’

    • A) SA to explore the strength of an association as a function of the strength of the unmeasured confounder

    • B) Answers the question “How strong must a confounder be to fully explain the observed association”

    • Several examples in Occupational Epi but also for claims data

Greenland S et al: Int Arch Occup Env Health 1994

Wang PS et al: J Am Geriatr Soc 2001

Foot in mouth award economist 04
Foot-in-Mouth Award (Economist ‘04):

“… there are known knowns; there are things we know we know. We also know that there are known unknowns; that is to say we know that there are some things we do not know. But there are also unknown unknowns – the ones we don’t know we don’t know. …, it is the latter category that tend to be the difficult ones.”

(Wisely unknowing) Donald Rumsfeld

A simple sensitivity analysis
A simple sensitivity analysis

  • The apparent RR is a function of the adjusted RR times ‘the imbalance of the unobserved confounder’

  • After solving for RR we can plug in values ofr the prevalence and strength of the confounder:

A made up example
A made-up example

  • Association between TNF-a blocking agents and NH lymphoma in RA patients

    • Let’s assume and observed RR of 2.0

    • Let’s assume 50% of RA patients have a more progressive immunologic disease

    • … and that more progressive disease is more likely to lead to NH lymphoma

    • Let’s now vary the imbalance of the hypothetical unobserved confounder

Pros and cons of array approach
Pros and cons of “Array approach”

  • Very easy to perform using Excel

  • Very informative to explore confounding with little prior knowledge


  • It usually does not really provide an answer to a specific research question

  • 4 parameters can vary -> in a 3-D plot 2 parameter have to be kept constant

  • The optical impression can be manipulated by choosing different ranges for the axes

Conclusion of array approach


Conclusion of “Array Approach”

  • Great tool but you need to be honest to yourself

  • For all but one tool that I present today:

    • Assuming conditional independence of CU and CM given the exposure status

    • If violated than residual bias may be overestimated

Hernan, Robins: Biometrics 1999





Drug exposure



More advanced techniques
More advanced techniques

  • Wouldn’t it be more interesting to know

    • How strong and imbalanced does a confounder have to be in order to fully explain the observed findings?




Wang et al: JAGS 2001;49:1685

Zolpidem use and hip fractures in older people.

The issue:

Are there any unmeasured factors that may lead to a preferred prescribing of zolpidem to people at higher risk for falling and fracturing?

> Frailty is a hard to measure concept in claims data





How do we do that
How do we do that?

  • We want to express as a function of , ARR, PC, PE



Walker AM: Observation and Inference. Epidemiology Resources Inc., Newton, 1991


Psaty et al: JAGS 1999;47:749

CCB use and acute MI.

The issue:

Are there any unmeasured factors that may lead to a preferred prescribing of CCB to people at higher risk for AMI?


ARR = 1.57

ARR = 1.30



  • Psaty et al. concluded that it is unlikely that an unmeasured confounder of that magnitude exists

  • However, the randomized trial ALLHAT showed no association between CCB use and AMI

  • Alternative explanations:

    • Joint residual confounding may be larger than anticipated from individual unmeasured confounders

    • Not an issue of residual confounding but other biases, e.g. control selection?

Pros and cons of rule out approach
Pros and cons of “Rule Out Approach”

  • Very easy to perform using Excel

  • Meaningful and easy to communicate interpretation

  • Study-specific interpretation


  • Still assuming conditional independence of CU and CM

  • “Rule Out” lacks any quantitative assessment of potential confounders that are unmeasured

External adjustment
External Adjustment

  • One step beyond sensitivity analyses

  • Using additional information not available in the main study

  • Often survey information

Strategies to adjust residual con founding using external information
Strategies to Adjust residual con-founding using external information

  • Survey information in a representative sample can be used to quantify the imbalance of risk factors that are not measured in claims among exposure groups

  • The association of such risk factors with the outcome can be assess from the medical literature (RCTs, observational studies)

Velentgas et al: PDS, under review

Schneeweiss et al: Epidemiology, in press 2004

How do we do that1
How do we do that?

  • We want to express ARR as a function of , , ARR, PC, PE



Walker AM: Observation an Inference. Epidemiology Resources Inc., Newton, 1991

Example cox 2 inhibitors use and mi
Example: COX-2 inhibitors use and MI

  • Ray et al., Lancet 2002:

    • >25mg roficoxib vs. non NSAID users, RR=1.9 (1.1-3.4)

    • Medicaid patients, new users

  • Solomon et al., Circulation in press:

    • >25mg roficoxib vs. non NSAID users, RR=1.6 (1.04-2.4)

    • Medicare patients with drug coverage through PACE

  • Can these associations be due to confounding by factors not measured in claims data?

    • e.g. BMI, OTC aspirin use, smoking, education etc.

In our example

[smoking,aspirin, BMI, etc.]




In our example:

From Survey data in a subsample

From medical literature



Acute MI


Where can we get detailed information on unmeasured confounders
Where can we get detailed information on unmeasured confounders?

  • MCBS: Medicare Current Beneficiary Survey

    • Representative Sample

    • 12,000 Medicare beneficiaries each year (majority > 65y)

    • Face-to-face interview in beneficiary’s home

    • ‘Cost and Use’ file include drug utilization

    • 98% response rate

    • >95% data completeness

    • Low cost ($900 / year)

    • Readily available, but 2-year lag time)

Unobserved confounders in our example
Unobserved confounders in our example confounders?

  • Independent predictors of MI:

    • Aspirin use

    • Smoking

    • BMI

    • Educational attainment

    • Income status

  • Expl. 2: Independent predictors of hip fracturs:

    • Cognitive impairment

    • Physical impairment

    • Restrictions in ADL (Rubinstein L)

Our survey population
Our survey population confounders?

  • 1999 MCBS

  • Restricted to >64 years

  • Restricted to community sample (no proxi interviews)

  • N = 8,785

Calculating bias
Calculating bias confounders?

More contrasts
More contrasts confounders?

What does it mean
What does it mean? confounders?

  • Ray et al.: RR of 1.9 is an underestimation of the unconfounded RR by 5% (max)

    • So the effect estimate corrected for 5 unobserved confounders would be about 2.0

  • Solomon et al.: RR of 1.6 would move to 1.7

Sensitivity of bias as a function of a misspecified rr cd
Sensitivity of Bias as a Function of a Misspecified RR confounders?CD :

Obesity (BMI >=30 vs. BMI<30)

Sensitivity towards a misspecified rr co from the literature
Sensitivity towards a misspecified RR confounders?CO from the literature:

OTC aspirin use (y/n)

Summary external adjustment
Summary External Adjustment confounders?

  • This method provides a quantitative assessment of the effect of selected unobserved confounders

  • Easy to use (Excel program available from author)

  • MCBS is available from CMS for $900 per annual survey

  • Should be more frequently used in Pharmacoepi studies using claims data

Limitations 1
Limitations (1) confounders?

  • Example is limited to 5 potential confounders

    • No lab values, physical activity, blood pressure etc.

    • What about the ‘unknow unknowns’?

  • We currently explore NHANES ’99/’00

    • Lab values, dietary suppl. (Ca2+),

    • Drug data quality?

  • To assess the bias we assume an exposure–disease association of 1 (null hypothesis)

    • The more the truth is away from the null the more bias in our bias estimate

    • However the less relevant unmeasured confounders become

Limitations 2
Limitations (2) confounders?

  • Validity depends on representativenes of sampling with regard to the unmeasured confounders

  • We could not consider the joint distribution of confounders

  • Limited to a binary world

Solving the main limitations
Solving the Main Limitations confounders?

  • Need a method

    • That addresses the joint distribution of several unmeasured confounders

    • That can handle binary, ordinal or normally distributed unmeasured confounders

  • Lin et al. (Biometrics 1998):

    • Can handle a single unmeasured covariate of any distribution

    • But can handle only 1 covariate

  • Sturmer, Schneeweiss et al. (AJE 2005 in press):

    • Propensity Score Calibration can handle multiple unmeasured covariates of any distribution