Epidemiologic methods fall 2009
This presentation is the property of its rightful owner.
Sponsored Links
1 / 67

Epidemiologic Methods - Fall 2009 PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on
  • Presentation posted in: General

Epidemiologic Methods - Fall 2009. Bias in Clinical Research: General Aspects and Focus on Selection Bias. Framework for understanding error in clinical research systematic error: threats to internal validity (bias) random error: sampling error (chance)

Download Presentation

Epidemiologic Methods - Fall 2009

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Epidemiologic methods fall 2009

Epidemiologic Methods - Fall 2009


Bias in clinical research general aspects and focus on selection bias

Bias in Clinical Research: General Aspects and Focus on Selection Bias

  • Framework for understanding error in clinical research

    • systematic error: threats to internal validity (bias)

    • random error: sampling error (chance)

  • Selection bias (a type of systematic error)

    • by study design:

      • descriptive

      • case-control

      • cross-sectional

      • longitudinal studies (observational or experimental)


Clinical research

Clinical Research:

Sample

Measure

(Intervene)

Analyze

Infer

  • Inference

    • Websters: the act of passing from sample data to generalizations, usually with calculated degrees of certainty

    • All we can do is make educated guesses about the soundness of our inferences

    • Those who are more educated will make better guesses


Epidemiologic methods fall 2009

  • Anyone can get an answer

  • The challenge is to tell if it is correct


Epidemiologic methods fall 2009

OTHER POPULATIONS

Inference

Disease

+ -

+

-

Inference

Exposure

REFERENCE/

TARGET/

SOURCE POPULATION

aka

STUDY BASE

Two types of inferences

STUDY SAMPLE


Epidemiologic methods fall 2009

20 to 65 year olds, in Europe

>65 years old

in U.S.

Inference

Disease

+ -

+

-

Inference

Exposure

20 to 65 year olds, in U.S., outside of San Francisco

San Franciscans, 20 to 65 years old

SAMPLE of San Franciscans,

20 to 65 yrs old


Epidemiologic methods fall 2009

Attempts in study design to enhance the second inference are often in conflict with goal of making a sound first inference

Most important inference is the first one

Disease

Without an accurate first inference, there is little point considering the second inference

+ -

+

-

Inference

Exposure

REFERENCE/

TARGET/

SOURCE POPULATION

aka

STUDY BASE

STUDY SAMPLE


Epidemiologic methods fall 2009

Error in Clinical Research

  • The goal of any study is make an accurate (true) inference, i.e.:

    • measure of disease occurrence in a descriptive study

    • measure of association between exposure and disease in an analytic study

  • Ways of getting the wrong answer:

    • systematic error; aka bias

      • any systematic process in the conduct of a study that causes a distortion from the truth in a predictable direction

      • captured in the validity of the inference

    • random error; aka chance or sampling error

      • occurs because we cannot study everyone (we must sample)

      • direction is random and not predictable

      • captured in the precision of the inference (e.g., SE and CI)


Validity and precision each shot at target represents a study sample of a given sample size

Validity and Precision: Each Shot at Target Represents a Study Sample of a Given Sample Size

Good Validity

Good Precision

Poor Validity

Poor Precision


Validity and precision

Validity and Precision

Poor Validity

Good Precision

Good Validity

Poor Precision


Validity and precision1

Random error (chance)

Validity and Precision

Random error (chance)

No

Systematic error

Systematic error (bias)

Poor Validity

Good Precision

Good Validity

Poor Precision


Performing an actual study you only have one shot

Performing an Actual Study:You Only Have One Shot

Only judgment can tell you about systematic error (validity)

Field of “statistics” can tell you the random error (precision) with formulae for confidence intervals

Judgment requires substantive and methodologic knowledge


Epidemiologic methods fall 2009

? EXTERNAL VALIDITY (generalizability)

OTHER POPULATIONS

Inference

Disease

+ -

+

-

Inference

? INTERNAL VALIDITY

Exposure

REFERENCE/

TARGET/

SOURCE POPULATION

Two Types of InferencesCorrespond to Two Types of Validity

STUDY SAMPLE


Two types of inferences correspond to two types of validity

Two Types of InferencesCorrespond to Two Types of Validity

  • Internal validity

    • Do the results obtained from the actual subjects accurately represent the target/reference/source population?

  • External validity (generalizability)

    • Do the results obtained from the actual subjects pertain to persons outside of the source population?

    • Internal validity is a prerequisite for external validity

  • “Validity” to us typically means internal validity

    • “Threat to validity” = threat to internal validity

    • Identifying threats to validity is a critical aspect of research


Epidemiologic methods fall 2009

Error in Clinical Research

  • The goal of any study is make an accurate (true) inference, i.e.:

    • measure of disease occurrence in a descriptive study

    • measure of association between exposure and disease in an analytic study

  • Ways of getting the wrong answer:

    • systematic error; aka bias

      • a systematic process in the conduct of a study that causes a distortion from the truth in a predictable direction

      • captured in the validity of the inference

    • random error; aka chance or sampling error

      • occurs because we cannot study everyone (we must sample)

      • direction is random and not predictable

      • captured in the precision of the inference (e.g., SE and CI)


Epidemiologic methods fall 2009

MetLife Is Settling Bias Lawsuit

BUSINESS/FINANCIAL DESK

August 30, 2002, Friday

MetLife said yesterday that it had reached a preliminary settlement of a class-action lawsuit accusing it of charging blacks more than whites for life insurance from 1901 to 1972.

MetLife, based in New York, did not say how much the settlement was worth but said it should be covered by the $250 million, before tax, that it set aside for the case in February.


Bias in webster s dictionary

“Bias” in Webster’s Dictionary

1: a line diagonal to the grain of a fabric; especially: a line at a 45° angle to the selvage often utilized in the cutting of garments for smoother fit2 a: a peculiarity in the shape of a bowl that causes it to swerve when rolled on the green b: the tendency of a bowl to swerve; also: the impulse causing this tendency c: the swerve of the bowl3 a: bent or tendencyb: an inclination of temperament or outlook; especially: a personal and sometimes unreasoned judgment : prejudice

c: an instance of such prejudice

d (1) : deviation of the expected value of a statistical estimate from the quantity it estimates

(2) : systematic error introduced into sampling or testing

4 a: a voltage applied to a device (as a transistor control electrode) to establish a reference level for operation b: a high-frequency voltage combined with an audio signal to reduce distortion in tape recording


Bias of priene 600 540 bc

Bias of Priene (600 - 540 BC)

  • One of the 7 sages of classical antiquity

  • Consulted by Croesus, king of Lydia, about the bestway to deploy warships against the Ionians

  • Bias wished to avoid bloodshed, so he misled Croesus, falselyadvising him that the Ionians were buying horses

  • Bias later confessed to Croesusthat he had lied.

  • Croesus was pleased with the way that he had been deceived byBias and made peace with the Ionians.

  • Bias = deviation from truth

BMJ 2002;324:1071


Classification schemes for error

Classification Schemes for Error

  • Szklo and Nieto

    • Bias

      • Selection Bias

      • Information/Measurement Bias

    • Confounding

    • Chance

  • Other Common Approach

    • Bias

      • Selection Bias

      • Information/Measurement Bias

      • Confounding Bias

    • Chance

“BIG 4”


Emerging terminology causal research

Emerging Terminology: “Causal Research”

  • Goal: Identify causal relationships

  • 6 ways a statistical association can occur

    • Chance

    • Selection bias

    • Measurement bias

    • Confounding

    • Reverse causation

    • True causal relationship

  • Process of causal research: rule out the first 5


Selection bias

Selection Bias

  • Technical definition

    • Bias that is caused when individuals have different probabilities of being included in the study according to relevant study characteristics: namely, the exposure and the outcome of interest

  • Easier definition

    • Bias that is caused by some kind of systematic problem in the process of selecting subjects initially or - in a longitudinal study - in the process that determines which subjects drop out of the study

      • Problem caused by:

        • Investigators: Faulty study design

        • Participants: By choosing not to participate/ending participation

        • (or both)


Selection bias in a descriptive study

Selection Bias in a Descriptive Study

  • Surveys re: 1948 Presidential election

    • various methods used to find subjects

    • largest % favored Dewey

  • General election results

    • Truman beat Dewey

  • Fault: Bad Study Design

  • Ushered in realization of the importance of representative (random) sampling


N 894 sample actual vote

N= 894 sample Actual vote

The San Francisco Chronicle

Should Gov. Davis be recalled?

Yes 4,717,006 (55%)

No 3,809,090 (45%)

Election polls provide rare opportunity to later look at truth

Based on a survey conducted in English and Spanish among random samples of people likely to vote in California’s Oct. 7 recall election


Epidemiologic methods fall 2009

Descriptive Study: Unbiased Sampling

No Selection Bias

Even dispersion of arrows

SOURCE POPULATION

STUDY SAMPLE


Epidemiologic methods fall 2009

Descriptive Study: Biased Sampling

Presence of Selection Bias

Uneven dispersion of arrows

e.g., Dewey backers were over-represented

SOURCE POPULATION

STUDY SAMPLE


Leukemia among observers of a nuclear bomb test

Leukemia Among Observers of a Nuclear Bomb Test

Caldwell et al. JAMA 1980

  • Smoky Atomic Test in Nevada

  • Outcome of 76% of troops at site was later found; occurrence of leukemia determined

82% contacted by the investigators

18% contacted the investigators on their own

4.4 greater incidence of leukemia than those contacted by the investigators

Fault: Study design (look back studies are inherently limited) + the participants (especially who chose not to participate)


Epidemiologic methods fall 2009

Geng et al. JAMA 2008

Mortality following initiation of antiretroviral therapy in Uganda

In the presence of 39% loss to follow-up at 3 years


Epidemiologic methods fall 2009

Mortality following initiation of antiretroviral therapy in Uganda

Accounting for losses to follow-up by tracking down vital status of a sample of the lost in the community

Corrected estimate

Selection bias

Naive estimate


Epidemiologic methods fall 2009

Analytic Study: Unbiased Sampling

No Selection Bias

Disease

Given that a person resides in one of the 4 cells in the source population, the selection probability is the probability he/she will be represented in that cell in the study sample.

+ -

+

-

Exposure

SOURCE POPULATION

For no selection bias to occur, selection probabilities cannot differ according to both exposure and disease

STUDY SAMPLE


Epidemiologic methods fall 2009

Analytic Study: Biased Sampling

Presence of Selection Bias

Diseased

Unequal selection probability isolated to one cell:

Underestimate of Exposure Effect

+ -

+

-

Exposed

SOURCE POPULATION

STUDY SAMPLE


Selection bias in case control studies

Selection Bias in Case-Control Studies

Coffee and cancer of the pancreas

MacMahon et al. N Eng J Med 1981; 304:630-3

Cases: patients with histologic diagnosis of pancreatic cancer in any of 11 large hospitals in Boston and Rhode Island between October 1974 and August 1979

What study base gave rise to these cases?

How should controls be selected?


Selection bias in a case control study

Selection Bias in a Case-Control Study

  • Coffee and cancer of the pancreas

  • MacMahon et al. N Eng J Med 1981; 304:630-3

  • Controls:

  • Other patients without pancreatic cancer under the care of the same physician of the cases with pancreatic cancer.

  • Patients with diseases known to be associated with smoking or alcohol consumption were excluded


Epidemiologic methods fall 2009

Coffee and cancer of the pancreas

MacMahon et al., (N Eng J Med 1981; 304:630-3)

CaseControl

Coffee:

> 1 cup day

No coffee

216 307

OR= (207/9) / (275/32) = 2.7 (95% CI, 1.2-6.5)

Biased?


Epidemiologic methods fall 2009

  • Relative to the true study base that gave rise to the cases, the:

  • Controls were:

  • Other patients under the care of the same physician at the time of an interview with a patient with pancreatic cancer

    • Most of the MDs were gastroenterologists whose other patients were likely advised to stop using coffee

  • Patients with diseases known to be associated with smoking or alcohol consumption were excluded

    • Smoking and alcohol use are correlated with coffee use; therefore, sample is relatively depleted of coffee users

    • Conclusion: Controls vastly depleted of coffee users compared to true study base

    • Fault: Investigators (Poor study design)


Epidemiologic methods fall 2009

Case-control Study of Coffee and Pancreatic Cancer:

Selection Bias

Cancer No cancer

coffee

no coffee

Bias: overestimate effect of coffee in causing cancer

SOURCE POPULATION

STUDY SAMPLE


Epidemiologic methods fall 2009

84

82

10

14

  • Coffee and cancer of the pancreas:

  • Use of population-based controls

  • Gold et al. Cancer 1985

CaseControl

Coffee:

> 1 cup day

No coffee

OR= (84/10) / (82/14) = 1.4 (95% CI, 0.55 - 3.8)


Epidemiologic methods fall 2009

Selection Bias in a Cross-sectional Study:Presence of exposure and disease at outset invites selection bias

Disease

+ -

Equal selection probability in all 4 cells:

No Selection Bias

+

-

Exposure

SOURCE POPULATION

STUDY SAMPLE


Epidemiologic methods fall 2009

Selection Bias in a Cross-sectional Study:Presence of exposure and disease at outset invites selection bias

Disease

+ -

Unequal selection probability:

Overestimate of Effect

+

-

Exposure

SOURCE POPULATION

STUDY SAMPLE


Epidemiologic methods fall 2009

Selection Bias in a Cross-sectional Study:Presence of exposure and disease at outset invites selection bias

Disease

+ -

Unequal selection probability:

Underestimate of Effect

+

-

Exposure

SOURCE POPULATION

STUDY SAMPLE


Epidemiologic methods fall 2009

Selection Bias in a Cross-sectional Study:Presence of exposure and disease at outset invites selection bias

Disease

+ -

Unequal selection probability:

Overestimate of Effect

+

-

Exposure

SOURCE POPULATION

STUDY SAMPLE


Epidemiologic methods fall 2009

Selection Bias in a Cross-sectional Study:Presence of exposure and disease at outset invites selection bias

Disease

+ -

Unequal selection probability:

Underestimate of Effect

+

-

Exposure

SOURCE POPULATION

STUDY SAMPLE


Epidemiologic methods fall 2009

Selection Bias in a Cross-sectional Study:Presence of exposure and disease at outset invites selection bias

Disease

+ -

Typically you don’t know the selection probabilities

+

-

Exposure

?

?

?

SOURCE POPULATION

?

STUDY SAMPLE


Epidemiologic methods fall 2009

Selection Bias in a Cross-sectional Study:Effect of Non-Responders

History of Heart Attack

+ -

+

-

Hyper-lipidemia

?

?

SOURCE POPULATION

?

25

347

?

Overall 82% Response

45

2312

Austin, AJE 1981

Survey of S. California adults

OR observed = 3.6

STUDY SAMPLE


Epidemiologic methods fall 2009

Selection Bias in a Cross-sectional Study:Effect of Non-Responders

History of Heart Attack

Investigators made the extra effort to track down and question the initial non-responders

+ -

+

-

30

401

Hyper-lipidemia

100%

100%

63

2807

100%

100%

CORRECTED

STUDY SAMPLE

SOURCE POPULATION

Selection probability

OR true = 3.3

Austin, AJE 1981

Survey of S. California adults


Epidemiologic methods fall 2009

Selection Bias in a Cross-sectional Study:Effect of Non-Responders

History of Heart Attack

Investigators made the extra effort to track down and question the initial non-responders

+ -

+

-

30

401

Hyper-lipidemia

100%

100%

63

2807

100%

100%

CORRECTED

STUDY SAMPLE

SOURCE POPULATION

OR true = 3.3

25

347

83%

87%

Response %

Selection bias

45

2312

Austin, AJE 1981

Survey of S. California adults

72%

83%

OR biased = 3.6

STUDY SAMPLE


Effect of unequal response probabilities in a cross sectional study

Effect of unequal response probabilities in a cross-sectional study

Fault: The Participants

(Study design is fine)

Austin, AJE 1981

Survey of S. California adults


Another mechanism for selection bias in cross sectional studies

Another Mechanism for Selection Bias in Cross-sectional Studies

  • Finding a diseased person in a cross-sectional study requires 2 things:

    • the disease occurred in the first place

    • person survived long enough to be sampled

  • Any factor found associated with a prevalent case of disease might be associated with disease development, survival with disease, or both

  • Assuming goal is to find factors associated with disease development (etiologic research), bias in prevalence ratio occurs any time that exposure under study is associated with survival with disease


Epidemiologic methods fall 2009

Cross-Sectional Study Design


Selection bias in a cross sectional study

Selection Bias in a Cross-Sectional Study

  • Is glutathione S-transferase class  deletion (GSTM1-null) polymorphism associated with increased risk of breast cancer?

  • With prevalent breast cancer, an association with GSTM1-null is seen depending upon the number of years since diagnosis

  • But not with brand new incident diagnoses

Kelsey et al. Canc Epi Bio Prev 1997


Epidemiologic methods fall 2009

Cross-sectional study of GSTM1 polymorphismand breast cancer

Breast Cancer

+ -

Bias: overestimate effect of GSTM-1 null polymorphism in causing breast cancer

null

GSTM1

pos.

SOURCE POPULATION

Fault: Study design

STUDY SAMPLE


Selection bias cohort studies rcts

Selection Bias: Cohort Studies/RCTs

  • Among initially selected subjects, selection bias “on the front end” less likely to occur compared to case-control or cross-sectional studies

    • Reason: study participants (exposed or unexposed; treatment vs placebo) are selected (theoretically) before the outcome occurs


Epidemiologic methods fall 2009

Cohort Study/RCT

At the outset, since disease has not occurred yet among initially selected subjects, there is typically no opportunity for disproportionate sampling with respect to exposure and disease. (We cannot yet draw the 4 arrows)

Disease

+ -

+

-

Exposure

SOURCE POPULATION

STUDY SAMPLE


Epidemiologic methods fall 2009

Cohort Study/RCT

All that is sampled is exposure status (the “margins”)

Even if disproportionate sampling of exposed or unexposed groups occurs, it will not result in selection bias when forming measures of association

Disease

+ -

+

-

A + B

Exposure

C + D

SOURCE POPULATION

a + b

c + d

STUDY SAMPLE


Selection bias cohort studies

Selection Bias: Cohort Studies

  • Selection bias can occur on the “front-end” of the cohort if diseased individuals are unknowingly entered into the cohort

  • e.g.:

    • Consider a cohort study of effect of exercise on all-cause mortality in persons initially thought to be completely healthy.

    • If some participants were enrolled had undiagnosed cardiovascular disease and as a consequence were more likely to exercise less, what would happen to the measure of association?


Epidemiologic methods fall 2009

Cohort Study of Exercise and Survival

Selection bias will lead to spurious protective effect of exercise (assuming truly no effect)

Death No death

exercise

no exercise

SOURCE

POPULATION

STUDY SAMPLE


Selection bias cohort studies rcts1

Selection Bias: Cohort Studies/RCTs

  • Most common form of selection bias does not occur with the process of initial selection of subjects

  • Instead, selection bias most commonly caused by forces that determine length of participation (who ultimately stays in the analysis; losses)

    • When those lost to follow-up have a different probability of the outcome than those who remain (i.e. informative censoring) in at least one of the exposure groups

      AND

    • Rate of informative censoring differs across exposure groups

      • Selection bias results


Selection bias cohort studies1

Selection Bias: Cohort Studies

e.g., Cohort study of progression to AIDS: IDU vs homosexual men

All the ingredients are present:

  • Informative censoring is present

    • getting sick is a common reason for loss to follow-up

    • persons who are lost to follow-up have greater AIDS incidence than those who remain (i.e., informative censoring)

  • Informative censoring is differential across exposure groups

    • IDU more likely to become lost to follow-up - at any level of feeling sick

    • i.e., the magnitude of informative censoring differs across exposure groups (IDU vs homosexual men)

  • Result: selection bias -- underestimates the incidence of AIDS in IDU relative to homosexual men


Effect of selection bias in a cohort study

Effect of Selection Bias in a Cohort Study

Effect of informative censoring in homosexual men group

Effect of informative censoring in IDU group

Probability of being AIDS-free

Selection bias

Survival assuming no informative censoring and no difference between IDU and homosexual men

Time


Epidemiologic methods fall 2009

Cohort Study of HIV Risk Group and AIDS Progression

Selection bias will lead to spurious underestimation of AIDS incidence in both exposure groups, more so in IDU group

AIDS No AIDS

IDU

Homo-sexual

men

SOURCE POPULATION

Fault: The Participants

(Study design is fine)

STUDY SAMPLE


Effect of losses to follow up in a cohort study

Effect of losses to follow-up in a cohort study

Naively Ignoring Losses

Tracking Down Vital Status on Losses

Determinants of survival after initiation of antiretroviral therapy in Africa

Bisson, PLoSOne, 2008


Selection bias in a randomized clinical trial

Selection Bias in a Randomized Clinical Trial

  • If randomization is performed correctly, then selection bias on the “front-end” of the study (i.e., differential inclusion of diseased individuals between arms) is not possible (other than by chance)

    • even if diseased individuals are unknowingly included, randomization typically ensures that this occurs evenly across treatment groups


Selection bias in a clinical trial

Selection Bias in a Clinical Trial

  • Losses to follow-up are the big unknown in clinical trials and the major potential cause of selection bias

  • e.g., Assume that:

    • a symptom-causing side effect of a drug is more common in persons “sick” from the disease under study

    • occurrence of the side effect is associated with more losses to follow-up

  • Then:

    • Compared to placebo, drug treatment group would be selectively depleted of the sickest persons (i.e., informative censoring)

    • Would make drug treatment group appear better


Effect of selection bias in an rct

Effect of Selection Bias in an RCT

Effect of informative censoring in drug group

Probability of non-disease

Survival assuming no informative censoring and no difference between drug and placebo

Time


Managing selection bias

Managing Selection Bias

  • Prevention and avoidance are critical

    • Unlike confounding where there are solutions in the analysis of the data, once the subjects are selected and their follow-up occurs, there are usually no easy fixes for selection bias

  • In case-control studies:

    • Follow the study base principle

  • In cross-sectional studies:

    • Strive for high response percentages

    • Be aware of how exposure in question affects disease survival

  • In longitudinal studies (cohorts/RCTs):

    • Screen for occult disease/precursors at baseline

    • Avoid losses to follow-up

    • Consider approaches to tracking down the lost


  • Login