EPI 5240: Introduction to Epidemiology Bias and Misclassification November 9, 2009

1. EPI 5240:Introduction to EpidemiologyBias and MisclassificationNovember 9, 2009 Dr. N. Birkett, Department of Epidemiology & Community Medicine, University of Ottawa

2. Session Overview • Importance of Bias • Broad classes of bias • Examples of common biases • Misclassification • What can we do about bias and misclassification?

3. Potential Reasons for an Association • Bias • Systematic error due to study design of its implementation • Incorrectly calibrated scale • Bad interviewing • Confounding • A mixing of the effects of exposure and a third factor on the outcome • Random error/chance • That’s what statistics are for  • TRUTH

4. Bias (1) • A systematic error in a study which leads to a distortion of the results. • Can be deliberate (fraud) or due to investigator prejudice • BUT, this is UNCOMMON • Most often, due to design weaknesses or problems with study execution. • A more serious issue with observational studies than RCT’s. • ALL studies contain some bias!

5. Bias (2) • Small biases do not automatically invalidate a study • If you claim that a bias does invalidate a study: • You need to show why this is true. • Just pointing out a bias is not sufficient. • Likelihood of bias invalidate study depends of strength of association • OR/RR=1.2 is more likely to be explained by a bias than is an OR/RR of 3.0 • Control bias by good design and execution • Statistics generally aren’t much help with biases

6. Bias (3) • Bias can affect the OR/RR in any way • Convert a ‘true’ effect to a ‘null’ effect • Convert a ‘null’ effect to a ‘true’ effect • Reduce the size of a ‘true’ effect • Bias towards the null (‘good’ bias) • Increase the size of a ‘true’ effect • Bias away from the null (‘bad’ bias) • Convert a ‘true’ effect which increase risk to an observed effect which appears to reduce risk • Or vice-versa

7. Bias (4)

8. Bias (5)

9. Bias (6) • Several hundred types of biases have been classified and named • Don’t memorize them all  • Two main sources for bias • Categories have over-lap (gray zones) • Selection bias • The process of selecting or retaining study subjects distorts the relationship under study • Information bias • The process of collecting information on the study participants distorts the relationship under study. • Confounding is sometimes included here

10. Selection Bias (1) • An error due to systematic differences in characteristics between those who are selected for study and those who are not. • Includes differences between groups of study subjects • Occurs mainly in case-control and historical cohort studies • Can arise from: • Problems with which people enter a study • Problems with retaining people in the study once they are enrolled. • Can affect • External validity • Internal validity

11. Selection Bias (2) • Volunteer bias • People who volunteer are generally different from the general population • Younger • Healthier • Higher SES and education level • Mainly affects external validity • Generally OK to use volunteers in RCTs, etc. since focus is on variable relationships not external validity. • Results can potentially be adjusted using statistical models • Post-stratification • Requires knowledge of distribution of key factors in general population • A risky approach

13. Selection Bias (3) • Healthy worker effect • A common research question is to look for health effects of a work place exposure • Common design is to compare outcomes in workers to a general population (SMR) • Generally, workers will have less disease than general population • Workers are pre-selected to be healthier • Workers who get ill often will retire or quit. • Generates a bias which leads to under-estimates of the risk when workers are compared to the general public. (OR/SMR often < 1.0) • Note: this is a different effect from losing track of workers who get the target illness (loss-to-follow-up)

14. Selection Bias (4) • Clinic/referral bias • One of the causes of hypertension is renal artery stenosis. • In the 1960’s, research showed that about 10% of hypertension was caused by renal artery stenosis • Routine patient evaluation with intravenous pyelogram (IVP) • Expensive; risk of severe adverse reactions • In the 1970’s, new research showed that less than 0.5% of hypertension had renal artery stenosis. • Why the discrepancy?

15. Selection Bias (5) • Clinic/referral bias (cont) • Early studies were based in specialty hypertension treatment clinics. • They do the research • They got referred the ‘difficult’ or ‘interesting’ cases. • Later studies were done in general practice or with the general public. • They got ‘everyone’. • Referral bias is at work.

16. Selection Bias (6) • Loss-to-follow-up • Most serious selection bias problem with RCT’s and cohort studies. • Losses occur at different points of time in the study and for different reasons. • Random losses produce reduced power but no bias • Losses related to probability of getting outcome can produce a serious bias.

17. Selection Bias (7) • Loss-to-follow-up (cont) • Consider this situation • RCT of new medication to treat cancer. • In truth, the drug doesn’t work. • It does have more side effects than the control treatment. • Experimental group • Patients can tolerate the side effects when healthy but, as they become terminally ill, the side effects interfere with their quality of life. • As a result, those who are about to die drop out of the study. • Standard therapy group • No side effects so remain in study to death • Impact: Mortality rate in the group with the new treatment will be very low, making the new drug look ‘better’ than standard therapy. BIAS AWAY FROM TRUTH  BAD

18. Selection Bias (8) • Control selection in case-control studies • The biggest threat to the internal validity of a case-control study. • Consider this example: • Question: Does smoking cause lung cancer? • Cases: All men treated at the Ottawa hospital with lung cancer. • Controls: Men admitted to the respiratory ward with advanced emphysema • OR = 1.00 • WHY?

19. Selection Bias (9) • Control selection in case-control studies (cont) • Smoking causes emphysema as well as lung cancer. • Hence, the controls will be mostly smokers just like the cases. • Selection bias has distorted the smoking rate in the underlying target population and produced a serious bias. • BIAS TOWARDS THE NULL

20. Selection Bias (10) • Self-Selection bias • Similar to Volunteer Bias • Occurs when participation relates to both exposure and outcome status. • Either initial agreement to participate OR loss-to-follow-up • Differential participation by ONE of these factors does NOT produce a bias • Aim for 80+% participation over course of entire study

21. Selection Bias (11) Observed (20% not exp participate) The TRUTH 700 1000 300 1500*.2=300 2000 1000 1000 CIRobs = 2.33 CIRtrue = 2.33

22. Selection Bias (12) Observed (20% of exp/dis take part) The TRUTH 140 440 700*.2=140 5440 1640 CIRobs = 1.06 CIRtrue = 2.33

25. Selection Bias (13) • Differential Surveillance Bias (e.g. Berkson’s bias) • Similar underlying basis as self-selection • Consider a case-control study of DVT and oral contraception use • Assume there is really no effect • Cases • Women aged 20-44 hospitalized for DVT • Controls • Women age 20-44 hospitalized for elective surgery • OR = 5.0. • Why?

26. Selection Bias (14) • MD’s believed that OC use caused DVT • Women using OC with symptoms of DVT were more likely to be admitted to hospital • Hospitalization probabilities:

27. Selection Bias (15) • Incidence-prevalence bias (Neyman bias) • Using ‘prevalent’ cases in a case-control study causes bias. • Two ways to get cases for a breast cancer study • Start on January 1, 2010 and recruit all newly diagnosed women with breast cancer • Incident cases • Recruit all women on the active treatment roster at the Ottawa Cancer Center on January 1, 2010. • Prevalent cases • Prevalent cases lead to bias • Must have survived until the point of recruitment. • Cases with slowly progressive disease • Cases with treatment response • Including prevalent cases can bias study: • identification of factors promoting survival rather than the disease.

29. Information Bias (1) • Also called Observation & Measurement Bias • A flaw in measuring exposure, outcome (or confounders) that results in differential quality (accuracy) of information in the comparison groups. • Measuring height using a misassembled rule • For categorical variables, is essentially the same as misclassification • Can affect ALL types of studies.

30. Information Bias (2) • Occurs after subjects have been recruited • Pertains to how the data is collected • Can bias results towards or away from the null • More commonly a problem with questionnaires than physical measures • Often results in incorrect classification of subjects to exposure or outcome groups • Misclassification

31. Misclassification (1) • Occurs when subject is placed into the wrong category • Classed as a case when didn’t have disease • Classed as a control when did have disease (more common) • Classed as exposed when subject was not exposed. • Classed as not exposed when subject was exposed. • Concept can be extended to multi-level categories and to continuous variables

34. Misclassification (2) The TRUTH • Consider a case-control study. • Subjects who are truly exposed have a 10% chance of being classed as unexposed • Subjects who are truly unexposed have a 20% chance of being classed as exposed. ORtrue = 3.5

35. Misclassification (3) The TRUTH Observed 62 30*.9=27 41 103 60*.9=54 38 59 97 70*.2=14 40*.2=8 ORtrue = 2.35 ORtrue = 3.5

36. Misclassification (4) • In this example, the OR is biased towards the null. • Non-differential misclassification • The misclassification rates don’t differ between cases and controls. • ‘always’ biases towards null • As long as test isn’t really bad • Ignores random properties (Epi II topic) • ‘good’ type of misclassification since OR isn’t over-estimated.

37. Misclassification (5) • Differential misclassification • The misclassification rates differ between cases and controls. • Recall bias is a good example. • Cases have better (less misclassified) exposure estimates than do controls. • The bias can be in any direction and (almost) to any extreme desired. • Re-do previous example but now, cases have no misclassification while controls have 20% of exposed subjects misclassified and 5% of the non-exposed subjects.

38. Misclassification (5A) Controls Cases Truth No errors in exposure Determination. OBS

39. Misclassification (6) The TRUTH Observed 30*.8=24 27.5 87.5 72.5 112.5 70*.05=3.5 ORtrue = 3.95 ORtrue = 3.5

40. Information Bias (2) • Recall bias • A serious problem for case-control studies • Not a problem for cohort studies • People who have been diagnosed with a disease are more likely to remember past exposure than controls. • A mother of a child born with a serious developmental problem will tend to ruminate on the pregnancy and what might have gone wrong • DIFFERENTIAL MISCLASSIFICATION

42. Information Bias (3) • How to control recall bias • Use a structured questionnaire • Use biological measures or records • Different methods of administration • Self-admin • Computer • Randomized response • Ask subjects for knowledge of study hypothesis • ‘Mask’ key questions

44. Information Bias (4) • Interviewer bias • If interviewer knows the study hypothesis and which subjects are cases and controls, they might probe harder for exposure in one group. • Also, interviewer attitude might influence response patterns. • In an HIV study, cases are homosexual while controls aren’t. If an interviewer were biased against homosexuality, cases may be less forthcoming with sensitive information. • Prevent by blinding interviewers

45. Information bias (5) • Sample Questions (good or bad?) • Have you ever smoked? • Do you get out of breath when you walk up a hill? • Tell me the name of the brand of condoms you have used since you first had sex? • Have you ever used oral contraceptives, condoms, diaphragms or OCD when having sex? • When did you stop using heroin? • Translation issues

46. Summary • What can we do about bias? • Prevention is the key approach • Good design • Careful attention to issues in the field work of the study • Good strategies to retain study participants. • Once you get to the study analysis, there are very few options to handle bias.