Biases in Studies of Screening Programs

1. Biases in Studies of Screening Programs Thomas B. Newman, MD, MPH June 29, 2012

2. Overview • Introduction • TN Biases • Definitions • Problems with observational studies • Volunteer bias • Lead time bias • Length bias • Stage migration bias • Overdiagnosis

3. Screening tests: TN Biases • “When your only tool is a hammer, you tend to see every problem as a nail.” • Clinical care accounts for 95% of spending but only 20% of determinants of health* • Biggest threats are public health threats • Interventions aimed at individuals are overemphasized because they are more profitable and we know how to do/sell them *Teutsch SM, Fielding JE. Comparative effectiveness: looking under the lamppost. JAMA 2011; 305:2225-6

4. Cultural characteristics "We live in a wasteful, technology driven, individualistic and death-denying culture." --George Annas, New Engl J Med, 1995

5. What is screening? • Common definition: testing to detect asymptomatic disease • Better definition*: application of a test to detect a potential disease or condition in people with no known signs or symptoms of that disease or condition. • Disease vs. condition • Asymptomatic vs. no known signs or symptoms *Common screening tests. David M. Eddy, editor. Philadelphia, PA: American College of Physicians, 1991

6. Screening tests may be history questions

7. Screening Spectrum Presymp-tomatic disease Recognized symptomatic disease Risk factor Unrecognized symptomatic disease Decreasing numbers labeled and treated Decreasing difficulty demonstrating benefit

8. Examples and overlap • Unrecognized symptomatic disease: vision and hearing problems in young children; iron deficiency anemia, depression • Presymptomatic disease: neonatal hypothyroidism, syphilis, HIV • Risk factor: hypercholesterolemia, hypertension • Somewhere between: prostate cancer, ductal carcinoma in situ of the breast, more severe hypertension

9. Evaluating Studies of Screening Mortality after Randomization D+ Screened D- R Mortaltiy after Randomization Not screened D+ D- Ideal Study: • Randomize patients to be screened or not • Compare outcomes in ENTIRE screened group to ENTIRE unscreened group

10. Observational studies: Patients are not randomized Compare outcomes in screened vs. unscreened patients Or among patients with disease: • Compare outcomes in those diagnosed by screening vs. those diagnosed by symptoms • Compare stage-specific survival with and without screening

11. KEY DIFFERENCE: Mortality vs. Survival • Mortality: denominator is a population, most of whom never get the disease • Survival: denominator is patients with the disease • Beware of any studies evaluating screening tests using survival

12. Possible Biases in Observational Studies of Screening Tests • Volunteer bias • Lead time bias • Length time bias • Stage migration bias • Overdiagnosis

13. Volunteer Bias People who volunteer for screening differ from those who do not Examples HIP Mammography study: Women who volunteered for mammography had lower heart disease death rates Multicenter Aneurysm Screening Study (EBD Problem 6.3) Men aged 65-74 were randomized to either receive an invitation for an abdominal ultrasound scan or not.

14. Multicenter Aneurysm Screening Study: Within Groups Result in Invited Group

15. Avoiding Volunteer Bias Randomize patients to screened and unscreened Otherwise, try to control for factors (confounders) associated with both screening and outcome • Examples: family history, level of health concern, other health behaviors, baseline health/illnesses

16. Lead Time Bias (zero-time bias) Screening identifies disease during a latent periodbefore it becomes symptomatic If survival is measured from time of diagnosis, screening will always improve survival even if treatment is ineffective

17. Lead time bias Source: EDITORIAL: Finding and Redefining Disease.Effective Clinical Practice, March/April 1999. Available at: ACP- Online http://www.acponline.org/journals/ecp/marapr99/primer.htm accessed 8/30/02

18. Avoiding Lead Time Bias Only occurs when survival from diagnosis is compared between diseased persons • Screened vs. not screened • Diagnosed by screening vs. by symptoms Avoiding lead time bias • Measure mortality, not survival • Count from date of randomization • Follow patients for a long time (20 years?) and use total, not e.g. 5-year survival

19. Length Bias (Different natural history bias) • Screening picks up prevalent disease • Prevalence = incidence x duration • Slowly growing tumors have greater duration in presymptomatic phase, therefore greater prevalence • Therefore, cases picked up by screening will be disproportionately those that are slow growing

20. Length bias Source: EDITORIAL: Finding and Redefining Disease.Effective Clinical Practice, March/April 1999. Available at: ACP- Online http://www.acponline.org/journals/ecp/marapr99/primer.htm

21. Length Bias Slower growing tumor with better prognosis ? Early detection Higher cure rate

22. Avoiding Length Bias Only present when • survival from diagnosis is compared • AND disease is heterogeneous Lead time bias usually present as well Avoiding length bias: • Compare mortality in the ENTIRE screened group to the ENTIRE unscreened group • Study disease subgroups with a uniform natural history

23. Stage migration bias Old tests New tests

24. Stage migration bias • Also called the "Will Rogers Phenomenon" • "When the Okies left Oklahoma and moved to California, they raised the average intelligence level in both states." -- Will Rogers • Documented with colon cancer at Yale • Other examples abound – the more you look for disease, the higher the prevalence and the better the prognosis Best reference on this topic: Black WC and Welch HG. Advances in diagnostic imaging and overestimation of disease prevalence and the benefits of therapy. NEJM 1993;328:1237-43.

25. Avoiding Stage Migration Bias The harder you look for disease, and the more advanced the technology the higher the prevalence, the higher the stage, and the better the (apparent) outcome for the stage Beware of stage migration in any stratified analysis Check OVERALL survival in screened vs. unscreened group More generally, do not stratify on factors distal in a causal pathway to the factor you wish to evaluate!

26. Overdiagnosis • Leads to diagnosis of “pseudodisease” • A condition that looks just like the disease, but never would have bothered the patient • Type I: Disease which would never cause symptoms • Type II: Preclinical disease in people who will die from another cause before disease presents

27. Overdiagnosis • In an individual treated patient it is impossible to distinguish pseudodisease from successfully treated asymptomatic disease • The Problem: • Treating pseudodisease will always look successful • Treating pseudodisease will always be (medically) harmful

28. Example: Mayo Lung Project • RCT of lung cancer screening • Enrollment 1971-76 • 9,211 male smokers randomized to two study arms • Intervention: chest x-ray and sputum cytology every 4 months for 6 years (75% compliance) • Control: Tests at trial entry, then a recommendation to receive the same tests annually *Marcus et al., JNCI 2000;92:1308-16

29. Mayo Lung Project Extended Follow-up Results* • Among those with lung cancer, intervention group had more cancers diagnosed at early stage and better survival *Marcus et al., JNCI 2000;92:1308-16

30. MLP Extended Follow-up Results* • Intervention group: slight increase in lung-cancer mortality (P=0.09 by 1996) *Marcus et al., JNCI 2000;92:1308-16

31. What happened? Black W. Overdiagnosis: an underrecognized cause of confusion and harm in cancer screening. JNCI 2000;92:1308-16 After 20 years of follow up, there was a significant increase (29%) in the total number of lung cancers in the screened group • Excess of tumors in early stage • No decrease in late stage tumors Overdiagnosis (pseudodisease)

32. Looking for Pseudodisease Appreciate the varying natural history of disease, and limits of diagnosis Impossible to distinguish from successful cure of (asymptomatic) disease in individual patient Few compelling stories of pseudodisease… Clues to pseudodisease: Higher cumulative incidence of disease in screened group No difference in overall mortality between screened and unscreened groups

33. More examples From Moynihan R et al.BMJ 2012;344:e3502, adapted from Welch HG and Black WC. JNCI: 2010;102:605-13

34. More examples From Moynihan R et al.BMJ 2012;344:e3502, adapted from Welch HG and Black WC. JNCI: 2010;102:605-13

35. More examples From Moynihan R et al.BMJ 2012;344:e3502, adapted from Welch HG and Black WC. JNCI: 2010;102:605-13

36. Review of biases Each year, 182,000 women are diagnosed with breast cancer and 43,300 die. One woman in eight either has or will develop breast cancer in her lifetime... If detected early, the five-year survival rate exceeds 95%. Mammograms are among the best early detection methods, yet 13 million women in the U.S. are 40 years old or older and have never had a mammogram. 39,800 Clicks per mammogram

37. Why is this misleading • Each year 43,000 die, 182,000 new cases suggests mortality is ~24% • 5-year survival > 95% with early detection suggests < 5% mortality, suggesting about 80% of these deaths preventable • Actual efficacy is closer < 20% for breast cancer mortality (lower for total mortality)

38. Questions?