Outcomes and AEs

1. Outcomes and AEs

2. Outline: Outcomes • Selecting a primary outcome • Surrogate measurements • Composite outcomes • Adverse experiences

3. Consultation • The company wants help designing a trial of a new treatment for osteoporosis in postmenopausal women. • “SERM”-like: can affect bones, breast cancer… other outcomes. • What outcomes should be assessed? • How should they assess safety?

4. Fracture Avoidance Trial (FAT) Potential osteoporosis outcomes • All non-vertebral fractures • Symptomatic vertebral fractures • New vertebral fractures detected by x-ray • Osteoporosis Quality of Life scale • Days of disability due to fracture • Height loss • Global index of risk of serious diseases.

5. How to start?

6. How to start? • Designate one “primary” and the others as “secondary” outcomes

7. Why one “primary” outcome? • To calculate sample size • Gives that outcome more credibility • The FDA requires that an outcome be “primary” in order to approve a drug for that indication

8. Which primary for FAT? Potential outcomes • All nonvertebral fractures • Symptomatic vertebral fractures • New vertebral fractures detected by x-ray • Osteoporosis Quality of Life scale • Days of disability due to fracture • Height loss • Global index of risk of serious diseases.

9. Considerations • Sample size and cost • Clinical importance • For companies: What does FDA need in order to approve an indication for prescribing the drug

10. Alternatives Improvement in BMD All fractures New ‘vert fx’ on x-ray Quality of Life Days of disability Height loss Sample size/duration 200/2 yrs 3,000/4 yrs 3,000/3 yrs 6,000/2 yrs 4,000/4 yrs 3,000/3 yrs Which Primary Outcome?Sample size

11. Why not make BMD the primary outcome? • Best choice to minimize cost • Issue: is it a good “surrogate marker” of clinical outcomes?

12. Criteria for validating a surrogate marker for treatments • Biologically plausible • Marker strongly predicts the clinical outcome

13. Risk of new vertebral fractures by quartile of BMD Hip BMD Spine BMD 9.9 10 9.5 10 8 8 6 6 5.2 % of women with fracture 4.6 4.2 3.9 4 4 2.7 1.8 2 2 0 0 1 2 3 4 1 2 3 4 Quartile of BMD Quartile of BMD

14. Criteria for validating a surrogate marker for treatments • Biologically plausible • Marker strongly predicts the clinical outcome • Treatment changes the marker • Treatment changes the rate of disease in the predicted direction * Prentice, 1989

15. Bone density • Biologically plausible: YES • Marker strongly predicts the clinical outcome: YES • Treatment changes the marker* • YES: treatment improves BMD~5% • Treatment changes the rate of disease in the predicted direction* • YES: other trials - 35-50% decrease risk * Prentice, 1989

16. Are we there yet? • BMD does all the right things a surrogate marker should do. • Anything else?

17. Bone density • Do changes in the surrogate (bone density) account for changes in reduction in the outcome (fractures) • This can only be determined by making the measurement in a randomized trial. * Prentice, 1989

18. Calcitonin Alendronate Etidronate Etidronate Estradiol 0 20 40 60 80 Decreased Vertebral Fracture Risk: Predicted from BMD vs. Observed 3 20 Predicted 13 5 8 Cummings, ASBMR 1997

19. Calcitonin Alendronate Etidronate Etidronate Estradiol 0 20 40 60 80 Decreased Vertebral Fracture Risk: Predicted from BMD vs. Observed 3 62 20 Predicted 50 Observed 13 58 5 56 8 61 Cummings, ASBMR 1997

20. Does BMD “Explain” the Reduction in Fracture Risk? Main methods • Freeman • For individual data from a trial • Meta-analysis • For aggregate data from multiple trials

21. Does BMD “Explain” the Reduction in Fracture Risk? Main methods • Freeman • For individual data from a trial • Estimates p = proportion of treatment effect “explained” by change in the marker • ß = coefficient for treatment • ß* = “ adjusted for change in the marker • (1 - ß* / ß)

22. Does BMD “Explain” the Reduction in Fracture Risk? • Freeman method applied to studies • FIT trial (alendronate): p = 0.16 • MORE trial (raloxifene): p = 0.05 • Very little of the treatment effects are due to individual improvements in spine BMD, as measured by DXA. • Stimulated research into other mechanisms

23. 1.5 1 Observed Relative Risk 0.5 0 0.75 0.8 0.85 0.9 0.95 1 1.05 Expected Relative Risk Relationship between improvement in spine BMD and decrease in risk of vertebral fracture Size of Study 500 1000 5000 Observed Each 1% gain in BMD reduces risk ~4%

24. 1.5 1 Observed Relative Risk 0.5 0 0.75 0.8 0.85 0.9 0.95 1 1.05 Expected Relative Risk Relationship between improvement in spine BMD and decrease in risk of vertebral fracture Size of Study 500 1000 5000 Expected Observed Each 1% gain in BMD reduces risk ~4% This slope is the same as predicted

25. 1.5 1 Observed Relative Risk 0.5 0 0.75 0.8 0.85 0.9 0.95 1 1.05 Expected Relative Risk Relationship between improvement in spine BMD and decrease in risk of vertebral fracture Size of Study 500 1000 5000 Expected 24% Observed Each 1% gain in BMD reduces risk 4% But antiresorptive treatment adds ~24% more

26. Surrogate markers that failed • Estrogen and cholesterol • Anti-arrhythmic drugs and sudden death • BMD and fracture

27. Limitations of surrogate markers is the main reason for relying on trials with clinical outcomes

28. Which primary outcome? • In trials that are not intended for FDA approval of a drug or new indication for its use, consider clinical importance and feasibility • Pharma does trials for FDA approval; approval requires meeting certain FDA guidelines. • FDA approves drugs ‘to treat osteoporosis’ if they drug reduce risk of vertebral fracture.

29. Alternatives Improvement in BMD Non-vertebral fxs New vert fx on x-ray Quality of Life scale Days of disability Height loss Global index Sample size/duration 200/2 yrs 3,000/4 yrs 3,000/3 yrs 6,000/2 yrs 4,000/4 yrs 3,000/3 yrs 12,000/5 yrs Which Primary Outcome for FAT?

30. Alternatives Improvement in BMD Non-vertebral fxs New vert fx on x-ray Quality of Life scale Days of disability Height loss Global index Sample size/duration 200/2 yrs 3,000/4 yrs 3,000/3 yrs 6,000/2 yrs 4,000/4 yrs 3,00/3 yrs 12,000/5 yrs Which Primary Outcome?

31. FAT 1° Outcome • Vertebral deformities assessed by x-ray • 3,000 women with osteoporosis for 3 years • Power =0.9; alpha =0.05

32. FAT 1° Outcome • Vertebral deformities assessed by x-ray • 3,000 women with osteoporosis for 3 years • Power =0.9; alpha =0.05 • Less important than most other outcomes, but the least expensive way to get FDA approval to market the drug.

33. How about combining events?

34. Composite endpoints Increase number of events Improve power… unless they dilute effect Must reflect the same (or very similar) underlying biology Combining all fractures Doubles the number of events How about combining events?

35. Composite endpoints Increase number of events Improve power… unless they dilute effect Must reflect the same (or very similar) underlying biology Combining all fractures Doubles the number of events Drugs decrease vert fxs 50%, others 0-25% Vert fxs are mostly due to low trabecular BMD, others mostly trauma. How about combining events?

36. FAT 1° Outcome • Vertebral deformities assessed by x-ray • 3,000 women with osteoporosis for 3 years • Power =0.9; alpha =0.05 • IF we combined vertebral deformities by x-ray and all types of fractures (ribs, wrist…etc.), the number of outcomes would more than double. • Would this be acceptable to the FDA • Should this be the primary outcome instead?

37. FAT 1° Outcome • Drugs have less effect on nonspine fractures. • Including them ‘dilutes’ the likely effect size (from 50 to ~25-30%) • Primary outcome: vertebral deformity on x-ray.

38. 2° Outcomes • Why are outcomes called “secondary?” • Less important, insufficient power • Penalized in the analysis phase • ConsiderProblem? Improved in BMD Decreased height loss Decreased hip fracture Decreased breast cancer

39. 2° Outcomes • ConsiderProblem Improved in BMD Less important Decreased height loss Less important Decreased hip fx Insufficient power Decreased breast ca Insufficient power

40. Treatment may reduce breast cancer risk • Ideally, the treatment would get an FDA indication for prevention of breast cancer and fractures. • How can you design the analysis plan for the trial to get both?

41. Adding Breast Cancer as a Co-primary Aim? • Why “co-primary aims” of a study? • Increases the credibility of the result” • FDA requires ‘primary aim’ for breast cancer? • Should the power be adjusted for having 2 primary outcomes?

42. Sharing Alpha • Setting p=.05 for all of the primary outcomes of trial. • Result will be called “significant” (drug approved) if any result is “statistically significant” at the specified level

43. Adding breast cancer • FDA requires “sharing” a total alpha = 0.05 • Co-primary outcomes; power =0.9 for both alpha • Breast cancer .048 • Vertebral deformities .003 • Why the difference in alpha?

44. Issues re: Adverse Events • Elicited vs. volunteered • Nuisance AEs • Attribution of cause

46. AEs

47. AEs for FAT? • The company’s standard approach: Record any symptoms or conditions the subject has experienced: ________________________________ _________________________________

48. AEs for FAT? • The company’s standard approach: Record any symptoms or conditions the subject has experienced: ________________________________ _________________________________ • What’s wrong with this approach?

49. An alternative? “Since your last visit, has a doctor told you you had (check all that apply)” __A blood clot in the leg (venous thrombosis) __ A blood clot in the lung (“P.E.”)… …for all possible diseases What’s wrong with this approach?

50. Pro elicited Approaches to AEsVolunteered vs. elicited Pro volunteered