1 / 41

Lessons about risk-benefit assessment from OMERACT

Lessons about risk-benefit assessment from OMERACT. Jasvinder Singh, MBBS, MPH Associate Professor of Medicine, UAB School of Medicine & VA Medical Center, Birmingham, Alabama Mayo Clinic School of Medicine, Rochester, MN. IMMPACT-XIV, Arlington, VA June 16-17, 2011. Acknowledgement.

sumi
Download Presentation

Lessons about risk-benefit assessment from OMERACT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lessons about risk-benefit assessment from OMERACT Jasvinder Singh, MBBS, MPH Associate Professor of Medicine, UAB School of Medicine & VA Medical Center, Birmingham, Alabama Mayo Clinic School of Medicine, Rochester, MN IMMPACT-XIV, Arlington, VA June 16-17, 2011

  2. Acknowledgement • Entire OMERACT Executive: Developed OMERACT 3 X 3 • Slides: Dr. Maarten Boers • Comments: Drs. Maarten Boers & Lee Simon

  3. Overview • Definitions • Summary and Discussions at 2008 OMERACT Drug Safety Summit • OMERACT approach • Benefit Risk Action Team (BRAT) approach • Pharmaceutical Research and Manufacturers of America • Questions and potential solutions • The Future

  4. Definitions: Risk and Benefit • Risk: • Oxford: “a situation involving exposure to danger” • Merriam-Webster: “possibility of loss or injury” • Medical context: an effect that is harmful to the patient’s or public’s health and which can relate to safety, efficacy or quality of a product. • Benefit: • Oxford: “an advantage or profit gained from something” • Merriam-Webster: “something that promotes well-being “

  5. Overview • Definitions • Summary and Discussions at 2008 OMERACT Drug Safety Summit • OMERACT approach • Benefit Risk Action Team (BRAT) approach • Pharmaceutical Research and Manufacturers of America • Questions and potential solutions • The Future

  6. OMERACT Drug Safety Summit (2008) • Assigning causality: • Bradford-Hill criteria + biological plausibility + Bayesian methods • No agreement on magnitude of frequency, i.e., when does something become a “signal” • Utility of large trials to define risk • Large, simple trials such as prospective randomized open blinded endpoint (PROBE) • Randomization to two effective treatments • Alternative: observational studies with confounding by indication and channeling bias

  7. OMERACT Drug Safety Summit (2008) • Utility of metanalyses of RCTs • Multiple small RCTs ?statistically equivalent to a large single trials with same denominator • 200-300 events needed for credible estimates • Role for indirect comparisons- unclear • Observational studies: may be included; • Selection and confounding bias issues • Outcome definition and measurement and follow-up

  8. OMERACT Drug Safety Summit (2008) • Postmarketing Surveillance • Essential for drug safety evaluation • All drugs in a class should be analyzed identically and concurrently • Desired components: • More than one defined population • Full protocol with outcomes assessed at regular intervals • Concurrent control subjects • Outcomes- patient-recoded + documented in electronic records • Pre-defined hypothesis • Oversight by FDA and data management council

  9. OMERACT Drug Safety Summit (2008) • Utilities of Drug registries • Advantages: real world experience, long-term FU • Disadvantages: lack of comparator group, data quality, patients not obligated to follow protocol, loss to follow-up • Pharmacoepidemiologic studies • Cohort studies generally better than case-control studies in providing risk estimates • Issues need to addressed • Misclassification bias • Validation of outcomes • Guidelines for confounding bias and methods for adjustment

  10. OMERACT Drug Safety Summit (2008) • Simple versus complex metric • Simple: OMERACT 3 x 3 • Benefit and risk categorized into 3 levels each- none, substantial, (near) remission or death • Complex: multicriteria decision analysis • Complex Frameworks: Quantitative methods • Decision analysis method • Conjoint analysis • Incremental net-benefit • BRAT approach

  11. OMERACT Drug Safety Summit (2008) • GRADE approach • Classifies evidence into 4 levels: high, moderate, low and very low based on • Study design, weaknesses, special strengths (large effect, dose response) • 2 recommendations: strong and weak • RCTs always starting at high and non-RCTs starting at low? • Other models of Risk: Nontreatment • Risk and value of available treatment versus nontreatment options • Type, intensity and severity of adverse event • Acute, subacute or chronic; manageable, treatable or not

  12. Overview • Definitions • Summary and Discussions at 2008 OMERACT Drug Safety Summit • OMERACT approach • Benefit Risk Action Team (BRAT) approach • Pharmaceutical Research and Manufacturers of America • Questions and potential solutions • The Future

  13. OMERACT 3 x3 • Create three categories of harm and benefit • Benefit: none/minimal, major, (near) remission • Harm: none/minimal, major, (near) death • Key components of this approach: • Harms versus benefits

  14. Problems in benefit-risk assessment • measurement of benefit is specific and evolved, but measurement of risk (or harm) is generic and more primitive • Benefit and harm not placed on a single scale single scale benefit-harm

  15. Conceptual problems • placing value judgments on scientific facts • values will vary depending on the perspective of the assessor • comparing benefit and risk involves: • comparing apples and pears • trading off (and discounting) long-term against short-term effects • assessing multiple benefits and risks • assessment mostly driven by the need to make decisions, whereas most research is ‘truth-driven’ single scale benefit-harm

  16. Practical problems • ...placing value judgments on scientific facts • Requires qualitative research • ...multiple comparisons and trade-offs • Difficult science • Requires decision analysis • ...mostly driven by decision-makers • Low speed of developments • Literature difficult to access single scale benefit-harm

  17. OMERACT 3 x 3 Table single scale benefit-harm

  18. Examples • COBRA trial • VIGOR trial single scale benefit-harm

  19. COBRA trial • RCT comparing combination therapy (COBRA) including step-down high-dose prednisolone with single drug therapy (sulfasalazine, SSZ) in early rheumatoid arthritis • Main result showed COBRA to be more effective and result in less side effects than SSZ single scale benefit-harm

  20. 3 x 3 COBRA Major benefit = Disease activity score <=3.7 (near) Remission = Disease activity score <=2.4 Severe Harm = treatment discontinuation due to serious harm, loss of efficacy or both (near) death = death or severe inacapacitation

  21. 2 x 2 COBRA ARR: Unqualified success: 80-54% = 26%; NNT = 4 ARR: Unmitigated failure: 5-14% = –9%; NNH = ? single scale benefit-harm

  22. Success/failure Unqualified success Unmitigated failure single scale benefit-harm

  23. VIGOR trial • large industry-sponsored trial to compare high-dose rofecoxib with naproxen in RA • object was GI safety, not efficacy • rofecoxib proved safer for GI, but less safe for cardiovascular events • Merck withdrew the drug voluntarily amidst much controversy • no access to individual patient data • safety and efficacy assumed independent single scale benefit-harm

  24. 2 x 2 VIGOR Data from original report and from Strand V. Lancet 2007;370:2138-51. Major harm is defined as cardiovascular event; gastrointestinal event; death. single scale benefit-harm

  25. Success/failure Unqualified success Unmitigated failure single scale benefit-harm

  26. Overview • Definitions • Summary and Discussions at 2008 OMERACT Drug Safety Summit • OMERACT approach • Benefit Risk Action Team (BRAT) approach • Pharmaceutical Research and Manufacturers of America • Questions and potential solutions • The Future

  27. STEP 1: Defining the decision context • Specifying the therapeutic context • Explicit definition of product, indication, target population, formulation, dosage and contraindications • Specifying the comparator • Standard of care, best in class, watchful waiting, placebo and nonpharmacological intervention • Defining the time horizon • Duration of exposure and time period over which benefit-risk events are measured • Specifying the stakeholder perspective • Sponsor, regulators, patients, physicians 1Coplan PM Clin Pharma Therap 2011; 89:2: 312-15

  28. Step 2: Identifying benefit and risk outcomes: Building the value tree • Identifying pool of candidate outcomes for the assessment • Deciding which outcomes to include or exclude from the framework • Documenting all critical assumptions for these inclusion and exclusion decisions • Value tree is a visual hierarchical presentation of key ideas, values or concepts 1Coplan PM Clin Pharma Therap 2011; 89:2: 312-15

  29. Step 3: Identifying data sources for the framework • Information may come from multiple data sources • A repository of all data, called data source table is kept 1Coplan PM Clin Pharma Therap 2011; 89:2: 312-15

  30. Step 4: Customizing the framework • Customization done based on data quality and limitations • Study end-points are organized in value tree at 2 levels • Body system category of the benefit or risk • The end-point measured in studies • Approaches to capturing level of severity of outcomes identified and the value tree is enhanced 1Coplan PM Clin Pharma Therap 2011; 89:2: 312-15

  31. Step 5: Assess relative importance of different outcomes • Assess relative weights • Stated choice methods • QALYs • Utilities • Value functions • Application of weighted approach in analysis • Net clinical benefit • Number needed to harm (NNH) • Multicriteria decision analysis 1Coplan PM Clin Pharma Therap 2011; 89:2: 312-15

  32. Step 6: Display and interpret key benefit-risk metrics • Key risk benefit table • Flexible table summarizing the key information needed to quantify outcomes in the value trees • Additional items to enhance key table • Heat map color coding • Embedded graphs 1Coplan PM Clin Pharma Therap 2011; 89:2: 312-15

  33. BRAT framework Example: Acute treatment of headache phase of migraine 1Levitan Clin Pharma Therap 2011; 89:2: 217-234

  34. Example 2: BRAT framework 1Levitan Clin Pharma Therap 2011; 89:2: 217-234

  35. Example 2: Key Benefit-risk table 1Levitan Clin Pharma Therap 2011; 89:2: 217-234

  36. Overview • Definitions • Summary and Discussions at 2008 OMERACT Drug Safety Summit • OMERACT approach • Benefit Risk Action Team (BRAT) approach • Pharmaceutical Research and Manufacturers of America • Questions and potential solutions • The Future

  37. Questions and potential solutions • How to come up with method (s) for risk-benefit assessment that is/are • Easily understood by regulators, patients and physicians • Take into account non-treatment or alternative treatments • Allow sensitivity analyses? • How to mandate and conduct efficient postmarketing surveillance studies? • Requirement by FDA and EMEA etc. • Funding • Preventing bias • Design and follow-up methodology

  38. Questions and potential solutions • Can efficacy trials be better designed to allow better short-term risk-benefit analysis? • What steps must be taken to use large databases to conduct postmarketing studies? • Standardization of methods for reduction of bias • Data integrity and privacy issues • How to achieve an agreement on methods by major regulators (FDA, EMEA)?

  39. Overview • Definitions • Summary and Discussions at 2008 OMERACT Drug Safety Summit • OMERACT approach • Benefit Risk Action Team (BRAT) approach • Pharmaceutical Research and Manufacturers of America • Questions and potential solutions • The Future

  40. Future • Collaborative networks of academia, regulatory agencies, patients and pharma • Use of preferences, values, cost and delay in drug approval (risks of alternatives) in risk-benefit analysis • Universally applicable methods • Ability to vary values, weights and perform sensitivity analyses • Utilize non-randomized studies • Design and conduct useful large simple comparative effectiveness trials

  41. Thank You

More Related