A way forward: better decisions and better research
1 / 49

Tim Carey MD MPH Spring 2013 - PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

A way forward: better decisions and better research Comparing treatments in the new health care environment. Tim Carey MD MPH Spring 2013. Support. NIAMS- National Institute of Arthritis and Musculoskeletal Disease NIH Clinical Translational Science Award to UNC

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Tim Carey MD MPH Spring 2013

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

A way forward: better decisions and better researchComparing treatments in the new health care environment

Tim Carey MD MPH

Spring 2013


  • NIAMS- National Institute of Arthritis and Musculoskeletal Disease

  • NIH Clinical Translational Science Award to UNC

  • NCMHD-National Center for Minority Health and Health Disparities

  • AHRQ-Agency for Healthcare Research and Quality

  • Health Resources and Services Administration

  • Glaxo Smith Kline Foundation

  • Robert Wood Johnson Foundation

  • DERP- Drug Effectiveness Review Project

  • Dissemination grant supported by the Neurontin Special Committee

Nothing new

  • Clinicians have always compared one treatment with another

  • Most conditions have therapeutic options

    • Meds vs stent vs bypass surgery for coronary artery disease

    • Surgery vs radiation vs surveillance for prostate cancer

    • Decompression vs fusion vs exercise for spine disease

    • Fluoxetine vs. paroxetine for depression

  • Increase in efficacious treatments, and especially expensive efficacious rx

    • Rise in healthcare costs has led to renewed emphasis on comparative effectiveness and cost-effectiveness

    • Direct to consumer advertising

    • Information overload for providers

  • Increased emphasis on comparing treatments

    • Medications with each other

    • Procedures with each other

    • Procedures compared with medications or physical treatments (exercise, PT, etc)

One problem(among many)

  • Fractures due to osteoporosis are common, disabling in the elderly

    • Conventional treatment physical therapy, pain control, bone-strengthening medications

  • Vertebroplasty was developed in late ‘90’s

    • Biologic rationale

  • Case series demonstrated marked improvement after procedure

  • > 1,000 procedures in 2007 in NC alone

  • 2 randomized trials in 2009 demonstrated minimal, if any advantage over sham injection, medical regimen

    • # of patients across both trials=220

Comparative Effectiveness Research

CER is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers and policy makers to make informed decisions that will improve health care at both the individual and population levels.

Institute of Medicine, 2009

Patient Centered Outcomes Research

  • Patient-Centered Outcomes Research (PCOR) helps people and their caregivers communicate and make informed health care decisions, allowing their voices to be heard in assessing the value of health care options. This research answers patient-centered questions such as:

  • “Given my personal characteristics, conditions and preferences, what should I expect will happen to me?”

  • What are my options and what are the potential benefits and harms of those options?”

  • What can I do to improve the outcomes that are most important to me?”

  • “How can clinicians and the care delivery systems they work in help me make the best decisions about my health and healthcare?”

    PCORI board-2012

What is being compared?

  • Similar, definable treatments?

  • Appropriate outcomes that matter to patients?

  • Are harms being searched for?

  • Is the comparison treatment the current state of the art treatment?

  • Encompasses comparing systems of care as well as drug A vs drug B

    • Care management

    • Self-care, exercise

    • Payment issues

    • Care integration

  • Patient and stakeholder preferences taken into account?

    • Relationship to patient-centered outcome research

Structuring CER

  • Population

  • Intervention

  • Comparator

  • Outcome

  • Timeframe

  • Setting

Schematic for patient outcomes research

Coke vs Pepsi

  • Risk of losing perspective- how well does treatment work at all for the condition

  • Is it an interesting question to compare two similar medications (or procedures)?

    • Two statins

    • Patent vs generic

    • Harm profiles

    • Drug vs procedure; invasive vs non-invasive

  • What criteria should drive research investment?

    • Secondary analysis vs primary data collection

    • Large, simple trials

    • Which conditions are most important to examine?

    • Who are the constituencies? Agencies, manufacturers, researchers, advocates, patients, the public?


Do antidepressants differ in efficacy and effectiveness for the treatment of major depressive disorder, dysthymia, and subsyndromal depression?

Included Medications















Results: Excluded Studies

62 studies excluded because of

poor internal validity

  • High loss to followup

  • Single blinding

  • No intention-to-treat analysis

  • No systematic literature search for systematic reviews

Major Depressive Disorder:Body of Evidence

  • 72 head-to-head trials (including 3 effectiveness trials) on 16,780 patients

  • 18 studies assessed quality of life

  • We conducted 4 meta-analyses and 62 adjusted indirect comparisons

    • Outcome of interest: response to treatment

Major Depressive Disorder:Evidence of Comparative Efficacy

  • Overall, no substantial differences in efficacy

  • Statistically significant results from meta-analyses: modest and likely not clinically important

  • No differences in quality of life

    Strength of evidence: moderate


Citalopram vs. Duloxetine

0.76 (0.39, 1.47)

Escitalopram vs. Duloxetine

0.97 (0.71, 1.33)

Fluoxetine vs. Duloxetine

1.12 (0.84, 1.50)

Fluvoxamine vs. Duloxetine

1.59 (0.30, 8.45)

Paroxetine vs. Duloxetine

1.50 (0.88, 2.53)

Sertraline vs. Duloxetine

1.27 (0.99, 1.64)


Citalopram vs. Mirtazapine

0.78 (0.40, 1.53)

Escitalopram vs. Mirtazapine

1.01 (0.74, 1.37)

Fluoxetine vs. Mirtazapine

0.87 (0.72, 1.06)

Fluvoxamine vs. Mirtazapine

1.64 (0.31, 8.76)

Paroxetine vs. Mirtazapine

1.08 (0.88, 1.33)

Sertraline vs. Mirtazapine

0.92 (0.74, 1.14)

Citalopram vs. Venlafaxine

0.79 (0.41, 1.52)

Escitalopram vs. Venlafaxine

1.02 (0.82, 1.26)

*Fluoxetine vs. Venlafaxine

1.21 (1.01, 1.24)

Fluvoxamine vs. Venlafaxine

1.66 (0.31, 8.81)

Paroxetine vs. Venlafaxine

1.05 (0.75, 1.49)

Sertraline vs. Venlafaxine

0.88 (0.72, 1.07)


Duloxetine vs. Venlafaxine

1.28 (0.86, 1.91)

Duloxetine vs. Mirtazapine

1.03 (0.79, 1.35)

Mirtazapine vs. Venlafaxine

1.01 (0.81, 1.27)







Favors SSRI Favors SSNRI

Favors SNRI

Favors SSRI


Favors SNRI

* Based on meta-analysis of head-to-head trials

Major Depressive Disorder: Evidence of Comparative Efficacy

  • Although efficacy is similar, second-generation antidepressants are not identical

    • Mirtazapine has a significantly faster onset of action than SSRIs

    • Bupropion has less effect on sexual functioning than SSRIs

    • Strength of evidence: moderate

Ongoing Issues for Clinicians and Patients in depression treatment

  • Multiple treatment options may be necessary for many patients:

    • 40% of patients do not achieve clinical response with initial treatment

    • 10% - 15% discontinue treatment because of adverse events

    • Antidepressants differ significantly in dosing regimens

    • Need for treatment of med-refractory patients

      • Add medication? Switch medication?

      • When to use non-drug therapy such as ECT?

Use of existing dataIts not just claims data anymore

  • Many important comparisons can’t or won’t be evaluated in trials

  • Improved methods to control for selection bias

    • Propensity score, instrumental variables, etc

  • Enhanced methods to link databases

    • Disease registry linked with claims

    • Patient reported outcomes linked with EHR, etc

  • Somewhat improved access to data

    • Governance and security issues

    • Cost of data acquisition decreasing(?)

  • Electronic health records

    • Great promise, still a lot to learn

The Weight of the Evidence

Dissemination and implementation challenges

  • Research reports are long, technical and full of jargon

  • Not all research is of equal quality

    • Potential for bias in CER research

  • Links with health information technology initiatives promising but still early

  • Providers and ‘prompt fatigue’

Translation: Consumers

Source: ConsumerReportsHealth.org

ApplicabilityDoes one size fit all?

  • Occasionally a research study applies to nearly everyone with a condition

  • Most analyses have only limited ability to assess treatment effectiveness in sub-populations

    • Age, gender, ethnicity, income, co-existing conditions, genetic factors, prior treatment

  • Large research networks, hopefully incorporating large amounts of insurance claims data and/or EHR data are promising….

  • Need to take patient preferences into account when applying research results

  • Need for more information in these areas should not be an excuse for anything goes

Source: Schumock & Pickard; Am J Health-Syst Pharm 2009

Pragmatic clinical trials

  • Prospective comparison of a community, clinical, or system level intervention and a relevant comparator in participants who are similar to those affected by the condition(s) under study and in settings that are similar to those in which the condition is typically treated.

  • Underused, promising to enhance applicability, can engage stakeholders, PBRN’s

  • Significant infrastructure needed to reduce cost:

    • Governance and IRB issues

    • Informatics

    • Methods issues

Next steps for CTSA’s to enhance pragmatic trials

  • Each CTSA to have standing access to communities, practices and stakeholders for trial development, as well as implementation and dissemination

  • Regulatory relief: Modify informed consent for broad effectiveness studies. Central or deemed IRB. Contracting relief as well.

  • Learn by doing- work with a set of pragmatic trials- relationship with the IC’s. Use selected pragmatic trials as basis for learning.

  • Data terminology issues across sites, prioritize the most important issues, need policies in place to assist the community, need community input.

  • Methods focus in study design and analysis of treatment/site/patient heterogeneity

Enhancing multi-site studies

  • Need for inexpensive, rapid data collection

  • Challenges of heterogeneity of environment across sites

    • Heterogeneity of intervention?

  • Trialists and PBRN’s may have different views of ‘trials’

  • Timing and type of engagement

    • Practice, patient, public communities

Importance: Why We Need to Identify and Prioritize Research Gaps from Systematic Reviews

  • Systematic reviews are the standard for evaluating the current state of scientific knowledge regarding a specific clinical or policy question.

  • Identification and prioritization of research gaps has the potential to lead to more rapid generation of subsequent research, informed by input from stakeholders

  • Current studies often use only very general terms to describe future research ‘more and larger studies should be performed.’

  • Audiences including researchers, funders, clinicians, advocates, and patients could use information about prioritized research gaps to understand areas of uncertainty and more quickly initiate studies.

Primary Research



Assessment of


Prioritization of

New Research

Funding Opportunities

Conduct New


Systematic Review Update

Existing Methods to Identify and Prioritize Research Gaps

Agency for Healthcare Research and Quality Future Research Needs

  • Extract research gaps from a systematic review, transforming them into prioritized research questions with aided by diverse stakeholder groups.

  • 7 steps common to AHRQ FRN projects.

Example: AHRQ Future research needs on ADHD

Key Questions from Comparative Effectiveness Review

20 research gaps from the review mapped to the key questions, presented to a group of 12 stakeholders, including funders, advocates, clinicians, regulators, researchers, and policymakers.

After stakeholder input, 29 research gaps. 8 gaps emerged as the top future research needs after two rounds of prioritization using an online prioritization tool.

The next two slides show the presentation of one gap from identification to study design.

Example: AHRQ FRN on ADHD

Identify Research Gap:

For children less than 6 years of age with disruptive behavior disorder or ADHD, limited data are available about the efficacy and effectiveness of psychosocial treatment programs (e.g., parent training and summer behavior treatment programs), alone or in combination with pharmacological interventions, compared with other psychosocial treatment programs, alone or in combination with pharmacological interventions. (KQ 1)

After One Round of Prioritization Apply PICOTS and Develop Research Question:

Research Question: For children less than 6 years of age with disruptive behavior disorder or ADHD, what is the comparative efficacy and effectiveness of specific psychosocial treatments alone compared with pharmacological treatments alone or in combination with psychosocial treatments for patient outcomes?


After Second Round of Prioritization Develop Study Design Considerations:

  • Randomized controlled trials

  • Randomized trials could be designed to test various components in a 2x2 matrix of psychosocial treatment variants (parent training, school-based intervention, combination, or pharmacological).

  • Advantages of study design for producing a valid result

    • Allows isolation of causal inferences related to the intervention being tested. Multiple-armed trials would allow testing of several hypotheses regarding relative efficacy of singular or combination treatment components.

  • Ability to recruit/availability of data

    • Common condition in this age group with uncertainty regarding treatment choice; all arms receive some treatment.

  • Resource use, size, and duration

    • Large sample size (N = 840; n = 210 per treatment arm) needed. Key outcomes such as school achievement will require follow-up of several years.

  • Ethical, legal, and social issues

  • Vulnerable population, careful informed consent will need to occur.

State of the Science

  • Multiple groups are currently conducting work in this area

    • Local priorities important to assess

  • Sufficient common aspects to serve as a consensus

    • Criteria for gaps identification

    • Broad social input

    • Need for stakeholder training

    • Explicit prioritization method - but multiple methods currently used

    • Decisions regarding study design considerations

  • Prioritization should never imply discouraging scientific creativity

  • Early evaluation of new technology is critical

What about costs?Cost-effectiveness analysis (CEA)

  • CEA integral to European efforts, especially NICE

  • Perspective is critical

  • Some organizations forbidden from conducting CEA under Affordable Care Act

  • Cost savings vs cost-effectiveness

  • Risks of back-of-the envelope CEA

Effectiveness and ‘value’

  • Deferred future costs

  • Value of information analysis

    • Models in both planning for and interpreting research

  • MedPAC analyses

  • Patient-Centered Outcome Research Institute (PCORI)

  • (Independent Payment Advisory Board-IPAB)

  • CMS activities

    • Coverage with evidence development

    • CMS Innovation Center

  • State-based initiatives

  • Importance of distinguishing between absolute and comparative savings

Who is sponsoring the work?The alphabet soup

  • Patient-Centered Outcomes Research Institute (PCORI)

    • Private entity, multi-stakeholder board

    • ~$550M budget by 2014

    • Strong focus on role of patient/caregiver in research and decision-making

  • FDA

    • Regulatory role, not research

  • NIH

    • Historically not involved with CER, interest significantly higher now

    • ALLHAT, CATIE, STAR*D, SPORT. More to come?

    • CTSA and ‘Type II’ (bench to bedside) translational research

  • AHRQ

    • Effective Health Care Program; EPC’s, DEcIDE, etc

    • Education, dissemination, collaboration with PCORI

  • Drug Effectiveness Review Program: state Medicaid agencies

  • Industry

    • Limited incentive

    • “Do you feel lucky”- some potential to game comparisons

    • Vast amounts of marketing

  • Public good, public guardian

    • Widespread recognition that current system is dysfunctional

    • Evidence takes too long and is too expensive to generate

    • The presence of research doesn’t mean that it will be used

    • FDA role likely to change

      • Avandia, Vioxx, stents, vertebroplasty, etc

    • CMS taking increasing role

    • State Medicaid programs form consortia, major pressure to constrain costs, technology utilization

    • Loud megaphone of marketing


    • Substantial, but heterogeneous, current activity

    • Need to train additional researchers

      • MD, PhD; no consensus yet on core competencies

      • Train clinicians, public, administrators in use of research

    • Dissemination of findings into practice

    • Relationship to health information technology initiatives

      • Need for transparent relationship with vendors

      • Information must be combined with ease of implementation

      • Several infrastructure proposals pending to utilize EHR data for research, form consortia- long term goals


    • Patient Centered Outcomes Research Institute www.pcori.org

    • Agency for Healthcare Research and Quality www.ahrq.gov

    • Consumer’s Union http://www.consumerreports.org/health/best-buy-drugs/index.htm

    • Cochrane collaboration

      • http://www.cochrane.org/

    • Drug Effectiveness Review Project DERP

      • http://www.ohsu.edu/drugeffectiveness/

    Comparative effectiveness research

    • (Sort of) new wine

      • Interest is predominantly driven by technology availability, payer interest, rising chronic disease burden

    • New bottle

      • Federal and payer interest likely to be great in the next few years

      • Focus on stakeholder engagement and prioritization

      • Dissemination and implementation

      • Pro-active discussion to avoid political problems

      • Critical will be to maintain equipoise

        • Some research will find that more expensive treatments may be a dominant strategy

    Thank you and questions?

    Go Heels

    Beat Duke

    IOM Report:Commissioned by Congress in the American Recovery and Reinvestment Act (ARRA) of 2009

    June 30, 2009

    Examples of “Highest Priority”

    • Compare the effectiveness of dissemination and translation techniques to facilitate the use of CER by patients, clinicians, payers, and others.

    • Compare the effectiveness of comprehensive care coordination programs, such as the medical home, and usual care in managing children and adults with severe chronic disease, especially in populations with known health disparities.

    IOM June 30, 2009

    Examples of “High Priority”

    • Compare the effectiveness of strategies for enhancing patients’ adherence to medication regimens.

    • Compare the effectiveness of diverse models of transition support services for adults with complex health care needs (e.g., the elderly, homeless, mentally challenged) after hospital discharge.

    IOM June 30, 2009

    Activity in North Carolina

    • Clinical Trials

      • All academic health centers, private sector (Quintiles, etc)

    • Drug Effectiveness Review Project (Medicaid consortium)- UNC

    • PCORI- limited activity to date, UNC and Duke participating

    • Evidence-based Practice Centers (AHRQ)

      • RTI-UNC; Duke

    • Secondary data analyses: Developing Evidence to Inform Decisions about Effectiveness (AHRQ DEcIDE network)

      • Duke; UNC

    • Dissemination activities

      • Lineberger Cancer Center, CTSA network at UNC and Duke

    • Centers for Evidence and Research on Therapeutics (AHRQ and FDA CERT)

      • Duke

  • Login