How to develop brief economic commentaries for cochrane intervention reviews cirs
This presentation is the property of its rightful owner.
Sponsored Links
1 / 47

How to develop brief economic commentaries for Cochrane intervention reviews (CIRs) PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on
  • Presentation posted in: General

19 th Cochrane Colloquium, Madrid, October 2011. How to develop brief economic commentaries for Cochrane intervention reviews (CIRs). A Campbell and Cochrane Economics Methods Group Methods Training Workshop www.c-cemg.org. Background.

Download Presentation

How to develop brief economic commentaries for Cochrane intervention reviews (CIRs)

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


How to develop brief economic commentaries for cochrane intervention reviews cirs

19th Cochrane Colloquium, Madrid, October 2011

How to develop brief economic commentaries for Cochrane intervention reviews (CIRs)

A Campbell and Cochrane Economics Methods Group

Methods Training Workshop

www.c-cemg.org


Background

Background

  • End-user decisions increasingly need to take account of evidence for resource use, costs, cost-effectiveness

  • Economic perspectives and evidence are either absent from CIRs or injudiciously treated

  • At best CIRs lack relevance and impact and at worst may mislead

  • Extending CIRs to include economic perspectives and evidence can increase applicability for end-user decisions

    Shemilt et al. Cochrane Handbook Chp 15: Incorporating economic evidence, 2008.

    Lavis et al. J Health Serv Res Policy2005; 10 (S1): 35-48.


Background1

Background

  • Integrated SR of economic evaluations requires specialist expertise/support and can be time and resource intensive

  • What can be achieved with little time and without specialist expertise?

  • Editorial Unit proposal for brief economic commentaries - place an ‘economic lens’ on health condition and interventions, without major additional resource or workload burden

  • Pilot study to develop and evaluate process


Pilot study

Pilot study

  • 38 new CIRs - CDSR Issue 1, 2011

  • Developed and applied process iteratively based on pre-specified study protocol

  • Recorded and analysed process and search yield data

  • Formulated 25 recommendations


Cost of illness studies

Cost-of-illness studies

  • Identify and estimate all the costs of a health condition in a defined population over a specified time period

  • Monetised estimates of the total economic burden of the health condition

  • Maximum amount potentially saved or gained if health condition were eradicated

  • Analytic perspective: direct health care costs > societal costs

  • Geographical scope: within-country region > country > world-region > global

  • Use to inform economic commentary integrated into ‘Background’ section of CIR - ‘Description of the condition’ and ‘Why it is important to do the review’


Economic evaluations

Economic evaluations

Combine costs with effects

Evidence for relative efficiency - joint distribution of costs and effects

Conducted either within the framework of a single empirical study

OR

Conducted within the framework of a decision model

Comparison between two or more alternatives in terms of costs.

Cost = Resource use x Unit cost

Drummond et al. Methods for the Economic Evaluation of Health Care Programmes (3rd edition), pp.11., 2005.


Nhs eed about

NHS EED - About


Nhs eed search interface

NHS EED - Search Interface


Nhs eed structured abstract

NHS EED - Structured abstract


Nhs eed citation only record

NHS EED - Citation-only record


Heed expert search interface

HEED - Expert search Interface


Heed free access via cochrane org

HEED - Free access via cochrane.org


Heed field coded abstract

HEED - Field-coded abstract


Heed citation only record

HEED - Citation-only record


Developing economic commentaries basic stages of process

Developing economic commentaries: Basic stages of process

1

2

3


Brito et al 2011 plain language summary

Brito et al, 2011Plain Language Summary

  • Unstable angina

  • Non ST-elevation myocardial infarction (Non-STEMI)

  • ST-elevation myocardial infarction (STEMI)

The use of unfractionated heparin and low molecular weight heparins greatly reduces the risk of mortality and morbidity in acute coronary syndromes. However, their use has been associated with a risk of adverse events such as major bleeding, which has prompted researchers to seek safer alternative anticoagulants such as the synthetic inhibitors of the Xa factor - a crucial enzyme in the coagulation cascade. We systematically reviewed the efficacy and safety of factor Xa inhibitors in treating acute coronary syndromes when compared to unfractionated heparins or low molecular weight heparins.

Comparators

Intervention

Health condition


Developing economic commentaries basic stages of process1

Developing economic commentaries: Basic stages of process

1

2

3


Heed search for cost of illness studies

HEED search for cost-of-illness studies

  • Aim to locate the few most useful records:-

    • Economic burden (cost-of-illness) of the health condition being addressed

    • Recent cost-of-illness studies or reviews of cost-of-illness studies

    • International comparisons or world-region > global estimates

    • Wider societal economic burden (alongside economic burden to health systems)

  • Based on keyword search terms designed to capture ‘Population’ concepts

  • Adapted from searches used to locate eligible efficacy or effectiveness studies


Heed search for cost of illness studies1

HEED search for cost-of-illness studies

EMBASE:-

  • Myocardial Ischemi$.mp.

  • angina.ti,ab.

  • myocardial infarct$.mp.

  • heart infarct$.mp.

  • acute coronar$.mp.

  • coronary syndrom$.mp.

  • (Preinfarct$ or pre infarct$).mp.

  • (STEMI or NONSTEMI or NON-STEMI or NSTEMI).mp.

  • ACS.ti,ab.

  • Retrieved 232 records

  • Pilot study mean was 158 (n=38)

HEED (Expert search):-

(AX= (myocardial AND ischemi*) OR angina OR (myocardial AND infarct*) OR (heart AND infarct*) OR (acute AND coronar*) OR (coronary AND syndrome*) OR preinfarct* OR (pre AND infarct*) OR stemi OR nonstemi OR non-stemi OR nstemi OR ACS) AND EE= (Cost AND of AND illness)


Developing economic commentaries basic stages of process2

Developing economic commentaries: Basic stages of process

1

2

3


Heed and nhs eed searches for economic evaluations

HEED and NHS EED searches for economic evaluations

  • Aim to locate all eligible economic evaluations

    • Analysis types: CEA, CUA, CBA or cost analysis

    • Analytic framework: Single empirical study or decision model

    • Compares the experimental intervention(s) with one or more eligible comparators…

    • …for an eligible population of patients (ref. health condition)

  • Based on (at least) keyword search terms designed to capture ‘Intervention’ concepts

  • Adapted from searches used to locate eligible efficacy or effectiveness studies


Heed and nhs eed searches for economic evaluations1

HEED and NHS EED searches for economic evaluations

EMBASE:-

  • (fondaparinuxor idraparinux or Arixtra or otamixaban or Razaxaban or Fonadaparin or Dx 9065$).mp.

  • xainhibit$.mp.

  • 10a inhibit$.mp.

  • xaantagonist$.mp.

  • 10a antagonist$.mp.

  • xablock$.mp.

  • factor x inhibit$.mp.

  • Fxainhibit$.mp.

  • vasoflux.mp.

  • Retrieved 40 records

  • Pilot study mean was 32 (n=38)

HEED (Expert search):-

AX= fondaparin* OR idraparinux OR arixtra OR otamixaban OR ((xa OR 10a) AND (inhibit* OR antagonist* OR block*)) OR ("factor x" AND inhibit*) OR (fxa AND inhibitor*) OR "vaso flux" OR razaxaban OR "dx 9065"

  • Retrieved 21 records

  • Pilot study mean was 23 (n=38)

NHS EED (Quick search):-

fondaparin* OR idraparinux OR arixtra OR otamixaban OR ((xa OR 10a) AND (inhibit* OR antagonist* OR block*)) OR ("factor x" NEAR inhibit*) OR (fxa NEAR inhibitor*) OR "vaso flux" OR razaxaban OR "dx 9065"


Developing economic commentaries basic stages of process3

Developing economic commentaries: Basic stages of process

1

2

3


Screening results of heed search for cost of illness studies

Screening results of HEED search for cost-of-illness studies

Brito et al, 2011

  • Records of cost-of-illness studies or reviews of cost-of-illness studies of the target health condition:

    • Unstable angina

    • Non-STEMI

    • STEMI

    • But ideally ACS (i.e. all 3 clinical entities combined)

  • At least single country-level estimates (ideally >)

  • Whole patient groups of interest (not sub-groups):-

    • Adults ≥ 18 years

  • Corresponding article published in a peer reviewed journal


Screening results of heed and nhs eed searches for economic evaluations

Screening results of HEED and NHS EED searches for economic evaluations

  • Duplicate screening by two researchers working independently (ideal)

  • Eligibility criteria same as main review – Population(s), Intervention(s), Comparison(s)

    • P: Adults ≥ 18 years with ACS

    • I: Factor Xa inhibitors

    • C: Unfractionated heparins or low molecular weight heparins

  • Refer to taxonomy of evaluation types and descriptions of analysis types and analytic frameworks in Cochrane Handbook, Chp 15, Section 15.1


Are heed and nhs eed records alone sufficient to determine eligibility

Are HEED and NHS EED records alone sufficient to determine eligibility?

Pilot study:

  • Cost-of-illness: corresponding full-texts needed for 52% of HEED/NHS EED records

  • Economic evaluations: corresponding full-texts needed for 24% of HEED/NHS EED records


Pilot study search and screening results

Pilot study: search and screening results

Cost-of-illness studies

  • At least one relevant cost-of-illness study identified for 70% of included CIRs

  • Average (mean) 23 per included CIR (Range = 0 to 66)

    Economic evaluations

  • At least one relevant economic evaluation identified for 28% of included CIRs

  • Average (mean) 1.4 per included CIR, NHS EED and HEED combined, de-duplicated (Range = 0 to 24)


Developing economic commentaries basic stages of process4

Developing economic commentaries: Basic stages of process

1

2

3


Are heed and nhs eed records alone sufficient to inform development of economic commentaries

Are HEED and NHS EED records alone sufficient to inform development of economic commentaries?

Pilot study:

  • Use both HEED field-coded abstracts and NHS EED structured abstracts where both are available

    • Check key information is consistent between records

    • Some key information reported in one record but not the other

  • Corresponding full-texts always needed in the case of citation-only records


Are heed and nhs eed records alone sufficient to inform development of economic commentaries1

Are HEED and NHS EED records alone sufficient to inform development of economic commentaries?

Pilot study:

  • Information in HEED field-coded abstracts and NHS EED structured abstracts is invariably useful, but not always sufficient

  • Retrieve and use corresponding full-texts if available

    • Useful supplementary information not included in the NHS EED or HEED records

    • Resolves occasional discrepancies in key information between HEED and NHS EED records (if both are available)


Economic commentary background

Economic commentary: ‘Background’

  • A brief, general introductory statement of the scale of economic burden to health care systems, patients and/or their families and/or society as a whole

    Stroke is the leading cause of sustained disability in the world today, placing a huge economic burden on health systems and society.

    The global economic burden of schizophrenia is high and the costs-of-illness are wide-ranging.

    The economic burden of Crohn’s Disease to society is substantial, comprising both direct medical costs and indirect costs, such as loss of productivity, sick pay, reduced productivity during paid work, early retirement and loss of leisure time.


Economic commentary background1

Economic commentary: ‘Background’

  • A brief, general introductory statement of the scale of economic burden to health care systems, patients and/or their families and/or society as a whole

  • Monetised estimate(s) of the scale of economic burden to health care systems

  • Monetised estimate(s) of the scale of economic burden to patients and/or their families

  • Monetised estimate(s) of the scale of economic burden to societies as a whole

  • Include details of currency and price year applicable to each monetised estimate

  • Cite all sources and include bibliographic details in ‘Additional references’ section

  • Make commentary as widely applicable as possible, contingent on information available in identified cost-of-illness studies


Economic commentary discussion

Economic commentary: ‘Discussion’

Overall focus:

  • To what extent is there a prima facie case that an intervention might be judged favourably (or unfavourably) from an economic point of view?

    Include details of:

  • Electronic health economics literature databases searched

  • Numbers of relevant economic evaluations identified for each eligible comparison

  • Primary types of analysis used

  • Frameworks used to assemble data, including source(s) of efficacy/effectiveness and safety/adverse effects data used (if applicable)


Economic commentary discussion1

Economic commentary: ‘Discussion’

Include details of:

  • Analytic perspective and time horizon adopted for costs and effects

  • Main cost categories included in each analysis

  • Currency and price year

  • Authors’ principal conclusions (base case analysis)

  • Uncertainty regarding authors’ principal conclusions (sensitivity analyses)

  • Cite all sources and include bibliographic details in ‘Additional references’ section


Economic commentary discussion standard form of words

Economic commentary: ‘Discussion’ – Standard form of words

Introductory:

To supplement the main systematic review of effects, we sought to identify economic evaluations which have compared [‘Intervention X’]with [‘Comparator Y’]. Systematic supplementary searches of the NHS Economic Evaluation Database and the Health Economic Evaluations Database identified [N] relevant economic evaluations.


Critical appraisa l

Critical appraisal

  • Not necessary to subject cost-of-illness studies selected to inform ‘Background’ commentary to formal critical appraisal

  • Not recommended that authors should necessarily subject economic evaluations to formal critical appraisal, but this fact should be stated explicitly as a caveat alongside the ‘Discussion’ commentary.

    It is important to highlight that we did not subject any of the [N] identified economic evaluations to any formal critical appraisal and we do not attempt to draw any firm or general conclusions regarding the relative costs or efficiency of [‘Intervention X’] compared with [‘Comparator Y’].


Economic commentary discussion example form of words

Economic commentary: ‘Discussion’ – Example form of words

Prima facie case that an intervention might be judged favourably (or unfavourably) from an economic point of view:

Lack of evidence

The apparent scarcity of relevant economic evaluations indicates that economic evidence regarding [‘Intervention X’] for [‘Health Condition Z’] is currently lacking.


Economic commentary discussion example form of words1

Economic commentary: ‘Discussion’ – Example form of words

Prima facie case that an intervention might be judged favourably (or unfavourably) from an economic point of view:

Equivocal findings between studies

However, it is clear that the available economic evidence for [‘Intervention X’] compared [‘Comparator Y’] in the treatment of patients with [‘Health Condition Z’] is, at best, equivocal.


Economic commentary discussion example form of words2

Economic commentary: ‘Discussion’ – Example form of words

Prima facie case that an intervention might be judged favourably (or unfavourably) from an economic point of view:

Consistent findings between studies [1]

However, the available economic evidence indicates that, from an economic perspective, use of [‘Intervention X’] is (at least) a promising strategy compared with [‘Comparator Y’] for the secondary prevention of [‘Health Condition Z’].


Economic commentary discussion example form of words3

Economic commentary: ‘Discussion’ – Example form of words

Prima facie case that an intervention might be judged favourably (or unfavourably) from an economic point of view:

Consistent findings between studies [2]

Taking into account these limitations, there was consistency between economic evaluations in the finding that short-term direct health care costs were, on average, lower amongst patients with[‘Health Condition Z’] who underwent [‘Intervention X’] compared with those who underwent [‘Comparator Y’]. When considered alongside the principal finding from our main review of intervention effects that there is no clear difference in perioperative outcomes and re-operation rates for disease recurrence between [‘Intervention X’] and [‘Comparator Y’], the available economic evidence indicates that, from an economic perspective, [‘Intervention X’] may be a promising surgical technique, as a comparably safe and lower cost alternative to [‘Comparator Y’], in patients with [‘Health Condition Z’].


Economic commentary discussion standard form of words1

Economic commentary: ‘Discussion’ – Standard form of words

Final caveat:

End users of this review will need to assess the extent to which methods and results of identified economic evaluations may be applicable (or transferable) to their own setting.


How much time does it take to develop brief economic commentaries

How much time does it take to develop brief economic commentaries?

Pilot study:

  • Aggregate (median) trainedresearcher time input (time on task) required to complete all stages of the process undertaken in the pilot study :

    • Design and execution of NHS EED and HEED search strategies

    • Processing search results

      • Initial screening of NHS EED and HEED records

      • Assessment of eligibility based on NHS EED and HEED records

      • Retrieval and assessment of full-text articles;

      • Classification of eligible economic evaluations

    • Development of economic commentaries

      = 210 minutes per review = 3 hours and 30 minutes (Mean = 245.6, s.d. = 140.3; Range = 93.0 to 450.0)


How much time does it take to develop brief economic commentaries1

How much time does it take to develop brief economic commentaries?

If all 25 recommendations were implemented (inc. independent screening and classification of economic evaluations by two researchers):

Estimated aggregate researcher time input (time on task) = 4-4.5 hours per review


Proposed criteria for prioritising cirs for development of brief economic commentaries

Proposed criteria for prioritising CIRs for development of brief economic commentaries

  • The comparator(s) being considered include alternative management strategies that are used in current practice (i.e. comparator(s) are not limited to placebo only)

  • Important cost differences are expected between the experimental intervention(s) and comparator(s)

  • The CIR is being updated (i.e. updates rather than new reviews)


Small group exercise

Small group exercise

Compose brief economic commentaries and integrate these into the existing text of Brito et al’s Factor Xa inhibitors for acute coronary syndromes.

Group 1

Brief commentary for ‘Background’ section based on 3 full-text articles of relevant cost-of-illness studies (corresponding HEED/NHS EED records were all citation-only)

Group 2

Brief commentary for ‘Discussion’ section based on 3 HEED field-coded abstract records and 3 corresponding NHS EED structured abstract records of eligible economic evaluations (corresponding full-text articles withheld)

Group 3

Brief commentary for ‘Discussion’ section based on 3 full-text articles of eligible economic evaluations (corresponding HEED field-coded abstract records and NHS EED structured abstract records withheld, with the exception of Maxwell 2009 – as the corresponding full-text could not be retrieved)


  • Login