The appraisal extraction and pooling of cost and cost effectiveness studies
This presentation is the property of its rightful owner.
Sponsored Links
1 / 87

The Appraisal, Extraction and Pooling of Cost and Cost Effectiveness Studies PowerPoint PPT Presentation


  • 109 Views
  • Uploaded on
  • Presentation posted in: General

The Appraisal, Extraction and Pooling of Cost and Cost Effectiveness Studies. JBI/CSRTP/2013-14/0004. Introduction. Recap of Introductory Module Developing a question (PICO ); Inclusion Criteria; Search Strategy; Selecting Studies for Retrieval.

Download Presentation

The Appraisal, Extraction and Pooling of Cost and Cost Effectiveness Studies

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The appraisal extraction and pooling of cost and cost effectiveness studies

The Appraisal, Extraction and Pooling of Cost and Cost Effectiveness Studies

JBI/CSRTP/2013-14/0004


Introduction

Introduction

  • Recap of Introductory Module

    • Developing a question (PICO);

    • Inclusion Criteria;

    • Search Strategy;

    • Selecting Studies for Retrieval.

  • This module considers how to appraise, extract and synthesize evidence fromCost and Cost Effectiveness studies.


Aim and objectives

Aim and Objectives

  • The objectives of this module are to prepare participants to:

    • critically appraise studies of cost and cost effectiveness,

    • extract data from cost and cost effectiveness studies,

    • summarize the results of cost and cost effectiveness studies.


Program overview

Program Overview


Session 1 introduction to review of evidence on cost and cost effectiveness

Session 1: Introduction to review of evidence on cost and cost effectiveness


Common study designs

Common study designs

  • Prospective experimental or quasi experimental effectiveness studies with cost or cost effectiveness components;

  • Modelling studies.


Four approaches to analysis

Four approaches to analysis

  • Cost-minimization analysis (CMA);

  • Cost-effectiveness analysis (CEA);

  • Cost-utility analysis (CUA);

  • Cost-benefit analysis (CBA).


Methods measures benefits

Methods, measures, benefits


Cost minimization analysis cma

Cost-minimization analysis (CMA)

In cost-minimization analysis (CMA) only the costs of the interventions are compared; the outcomes are assumed to be equivalent.


Pico questions cost minimization

PICO Questions – Cost Minimization

What is the evidence on costs (direct and indirect) of laparoscopic compared to open appendectomy for patients aged 15 years or over (assuming the long-term outcome is the same in both groups)?


Cost effectiveness analysis

Cost-effectiveness analysis

Costs are measured in monetary units;

The outcome is common to both alternatives but the effect size and direction may vary;

Outcomes are measured in natural/clinical units;

(e.g. mortality, myocardial infarctions, lung function, weight, bleeds).


Cost effectiveness plane

Cost–effectiveness Plane

  • A four-quadrant figure of cost difference plotted against effect difference:

    • quadrant I, intervention more effective and more costly than comparator;

    • quadrant II, intervention more effective and less costly than comparator;

    • quadrant III, intervention less effective and less costly than comparator; and

    • quadrant IV, intervention less effective and more costly than comparator. (Culyer, 2005:77-78)


Cost effectiveness plane1

Cost Effectiveness Plane

Q4

Q1

Q2

Q3


Pico questions cost effectiveness

PICO Questions – Cost Effectiveness

What is the cost effectiveness of percutaneous coronary intervention with drug-eluting stents (PES) compared to bare-metal stents (BMS) to reduce angina symptoms for patients undergoing single-vessel percutaneous coronary intervention?


Cost utility analysis

Cost-utility analysis

Costs are measured in monetary units;

Outcomes are common to both alternatives;

Effect size and direction may vary;

Outcomes are measured as healthy years (typically measured as quality-adjusted life-years (QALYs)).


Outcome measures for cua

Outcome measures for CUA

The primary outcome for CUA is expressed as quality-adjusted life-years (QALYs);

Other generic outcome measures for CUA:

Disability-adjusted life-year (DALY);

Healthy years equivalent (HYE);

Saved-young-life-equivalent.

(Drummond et al., 2005:14)


Pico questions cost utility

PICO Questions – Cost Utility

What is the cost-utility of the cochlear implant in adults (age >18 years) with profound bilateral, post-lingual deafness compared with no intervention?


Cost benefit analysis

Cost-benefit analysis

Costs are measured in monetary units;

Outcomes are identified as single or multiple effects;

The effects are not necessarily common to both alternatives;

Outcomes are measured in monetary units.


Pico questions cost benefit

PICO Questions – Cost Benefit

What is the cost-benefit of donepezil compared to galantamine for cognitive function in patients with mild to moderate Alzheimer’s disease?


Searching for evidence

Searching for Evidence

Cost and Cost Effectiveness keywords;

Clinical keywords;

General databases;

Specific databases.


Specific economic databases

Specific economic databases

  • NHS Economic Evaluation Database (NHS EED);

  • Health Economic Evaluation Database (HEED);

  • Cost-effectiveness Analysis (CEA) Registry;

  • Health Technology Assessment (HTA) database;

  • Paediatric Economic Database Evaluation (PEDE);

  • European Network of Health Economic Evaluation Databases (EURONHEED);

  • COnnaissance et Decision en Economie de la Sante (CODECS).


Group work 1 identification of economic evaluation study designs

Group Work 1: Identification of Economic Evaluation Study Designs

  • Refer to Workbook.

  • Report back


Session 2 critical appraisal of cost and cost effectiveness studies

Session 2: Critical Appraisal of Cost and Cost Effectiveness Studies


Why critically appraise

Why Critically Appraise?

Combining results of poor quality research may lead to misleading understandings of issues explored.

1004 references

172 duplicates

832 references

Scanned Ti/Ab

715 do not meet

Incl. criteria

117 studies

retrieved

82 do not meet

Incl. criteria

35 studies for

Critical Appraisal


The critical appraisal process

The Critical Appraisal Process

Every review must set out to use an explicit appraisal process. Essentially,

A good understanding of research design is required in appraisers; and

The use of an agreed checklist is usual.


Critical appraisal of cost and cost effectiveness evidence

Critical appraisal of cost and cost effectiveness evidence

Primary purpose of critical appraisal is to assess a study’s quality and determine the extent to which a study has excluded the possibility of systematic flaws in its design, conduct and analysis.


Jbi critical appraisal checklist for cost and cost effectiveness studies

JBI Critical Appraisal Checklist for cost and cost effectiveness studies

Is there a well defined question?

Is there a comprehensive description of alternatives?

Are all important and relevant costs and outcomes for each alternative identified?

Has clinical effectiveness been established?

Are costs and outcomes measured accurately?

Are costs and outcomes valued credibly?


Jbi critical appraisal checklist for cost and cost effectiveness studies1

JBI Critical Appraisal Checklist for cost and cost effectiveness studies

Are costs and outcomes adjusted for differential timing?

Is there an incremental analysis of costs and consequences?

Were sensitivity analyses conducted to investigate uncertainty in estimates of cost or consequences?

Do study results include all issues of concern to users?

Are the results generalizable to the setting of interest in the review?


Jbi economic evidence appraisal

JBI Economic evidence appraisal

  • Is there a well defined question?

    • Costs and effects;

    • Comparison of alternatives;

    • Perspective of the analysis (including the decision-making context).


Effects

Effects

Mortality measurements;

Morbidity measurements;

Health-related quality of life measurements.


Perspective

Perspective

The ‘viewpoint’ adopted for the purposes of an economic appraisal (cost–effectiveness, cost–utility studies and so on) which defines the scope and character of the costs and benefits to be examined.


Perspectives

Perspectives

Societal perspective;

Health sector perspective;

Other sector perspective;

Health insurance perspective;

Hospital perspective;

Patient perspective.


Jbi economic evidence appraisal1

Is there a comprehensive description of alternatives?

Important alternatives

Do-nothing alternative

JBI Economic evidence appraisal


Jbi economic evidence appraisal2

Are all important and relevant costs and outcomes for each alternative identified?

Was the range wide enough for the research question;

Does it cover all relevant perspectives;

Were capital as well as operating costs included.

JBI Economic Evidence Appraisal


Typical classification of costs

Typical classification of costs

  • Functional costs;

  • Financial and economic costs;

  • Direct, indirect and intangible costs;

  • Capital and recurrent costs;

  • Fixed and variable costs;

  • Opportunity costs.


Functional costs

Functional costs

Can be classified into categories:

personnel;

buildings and space;

equipment;

supplies and pharmaceuticals;

transportation;

training;

information, education and communication.


Financial and economic costs

Financial and economic costs

Financial costs are defined as the actual money spent on the resources;

Inclusion of the costs of all resources, regardless of their financial cost is known as the economic cost.


Direct indirect and intangible costs

Direct, indirect and intangible Costs

  • Direct costs are associated directly with a healthcare intervention (e.g. drugs, staffing);

  • Indirect costs refer to the productivity gains or losses (e.g. time off work, illness);

  • Intangible costs refer to the non-monetary assets that can not be readily seen (e.g. anxiety, fatigue, pain or suffering from an illness or treatment).


Jbi economic evidence appraisal3

JBI Economic Evidence Appraisal

  • Has clinical effectiveness been established?

    • Was this through experimental research?

      • If so did the trial protocol reflect what would happen in regular practice?

    • Was effectiveness established through a synthesis of clinical studies?

    • Were observational data or assumptions used to establish effectiveness?

      • If so what were the potential biases in results?


Jbi economic evidence appraisal4

JBI Economic Evidence Appraisal

  • Are costs and outcomes measured accurately?

    • Were any of the identified items omitted from the measurement?

      • If so does this mean that they carried no weight in the subsequent analysis?

    • Were there any special circumstances (e.g. joint use of resources) that made measurement difficult?

      • If so, were these circumstances handled appropriately?


Jbi economic evidence appraisal5

JBI Economic Evidence Appraisal

  • Are costs and outcomes valued credibly?

    • Were the sources of all values clearly identified?

      • Possible sources include market values, patient or client preferences and views, policy maker’s views and health professional’s judgements;

    • Were market values employed for changes involving resources gained or depleted?

    • Where market values were absent (e.g. volunteer labour) or did not reflect actual values (such as clinic space donated at a reduced rate) were adjustments made to approximate market values?


Jbi economic evidence appraisal6

JBI Economic Evidence Appraisal

  • Are costs and outcomes adjusted for differential timing?

    • Were costs and outcomes that occur in the future ‘discounted’ to their present values?

    • Was there any justification given for the discount rate used?


Jbi economic evidence appraisal7

JBI Economic Evidence Appraisal

  • Is there an incremental analysis of costs and consequences?

    • Were the additional (incremental) costs generated by one alternative over another compared to the additional effects, benefits or utilities generated?


Jbi economic evidence appraisal8

JBI Economic Evidence Appraisal

  • Were sensitivity analyses conducted to investigate uncertainty in estimates of costs or outcomes?

    • If a sensitivity analysis was employed, was justification provided for the range of values (or for key study parameters)?

    • Were the study results sensitive to changes in the values (within the assumed range for sensitivity analysis or within the confidence interval around the ratio of costs to outcomes)?


Jbi economic evidence appraisal9

JBI Economic Evidence Appraisal

  • Do study results include all issues of concern to users?

    • Are the results of cost and effect for the alternative interventions?

    • Do they clearly specify the relative size of the effects for the interventions?

    • Do they clearly show how costs differ for the two interventions?

    • Can we use them with the Cost Effectiveness Plane?


The appraisal extraction and pooling of cost and cost effectiveness studies

  • Did the study take account of other important factors in the choice or decision under consideration (e.g. distribution of costs or outcomes or relevant ethical issues)?

  • Did the study discuss issues of implementation such as the feasibility of adopting the preferred program given existing financial or other constraints and whether any freed resources could be re-deployed to other worthwhile programs?


Jbi economic evidence appraisal10

JBI Economic Evidence Appraisal

  • Are the results generalizable to the setting of interest in the review?

    • Did the study make clear that the findings on costs and effects were generated in a specific setting using particular assumptions?

    • Was the generalizability of the results to other settings and patients/client groups discussed?


Group work 2 critical appraisal of evidence from economic evaluation studies

Group Work 2: Critical Appraisal of evidence from economic evaluation studies

  • Workbook, refer pages 9-11;

  • Report back.


Session 3 study data and data extraction

Session 3: Study data and Data Extraction


Data most frequently extracted

Data most frequently extracted

1004 references

172 duplicates

832 references

Scanned Ti/Ab

715 do not meet

Incl. criteria

117 studies

retrieved

82 do not meet

Incl. criteria

35 studies for

Critical Appraisal

26 studies incl.

in review


Considerations in data extraction

Considerations in Data Extraction

Source - citation and contact details;

Methods - study design, concerns about flaws;

Participants –number, characteristics and suitability for inclusion;

Interventions - description of interests;

Outcomes - outcomes and time points;

Results - for each outcome of interest;

Miscellaneous - funding source, etc.


Actuari data extraction

ACTUARI: Data Extraction

  • Interventions and Comparator;

  • Setting;

  • Geographical context;

  • Participants;

  • Source of effectiveness data;

  • Author’s conclusion/s;

  • Reviewer’s comments;

  • Clinical effectiveness results;

  • Economic results.


First level extraction

First level extraction


Actuari extracting data from economic studies

ACTUARI: Extracting data from economic studies

  • ACTUARI data extraction;

  • Four options available for economic evaluation methods.


Second level extraction

Second level extraction


Group work 3

Group Work 3

  • Data Extraction from economic evaluation studies;

  • Refer to Workbook;

  • Report back.


Session 4 protocol development in crems

Session 4: Protocol Development in CReMS


Group work 4

Group Work 4

  • Develop a draft protocol in CReMS;

  • Refer to Workbook.


Session 5 synthesis reporting cost and cost effectiveness evidence

Session 5: Synthesis/Reporting cost and cost effectiveness evidence


Synthesis reporting economic evidence

Synthesis/Reporting economic evidence

  • Presentation of results of synthesis:

    • Tables of results;

    • Narrative summary;

    • Hierarchical decision matrix.


Tabular summary of economic evidence

Tabular summary of economic evidence


Narrative summary of economic evidence

Narrative summary of economic evidence

  • “...The median and mean willingness to pay for a 25% reduction in symptoms were $US27 and $US87 per month (1997 values), respectively. Median and mean estimates nearly tripled for a 50% reduction. ...Willingness to pay of patients with urinary symptoms was between £74 and £92 per year (1999/2000 values) for complete continence with no adverse effects, substantially lower than in the Swedish[58] and US[40] studies. Individuals without symptoms valued this outcome at only between £14 and £21 per year.”


Actuari decision matrix summary of economic evidence

ACTUARI decision matrix summary of economic evidence


Session 6 appraisal extraction and synthesis using jbi actuari

Session 6: Appraisal, Extraction and Synthesis using JBI-ACTUARI


Analysis of cost technology and utilization assessment and review instrument actuari

Analysis of Cost, Technology and Utilization Assessment and Review Instrument (ACTUARI)


Session 7 actuari trial

Session 7: ACTUARI trial


Group work 5

JBI ACTUARI Software Trial.

Group Work 5:


Session 8 protocol completion

Session 8: Protocol completion


Session 9 assessment

Session 9: Assessment


Session 10 protocol presentations

Session 10: Protocol Presentations


  • Login