Systematising the process the role of psms in informing model structure
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

Systematising the process : the role of PSMs in informing model structure PowerPoint PPT Presentation


  • 55 Views
  • Uploaded on
  • Presentation posted in: General

Systematising the process : the role of PSMs in informing model structure. Jim Chilcott Technical Director, ScHARR-TAG Suzy Paisley DoH Research Scientist in Evidence Synthesis Paul Tappenden Senior OR Analyst Consensus Working Group on the Use of Evidence in Economic Decision Models

Download Presentation

Systematising the process : the role of PSMs in informing model structure

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Systematising the process the role of psms in informing model structure

Systematising the process : the role of PSMs in informing model structure

Jim Chilcott

Technical Director, ScHARR-TAG

Suzy Paisley

DoH Research Scientist in Evidence Synthesis

Paul Tappenden

Senior OR Analyst

Consensus Working Group on the Use of Evidence in Economic Decision Models

MRC HSRC Workshop 26 September 2005


Before we start

Before we start…


Today s presentation

Today’s presentation

  • Define ‘structuring’ within the modelling process

  • Describe current practice

    • Evidence used

    • Tasks undertaken

  • Discuss ‘What role problem structuring methods (PSMs)?’

Consensus Working Group on the Use of Evidence in Economic Decision Models


Defining model structuring

Defining model structuring

  • Disease and treatment pathway

  • Model scope

  • Modelling method

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice hta guidelines

Current practice:HTA Guidelines

HTA guidelines(n=32)

None17(53%)

Description2(6%)

Justification4(13%)

Sensitivity analysis5(16%)

Not available4(13%)

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice modelling guidelines philips et al 2004

Statement of problem

Justification of approach

Scope / perspective

Structural assumptions

Consistency

Strategies / comparators

Model type

Time horizon

Health states

Cycle length

Parsimony

Current practice:Modelling guidelines (Philips et al.,2004)

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice reported cooper et al 2005

Current practice:Reported (Cooper et al., 2005)

HTA monographs (n=42)

Diagram of model26(62%)

Model structure

and assumptions discussed5(12%)

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice observed

Current practice:Observed

  • 8 case study projects

  • Description model structuring process

  • Short, semi-structured interviews

  • Examination of corresponding study reports

Consensus Working Group on the Use of Evidence in Economic Decision Models


Evidence consulted

Evidence consulted

Consensus Working Group on the Use of Evidence in Economic Decision Models


Reporting

Reporting

Consensus Working Group on the Use of Evidence in Economic Decision Models


Summary of themes

Summary of themes

  • Model structure amended according to data availability

  • Clinician input (1st order validation)

  • Existing models consulted as 1st step

  • Iterative structuring process

  • Complexity of structure shifted both ways

  • Very limited reporting process of development

  • MSc dissertation provided most transparent reporting

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice evidence used

Clinical expertise

Published evidence

Existing models

Natural history

Treatment pathway

Modelling method

Effectiveness evidence

Current practice:Evidence used

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice tasks undertaken

Current practice:Tasks undertaken

  • Iteration

  • Consultation

  • Evaluation and comparison

  • Feedback and clarification

  • Decisions regarding simplicity, complexity, feasibility and inconsistency

Consensus Working Group on the Use of Evidence in Economic Decision Models


Why psms

Why PSMs?

  • Lack of transparency

  • Data modelling vs problem modelling

  • Validity of model (unquestioned question)

  • Issues that ‘can’t be modelled’

  • ‘Not qualified to comment’

Consensus Working Group on the Use of Evidence in Economic Decision Models


Why psms1

Why PSMs?

  • Data led models vs evidence informed models

  • Use of expert opinion

  • Implicit use of PSM methods

  • Observed usefulness of action research process

Consensus Working Group on the Use of Evidence in Economic Decision Models


Psms msms

?

PSMs / MSMs

MessesProblems

SODA

SSM

SCA

Robustness Analysis

Drama Theory

Decision Conferencing

Decision analysis

Markov

Queuing theory

Simulation

Game Theory

Consensus Working Group on the Use of Evidence in Economic Decision Models


Characteristics of psms

Characteristics of PSMs

  • Techniques to facilitate exploration / definition of a problem situation

  • Group based

  • Uncovering options

  • Enabling group action

  • Accepting uncertainty

  • Managing conflict

  • Facilitator - analyst

Consensus Working Group on the Use of Evidence in Economic Decision Models


Systematising the process the role of psms in informing model structure

SODA

Individual interviews =>

Merge maps =>

Group workshops =>

Commitment to action

Cognitive maps

Group map

Expand/refine map

Group ownership

Consensus Working Group on the Use of Evidence in Economic Decision Models


Systematising the process the role of psms in informing model structure

SSM

Consensus Working Group on the Use of Evidence in Economic Decision Models


Systematising the process the role of psms in informing model structure

Pros

  • Problem structuring, not problem solving

  • Gathering information, generating ideas

  • Retain flexibility in early stages

  • Demonstrate understanding of complexity, diversity and inconsistency

  • Prioritising and simplifying

Consensus Working Group on the Use of Evidence in Economic Decision Models


Systematising the process the role of psms in informing model structure

Cons

  • Methods development required

  • Adaptation to HTA context

  • New skills

  • Ownership

  • Facilitation of group decision making

  • Reproducibility

Consensus Working Group on the Use of Evidence in Economic Decision Models


Developing a mixed method

Developing a mixed method

Qualitative model structuring process to inform quantitative decision-making process.

‘Soft’ qualitative structuring process

‘Hard’ analytic

methods

Consensus Working Group on the Use of Evidence in Economic Decision Models


  • Login