Systematising the process the role of psms in informing model structure
Download
1 / 22

Systematising the process : the role of PSMs in informing model structure - PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on

Systematising the process : the role of PSMs in informing model structure. Jim Chilcott Technical Director, ScHARR-TAG Suzy Paisley DoH Research Scientist in Evidence Synthesis Paul Tappenden Senior OR Analyst Consensus Working Group on the Use of Evidence in Economic Decision Models

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Systematising the process : the role of PSMs in informing model structure ' - tarika


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Systematising the process the role of psms in informing model structure

Systematising the process : the role of PSMs in informing model structure

Jim Chilcott

Technical Director, ScHARR-TAG

Suzy Paisley

DoH Research Scientist in Evidence Synthesis

Paul Tappenden

Senior OR Analyst

Consensus Working Group on the Use of Evidence in Economic Decision Models

MRC HSRC Workshop 26 September 2005


Before we start

Before we start… model structure


Today s presentation
Today’s presentation model structure

  • Define ‘structuring’ within the modelling process

  • Describe current practice

    • Evidence used

    • Tasks undertaken

  • Discuss ‘What role problem structuring methods (PSMs)?’

Consensus Working Group on the Use of Evidence in Economic Decision Models


Defining model structuring
Defining model structuring model structure

  • Disease and treatment pathway

  • Model scope

  • Modelling method

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice hta guidelines
Current practice: model structure HTA Guidelines

HTA guidelines (n=32)

None 17 (53%)

Description 2 (6%)

Justification 4 (13%)

Sensitivity analysis 5 (16%)

Not available 4 (13%)

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice modelling guidelines philips et al 2004

Statement of problem model structure

Justification of approach

Scope / perspective

Structural assumptions

Consistency

Strategies / comparators

Model type

Time horizon

Health states

Cycle length

Parsimony

Current practice:Modelling guidelines (Philips et al.,2004)

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice reported cooper et al 2005
Current practice: model structure Reported (Cooper et al., 2005)

HTA monographs (n=42)

Diagram of model 26 (62%)

Model structure

and assumptions discussed 5 (12%)

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice observed
Current practice: model structure Observed

  • 8 case study projects

  • Description model structuring process

  • Short, semi-structured interviews

  • Examination of corresponding study reports

Consensus Working Group on the Use of Evidence in Economic Decision Models


Evidence consulted
Evidence consulted model structure

Consensus Working Group on the Use of Evidence in Economic Decision Models


Reporting
Reporting model structure

Consensus Working Group on the Use of Evidence in Economic Decision Models


Summary of themes
Summary of themes model structure

  • Model structure amended according to data availability

  • Clinician input (1st order validation)

  • Existing models consulted as 1st step

  • Iterative structuring process

  • Complexity of structure shifted both ways

  • Very limited reporting process of development

  • MSc dissertation provided most transparent reporting

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice evidence used

Clinical expertise model structure

Published evidence

Existing models

Natural history

Treatment pathway

Modelling method

Effectiveness evidence

Current practice:Evidence used

Consensus Working Group on the Use of Evidence in Economic Decision Models


Current practice tasks undertaken
Current practice: model structure Tasks undertaken

  • Iteration

  • Consultation

  • Evaluation and comparison

  • Feedback and clarification

  • Decisions regarding simplicity, complexity, feasibility and inconsistency

Consensus Working Group on the Use of Evidence in Economic Decision Models


Why psms
Why PSMs? model structure

  • Lack of transparency

  • Data modelling vs problem modelling

  • Validity of model (unquestioned question)

  • Issues that ‘can’t be modelled’

  • ‘Not qualified to comment’

Consensus Working Group on the Use of Evidence in Economic Decision Models


Why psms1
Why PSMs? model structure

  • Data led models vs evidence informed models

  • Use of expert opinion

  • Implicit use of PSM methods

  • Observed usefulness of action research process

Consensus Working Group on the Use of Evidence in Economic Decision Models


Psms msms

? model structure

PSMs / MSMs

Messes Problems

SODA

SSM

SCA

Robustness Analysis

Drama Theory

Decision Conferencing

Decision analysis

Markov

Queuing theory

Simulation

Game Theory

Consensus Working Group on the Use of Evidence in Economic Decision Models


Characteristics of psms
Characteristics of PSMs model structure

  • Techniques to facilitate exploration / definition of a problem situation

  • Group based

  • Uncovering options

  • Enabling group action

  • Accepting uncertainty

  • Managing conflict

  • Facilitator - analyst

Consensus Working Group on the Use of Evidence in Economic Decision Models


SODA model structure

Individual interviews =>

Merge maps =>

Group workshops =>

Commitment to action

Cognitive maps

Group map

Expand/refine map

Group ownership

Consensus Working Group on the Use of Evidence in Economic Decision Models


SSM model structure

Consensus Working Group on the Use of Evidence in Economic Decision Models


Pros model structure

  • Problem structuring, not problem solving

  • Gathering information, generating ideas

  • Retain flexibility in early stages

  • Demonstrate understanding of complexity, diversity and inconsistency

  • Prioritising and simplifying

Consensus Working Group on the Use of Evidence in Economic Decision Models


Cons model structure

  • Methods development required

  • Adaptation to HTA context

  • New skills

  • Ownership

  • Facilitation of group decision making

  • Reproducibility

Consensus Working Group on the Use of Evidence in Economic Decision Models


Developing a mixed method
Developing a mixed method model structure

Qualitative model structuring process to inform quantitative decision-making process.

‘Soft’ qualitative structuring process

‘Hard’ analytic

methods

Consensus Working Group on the Use of Evidence in Economic Decision Models


ad