NICE Scientific advice & Diagnostics assessment programme

1. NICE Scientific advice & Diagnostics assessment programme How to generate evidence to support value claims for diagnostics Dr Grace Jennings and Dr Sarah Byron September 23, 2014

2. What is NICE? An independent institute that identifies how to: • prevent, diagnose and treat disease and ill health in most effective ways • reduce inequalities and variation • ensure quality and value for money for the NHS Guidance & Quality Standards: Pharmaceuticals, Devices/Diagnostics, Clinical care, Social Care, Safe Staffing, Interventional Procedures, Public Health

4. NICE’s Procedural Principles Accountability for reasonableness

5. A NICE Process Independent review Evidence submissions Decision Committee Input from topic experts Stakeholder Perspectives Public Consultation Decision

6. Scarce resources DIFFICULT CHOICES FOR DECISION MAKERS HTA

7. Health Technology Assessment (HTA) Evidence Product Market Policy-making Evidence-based way of guiding the efficient allocation of health care resources

8. Benefits of engaging with NICE

9. How NICE makes decisions

10. Two key questions asked by NICE • How well does the technology work compared to established practice in the National Health Service (NHS)? • How much does this course of action cost compared to established practice in the NHS?

11. Decision-making at NICE

12. Value Proposition Impact on health system resources Incremental benefit for patients Value varies depending on your perspective

13. Understand the perspective of your decision maker and know what question they want to answer NICE takes the perspective of the National Health Service (NHS) and Personal Social Services (PSS) Perspective

14. Defining the clinical question: Components • Population • Intervention • Comparator • Outcomes PICO

15. Putting your case together

16. Navigating NICE

17. Why seek scientific advice? Product developers may be interested in ensuring their clinical development studies and other plans generate the evidence which is relevant to NICE • Increase the likelihood that clinical trials and other research activities undertaken meet NICE evidence requirements

18. Typical issues raised for advice Clinical Trial Programme • Study population • Comparators • Acceptability of endpoints, surrogate endpoints • Trial design • Appropriateness of health-related quality of life and other Patient Reported Outcomes • Positioning in the clinical pathway Economic Evaluation • Plans for using specific economic models • Sources of data , observational studies, analyses

19. Additional issues raised in previous diagnostics advice projects • Study design • Diagnostic cohort design, case-control study, cluster randomisation trial • Sensitivity and specificity • Diagnostic accuracy • Stratification by risk factor • Clinician blinding • Outcomes relevant to NICE • Role of new diagnostic in existing treatment pathway • Assessment of cost-effectiveness • Model structure and input assumptions

20. Key stages of advice processes

21. Advice for developers of screening tests • Screening not part of NICE remit • Screening and diagnostic tests often similar • Diagnostic test may be used as screening test in treatment pathway • National Screening Centre • Part of Public Health England • Areas of screening needs explored • Models developed by external academic group • New initiative – advice from NICE Scientific Advice in conjunction with NSC for screening tests

22. Questions?

23. Programmes at NICE MedicalTechnologies Evaluation Programme (MTEP) Technology Appraisals (TA) Diagnostics Assessment Programme (DAP)

24. Topic Selection • STEP 1 • The Company submits a notification form to Medical Technologies Evaluation Programme that details: • Product description • Patient population • Current management and comparator(s) • Claimed patient benefit • Claimed healthcare system benefit • Claimed sustainability benefit • Costs • Patient safety

25. STEP 2. NICE produces a briefing note for MTAC which decides whether the technology is suitable for the evaluation at NICE STEP 3. If selected, MTAC routes the technology to the appropriate Programme. Notifying a product to NICE Process takes ~10 weeks Questions about the process and eligibility? Contact MTEP:medtech@nice.org.uk

26. MTEP vs DAP DAP and MTEP encourage further research into promising technologies

27. Diagnostics Assessment Programme • Specialist programme to undertake complex assessments of diagnostic technologies • Decision making by independent Diagnostics Advisory Committee • Assessment of single or multiple technologies • No formal manufacturer submission required • Systematic review of evidence and modelling to estimate outcome benefits and cost effectiveness is undertaken as part of the assessment

28. Overview of assessment process

29. Two key questions asked by NICE • How well does the technology work compared to established practice in the National Health Service (NHS)? • How much does this course of action cost compared to established practice in the NHS?

30. Scoping Single technology notified and referred to DAP • DAP technical lead • Diagnostic pathway • Care pathway • Alternative technologies • Relevant population(s) • Costs • Outcomes • Potential equality issues • Potential implementation barriers Draft scope Scoping workshop Registered stakeholders Revised scope ASG meeting Specialist Committee members Standing DAC member Final scope Assessment of single or multiple technologies

31. 1. How is the condition managed in the NHS? 2. Where does my product fit in the care pathway? 3. What does my product deliver?

32. Understanding benefits: diagnostics False positive? False negative?

33. Understanding benefits: diagnostics • The treatment pathway or the range of pathways must be understood for the value of the diagnostic to be assessed • Test side effects should be included • Patient benefits rarely arise from the diagnostic directly – they come mainly from treatments informed by the diagnostic

34. Assessment

35. What data do you need? Regulator HTA Evidence on clinical effectiveness (compared to established practice): trials, evidence synthesis Evidence on cost effectiveness: trials, modelling Evidence on relative safety/adverse events • Product safety: laboratory testing with clinical trial data for devices with greatest risk • Evidence of efficacy: does the device meet its intended purpose? (not necessarily from comparative studies) Trueman P et al 2011

36. Evidence considerations • The ideal evidence would be a good quality ‘end-to-end’ study – follows patients from testing, through treatment, to final outcomes • Typically not available for diagnostics • Search for data on test accuracy, direct outcomes from the test, indirect health outcomes from the test result, and costs • Identified evidence can then be combined through a linked evidence approach Impact on treatment decisions Diagnostic accuracy Impact on outcomes

37. Evidence hierarchy

38. Study design • Outcomes: patient focussed outcomes are particularly important, as opposed to intermediate or surrogate outcomes • e.g. a reduction in tumour size will be given less weight than evidence about clinical benefit such as improved survival or quality of life • Size: Studies with larger numbers of patients will usually be preferred as estimates of benefits and harms will be more accurate • Duration: Studies should have sufficient follow up to capture final outcomes where possible • e.g. very important for prognostic tests

39. Diagnostic tests: Outcomes data Identify studies on the effectiveness of those subsequent treatments Test side effects should be included Use a systematic approach to identifying relevant studies Ideally comparative ‘end-to-end’ clinical studies including the test and subsequent treatments should be conducted Not possible

40. Diagnostic tests: Outcomes data • Measurements of test accuracy are necessary:

41. Diagnostic tests: Outcomes data Cut off points

42. Example: SonoVue (sulphur hexafluoride microbubbles) Contrast agent for contrast-enhanced ultrasound imaging of the liver Characterising incidentally detected focal liver lesions Detection of potential liver metastases Characterising focal liver lesions (cirrhosis) No end-to-end studies available High quality accuracy data – SonoVue vs CT and MRI Relevant evidence on care pathway and outcomes

43. Characterising incidentally detected focal liver lesions Adoption recommendation Cost effective Adoption recommendations where CT and MRI not appropriate + Detection of potential liver metastases Slightly less cost effective than CT and MRI Characterising focal liver lesions (cirrhosis) Research recommendations to explore potential broader applicability

44. Decision making

45. Recommendations • Adoption recommendations • Research recommendations • Not recommended

46. Guidance development • Decision making in presence of uncertainty • Public consultation can change decision making • Clarity in recommendations on indication • Rule-in / rule-out / diagnosis / monitoring • Setting • Supported by evidence, minimise risk of indication creep and inappropriate use of tests that may lead to misdiagnosis • Cost-effective use of NHS resources • ‘Committee considerations’ describe uncertainties and rationale behind decision-making.

47. Examples of recommendations Diagnostic Guidance (DG5) SonoVue (sulphur hexafluoride microbubbles) – contrast agent for contrast-enhanced ultrasound imaging of the liver Contrast-enhanced ultrasound with SonoVue is recommended for characterising incidentally detected focal liver lesions in adults in whom an unenhanced ultrasound scan is inconclusive. An unenhanced ultrasound scan in which a focal liver lesion is detected, but not characterised, is defined as inconclusive.

48. Recommendations • Adoption recommendations • Implementation support • Health Technologies Adoption Programme • Research recommendations • NICE research commissioning • Not recommended

49. NICE: Companion diagnostics

50. NICE: Companion diagnostics • Alternative technologies • Timing is an important factor • Appraisal programme coordinates publication of guidance with market authorisation of drug • CDx may be developed simultaneously with drug • CDx can come to market years after drug approval or may already be available • Rapid development of CDx and supporting technologies