The role of simulation and modelling in health care

1. The role of simulation and modelling in health care Martin Utley, Clinical Operational Research Unit University College London

3. My HEP credentials My lack of HEP credentials ZEUS PhD 1996 "A first study of the structure of the virtual photon at HERA“ under Professor David Saxon at the University of Glasgow. This work concerned the contribution of resolved photon processes to the dijet cross-section in photoproduction events with virtual and quasi-real photons.

5. My understanding of MC simulation methods in HEP

6. Measured / postulated distributions Theoretical underpinning Immutable laws of nature (current best guess) Alternative models for certain physical processes Universally accepted models for other physical processes Unscrambling the impact on results of how measurements are made

7. ASIDE Gallivan S, Stark J, Pagel C, Williams G, Williams WG, Dead reckoning: can we trust estimates of mortality rates in clinical databases? Eur J Cardiothorac Surg. 2008 Mar;33(3):334-40 In 6 / 1198 reported deaths, the patient lived. In 4 / 724 reported survivals, the patient died. Cooper H, Findlay G, Martin IC, Mason DG, Mason M, Utley M, National Confidential Enquiry into Patient Outcome and Death (2008).

8. Measured / postulated distributions Theoretical underpinning Immutable laws of nature (current best guess) Alternative models for certain physical processes Universally accepted models for other physical processes Unscrambling the impact on results of how measurements are made Entire endeavour is concerned with testing, refining and verifying models

9. image courtesy of Dr Sally Barrington There are extensive parallels between detector physics and medical imaging. This is not what I do.

10. Clinical Operational Research Unit (CORU) Dr Martin Utley Director Tel: +44 (0)20 7679 4508 Fax: +44 (0)20 7813 2814 E-Mail: m.utley@ucl.ac.uk Web: www.ucl.ac.uk/operational-research UCL Department of Mathematics University College London Gower Street London WC1E 6BT

11. What counts for evidence in health care Combined analysis of many randomised controlled trials Single randomised controlled trial Epidemiological studies ? Follow up studies modelling Anecdote

12. prediction of risk estimating benefits of treatment decision support calculation of health insurance premiums The scope of simulation and modelling in health care capacity planning identifying bottlenecks scheduling staff rosters operation of emergency departments identifying what services to offer evaluating national policy design of screening programmes structure of services deciding whether to buy new drugs emergency planning

14. UK National Health Service • Free for all at point of access • Funded via general taxation • Under political control • Third largest organisation in the world

15. The different roles of modelling illustrating a point generating insight informing decisions making decisions?

16. Booked admissions policy • Before 2001, short notice cancellations of elective operations were frequent. • Government put in place policy whereby patients were given a firm commitment to date of surgery. • Little thought was given to implications.

17. Simple model Full attendance No emergency admissions How many beds are needed to honour all commitments?

18. Gallivan et al BMJ 2002;324:280-2. Expectation Probability .25 .20 .15 Making a firm commitment to patients requires reserve bed capacity .10 .05 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Number of beds required

19. Hospital 1 Hospital 2 Hospital 1 Hospital 2 TC Introduction of Treatment Centres to UK New hospitals to separate routine elective cases from complex & emergency cases

20. How could a TC affect efficiency? Genuinely reducing length of stay Management of patients Genuinely reducing variability in length of stay Gains in efficiency for whole system? Economies of Scale Structure and organisation of service Reducing variability in length of stay through patient selection Note: our work was limited to study effects associated with organisation of services

21. Hospital 1 Hospital 2 Hospital 1 Hospital 2 TC Compare capacity requirements We evaluated a large number of hypothetical scenarios... ...to identify circumstances in which a TC might be an efficient use of capacity

22. Level of emergencies 0% 10% 20% Efficiency of system with TC Worse Marginally better Better Much better

23. Findings • In many circumstances, there is no theoretical benefit associated with treatment centres*. • Theoretical benefits exist if there is successful identification of shorter-stay patients and a number of participating non-TC hospitals. Benefits rely on cooperation between providers. Meanwhile, other policies foster competition. *in terms of the efficient use of capacity – there are a whole host of other considerations Utley et al, Health Care Management Science, Sept 2008

24. Current occupancy Current occupancy ? Variable demand for beds Future demand for beds ? ? Short term forecasting of PICU bed demand to assist managers

25. Consider staffing an extra bed in 3 days? Probability Chance that demand for paediatric intensive care unit exceeds 9 beds or 10 beds Pagel and Utley, ORAHS proceedings, forthcoming

26. Front page of British Medical Journal last week Mathematical modelling study £500 M decision concerning national vaccination programme Decision based on cost-per-QALY Jit et al, BMJ, Aug 9 2008

27. HPV modelling Natural history of infection Woodman et al. The natural history of cervical HPV infection: unresolved issues. Nature Reviews Cancer, 2007; 7:11.

28. HPV modelling Natural history of infection – model structure 1 Susceptible HPV infected Can calibrate transition rates for this model to be consistent with empirical “cross-sectional” data. CIN1 CIN2 CIN3 Cancer

29. HPV modelling Natural history of infection – model structure 2 Susceptible HPV infected Can calibrate transition rates for this model to be consistent with empirical “cross-sectional” data. CIN1 CIN2 CIN3 Cancer

30. HPV modelling Natural history of infection – model structure 3 Susceptible HPV infected and this model. CIN1 CIN2 CIN3 Cancer

31. HPV modelling Natural history of infection – model structure 4 Susceptible HPV infected Resistant CIN1 CIN2 and this one. CIN3 Cancer

32. HPV modelling Natural history of infection – model structure 5 Susceptible HPV infected Resistant CIN1 CIN2 you get the idea.. CIN3 Cancer

33. HPV modelling MC approach to account for a myriad of uncertainties Economic model Cost of screening Cost of cancer treatment Cost of warts treatment Vaccine cost QALY loss due to screening QALY loss due to cancer QALY loss due to warts Cancer mortality rate Screening accuracy 250,000 different analytical models

34. Different tools used in health care modelling Discrete event simulation Monte Carlo simulation System dynamics Queueing theory Game theory Decision analysis Stochastic analysis Optimisation techniques Mathematical programming Hybrid methods

35. Restructuring services for common mental health problems

36. Snapshot of 1 year Simulation of traditional care Animated simulations facilitate engagement with clinicians and managers origins in simulation of shop floor / industrial processes

37. Simulation of proposed care Permits modelling of complex decision rules... ... and queues and feedback

38. Example of output from a model WARNING For illustration purposes only Different allocation of same resources can give better system performance *for patients completing treatment

39. Pitfalls to simulation in health care Model development almost too easy - insufficient thought given to purpose of model detail added solely because it can be added modellers can start to believe their models. If you think 19 free parameters is untidy, you should see some of the models developed in healthcare

40. Modelling in HEP Modelling in NHS

41. Clients Decisions are made by politicians, health care managers and clinicians... ... reason vies with political dogma, professional rivalries & financial incentives*. * oh, and management consultants

42. A fundamental difference Decays and scatters are truly random processes. When can uncertain processes be modelled as random? Consider a sexually transmitted disease Random process or determined by characteristics of the individual? Important when considering multiple interactions.

43. Time dependence of models Physical laws are either static or time dependence is an intrinsic part of the model... ...patterns of sexual mixing among the young are subject to change.

44. Validating* models in health care B model vs. reality A Intervention of interest World before intervention of interest * as opposed to calibration or tuning

45. Validating models in health care B model A Intervention of interest vs. C reality Intervention of interest + a whole lot more

46. George Box said... “All models are wrong... ...some are useful”

47. A modeller’s checklist All models arewrong ...some areuseful ? What counts as useful?

48. One response but decisions will get made, with or without our input.

49. Are you a bright, financially secure, analytical thinker with exemplary people skills and the belief that you can improve the NHS? I hate you.

50. END