Use of Models to Evaluate Health PolicyLessons from the National Audit Office Quantitative Modelling in the Management of Health Care Tom McBride 30th March 2010
What does the National Audit Office do? • Scrutinise the economy, efficiency and effectiveness of public spending. • Totally independent of Government. • Audit the accounts of all Central Government bodies; • AND produce around 60 value-for-money reports each year (6 on health issues) • Work with Parliament’s Public Accounts Committee (PAC), whose hearings are based on our work.
The NAO Value for Money Cycle Follow up Strategy Government Response Planning ? The vfm cycle Committee of Public Accounts Fieldwork/ Examination Drafting Publication Clearance
Aim of the Presentation • Brief outline of some recent health models • Discuss advantages of using models in policy evaluation • Present the challenges of using models to evaluate and inform policy development
End of Life CareNovember 2008 • Used a Markov model to examine the financial consequences and patient benefits of allowing more people with terminal conditions to be cared for and die at home.
End of Life CareConclusions • Found that for cancer alone around £100 million could be released by delivering relatively small reductions in emergency admissions and length of hospital stays
Healthcare Associated InfectionsJune 2009 • Used a systems dynamic model to examine the impact of various interventions designed to limit the spread of c.difficile by modelling the spread of an infection through a hospital
Healthcare Associated InfectionsConclusions • Found that more frequent hand washing, bed and toilet cleaning had little effect on their own. All had to be increased to contain the spread of an infection • Showed that identifying and isolating patients has the greatest effect on limiting the outbreak
Supporting People with AutismJune 2009 • Used a decision tree to explore the costs and benefits of providing specialist health, social care and employment services for adults with high-functioning autism • Compares current service provision with a counterfactual where specialist support is available throughout England.
Supporting People with AutismConclusions • Identification of 4% of target group would be cost neutral; 8% identification could save the taxpayer around £67 million per year • Savings would come from decreased benefit payments, increased tax revenue and decreased use of supportive accommodation
Rheumatoid ArthritisJuly 2009 • First model compares current cost of services with more rapid identification and treatment in primary care • Second model looks at the impact that more rapid identification and treatment across all care would have on NHS costs, productivity and quality of life
Rheumatoid ArthritisConclusions • Increasing people treated within 3 months by 10% would cost NHS £11m over 5 years. This would become cost neutral after 9 years • However, this could also lead to £31m in productivity gains for the economy and a gain of 4% in QALYs
Progress in Improving Stroke CareFebruary 2010 • Used to discreet event simulation to model a cohort of patients for 10 years from the onset of stroke through 3 distinct phases • Time to admission • Inpatient care • Long-term care
Progress in Improving Stroke CareConclusion • Services have improved quality of life for stroke victims since report in 2005 (£5.5k per QALY). NICE threshold £30k so progress assessed as achieving VfM
Advantages • Allow the robust comparison of counterfactuals to conclude on VfM • Allow us to quantify the likely outcome of recommendations on service reconfiguration • Add value for our clients • Increased external profile
Challenges • Technical output hard to explain to lay person in a short report • Teams have made innovative use of appendices and web products • We need to convince our audience that results of models are robust and useful • Involve Department’s technical staff to provide Civil Servants with assurance • Ensure robust internal and external quality assurance processes
Challenges • Resource and time intensive • True but good planning can ensure work meets deadlines • Even simple models can provide insight • Lack of data • Ability to do primary research and data gathering is limited so need to ensure that these approaches are planed in at the start and used where appropriate
Challenges • Benefits – either financial or service improvements - may not accrue to the department incurring the cost • Our remit is to illustrate the potential benefits to the exchequer or the economy • Challenge to government in how this conflict can be resolved
Next Steps for the NAO • Develop our capacity to use models in health studies and other areas • Increase engagement with academic community • Recruit a temporary academic research fellow under the ESRC placement scheme • Submit certain work for peer review
Contact • www.nao.org.uk • email@example.com • 0044 (0)20 7798 7981