A ‘ PROM’ising Future

1. A ‘PROM’ising Future The Relationship Between Patient Reported and Other Process Outcomes at Trust Level Esther Kwong Academic F2 Dept Primary Care and Public Health Project Supervisor: Dr Paul Aylin Educational Supervisor: Dr Graham Easton Dept Primary Care and Public Health

2. Contents The Context –Setting the Scene for PROMs NHS What are Outcomes Measuring Patient Reported Outcomes National PROMs Program Overview Research Question Methods Results Discussion & Conclusions What have I learnt…

3. The Past “Despite a century of developments in medical technology, and vast improvements in the ability of medical science to prevent, diagnose and treat disease and ill health, attempts to measure the outputs of health care in terms of their impact on patients’ health have not progressed beyond Florence Nightingale’s time.” Getting the most out of PROMs Kings Fund 2010

4. The Context

5. Setting the Scene for PROMS Darzi Review “High Quality for All” NHS White Paper Equity and Excellence: Liberating the NHS Appraisal of new technologies – PRO data incorporated in the evaluation of new technologies Routine measurement of pre/post elective surgery PROMs Since April 2009 DH Long Term Conditions PROMs Pilot Since 2010

6. Patient Centred If quality is to be at the heart of everything we do, it must be understood from the perspective of patients. Patients pay regard both to clinical outcomes and their experience of the service... Lord Darzi The ultimate measure by which to judge the quality of medical effort is whether it helps patients, as they see it. Donald Berwick

7. What Are Outcomes Traditional Ways of Planning = Measuring in terms of OUTPUT • Quantifying what is produced, implemented, provided, and developed in the health service Increasing Focus = Measuring in terms of OUTCOME • Quantifying extent of any health impact on patients • Change in various dimensions ~physiological (e.g. functional status) or psychological (e.g. attitudes) • Can be harnessed from different sources

8. Sources of Outcomes

9. Measuring Patient Reported Outcomes Patient Reported Outcomes (PRO) • Health status as perceived by the patient Patient Reported Outcomes Measures (PROMs) • Measurement tools to harness this information • Can be used in two points in time to record change in health status • Can be assessed against patient progress or health interventions received • Various types available • Much dedicated research and analysis on validating questionnaire types

10. Types of PROMS

11. National PROMs Program Overview Since 1 April 2009 Providers required to collect and report PROMs Four key NHS funded elective interventions Unilateral hip replacements Unilateral knee replacements Groin hernia surgery Varicose vein surgery Expected to invite patients to complete a pre-operative PROMs questionnaire (Q1) Post-operative questionnaires (Q2) are then sent to patients following their operation after a specified time period.

12. … the NHS will be the first health care system in the world to measure what it produces in terms of health, rather than in terms of the production of health care. Getting the most out of PROMs Kings Fund …the NHS will be the first health care system in the world to measure what it produces in terms of health, rather than in terms of the production of health care. Getting the most out of PROMs Kings Fund

13. PROMs Used for the National Program

14. Research Topic Aim: To explore the relationship between routinely collected patient reported and other process outcomes at trust level Null Hypothesis: There is no relationship between patient reported and other process outcomes at trust level Methods: Aggregate analysis conducted using STATA 11 on trust level data

15. Participation and Coverage 2010 Participation rate of 69.7%. 245,488 eligible hospital episodes 171,080 pre-operative questionnaires returned Return rate of 75.8% 147, 974 post-operative questionnaires sent out 112,163 returned

16. National PROMs Key Final Results 09-10 Overview Oxford Hip and Knee Score 95.7% of hip replacement respondents 91.4% of knee replacement respondents Recorded an improvement following operation EQ-5D Index score 87.2% of hip replacement respondents 77.6% of knee replacement respondents Recorded an increase in general health following operation

17. Data Sources Dr Foster Data Orthopaedic Revision Rates Orthopaedic Readmission Caseload Staff to bed ratio HSMR National Joint Registry Data Orthopaedic Procedures Caseload HES Inpatient Data Elective Surgery Waiting Times Emergency Admission Caseload

18. Data Comparison PROMs Outcomes 2010 Data- Hip and Knee Data Case Adjusted Health Gain (Q2-Q1) EQ5D Index EQ5D Visual Analogue Scale Oxford Hip/ Knee Score Other Process Outcomes/ Hospital Indicators Compared Hospital Standardised Mortality Ratios Dr Foster Orthopaedic Revision Relative Rate Dr Foster Orthopaedic Readmission Caseload National Joint Registry Data Orthopaedic Procedure Caseload Hospital Episode Statistics Elective Surgery Waiting times Hospital Episode Statistics Emergency Admissions Caseload Hospital Staff to Bed Ratios

19. Descriptive Results Hip

20. Correlations Hip

21. Regression Hip

22. Description Results Knee

23. Correlations Knee

24. Regression Knee

25. Discussion – Important Findings 1 Weak positive correlations between Hip and Orthopaedic Procedures Caseload and all Hip PROMs health gain • Suggests the more procedures a trust does the better its quality of hip replacement procedure perceived by patient • This is an expected correlation direction Weak negative correlations between Emergency Admission Caseload and all Knee PROMs health gain • Suggests the more emergency admissions a trust has the worse the patient perceived outcome for a knee replacement procedure • unexpected correlation direction, warrants further exploration into relational factors – such as trust specialisation and quality relationship

26. Discussion -Important Findings 2 Weak Positive Correlation Between EQ5D Visual Analogue Scale and Oxford Hip Score health gain for hip patients and Waiting Times for Elective Surgery • Suggests the longer a patient waits for elective surgery in a trust the more health gain perceived from hip operation • Unexpected correlation direction • Disease progression factors are adjusted for • May be explained by expectation management ‘Patient Satisfaction = Patient Experience - Patient Expectation’ • Longer waiting times may decrease expectation affecting perceived outcome • Lead time difference

27. Discussions Limitations Recruitment Bias LSHTM Report to Dept of Health on PROMs recorded correlation coefficient of -0.38 between EQ5D score and every 20% increased recruitment, suggesting low recruitment rates can introduce bias The report recommended a target recruitment rate of 80% Response Bias Studies suggest non responders were younger in all PROMs, This is particularly evidenced in orthopaedic PROMs Patient Reported Outcome Measures (PROMs) in Elective Surgery Report to the Department of Health, London School of Hygiene and Tropical Medicine

28. Conclusions Weak/ Lack of Correlations suggests Patient Reported Outcomes are capturing an added dimension of quality that traditional process outcome and clinical indicators were not measuring Weak correlations findings at trust level maybe due to aggregation, this could eliminate clinical variation within and between hospital services as well as patient characteristics • Evidence from clinical governance concluded acute hospitals services were ‘A mix of good and bad’ • Analysis of PROMs at clinical level and unadjusted data may provide further explanations and strengthen correlations Lack of evidence/ data available for statistical relationship significance for correlations • Further work building larger aggregate data set on PROMs • Analysis on new PROMs data, or analysis spanning two years of PROMs data

29. What I learnt from this Academic Rotation Teaching Formal Teaching courses/ training Clinical Methods teaching 3rd year Imperial Students Problem Base Learning facilitator GP Wealth of clinical experiences Primary care setting exposure Consultation simulation training Nature of Rotation 2 days a week in GP surgery clinical duties 3 days a week dept based research and teaching activities Research Literature search on PROMs Insight in health services research Handling aggregate data Data management Statistical analysis on STATA Seminars and Journal Club Experience of life as an academic!

30. References Bevan G , Skeller M, 2011 Competition between hospital and clinical quality BMJ 2011; 342:d3589 Berwick D, Hiatt H, Janeway P, Smith R. 1997 An ethical code for everybody in health care BMJ 1997;315:1633 Black N, Browne J, Cairns J. 2006. Health care productivity. British Medical Journal 333: 312–313. Brooks R, Rabin R, de Charro F. 2003. The Measurement and Valuation of Health Status using EQ-5D: A European Perspective. Kluwer: Dordrecht. Browne J, Jamieson L, Lawsey, J, van derMeulen J, Black N, Cairns J, Lamping D, Smith, S, Copley L,Horrockes, J. 2007. Patient Reported Outcome Measures (PROMs) in Elective Surgery. Report to theDepartment of Health. Available from: www.lshtm.ac.uk/hsru/research/PROMs-Report-12-Dec-07.pdf. Burge P, Devlin N, Appleby J, Gallo F, Nason E, Ling T. 2006. Understanding patients’ choices at the point of referral. Technical report TR359-DOH, Cambridge: RAND Europe. Available from: www.rand.org/pubs/technical_reports/TR359/. Darzi L. 2008. High Quality Care for All. NHS Next Stage Review: Final Report, Department of Health, London. Available from: www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/ DH_085825. Dawson J, Rogers K, Doll H, Using Patient Reported Outcomes Routinely: An example in context of Shoulder surgery Open Epidemiology Journal 2010, 3,42-52 Department of Health. 2008. Guidance of the Routine Collection of Patient Reported Outcome Measures (PROMs).Department of Health document DH_081179[1].pdf. Devlin N, Appleby J. 2010. Getting the Most Out of PROMs: Putting Health Outcomes at the Heart of NHS Decision Making. Kings Fund/Office of Health Economics: London. Dolan P. 1997. Modelling valuations for EuroQol health states. Medical Care 35(11): 1095–1108.2010). Greenhalgh J, Long A, Flynn Rob, 2004 The use of patient reported outcome measures in routine clinical practice: Lack of theory or lack or impact. Social Science and Medicine 60 (2005) 833-843 EuroQol Foundation. Springer: Rotterdam. Hospital Episode Statistics: Finalised Patient Reported Outcome Measures (PROMs) in England: April 2009 – March 2010 Isis Outcomes Patient Reported Outcome Measures from the University of Oxford, Orthopaedic Pros http://www.isis-innovation.com/outcomes/orthopaedic/ London School of Hygiene and Tropical Medicine Patient Reported Outcomes on Elective Surgery, Report To Department of Health Dec 2007, http://www.lshtm.ac.uk/php/hsrp/research/proms_report_12_dec_07.pdf NHS North West. 2010. Advancing quality. Available from: www.advancingqualitynw.nhs.uk National Council on Ageing and Older People, 1998 health promotion strategy for older people in Ireland. Adding years to life and life to years National Network of Libraries of Medicine Guide 3: Define Measurable Goals, Outputs and Outcomes http://nnlm.gov/outreach/community/goals.html Office of Health Economics. 2008. NHS Outcomes, Performance and Productivity. Report of the Office of Health Economics Commission. OHE: London. Szende A, Oppe M, Devlin N. 2007. EQ-5D Valuation Sets: An Inventory, Comparative Review and Users’ Guide.

31. Acknowledgements I want to thank all those in the department who have contributed their expertise and advice towards this project and towards my educational development Dr Paul Aylin, Dr Graham Easton, Dr Jenny Lebus, Dr Michael Soljak, Dr Sonia Saxena Dr Fiona Hamilton, Dr Matthew Harris, Dr EszterVamos Elizabeth Cecil, FarzanRahman, Dr GhasemYadegarfar