Data-Driven Health System Performance Evaluation for PCTs

1. Using Data to Drive Health System Performance Commissioned from Ovations by the National Primary and Care Trust Development Programme

2. Health Care Evaluation & Measurement for PCTs Some Key Evaluation Terms Acceptable Accessible Accomplishment Accountable Accuracy Analysis Appropriate Available Cohorts Comparison Content Context Control Cost Data: primary and secondary Goals Judgment Metrics Norms Objectives Outcomes Outputs Precision Process Purpose Quality Quantity Recording Reliability Reporting Standards Synthesis Time Timelines Validity Value Weighting Worth

3. Health Care Evaluation & Measurement for PCTs Things to Consider • Comparing and contrasting various types of evaluation • Identification of problems that may hinder an effective evaluation • Reasons to conduct an evaluation • The planning and conducting of an evaluation • Defining metrics and methods to measure and report evaluation findings

4. Health Care EvaluationKey Ideas for PCTs • Evaluation is critical for all health care, and prevention/ health promotion initiatives. • Evaluation must be designed early in the process of health care planning. Remember PRECEDE/PROCEED! • The process of designing an evaluation must be a collaborative effort of all stakeholders. • Evaluation ultimately becomes “judgment.” Who has the power to decide? • Evaluation does not need to be a formal academic study • Do you report successes, failures, strengths, weaknesses?

5. Health Care EvaluationPurposes for PCTs 1. To determine achievement of objectives related to improved health status 2. To improve health program implementation 3. To provide accountability to funders 4. To increase community support 5. To contribute to the scientific base for community. public health interventions 6. To impact policy decisions

6. Health Care EvaluationDefinition Evaluation: Determining the value or worth of the health care initiative against a standard of acceptability. To examine or judge. (The key is who establishes the standard and who judges!)

7. Health Care EvaluationElements • Context: What, when , where, and who • Process: How care is organized and delivered • Content: Program elements to be provided and why - available birdseed • Output: How many times did the bird flap its wings? • Outcome: Did the bird fly? • Impact:How high? How far? Where?

8. Health Care EvaluationTypes Process Evaluation: Examines the procedures and tasks involved during the implementation of a program.

9. Health Care EvaluationTypes Process Evaluation • When to use: As soon as the health initiative begins • What it shows: How well a program is working as it goes • Why is it useful: Identifies early problems

10. Health Care EvaluationTypes Outcome Evaluation: Used to obtain descriptive data on a project and to document short-term results. Focuses on an ultimate goal of a health care program or treatment. Generally measured by vital statistics in a population.

11. Health Care Evaluation Types Outcome Evaluation • When to use: For ongoing programs at appropriate intervals or for one time programs when program is complete • What it shows: Has program reached its ultimate goal. • Why is it useful: Learn from successes and for future funding.

12. Health Care Evaluation Types Impact Evaluation: Is the most comprehensive type of evaluation because it focuses on the long-range results and the resultant improvements in health status. • Impact evaluation is the most costly. • Information obtained from an impact evaluation can include changes in e.g.,morbidity and mortality.

13. Health Care EvaluationTypes Impact Evaluation • When to use: After the health program has made “contact” with at least one person or a population • What it shows: Changes in knowledge, attitudes, and beliefs • Why is it useful: Allows management to modify resources effectively

14. Health Care Evaluation Lets pick a population health programme idea for your specific PCT and consider aspects of process, outcome and impact evaluation.

15. Metrics and Methods to Measure and ReportIndicators of Hospital Utilisation • Assumption: > Hospital admission is an adverse event that should be avoided whenever possible > Not all admissions are preventable • Two major indicators > Admissions per 1000 > Days per 1000

16. Metrics and Methods to Measure and ReportIndicators of Hospital Utilisation • Admissions per 1000 > In part measures a systems success at preventing acute disease episodes or caring for them in alternative systems > Considered a prevention indicator • Days per 1000 > Reflects both frequency of admission and length of stay > Measure episode characteristics

17. Metrics and Methods to Measure and ReportIndicators of Hospital Utilisation • Both measures are typically collected separately for: > Adult and pediatric medical > Surgical > ICU > Obstetrics > Boarder babies (newborn days after mother’s discharge) > Mental health > Substance abuse

18. Metrics and Methods to Measure and ReportRisk Adjustment • Prospective vs. retrospective • In general populations the ‘theoretical ceiling to predication is between 18-22% >This is higher in sicker, chronic populations • Demographic models >Typically explain 2-5% of variance in utilization

19. Metrics and Methods to Measure and ReportRisk Adjustment • Adding prior use increase the predicative ability of models to between 8 – 14% > Some antidotial evidence that adding Rx and Lab can increase predicative ability to the 18-22% range • Adding health status changes the weights but does not increase the predicative ability of models

20. Metrics and Methods to Measure and ReportTraditional Health Care Performance Indicators • Finances > Revenue and costs • Clinical Activities • Quality of Care > Includes access

21. Metrics and Methods to Measure and ReportGuidelines for Selecting Indicators • The ability of systems to produce the data required to generate the indicator • The perceived importance of the indicator to the development and assessment of the system • The current utility of the indicator in organizations within the system • The comparability of the indicator across systems • The uniqueness of the applicability of the indicator

22. Metrics and Methods to Measure and ReportIndicators of System Performance • Population Health > SF-30, SF-12, etc > Mental Health Index (such as the DIS) > Disease specific incidence and prevalence rates > Disease specific outcomes of care (SF-36, ADLs, IADLs)

23. Metrics and Methods to Measure and ReportIndicators of System Performance • Quality of Care > Compliance with guidelines by type of provider > Incidence of Sentinel Events: - Unplanned re-visits - Avoidable hospitalisations - Hospital re-admission - Mortality

24. Metrics and Methods to Measure and ReportIndicators of System Performance • Quality of Care (cont.): > Peer review > Disease specific mortality - Surgical - Infant mortality - Deaths due to cervical cancer > Frequency of preventable acute episodes for chronic conditions

25. Metrics and Methods to Measure and ReportIndicators of System Performance • Quality of Care (cont.): > RX - Inappropriate poly pharmacy - ADRs • Episode Prevention

29. Calculating Bed Days • Works for any given time period (e.g. day, month to date, year to date) • [A/ (B/365)] /(C/1000) > A = Gross bed days per unit of time > B = Days per unit of time > C = population size

30. Knowledge ManagementPlanning Cycle for Driving PCT Health Care System Performance Social/Health Indicators Evaluation Data & Analysis Implementation Useful Information Operations Planning Knowledge Management Strategy