Conducting a PIP: The Performance Improvement Project Manual

1. 1 Conducting a PIP: The Performance Improvement Project Manual Dow A. Wieman, Ph.D. The Evaluation Center @HSRI Robert Egnew, M.S.W., M.P.H National Association of County Behavioral Health Directors Ed Diksa, Sc.D. California Institute for Mental Health

2. 2  The Evaluation Center @HSRI Human Services Research Institute Cambridge, MA A National Technical Assistance Center for Evaluation of Mental Health Systems Funded by the Center for Mental Health Services,Substance Abuse and Mental Health Services Administration

3. 3 Conducting a PIP: The Performance Improvement Project Manual Judy Ashley Sheila Baler Eydie Dominguez Fred Hawley Darcy Johnson Gail Kinnamon Diane Koditek John Lessley Beth Martinez Rita McCabe-Hax Kenneth Meibert Timothy Mullins Sandra Naylor Goodwin Madelyn Schlaepfer

4. 4 Objectives Gain familiarity with the PIP manual Understand basic principles of quality improvement Gain practical (hands-on) understanding of PIP concepts and methodology Supplements, not supplants, DMH contract and EQRO protocol, guidance, training, and oversight

5. 5 Basic Principle I Experienced managers, clinicians, consumers and others �in the field� usually know instinctively when behavioral health care is of the highest quality and, alternatively, when there are gaps between what is possible and what is currently provided. This Manual describes methods for converting this intuitive knowledge into effective action by means of Performance Improvement Projects.

6. 6 Basic Principle II: Quality Improvement Addresses gaps between what is possible and what is currently provided Not �exhortation, blaming and trying harder.� Data-driven and cyclical

7. 7 Basic Principle III�Error Error as defects (Quality gaps) Measurement error Control charts: range of acceptable errorControl charts: range of acceptable error

8. 8 The PIP Process How do I: Plan for a PIP? Develop the data collection plan? Decide whether and how to sample? Decide whether and how to conduct a survey? Develop the data management and analysis plan? Develop the improvement strategy? Implement the improvement intervention? Report the results of the PIP? Sustain the improvement? Evaluate the PIP?

9. 9 How do I identify study topics? A study topic is an area of concern. May be administrative or clinical (satisfaction and process or mh and functional status) May be one required by DMH contract (capacity, access, satisfaction, service system, continuity of care, provider relations) Continuous data collection�.

10. 10 How do I identify study topics? (cont.) Continuous collection of data: Socio-demographic characteristics Utilization by subpopulations High-volume, high-risk services Known problem areas Sentinel events (death, serious injury)

11. 11 Other sources: Previous initiatives Consumers/families Providers Compliance UM data Conferences and literature Other organizations

12. 12 IOM Quality Chasm: 6 Aims Safe: Freedom from accident or injury. Effective: Disciplined use of systematically acquired evidence Patient-centered: Health care that establishes a partnership among providers, patients, and their families Timely: Individuals are able to obtain needed care and delays in obtaining care are minimized. Efficient: Continual reduction of waste in health care. Equitable: The health care system should work to improve the health status of all, reduce health disparities among subgroups.

13. 13 How do I prioritize study topics? Relevance to population Prevalence, volume or need Extent of risk Meaningful difference Improvement potential

14. 14 Practical considerations� Ease of implementation Burden Acceptability Available data Available standards

15. 15 Ethical considerations� Requirements less stringent than research BUT� Some recommend higher standards (risk)

16. 16 How do I define the study question? Clear: Write it out Simple: Keep digging (RCA) Specific: Population, setting, problem, outcomes Answerable: Available and usable data

17. 17 How do I choose study indicators? �Defined, measurable variables which reflect a discrete event or a status, used to measure performance.� (EQRO Protocol) Event: process e.g. screening Status: measurable level e.g. functioningEvent: process e.g. screening Status: measurable level e.g. functioning

18. 18 How do I choose study indicators? Often a rate (quality gap) Sources: Practice guidelines, EBPs (fidelity), established process measures (www.cqaimh.org), benchmarks Also: Controllable outcomes

19. 19 What makes a good indicator? Objective (unbiased) Clear and unambiguous Current Reliable Valid Actionable

20. 20 How do I establish indicator criteria? The set of rules (�measure specifications�) describing how the indicator is to be applied. Study population (diagnosis, enrollment, etc.) Service definition (criteria for receiving) Units of measure Data collection methods Numerator and denominator

21. 21 II. What should the data collection plan include? What type of data do I need? How should the data be collected? When should the data be collected? Who should collect the data?

22. 22 What type of data do I need? Availability and quality of data to: Address the study question Meet indicator criteria Ready made is best Many sources

23. 23 How should the data be collected? Keep it simple Keep it easy (data entry) Keep it clear (instructions)

24. 24 When should the data be collected? Baseline Post-intervention Follow-up

25. 25 Who should collect the data? Type of data Chart reviews MIS Survey Available skills

26. 26 Pilot Test What type of data do I need? How should the data be collected? When should the data be collected? Who should collect the data?

27. 27 Phase III: To Sample Or Not? May be less costly, more current May be any unit of information Must be representative (error)

28. 28 Phase IV: To Survey Or Not? Required for satisfaction Response rates: Must be representative (error) Should be actionable

29. 29 Phase V: What should I include in the data management/analysis plan? What is a data archive? What is required for the analysis? Simplicity Case-mix adjustment Standards, means, norms, benchmarks

30. 30 Phase VI: How do I develop the improvement strategy? An intervention designed to change behavior at an institutional, practitioner or beneficiary level (EQRO Protocol)

31. 31 Phase VI: How do I develop the improvement strategy? Barrier analysis Avoid confounding (KISS) Sources for interventions: Stakeholders High performers Research studies Disease management models Promising practices

32. 32 Phase VII: How do I implement the improvement intervention? Practical Based on RCA, stakeholder input Addresses human factors Organizational readiness Leadership Buy-in Data packaging and presentation

33. 33 Phase VII: How do I implement the improvement intervention (cont)? Pilot test, modify Assess requirements for sustainability

34. 34 Phase VIII: How do I report results of my PIP? What should I include in the Report? What should I consider in Report design? How should I disseminate the Report?

35. 35 What should I consider in the report? Credibility and Utility Consider your audience (multiple formats, media) Consider templates (e.g. NCQA)

36. 36 What should I consider in the design of the Report? Focus on study question Hierarchy of importance (may vary among stakeholders) Multiple formats, media Pilot test

37. 37 What should I consider in disseminating the Report? Develop strategy early Consider goals, target accordingly: Demonstrate Compliance Facilitate change Sustain improvement Achieve buy-in to PIP process

38. 38 IX: How do I evaluate the Pip process? Don�t neglect: PIP PIP Process (implementation) and Impact Evaluation Don�t forget satisfaction

39. 39 Excerpt: Logic Model from National Goals and Core Measures of Cancer Care Quality (McGlynn, EA and J. Malin, 2002)

40. 40 Study Topic Racial disparities in the psychopharmacological treatment of persons with schizophrenia

41. 41 Racial disparities in the psychopharmacological treatment of persons with schizophrenia Modeled after: Dickey, B. Normand, S-L. Hermann, RC, Eisen, SV. Cortes, DE. Cleary, PD, Ware, N. Guideline Recommendations for Treatment of Schizophrenia: The Impact of Managed Care. Archives of General Psychiatry. 60 (Apr), 2003 (NOTE: The following examples are not intended to represent the design or results of this study)

42. 42 Study Question(s) How does the appropriateness of psychopharmacological treatment compare for whites and minorities? How do white and minorities compare in rating the interpersonal aspects of their pharmacological treatment? How do whites and minorities compare in assessment of outcomes of their pharmacological treatment? Process, satisfaction, outcomes (self-reported)Process, satisfaction, outcomes (self-reported)

43. 43 Study Questions How does the appropriateness of psychopharmacological treatment compare for whites and minorities? or How does adherence to treatment guidelines (Schizophrenia PORT) compare for whites and minorities? What is appropriate?What is appropriate?

44. 44 How does adherence to treatment guidelines (Schizophrenia PORT) compare for whites and minorities? Relevance to population? Prevalence, volume or extent of need? Extent of risk? Meaningful difference? Can PIP achieve improvement? Which recommendations?: most likely to make a differenceWhich recommendations?: most likely to make a difference

45. 45 Dosage for Acute Episodes How does adherence to PORT guidelines for daily medication dosage for acute episodes (300-1000 CPZ units) compare for whites and minorities? What location?: bigger problem in outpatient but data more available in inpatient What location?: bigger problem in outpatient but data more available in inpatient

46. 46 Dosage for Acute Episodes on Inpatient Service How does adherence to PORT guidelines for daily medication dosage for acute episodes (300-1000 CPZ units) compare for whites and minorities treated on the inpatient service? Conduct a Pilot study 40 patientsConduct a Pilot study 40 patients

47. 47 Dose within PORT Range White vs. Non-White There is disparity�but neither is good. What is going on? Focus groupThere is disparity�but neither is good. What is going on? Focus group

48. 48 Focus groups Barrier analysis: What is the problem? 1) 2) 3) 4) Lack of knowledge Lack of time to assess appropriate dose Lack of time to adjust dose Atypical patients Lack of knowledge Lack of time to assess appropriate dose Lack of time to adjust dose Atypical patients

49. 49 Possible study questions: 1) lack of knowledge Will initial and annual training, with reminders, for physicians and nurses on PORT guidelines and disparities in treatment improve the proportion of patients with schizophrenia receiving appropriate dose of medication for inpatient treatment of acute episode?

50. 50 Possible study questions: 2) Lack of time to assess Will additional information on the intake screening form, and earlier scheduling of psyhcopharm assessment improve the proportion of patients with schizophrenia receiving appropriate dose of medication for inpatient treatment of acute episode? Whatever the question, you are now prepared to proceed�Whatever the question, you are now prepared to proceed�

51. 51 Proceed with PIP Decide on indicators: PORT Guidelines Define indicator criteria: Denominator age 21-64 with schizophrenia eligible for psychopharm admitted to inpatient during period, stratified by race. Numerator: number discharged on dose within PORT guidelines stratified by race Dose at intake and discharge Data: Chart review�random sample for baseline, again six months later. Data analysis methods. Implementation: Present data to clinical leadership, line staff, develop plan Present results, decide on follow-up action, etc.

52. 52 Contact Information Dow Wieman The Evaluation Center@ HSRI www.tecathsri.org dwieman@hsri.org 978 283 3245

53. 53 Questions and discussion