Conducting a PIP: The Performance Improvement Project Manual

1. Conducting a PIP: The Performance Improvement Project Manual Dow A. Wieman, Ph.D. The Evaluation Center @HSRI Robert Egnew, M.S.W., M.P.H National Association of County Behavioral Health Directors Ed Diksa, Sc.D. California Institute for Mental Health

2. The Evaluation Center @HSRI Human Services Research Institute Cambridge, MA A National Technical Assistance Center for Evaluation of Mental Health Systems Funded by the Center for Mental Health Services,Substance Abuse and Mental Health Services Administration

3. Conducting a PIP: The Performance Improvement Project Manual Judy Ashley Sheila Baler Eydie Dominguez Fred Hawley Darcy Johnson Gail Kinnamon Diane Koditek John Lessley Beth Martinez Rita McCabe-Hax Kenneth Meibert Timothy Mullins Sandra Naylor Goodwin Madelyn Schlaepfer

4. Objectives Gain familiarity with the PIP manual Understand basic principles of quality improvement Gain practical (hands-on) understanding of PIP concepts and methodology Supplements, not supplants, DMH contract and EQRO protocol, guidance, training, and oversight

5. Basic Principle I Experienced managers, clinicians, consumers and others “in the field” usually know instinctively when behavioral health care is of the highest quality and, alternatively, when there are gaps between what is possible and what is currently provided. This Manual describes methods for converting this intuitive knowledge into effective action by means of Performance Improvement Projects.

6. Basic Principle II: Quality Improvement • Addresses gaps between what is possibleandwhat is currently provided • Not “exhortation, blaming and trying harder.” • Data-driven and cyclical

7. Basic Principle III—Error • Error as defects (Quality gaps) • Measurement error

8. The PIP Process How do I: • Plan for a PIP? • Develop the data collection plan? • Decide whether and how to sample? • Decide whether and how to conduct a survey? • Develop the data management and analysis plan? • Develop the improvement strategy? • Implement the improvement intervention? • Report the results of the PIP? • Sustain the improvement? • Evaluate the PIP?

9. How do I identify study topics? A study topic is an area of concern. • May be administrative or clinical (satisfaction and process or mh and functional status) • May be one required by DMH contract (capacity, access, satisfaction, service system, continuity of care, provider relations) • Continuous data collection….

10. How do I identify study topics? (cont.) Continuous collection of data: • Socio-demographic characteristics • Utilization by subpopulations • High-volume, high-risk services • Known problem areas • Sentinel events (death, serious injury)

11. How do I identify study topics? (cont.) Other sources: • Previous initiatives • Consumers/families • Providers • Compliance • UM data • Conferences and literature • Other organizations

12. IOM Quality Chasm: 6 Aims • Safe:Freedom from accident or injury. • Effective:Disciplined use of systematically acquired evidence • Patient-centered:Health care that establishes a partnership among providers, patients, and their families • Timely:Individuals are able to obtain needed care and delays in obtaining care are minimized. • Efficient:Continual reduction of waste in health care. • Equitable:The health care system should work to improve the health status of all, reduce health disparities among subgroups.

13. How do I prioritize study topics? • Relevance to population • Prevalence, volume or need • Extent of risk • Meaningful difference • Improvement potential

14. Practical considerations… • Ease of implementation • Burden • Acceptability • Available data • Available standards

15. Ethical considerations… Requirements less stringent than research BUT… Some recommend higher standards (risk)

16. How do I define the study question? • Clear: Write it out • Simple: Keep digging (RCA) • Specific: Population, setting, problem, outcomes • Answerable: Available and usable data

17. How do I choose study indicators? “Defined, measurable variables which reflect a discrete event or a status, used to measure performance.” (EQRO Protocol)

18. How do I choose study indicators? • Oftena rate (quality gap) • Sources: Practice guidelines, EBPs (fidelity), established process measures (www.cqaimh.org), benchmarks • Also: Controllable outcomes

19. What makes a good indicator? • Objective (unbiased) • Clear and unambiguous • Current • Reliable • Valid • Actionable

20. How do I establish indicator criteria? The set of rules (“measure specifications”) describing how the indicator is to be applied. • Study population (diagnosis, enrollment, etc.) • Service definition (criteria for receiving) • Units of measure • Data collection methods • Numerator and denominator

21. II. What should the data collection plan include? • What type of data do I need? • How should the data be collected? • When should the data be collected? • Who should collect the data?

22. What type of data do I need? Availability and quality of data to: • Address the study question • Meet indicator criteria Ready made is best Many sources

23. How should the data be collected? • Keep it simple • Keep it easy (data entry) • Keep it clear (instructions)

24. When should the data be collected? • Baseline • Post-intervention • Follow-up

25. Who should collect the data? Type of data • Chart reviews • MIS • Survey Available skills

26. Pilot Test • What type of data do I need? • How should the data be collected? • When should the data be collected? • Who should collect the data?

27. Phase III: To Sample Or Not? • May be less costly, more current • May be any unit of information • Must be representative (error)

28. Phase IV: To Survey Or Not? • Required for satisfaction • Response rates: Must be representative (error) • Should be actionable

29. Phase V: What should I include in the data management/analysis plan? What is a data archive? What is required for the analysis? • Simplicity • Case-mix adjustment • Standards, means, norms, benchmarks

30. Phase VI: How do I develop the improvement strategy? An intervention designed to change behavior at an institutional, practitioner or beneficiary level (EQRO Protocol)

31. Phase VI: How do I develop the improvement strategy? Barrier analysis Avoid confounding (KISS) Sources for interventions: • Stakeholders • High performers • Research studies • Disease management models • Promising practices

32. Phase VII: How do I implement the improvement intervention? • Practical • Based on RCA, stakeholder input • Addresses human factors Organizational readiness Leadership Buy-in Data packaging and presentation

33. Phase VII: How do I implement the improvement intervention (cont)? • Pilot test, modify • Assess requirements for sustainability

34. Phase VIII: How do I report results of my PIP? • What should I include in the Report? • What should I consider in Report design? • How should I disseminate the Report?

35. What should I consider in the report? • Credibility and Utility • Consider your audience (multiple formats, media) • Consider templates (e.g. NCQA)

36. What should I consider in the design of the Report? • Focus on study question • Hierarchy of importance (may vary among stakeholders) • Multiple formats, media • Pilot test

37. What should I consider in disseminating the Report? Develop strategy early Consider goals, target accordingly: • Demonstrate Compliance • Facilitate change • Sustain improvement • Achieve buy-in to PIP process

38. IX: How do I evaluate the Pip process? • Don’t neglect: PIP PIP • Process (implementation) and Impact Evaluation • Don’t forget satisfaction

39. Excerpt: Logic Modelfrom National Goals and Core Measures of Cancer Care Quality(McGlynn, EA and J. Malin, 2002)

40. Study Topic Racial disparities in the psychopharmacological treatment of persons with schizophrenia

41. Racial disparities in the psychopharmacological treatment of persons with schizophrenia Modeled after: Dickey, B. Normand, S-L. Hermann, RC, Eisen, SV. Cortes, DE. Cleary, PD, Ware, N. Guideline Recommendations for Treatment of Schizophrenia: The Impact of Managed Care. Archives of General Psychiatry. 60 (Apr), 2003 (NOTE: The following examples are not intended to represent the design or results of this study)

42. Study Question(s) • How does the appropriateness of psychopharmacological treatment compare for whites and minorities? • How do white and minorities compare in rating the interpersonal aspects of their pharmacological treatment? • How do whites and minorities compare in assessment of outcomes of their pharmacological treatment?

43. Study Questions How does the appropriateness of psychopharmacological treatment compare for whites and minorities? or How does adherence to treatment guidelines (Schizophrenia PORT) compare for whites and minorities?

44. How does adherence to treatment guidelines (Schizophrenia PORT) compare for whites and minorities? Relevance to population? Prevalence, volume or extent of need? Extent of risk? Meaningful difference? Can PIP achieve improvement?

45. Dosage for Acute Episodes How does adherence to PORT guidelines for daily medication dosage for acute episodes (300-1000 CPZ units) compare for whites and minorities?

46. Dosage for Acute Episodes on Inpatient Service How does adherence to PORT guidelines for daily medication dosage for acute episodes (300-1000 CPZ units) compare for whites and minorities treated on the inpatient service?

47. Dose within PORT Range White vs. Non-White

48. Focus groups Barrier analysis: What is the problem? 1) 2) 3) 4)

49. Possible study questions: 1) lack of knowledge Will initial and annual training, with reminders, for physicians and nurses on PORT guidelines and disparities in treatment improve the proportion of patients with schizophrenia receiving appropriate dose of medication for inpatient treatment of acute episode?

50. Possible study questions: 2) Lack of time to assess Will additional information on the intake screening form, and earlier scheduling of psyhcopharm assessment improve the proportion of patients with schizophrenia receiving appropriate dose of medication for inpatient treatment of acute episode?