NRSA AHRQ Workshop: Quality and Quality Improvement

1. NRSA AHRQ Workshop:Quality and Quality Improvement Carole Lannon MD MPH Peter Margolis MD PhD

2. Session Objectives Overview: what is driving the quality momentum? What do we know about quality improvement: Examples: VON, ESRD, The challenges of evaluation An in-depth example of successful QI Methods exist to improve quality Key components of successful QI methods

3. Session objectives Overview: what is driving the quality momentum? What is quality and quality improvement? What changes clinical care? A conceptual framework for improvement An in-depth example of successful QI Challenges with evaluation Next

5. The Chain of Effect in Improving Health Care Quality

6. What is driving the quality momentum ?

7. “Drivers” in the current environment Clinical teams’ motivation to provide best care for patients Consumers (patients and families) expectations to receive best care Leadership Raising awareness and building will Supporting change and spreading key improvements Maintenance of certification and ACGME competencies Public and private emphasis on use of quality measures Pay-for-performance programs

8. Board certification of physicians Certification: Until ~1990: Knowledge-based one-time test Re-certification: 1990s: episodic cognitive examination every 6-10 yrs Maintenance of certification Continuous rather than episodic Focus is on six primary competencies

10. Peer & Patient Surveys Practice Assessment and Quality Improvement Component

11. National emphasis on measures and P4P National Quality Forum Set consensus-based national standards for measurement and public reporting of healthcare performance data National Committee on Quality Assurance Joint Commission for the Accreditation of Healthcare Organizations AMA Consortium on Performance Measures Ambulatory Care Quality Alliance Leapfrog Group

12. National emphasis on P4P and HIT P4P Link quality, measures, and reimbursement Over 100 programs underway; strong push from CMS for legislative mandate Bridges to Excellence CMS/Premier Integrated Health Association Office of the National Coordinator for Health Information Technology

13. 2004 National Reports on Quality and Disparities Second annual reports focus on quality of and disparities in health care in America AHRQ

14. Key Question What do we mean by Quality? What is the difference between Quality and Quality Improvement?

19. Percent of Patients Meeting the NKF-DOQI Target Urea Reduction Ratio of 65%figure 4.25, prevalent hemodialysis patients, 1999, by HSA

20. Quality of Care (ESRD)

21. Adequacy of Hemodialysis

22. “Adults received (only) 54.9% of recommended care….strategies to reduce these deficits are warranted” -McGlynn NEJM, 2003

23. AHRQ taxonomy ‘QI strategies’ Provider reminder systems Audit and feedback Provider education Patient education Promotion of self-management Patient reminder systems Organizational change Financial incentives, regulation, and policy

24. What Changes Clinical Care? Let me talk briefly about the conceptual approach to supporting improvement in primary care. This slide shows a summary of what is know about what works in supporting change in practice from the work of Dave Davis. On the vertical axis are various types of interventions, traditional CME, educational materials, audit and feedback, patient mediated interventions, reminders, opinion leaders, educational outreach. Across the bottom are the proportion of studies in which change in practice took place. As Denise has described, traditional educational interventions are not especially effective at promoting change. There appears to be a gradient that involves measurement (audit and feedback), patient focused interventions, communication (opinion leaders) and more tailored approaches (educational outreach). Let me talk briefly about the conceptual approach to supporting improvement in primary care. This slide shows a summary of what is know about what works in supporting change in practice from the work of Dave Davis. On the vertical axis are various types of interventions, traditional CME, educational materials, audit and feedback, patient mediated interventions, reminders, opinion leaders, educational outreach. Across the bottom are the proportion of studies in which change in practice took place. As Denise has described, traditional educational interventions are not especially effective at promoting change. There appears to be a gradient that involves measurement (audit and feedback), patient focused interventions, communication (opinion leaders) and more tailored approaches (educational outreach).

25. Effectiveness of interventions in improving physician behavior or health outcomes Little effect Didactic lecture-based, mailed unsolicited materials Moderately effective Audit and feedback, especially if delivered by peers or opinion leaders Relatively strong Reminder systems, academic detailing, and multiple interventions

26. What increases effectiveness of CME Active (interactive) learning opportunities Longitudinal or sequenced learning Enabling methods to facilitate implementation in the practice setting (e.g., tools, strategies) (Davis, 1999, JAMA)

27. Most effective CME Assessment of learning needs Interaction among clinician-learners with opportunity to practice the skills learned Sequenced and multifaceted educational opportunities Mazmanian and Davis JAMA 2002: 288; 1057-1060

28. Paradigm shift! Continuing education Measurement

29. Partnership for Quality

30. Project Description Setting: 10 American Academy of Pediatrics state chapters Technical assistance and support at local level: Chapter leadership Monthly calls Templates Practices: 6-hour chapter workshop Monthly conference calls (clinical, measurement, and improvement) Listserv Use of on-line CE program measurement and improvement tool Use satisfies Part IV ABP maintenance of certification

32. “Asking busy clinicians to collect data is like adding a brick to a full backpack”

33. Performance measurement Doable Track changes in normal care setting Valuable Provide baseline for comparison and document improved outcomes Identify opportunities for improvement Identify processes on which to focus effort Practical Economic to obtain Can be imbedded in work Contributes to sustaining gains

34. Achieving quality through systems change

35. Deming’s System of Profound Knowledge

36. Deming’s system of profound* knowledge Appreciation of a system Understanding variation Human psychology of change Theory of knowledge

37. The relationship between improvement science (Deming) and HSR Improvement science Appreciation of a system Knowledge of variation Theory of knowledge Psychology of change HSR Conceptual model Epidemiology, biostatistics Scientific methods, testing changes Behavior change theory

38. An Example Today’s emphasis is using measurement for improvement. A key point is that the core methods represent the scientific method so I selected an example that connects the measurement in research and measurement for improvement. Here’s how a researcher would use these data. Here’s how we would use it as someone working on process improvement Key points Focus on ideas to action (Real time, Change the future for current patients) Match research question to question. How much evidence do I need to change practice?Today’s emphasis is using measurement for improvement. A key point is that the core methods represent the scientific method so I selected an example that connects the measurement in research and measurement for improvement. Here’s how a researcher would use these data. Here’s how we would use it as someone working on process improvement Key points Focus on ideas to action (Real time, Change the future for current patients) Match research question to question. How much evidence do I need to change practice?

39. CF physicians are justifiably proud

40. What is a System? “A network of interdependent components that work together to accomplish a shared aim.” (WE Demming) The aim of the CF system To assure the development of the means to control and cure cystic fibrosis (CF Foundation).

41. Deming’s System of Profound Knowledge

42. System of Care for Children with CF Cystic Fibrosis Foundation Sets research agenda (e.g., genomics) Collaboration with industry, NIH Provides significant portion of funding Assures availability of high quality care Accreditation of care centers (provider team, support services, laboratory procedures, diagnostic testing) Provides funding for these mandates CF system is moving at a speed much faster than other diseases. Many of the advances learned in CF are being applied to other conditions. Now we are seeing this in the area of QI>CF system is moving at a speed much faster than other diseases. Many of the advances learned in CF are being applied to other conditions. Now we are seeing this in the area of QI>

43. System of Care for Children with CF Disseminates new knowledge NA Cystic Fibrosis meeting (>3,000 people from all disciplines) Consensus statement and guideline development Tracks patient outcomes National data registry Advocacy CF system is moving at a speed much faster than other diseases. Many of the advances learned in CF are being applied to other conditions. Now we are seeing this in the area of QI>CF system is moving at a speed much faster than other diseases. Many of the advances learned in CF are being applied to other conditions. Now we are seeing this in the area of QI>

44. The CF Registry Contains demographic and clinical data on all patients attending accredited care centers Content and use have evolved over 20 years Initially, describe the CF population Hypothesis generation Epidemiologic risk factors Disease progression and pathogenesis Since 1998 Define variability in outcomes among centers Clinical information system to support quality improvement

45. Variation in outcomes among CF Care Centers

46. Survival of Children with Cystic FibrosisUniversity of Minnesota Data

47. Our Initial Responses as Clinicians It can’t be true because: We are world famous experts and/or researchers at cutting edge of CF care We work really hard and our intentions are pure It’s not true: The data are wrong The interpretation is wrong Our patients are sicker than theirs We are trained in research to be skeptics. As practitioners we all think we are doing a good job. You want to know what quality is. Just watch me.We are trained in research to be skeptics. As practitioners we all think we are doing a good job. You want to know what quality is. Just watch me.

48. It’s the system! So….what’s an improvement focus. Is it an outlier?So….what’s an improvement focus. Is it an outlier?

49. The CF System at the University of Minnesota Multi-disciplinary educational program for newly diagnosed patients and families Written guidelines and standards for outpatient care Detailed planning before clinic visits Pre-visit patient questionnaires to identify needs Aggressive approach to nutrition Database of all patients linked to lab Regular review of overall population performance There are methods to assess whether MN is an outlier. There are methods to assess whether MN is an outlier.

50. Maybe we can use some Minnesota ideas?

51. Examples of QI Studies How effective are specific changes (e.g., reminder/recall) in care processes? How effective are QI methods in promoting changes in care delivery? Which process changes are appropriate in different clinical settings? (e.g., large vs. small practices)

52. Matching the Study Design to the Question A framework for QI studies/projects

53. Comparing Research and Improvement Studies

55. How to use the model Provides framework for conducting tests Goal: improve care How: Conduct tests to learn which changes to current system will result in improvement Initial focus on aim and measures

56. Building Belief in a Change

57. Cystic Fibrosis Aim Nutritional goal: reduce the proportion of children in nutritional failure by increasing use of nutritional status classification, self management goals, and appropriate medical interventions ETS exposure goal: eliminate ETS exposure by documenting parental smoking, prescribing a smoke free environment, and counseling parents on smoking cessation

58. How will we know if a change is an improvement? “All improvement requires change, but not all change is an improvement”

59. Principles of QI measurement Enable participants to observe gaps between desired and observed outcomes Facilitate priority setting Track performance over time Facilitate communication between teams working toward common goals

62. Building a Measurement System Attributes Small number (<8) of key measures that refer to the Collaborative goals Balance – together describe a great system of care Ideally – clearly defined for data collection and reporting (but not necessary)

63. CF Goals Improve nutritional status Reduce exposure to environmental tobacco smoke

64. CF Collaborative Measures

66. Building Belief in a Change

67. Evaluating Effectiveness Graphical Methods

68. Improvement in Cycle Time

71. Improvement in Cycle Time

72. Improvement in Cycle Time

73. Improvement in Cycle Time

77. Impact of improvement efforts on nutritional status at UNC

78. Comparing Research and Improvement Studies

79. Many studies of QI interventions do not show improvement…why?

81. Variability in changes and methods Changes Evidence-based Format Method Format Dose Frequency Combination Timing

82. Evaluation of QI Interventions Uncontrolled before/after* Time series (A baseline, B intervention) Controlled time series (AA, AB) Multiple time periods Cluster randomized controlled trial Factorial design

84. Many opportunities for learning and research

85. Next steps

86. If AHRQ offered a series of distance-learning activities on implementing and evaluating quality improvement strategies, would you be interested in participating?

87. What would be helpful? Conference calls Powerpoint Pre-work: readings Frequency Archive powerpoint/audiotape Series on specific topics Topics? QI strategies Design of QI interventions Evaluation