Presented at AcademyHealth 2007 Annual Research Meeting Orlando, FL June 5, 2007

1. Did the Premier Demonstration Project Make a Difference?: Assessing the National Impact of P4P and Public Reporting Initiatives Presented at AcademyHealth 2007 Annual Research Meeting Orlando, FL June 5, 2007 Stephen R. Grossbart, Ph.D. Vice President, Quality Management Catholic Healthcare Partners srgrossbart@health-partners.org

2. Research Objectives • Identify the impact of the Center for Medicare and Medicaid Services’ (CMS) “Premier Hospital Quality Incentive Demonstration Project” (HQID) on quality improvement compared to other hospitals that participated in the Hospital Quality Alliance’s (HQA) public reporting initiative between 2004 and 2006.

3. Previous Research Richard A. Norling. How hospitals and health systems fit into the pay for performance puzzle. National Pay for Performance Summit. Beverly Hills, CA, Feb 15, 2007

4. Grossbart, 2006 Med Care Res Rev, Mar 2006: • Heart failure composite score improved 19% in acute care hospitals within a single multi-hospital system • Demonstration participants outperformed matched cohort in “non-starter set” HQA measures • “The alignment of financial incentives with performance has significantly affected the rate of quality improvement, resulting in an overall higher quality of care”

5. Latest findings on Premier P4P NEJM, Feb 1, 2007: • Participants improved 2.6% to 4.1% more than matched control group for 10 HQA “starter set” measures • “Financial incentives can modestly increase improvement in quality among hospitals already engaged in public reporting.” Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, Bratzler DW. Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007; 356:486-96

6. Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, Bratzler DW. Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007; 356:486-96

7. What are we looking for from P4P? Arnold Milstein, MD, MPH. You Want Me to Pay More? How Does P4P Fit With What Mainstream Purchasers Seek?. Beverly Hills, CA, National Pay for Performance Summit, February 8, 2006. Reproduced with permission of Arnold Milstein, MD (Mercer)

8. Study Design • The study compares the impact of the Premier HQID and the HQA’s public reporting initiative on quality performance between 2004 and 2006. • 232 hospital participants in the Premier HQID are compared to 3,044 hospitals not in the project that submitted quality data as part of the HQA’s reporting initiative. • Focus on four clinical areas that are partly or fully included in the HQID: acute myocardial infarction (AMI), heart failure, pneumonia, and surgical care improvement project (SCIP). • Reviewed both “starter set” and “non-starter set” measures submitted to HQA. Other studies have not focused on “non-starter set.”

9. Study Design • Based on publicly available data available on the CMS Hospital Compare web site. Data is reported quarterly on a rolling 12-month basis. The study identified from a database of 4,661 hospitals submitting data as part of the HQA a total of 3,276 hospitals that submitted data for all available quarters in the study (Quarter 1, 2004 – Quarter 1, 2006) and had complete data describing hospital characteristics. • Hospital characteristics were available from a variety of sources including MedPAR and American Hospital Association Guide. The study controls for variation in hospital characteristics by using multivariate regression to control for hospital size (measured as staffed beds), the existence of graduate medical education programs (Academic Medical Center, Council of Teaching Hospitals Members, Other GME, non-teaching), trauma centers, urban locations, and open heart programs to assess performance on overall quality using a composite opportunity quality score that was developed by Hospital Core Performance Measurement Project for the Rhode Island Public Reporting Program for Health Care Services and modified by Premier, Inc. for use in their demonstration project. (Premier, 2003)

10. Measuring Composite Quality Score (CQS) Measure of Improvement between time 1 (Q2 2004) and time 8 (Q1 2006) CQS Improvement(t1-t8) = a′ + b ′*P4PDEMO + c ′*SBEDS + d ′*GME + e ′*TRAUMA + f ′*URBAN + g ′*OHEART + u′ Measure of Change in CQS over Time for each Cohort (CMS/Premier participants vs. non-participants) CQS Score = a′ + b ′*TIME + c ′*SBEDS + d ′*GME + e ′*TRAUMA + f ′*URBAN + g ′*OHEART + u′

11. Population Studied • Hospitals submitting performance measure data as part of the HQA reporting initiative for up to twenty-one performance measures for AMI, heart failure, pneumonia, and SCIP. A total of 3,276 hospitals had sufficient data for analysis, including 232 of the 260 hospitals included in the Premier HQID. All data is available from public sources.

12. Acute Myocardial Infarction (Heart Attack) AMI-1 Aspirin at arrival* AMI-2 Aspirin at discharge* AMI-3 ACEI/ARB given for LVSD* AMI-4 Adult smoking cessation advice AMI-5 Beta blocker at discharge AMI-6 Beta blocker at arrival* AMI-7a Fibrinolysis therapy within 30 min of arrival AMI-8a Primary percutaneous coronary intervention within 90 minutes of arrival Heart Failure HF-1 Discharge instructions HF-2 LVF Assessment * HF-3 ACEI/ARB given for LVSD* HF-4 Adult smoking cessation advice Pneumonia PN-1 Oxygenation assessment* PN-2 Pneumococcal vaccination* PN-3b Initial blood culture in ER* PN-4 Adult smoking cessation advice PN-5b Antibiotic within 4 hours of arrival PN-6 Antibiotic selection PN-7 Influenza vaccination Surgical Care Improvement Project (colon surgery, hip and knee arthroplasty, abdominal and vaginal hysterectomy, cardiac surgery (including coronary artery bypass grafts (CABG)) and vascular surgery) SCIP-Inf-1a Antibiotic within 1 hr of incision-Overall SCIP-Inf-3a Antibiotic disc. within 24 hrs-Overall (48 hrs for CABG) Current measures collected and publicly reported (through Q3 2006) • *Hospital Quality Alliance "Starter Set Measures"

13. p-value < .001 p-value = .023 p-value < .001 p-value = .001 p-value = not calculated

14. p-value < .001 p-value = .015

18. National Impact of P4P • Based on publicly reported data available in Medicare’s Hospital Compare • Premier P4P participants had a more rapid increase in overall performance improvement in all major clinical areas than other hospitals submitting to Hospital Quality Alliance (HQA) (p-value < .001) • P4P had slight impact on performance for HQA publicly reported ten-measure “Starter Set” (p-value .015; Lindenauer, et. al., 2007) • Difference most pronounced with other HQA (non-starter set) measures (p-value < .001) • P4P participants had higher performance rates in early 2004 and increased the spread through 2005

19. Limitations • Limits on data quality and availability on CMS Hospital Compare • Accurate identification of hospital characteristics (e.g., size, services, infrastructure) • Lack of information on performance improvement infrastructure at hospitals • Potential serial correlation in time series analysis • Surgical data incomplete

20. Conclusions • When hospitals voluntarily agreed to submit data to CMS for the Hospital Quality Alliance’s reporting initiative, focus increased dramatically on 10 key “starter set” measures. The pressure of public reporting alone improved performance in hospitals across the nation. • There was little difference between hospitals in the pay-for-performance demonstration project and those only participating in the HQA reporting initiative. In contrast, in areas of clinical measurement not included in the public reporting initiative (non-starter set measures of AMI, pneumonia, heart failures, and SCIP), participants in the Premier HQID, dramatically outperformed non-participants.

21. Conclusions • The pay-for-performance demonstration clearly accelerated performance improvement among participants across a diverse set of measures, however, the influence of public reporting, approached that of the pay-for-performance demonstration on the more limited set of HQA measures that were part of the original public reporting initiative. Improvement in general was limited for most hospitals for only the 10 measure “starter set.” Other closely related measures in the areas of AMI, heart failure, and pneumonia did not experience the rate of improvement that participants in the Premier demonstration were able to post for the larger measure set that represented a comprehensive bundle of evidence-based care and included in the pay-for-performance project.

22. Implications for Policy, Delivery, or Practice • This analysis provides evidence that both pay-for-performance and public reporting initiatives accelerated performance improvement among the nation’s hospitals. The analysis also points out the need for broadly defined sets of measures that represent performance across the full continuum of care within each clinical condition. Understanding how providers will optimize performance is a key to designing future public reporting initiatives and pay-for-performance programs. This analysis demonstrates the need for broadly defined initiatives around clinical needs of patients to ensure that the full “bundle” of evidence-based aspects of care is delivered to target populations. • Raises concerns about “sub-optimization.” Performance should be assessed based on delivery of all appropriate care using “bundles” rather than using composites that may reward sub-optimal care. Bundles more likely to stimulate re-engineered of care delivery processes.

23. References Grossbart SR. What's the Return? Assessing the Effect of "Pay-for-Performance" Initiatives on the Quality of Care Delivery. Med Care Res Rev, Feb 2006; 63: 29S - 48S. Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, Bratzler DW. Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007; 356:486-96 Milstein A. You Want Me to Pay More? How Does P4P Fit With What Mainstream Purchasers Seek?. Beverly Hills, CA, National Pay for Performance Summit, Feb 8, 2006. Norling RA. How hospitals and health systems fit into the pay for performance puzzle. National Pay for Performance Summit. Beverly Hills, CA, Feb 15, 2007. Premier, Inc. CMS HQID project composite quality score methodology overview. Charlotte, NC, 2003.