Part 07 l.jpg
Advertisement
This presentation is the property of its rightful owner.
1 / 38

PART 07 PowerPoint PPT Presentation

PART 07 Evaluating Hospital Performance PERFORMANCE MEASURES Patient outcomes Mortality, morbidity, satisfaction with care 30-day mortality among heart attack patients ( Normand et al JAMA 1996, JASA 1997) Process Medication & test administration, costs

Download Presentation

PART 07

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Part 07 l.jpg

PART 07

Evaluating Hospital Performance

BIO656--Multilevel Models


Performance measures l.jpg

PERFORMANCE MEASURES

Patient outcomes

  • Mortality, morbidity, satisfaction with care

    • 30-day mortality among heart attack patients (Normand et al JAMA 1996, JASA 1997)

      Process

  • Medication & test administration, costs

    • Laboratory costs for diabetic patients

    • Number of physician visits

      • Hofer et al JAMA, 1999

    • Palmer et al. (1996)

BIO656--Multilevel Models


Data structure l.jpg

DATA STRUCTURE

Multi-level

  • Patients nested in physicians, hospitals, HMOs, ...

  • Providers clustered by health care systems, market areas, geographic areas

  • Covariates at different levels of aggregation:

    • patient, physician, hospital, ...

      Variation in variability

  • Statistical stability varies over physicians, hospitals, ..

BIO656--Multilevel Models


Mlms are effective l.jpg

MLMs are Effective

Correlation at many levels

  • Hospital practices may induce a strong correlation among patient outcomes within hospitals even after accounting for patient characteristics

    Structuring estimation

  • Stabilizing noisy estimates

  • Balancing SEs

  • Estimating ranks and other non-standard summaries

BIO656--Multilevel Models


The cooperative cardiovascular project ccp l.jpg

The Cooperative Cardiovascular Project (CCP)

  • Abstracted medical records for patients discharged from hospitals located in Alabama, Connecticut, Iowa, and Wisconsin (June 1992May 1993)

  • 3,269 patients hospitalized in 122 hospitals in four US States for Acute Myocardial Infarction

BIO656--Multilevel Models


Goals l.jpg

GOALS

  • Identify “aberrant” hospitals with respect to several performance measures

  • Report the statistical uncertainty associated with ranking of the “worst hospitals”

  • Investigate if hospital characteristics explain variation in hospital-specific mortality rates

BIO656--Multilevel Models


Slide7 l.jpg

DATA

Outcome

  • Mortality within 30-days of hospital admission

    Patient characteristics

  • Admission severity index constructed on the basis of 34 patient attributes

    Hospital characteristics

  • Urban/Rural

  • (Non academic)/(versus academic)

  • Number of beds

BIO656--Multilevel Models


Why adjust for case mix patient characteristics l.jpg

Why adjust for case mix?(patient characteristics)

  • Irrespective of quality of care, older/sicker patients with multiple diseases have increased need of health care services and poorer health outcomes

  • Without adjustment, physicians/hospitals who treat relatively more of these patients will appear to provide more expensive and lower quality care than those who see relatively younger/healthier patients

  • If there is inadequate case mix adjustment,

    evaluations will be unfair

  • But, need to avoid over adjusting

BIO656--Multilevel Models


Case mix adjustment l.jpg

Case-mix Adjustment

Compute hospital-specific, expected mortality by:

  • estimating a patient-level mortality model using

    all hospitals

    2. averaging the model-produced probabilities for all patients within a hospital

  • Hospitals with “higher-than-expected” mortality rates can be flagged as institutions with potential quality problems, but need to account for uncertainty

  • Need to be careful, if also adjusting for hospital characteristics

    • May adjust away the important signal

  • BIO656--Multilevel Models


    Slide10 l.jpg

    • Wrong SEs

    • Test-based

    (as we know, very poor approach)

    BIO656--Multilevel Models


    Slide11 l.jpg

    Hospital Profiling of Mortality Rates Acute Myocardial Infarction Patients(Normand et al. JAMA 1996, JASA 1997)

    BIO656--Multilevel Models


    Hierarchical logistic regression l.jpg

    Hierarchical logistic regression

    I: Patient within-provider

    • Patient-level logistic regression model with random intercept & slope

      II: Between-provider

    • Hospital-specific random effects are regressed on hospital-specific characteristics

      • Explicit regression

    BIO656--Multilevel Models


    Admission severity index normand et al 1997 jasa l.jpg

    Admission severity index(Normand et al. 1997 JASA)

    BIO656--Multilevel Models


    Slide14 l.jpg

    sevbar

    0 + 1(sevij – sevbar)

    0

    1

    BIO656--Multilevel Models


    Slide15 l.jpg

    we use b0i + b1i(...)

    BIO656--Multilevel Models


    Slide16 l.jpg

    b0i = *00 + N(..), etc.

    Interpretation of parameters is different for the two levels

    BIO656--Multilevel Models


    Results l.jpg

    RESULTS

    • Estimates of regression coefficients under three models:

      • Random intercept only

      • Random intercept and random slope

      • Random intercept, random slope, and hospital covariates

    • Hospital performance measures

    BIO656--Multilevel Models


    Slide18 l.jpg

    Normand et al. JASA 1997

    BIO656--Multilevel Models


    30 day mortality 2 5 th and 97 5 th percentiles for a patient of average admission severity l.jpg

    30-DAY MORTALITY2.5th and 97.5th percentiles for a patient of average admission severity

    Exchangeable model

    • Random intercept and slope, no hospital covariates

      log(odds): (-1.87,-1.56)

      probability,scale: (0.13, 0.17)

      Covariate (non-exchangeable) model

    • Random intercept and slope, with hospital covariates

    • Patient treated in a large, urban academic hospital

      log(odds): (-2.15,-1.45)

      probability scale: (0.10,0.19)

    BIO656--Multilevel Models


    Effect of hospital characteristics on baseline log odds of 30 day mortality l.jpg

    Effect of hospital characteristics on baseline log-odds of 30-day mortality

    • For an average patient, rural hospitals have a higher odds ratio than urban hospitals

      • Indicates between-hospital differences in the baseline mortality rates

      • Case-mix adjustment may be able to remove some of this difference

    BIO656--Multilevel Models


    Estimates of stage ii regression coefficients intercepts l.jpg

    Estimates of Stage-II regression coefficientsIntercepts

    BIO656--Multilevel Models


    Effect of hospital characteristics on association between severity and mortality slopes l.jpg

    Effect of hospital characteristics on association between severity and mortality(slopes)

    • The association between severity and mortality is modified by hospital size

    • Medium-sized hospitals have smaller severity/mortality associations than large hospitals

      • Indicates that the effect of clinical burden (patient severity) on mortality differs across hospitals

    BIO656--Multilevel Models


    Estimates of stage ii regression coefficients slopes l.jpg

    Estimates of Stage IIregression coefficientsSlopes

    BIO656--Multilevel Models


    Homework is on front table l.jpg

    Homework is on front table

    BIO656--Multilevel Models


    Slide25 l.jpg

    Observed and risk-adjusted hospital mortality rates

    Urban Hospitals

    Histogram displays (observed – adjusted)

    BIO656--Multilevel Models


    Slide26 l.jpg

    Observed and risk-adjusted hospital mortality rates

    Rural Hospitals

    Histogram displays (observed – adjusted)

    Substantial adjustment for severity

    BIO656--Multilevel Models


    Findings l.jpg

    FINDINGS

    • There is substantial adjustment for admission severity

    • Generally, urban hospitals are adjusted less than rural

    • There is less variability in observed or adjusted estimated rates for urban hospitals than for rural hospitals

      Can you explain why?

    BIO656--Multilevel Models


    Slide28 l.jpg

    Normand et al. JASA 1997

    BIO656--Multilevel Models


    Slide29 l.jpg

    Average the probabilities

    Don’t average the covariates

    BIO656--Multilevel Models


    Slide30 l.jpg

    k denotes a draw from the posterior

    BIO656--Multilevel Models


    Plug in the average covariate l.jpg

    Plug in the average covariate

    Keep the hospital variation

    BIO656--Multilevel Models


    Slide32 l.jpg

    BIO656--Multilevel Models


    Comparing measures of hospital performance l.jpg

    Comparing measures of hospital performance

    Three measures of hospital performance

    • Probability of a large difference between adjusted and standardized mortality rates

    • Probability of excess mortality for the average patient

    • Z-score

    BIO656--Multilevel Models


    Slide34 l.jpg

    Hospital Rankings: Normand et al 1997 JASA

    BIO656--Multilevel Models


    Hospital ranks l.jpg

    Hospital Ranks

    • There was moderate disagreement among the criteria for classifying hospitals as “aberrant”

    • Nevertheless, hospital 1 is ranked worst

    • It is rural, medium sized non-academic with an observed mortality rate of 35%, and adjusted rate of 28%

    BIO656--Multilevel Models


    Adjusting for hospital level charateristics l.jpg

    Adjusting for hospital-level charateristics

    Changes the comparison group in “as compared to what?”

    • All hospitals (unadjusted at hospital level)

    • Hospitals of a similar size, urbanicity, ...

    • Percent of physicians who are board certified

    • Hospitals with a similar death rate 

      Variance reduction and goodness of fit

      should not be the primary considerations

    • “As compared to what?” must dominate

    BIO656--Multilevel Models


    Discussion l.jpg

    Discussion

    • Profiling medical providers is multi-faced and data intensive process with substantial implications for health care practice, management, and policy

    • Major issues include data quality and availability, choice of performance measures, formulation of statistical models (including adjustments), reporting results

    • The ranking approaches and summaries used by Normand and colleagues are very good, but some improvement is possible

    BIO656--Multilevel Models


    Multi level models address key technical conceptual profiling issues including l.jpg

    Multi-level models address key technical & conceptual profiling issues, including

    • Adjusting for patient severity

    • Accounting for within-provider correlations

    • Accounting for differential sample sizes at all levels

    • Stabilize estimates

    • Structure ranking and other, derived comparisons

    BIO656--Multilevel Models


  • Login