refinement and validation of the ahrq patient safety indicators n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Refinement and Validation of the AHRQ Patient Safety Indicators PowerPoint Presentation
Download Presentation
Refinement and Validation of the AHRQ Patient Safety Indicators

Loading in 2 Seconds...

play fullscreen
1 / 32

Refinement and Validation of the AHRQ Patient Safety Indicators - PowerPoint PPT Presentation


  • 148 Views
  • Uploaded on

Refinement and Validation of the AHRQ Patient Safety Indicators. EPC Team (PSI Development) PI: Kathryn McDonald, M.M., Stanford Patrick Romano, M.D., M.P.H, UC Davis Jeffrey Geppert, J.D., Ed.M., Stanford Sheryl Davies, M.A., Stanford

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Refinement and Validation of the AHRQ Patient Safety Indicators' - regis


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
refinement and validation of the ahrq patient safety indicators

Refinement and Validation of the AHRQ Patient Safety Indicators

EPC Team (PSI Development)

PI: Kathryn McDonald, M.M., Stanford

Patrick Romano, M.D., M.P.H, UC Davis

Jeffrey Geppert, J.D., Ed.M., Stanford

Sheryl Davies, M.A., Stanford

Bradford Duncan, M.D., M.A., Stanford Kaveh G. Shojania, M.D., UCSF

Support of Quality Indicators

PI: Kathryn McDonald, M.M., Stanford

Patrick Romano, M.D., M.P.H, UC Davis

Jeffrey Geppert, J.D. Ed.M., Stanford

Sheryl Davies, M.A., Stanford

Mark Gritz, PhD, Battelle

Greg Hubert, Battelle

Denise Remus, Ph.D., RN, AHRQ

Developed by UC-Stanford Evidence Based Practice Center

Funded by the Agency for Healthcare Research and Quality

acknowledgments
Acknowledgments

Funded by AHRQ

Contract No. 290-97-0013

Support of Quality Indicators Contract No. 290-02-0007

Data used for analyses:

Nationwide Inpatient Sample (NIS), 1995-2000. Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality

State Inpatient Databases (SID), 1997 (19 states). Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality

For more information:

http://www.qualityindicators.ahrq.gov

acknowledgments1
Acknowledgments
  • We gratefully acknowledge the data organizations in participating states that contributed data to HCUP and that we used in this study: the Arizona Department of Health Services; California Office of Statewide Health and Development; Colorado Health and Hospital Association; CHIME, Inc. (Connecticut); Florida Agency for Health Care Administration; Georgia Hospital Association; Hawaii Health Information Corporation; Illinois Health Care Cost Containment Council; Iowa Hospital Association; Kansas Hospital Association; Maryland Health Services Cost Review Commission; Massachusetts Division of Health Care Finance and Policy; Missouri Hospital Industry Data Institute; New Jersey Department of Health and Senior Services; New York State Department of Health; Oregon Association of Hospitals and Health Systems; Pennsylvania Health Care Cost Containment Council; South Carolina State Budget and Control Board; Tennessee Hospital Association; Utah Department of Health; Washington State Department of Health; and Wisconsin Department of Health and Family Service.
rationale for the psis
Rationale for the PSIs

Background: Perceived need for an inexpensive patient safety surveillance system based on readily available data

UC-Stanford EPC charge: To review and improve the evidence base related to potential patient safety indicators (PSIs) that can be ascertained from data elements in a standardized, multi-state health data system, the Healthcare Cost and Utilization Project (HCUP).

literature review to find candidate indicators
Literature review to find candidate indicators
  • MEDLINE/EMBASE search guided by medical librarians at Stanford and NCPCRD (UK)
    • Few examples described in peer reviewed journals
  • Iezzoni et al.’s Complications Screening Program (CSP)
  • Miller et al.’s Patient Safety Indicators
  • Review of ICD-9-CM code book
  • Codes from above sources were grouped into clinically coherent indicators with appropriate denominators
structure of indicators
Structure of indicators
  • All definitions were created using ICD-9-CM diagnosis and procedure codes (along with DRG, MDC, sex, age and procedure dates)
  • Numerator of each indicator is the number of cases with the complication of interest (e.g., Postop DVT/PE)
  • Denominator of each indicator is the number of hospitalizations (or patients) considered to be at risk (e.g. elective surgical patients)
  • Exclusions were defined to restrict the denominator to patients for whom the complication was less likely to have been present at admission, and more likely to have been preventable
  • The indicator “rate” is the numerator/denominator
psi assessment methods
PSI assessment methods
  • Literature review to gather data on coding and construct validity
  • ICD-9-CM coding consultant review (face validity)
  • Clinical panel review (face validity)
  • Empirical analyses of nationwide rates, hospital variation, impact of risk adjustment, and relationships among indicators
clinical panel review
Clinical panel review
  • Intended to establish consensual validity
  • Modified RAND/UCLA Appropriateness Method
  • Physicians of various specialties/subspecialties, nurses, other specialized professionals (e.g., midwife, pharmacist)
  • Potential indicators were rated by 8 multispecialty panels; surgical indicators were also rated by 3 surgical panels
  • All panelists rated all assigned indicators (1-9) on:
    • Overall usefulness
    • Likelihood of identifying the occurrence of an adverse event or complication (i.e., not present at admission)
    • Likelihood of being preventable (i.e., not an expected result of underlying conditions)
    • Likelihood of being due to medical error or negligence (i.e., not just lack of ideal or perfect care)
    • Likelihood of being clearly charted in the medical record
    • Extent to which indicator is subject to bias due to case mix
evaluation framework
Evaluation framework

Medical error and complications continuum

Medical error

Nonpreventable

Complications

  • Pre-conference ratings and comments/suggestions
  • Individual ratings returned to panelists with distribution of ratings and other panelists’ comments/suggestions
  • Telephone conference call moderated by PI and attended by note-taker, focusing on high-variability items and panelists’ suggestions (90-120 mins)
  • Suggestions adopted only by consensus
  • Post-conference ratings and comments/ suggestions
final selection of indicators
Final selection of indicators
  • Retained indicators for which “overall usefulness” rating was “Acceptable” or “Acceptable-” :
    • Median score 7-9
    • Definite or indeterminate agreement
  • Excluded indicators rated “Unclear,” “Unclear-,” or “Unacceptable”:
    • Median score <7, OR
    • At least 2 panelists rated the indicator in each of the extreme 3-point ranges
psis reviewed
PSIs reviewed
  • 48 indicators reviewed in total
    • 37 reviewed by multispecialty panel
    • 15 of those reviewed by surgical panel
  • 20 “accepted” based on face validity
    • 2 dropped due to operational concerns
  • 17 “experimental” or promising indicators
  • 11 rejected
accepted psis
Selected postoperative complications

Postoperative thromboembolism

Postoperative respiratory failure

Postoperative sepsis

Postoperative physiologic and metabolic derangements

Postoperative abdominopelvic wound dehiscence

Postoperative hip fracture

Postoperative hemorrhage or hematoma

Selected technical adverse events

Decubitus ulcer

Selected infections due to medical care

Technical difficulty with procedures

Iatrogenic pneumothorax

Accidental puncture or laceration

Foreign body left in during procedure

Other

Complications of anesthesia

Death in low mortality DRGs

Failure to rescue

Transfusion reaction

Obstetric trauma and birth trauma

Birth trauma – injury to neonate

Obstetric trauma – vaginal delivery with instrument

Obstetric trauma – vaginal delivery without instrument

Obstetric trauma – cesarean section delivery

“Accepted” PSIs
national trends in psi rates
National trends in PSI rates
  • Nationwide Inpatient Sample (NIS), 1995-2000
  • 7.5 million discharges/1,000 hospitals/28 States
  • Approximates 20% sample of nonfederal acute care hospitals
  • Discharge level weights applied to generate national estimates for each year
  • Adjusted for age, gender, age-gender inter-actions, comorbidities, and DRG clusters
  • 1,121,000 potential safety-related events affecting 1,070,000 hospitalizations

Romano et al., Health Affairs 2003; 22(2):154-166

national trends 1995 2000
National trends 1995-2000

Romano, PS, Geppert, JJ, Davies, SM, Miller, M et al. A National Profile of Patient Safety in US Hospitals Based on Administrative Data, Health Affairs 2003;22(2):154-166.

national trends 1995 20001
National trends 1995-2000

Romano, PS, Geppert, JJ, Davies, SM, Miller, M et al. A National Profile of Patient Safety in US Hospitals Based on Administrative Data, Health Affairs 2003;22(2):154-166.

national trends 1995 20002
National trends 1995-2000

Romano, PS, Geppert, JJ, Davies, SM, Miller, M et al. A National Profile of Patient Safety in US Hospitals Based on Administrative Data, Health Affairs 2003;22(2):154-166.

national trends 1995 20003
National trends 1995-2000

Romano, PS, Geppert, JJ, Davies, SM, Miller, M et al. A National Profile of Patient Safety in US Hospitals Based on Administrative Data, Health Affairs 2003;22(2):154-166.

national trends 1995 20004
National trends 1995-2000

Romano, PS, Geppert, JJ, Davies, SM, Miller, M et al. A National Profile of Patient Safety in US Hospitals Based on Administrative Data, Health Affairs 2003;22(2):154-166.

research policy question
Research/Policy Question

Why are some PSIs increasing in incidence over time while others are decreasing?

  • Selective changes in coding practice
  • Changes in severity of illness or underlying risk of potential safety-related events
  • True changes in quality due to technical improvements in surgical or nursing technique, counterbalanced by inadequate staffing to prevent some complications
standard deviation of hospital effects 1997 sid
Standard deviation of hospital effects: 1997 SID

PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

ratio of hospital level signal to total hospital variation 1997 sid
Ratio of hospital-level signal to total hospital variation: 1997 SID

PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

year to year correlation of hospital effects 1996 97 florida sid
Year-to-year correlation of hospital effects: 1996-97 Florida SID

PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

risk adjustment methods
Risk adjustment methods
  • Must use only administrative data
  • APR-DRGs and other canned packages may adjust for complications
  • Final model
    • DRGs (complication DRGs aggregated)
    • Modified Comorbidity Index based on list developed by Elixhauser et al.
    • Age, Sex, Age-Sex interactions
psis loading on catheter related and technical complications factor 1
PSIs loading on “catheter-related and technical complications” (factor 1)

PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

psis loading on post intraoperative complications factor 2
PSIs loading on “post/intraoperative complications” (factor 2)

PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

psis loading on neither factor 1 variance explained
PSIs loading on neither factor (<1% variance explained)

PSI Technical Review at http://www.qualityindicators.ahrq.gov/data/hcup/psi.htm

conclusions
Conclusions
  • Administrative data are appealing, but the development of indicators is time-consuming
  • Variations across hospitals and over time merit further exploration
  • Potentially useful screening tool for providers, provider associations, and health data agencies to identify possible safety problems
  • Ongoing support and validation work expected to offer many more insights into opportunities and obstacles in using administrative data for patient safety surveillance