change starts here n.
Skip this Video
Loading SlideShow in 5 Seconds..
Change Starts Here . PowerPoint Presentation
Download Presentation
Change Starts Here .

Loading in 2 Seconds...

play fullscreen
1 / 14

Change Starts Here . - PowerPoint PPT Presentation

  • Uploaded on

Change Starts Here . The One about Outcomes and Indicators ICPC National Coordinating Center.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Change Starts Here .

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
change starts here

Change Starts Here.

The One about Outcomes and Indicators

ICPC National Coordinating Center

This material was prepared by CFMC (PM-4010-080 CO 2011), the Medicare Quality Improvement Organization for Colorado under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy.

recap measurement for ic 4
Recap: measurement for IC-4
  • Time series outcomes
    • Effect on root cause/driver
    • Success of the intervention
      • Rates; scores; rating scales
      • Best-fit line or other signal indicating improvement
      • What to do about outcomes not well portrayed as time-series
  • Intervention implementation
    • Reach/dosage of an intervention
    • Who was affected?
      • Counts
      • Rates among eligible population (offered, refused, completed)
recap suggested approach
Recap: suggested approach
  • Map out a detailed, community-level logic model of the intervention strategy.
  • Select and operationalizeoutcomes and processes from the logic model.
  • Develop and enforce the system for tracking implementation and outcome.
  • Effectively report time series data.
who should participate in outcomes selection and measurement
Who should participate in outcomes selection and measurement?
  • Considerations
    • Who will implement the interventions?
    • Who will be collecting data?
    • Community stakeholders
      • To whom will the observed changes will be meaningful?
  • Resources brought by the QIO
    • Building capacity for outcome measurement
      • Logic models
      • Data collection
    • Technical assistance; analytic support
      • Data management and reporting
      • Detecting improvement (tests of trend, run/control charts)
recap logic model outcomes
Recap: logic model outcomes

Expected short-, medium-, and long-term changes and improvements

  • Short-term
    • Specific improvements in the targeted driver or root cause
  • Medium-term
    • Related outcomes along the causal path
  • Long-term
    • Improved care transitions
    • Avoided readmission
    • Improved health care utilization
  • Implications of potential negative changes or non-changes

Short-/medium-term outcomes are measured for IC-4.

recap outcomes selection
Recap: outcomes selection

SMART objectives guide outcome selection.


Concrete; represents what, or who, is expected to change


Can be seen, heard, counted, etc.


Is likely to be achieved


Generates meaningful, valued results


Has an acceptable target date


going further attainability moveabilty
Going further: attainability, moveabilty

Short-term outcomes are more likely to show movement…

…but consider downstream (medium-term) outcomes if short-term outcomes are not feasibly measured.

Outputs Short-term outcomes --------------- Long-term outcomes


What does the outcome look like? How would we know it improved?

  • Create operational definitions.
    • Specifics of how the outcome is measured
  • Make it a number. Quantitative data can be plotted over time or compared across groups.
    • Rate, percentage
      • Numerator and denominator
    • Score (continuous)
      • e.g., 0 to 100 points
    • Rating scale (ordinal)
      • e.g., [0 - ‘never’] -- [1 - ‘sometimes’] -- [2 - ‘always’]

Simple counts are not very informative on their own. Find a way to use them as the numerator of a rate or percentage.

timing and duration
Timing and duration

When will improvement be detected?

  • Considerations
    • How long should it take to observe an effect?
    • What should the effect look like?
      • Abrupt, sustained improvement
      • Rises and falls, with gradual trend towards improvement
  • IC-4: ≥4 quarters of data within 18 months of community engagement
    • Ensure that the measurement period includes pre-intervention baseline data.
  • Measure frequently
    • The more data points, the better. Monthly indicators lend themselves to run/control charts.
level of analysis
Level of analysis

Where will improvement be seen?

  • Provider-/initiative-level
    • A change occurs across several events, people, or organizations.
      • Process improvements (re: standardization; information transfer)
        • e.g., inpatient satisfaction with discharge information
      • Utilization outcomes
        • e.g, 7-day readmission rates; 30-day primary care follow-up rates
  • Patient-level
    • Something happens to an individual person related to his/her patient experience.
      • Patient activation (e,g., change in PAM score)
      • Utilization(e.g. , prevented readmission)
  • Patient-level data should be aggregated to provider-/initiative-level for reporting.
collecting data
Collecting data
  • Considerations
    • What method?
      • Sampling
      • Survey, case review, etc.
    • Who collects the data? Who analyzes it?
    • How will data quality be ensured?
  • Make it explicit.
    • Standardization, data specifications
    • Accountability
  • Don’t forget to track implementation (process) measures, as well.
    • Counts, rates among eligible population (offered, refused, completed)
more to come
More to come
  • Detecting and reporting improvement
  • Context and reasons for success/failure
  • Toolkit – measurement

  • Measuring Program Outcomes: A Practical Approach


  • ICPCA NCC contact: Tom Ventura



The ICPC National Coordinating Center –

Change Starts Here.