1 / 17

Change Starts Here .

Change Starts Here. The One about Demonstrating Change ICPC National Coordinating Center.

reid
Download Presentation

Change Starts Here .

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Change Starts Here. The One about Demonstrating Change ICPC National Coordinating Center This material was prepared by CFMC (PM-4010-075 CO 2011), the Medicare Quality Improvement Organization for Colorado under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy.

  2. Recap: measurement for IC-4 • Time series outcomes • Effect on root cause/driver • Success of the intervention • Rates; scores; rating scales • Best-fit line or other signal indicating improvement • What to do about outcomes not well portrayed as time-series • Intervention implementation • Reach/dosage of an intervention • Who was affected? • Counts • Rates among eligible population (offered, refused, completed)

  3. Recap: suggested approach • Map out a detailed, community-level logic model of the intervention strategy. • Select and operationalizeoutcomes and processes from the logic model. • Develop and enforce the system for tracking implementation and outcome. • Effectively report time series data.

  4. Recap: timing and duration When will improvement be detected? • Considerations • How long should it take to observe an effect? • What should the effect look like? • IC-4: ≥4 quarters of data within 18 months of community engagement • Ensure that the measurement period includes pre-intervention baseline data. • Measure frequently • The more data points, the better. Monthly indicators lend themselves to run/control charts.

  5. What’s this all about? • Purpose • Confidently demonstrate that the interventions were effective. • Link interventions to observed changes in readmissions. • Validate or revise the logic model based on short-term outcomes.

  6. Be thoughtful and careful • Pitfalls • Waiting too long to begin the process • Checking progress infrequently • Going it alone • Advice • Rapid cycle improvement • Measure frequently, revise accordingly • Analytic support • NCC support

  7. Tracking outcomes • Time series • Run chart or control chart (≥12 data points) • Trend line (fewer data points) • Cross-sectional; cohort • Group comparison • Intervention vs. no intervention • Beneficiary-level • Difference between pre- and post-intervention

  8. Detecting improvement Does the intervention have an effect? • Run chart: special cause • Co-occurrence with intervention deployment • Best-fit trend line: statistical significance • Between-group or pre-/post-intervention differences achieved and sustained

  9. Time series: run charts • Run chart • Data points overlaid against the median • Simple patterns suggest ‘special cause’ • Runs(consecutive points above or below median) • Number of runs • Length of run • Consecutive data points continually increasing/decreasing • Very high or low point • Resources: • Perla, Provost & Murray (2011) • http://qualitysafety.bmj.com/content/20/1/46.full.pdf

  10. Run charts Perla, Provost & Murray (2011)

  11. Time series: best-fit line • Best-fit line among available data points • Cochran-Armitage test • Statistical significance: slope of the line is different from zero • Requires analytic support • Resources: • SAS documentation • SAS paper: Liu (2007) • www.lexjansen.com/pharmasug/2007/sp/sp05.pdf

  12. Cross sectional and cohort data Not well portrayed as time series, per se… • Group comparisons • Separate plot for each group on the same time-series graph • Pre- and post-intervention change • e.g., Patient Activation Measure • Consider ways to make it a group comparison • Pre-intervention vs. post-intervention scores • Sample of individuals who did not receive intervention • Requires analytic support

  13. What if we don’t see improvement? • Refer to the logic model • External factors • Challenged assumptions • What factors influenced the outcome? • What adjustments could have been made? • What other outcomes may have been measured?

  14. Document what was learned • Regardless of success or failure • Context • What made the intervention successful or unsuccessful in your setting/community?

  15. Additional resources • Toolkit – measurement http://www.cfmc.org/caretransitions/toolkit_measure.htm • Runcharts: Perla, Provost & Murray (2011) http://qualitysafety.bmj.com/content/20/1/46.full.pdf • ICPCA NCC contact: Tom Ventura tventura@coqio.sdps.org 303-784-5766

  16. Questions? CO-ICPCTechnical@coqio.sdps.org The ICPC National Coordinating Center – www.cfmc.org/caretransitions Change Starts Here.

More Related