Displaying Data in Meaningful Ways & Interpreting Quality Data

1. Displaying Data in Meaningful Ways & Interpreting Quality Data J.R. (Jefferson) Rowland, M.Sc. Epidemiologist, COTR Boston Regional Office – DQI Jefferson.Rowland@cms.hhs.gov Re-organizing Complex Information to Simplify and Accelerate a Common Understanding - or the ‘Main Point’ - of Months (or years!) of work in 5 seconds. CMS

2. Rules for using graphs, charts and tables 1) Anything worth doing is worth overdoing … 1a) Except graphs and tables … 1.a. i) Unless you’re the USRDS …(then see # 1)

3. In the last three years the Memphis vaccine rate has steadily increased from 55 to 70 percent while Detroit remained stuck at about 53%. Vaccine Program Results to Date

4. Objective: Share 3 Guiding Principles CMS Make it Simple – complexity is the enemy, yet the result of many ‘displays’ Make your Point – be the interpreter, don’t make your audience interpret Leave ‘em Wanting ‘Next’ Not ‘More’ – anticipating the next ‘report’ or, maybe, taking ‘ownership’ of that next data point

5. Exercise: Don’t tell me, show me. CMS 2 groups (split the room) No peeking (yes you…cheater) You are sitting in a planning meeting, eating a ‘slice’ with one hand, flipping pages with the other (yeah, we know, you’re a busy person) trying to ‘catch up’ and participate. Clinical areas for focus next year is ‘on the table’…

6. CMS

7. Chapter 4 – Section III. ADEQUACY OF HEMODIALYSIS Network Rates The Networks continue to work to improve URR rates for all Medicare hemodialysis patients as part of their core indicator improvement program. Each Network develops targets based on prior accomplishments and sets annual target rates for improvement. Local improvement efforts are quantified in Improvement Plans (IPs) and approved by their project officers. Networks continue to increase the rate of adequate dialysis (URR >= .65) and in the last two years improved ‘compliance’ rates (absolute improvement) by 3% nationally, a noticeable flattening of the slope from earlier years. A slow down in the rate of the ‘improvement rate’ is evidenced in the graph below. The actual rate (81%) of Network improvement in the URR >=.65 rate is lower than predicted by the model during the last few years of Network improvement work. Although the predicted national rate in 2002 (85%), using the logarithm model, indicates a slowing of the impact of Network efforts, individual Network results show variation in this overall trend. The Networks have provided various explanations for these results that include their patients are sicker, their nephrologists are overworked, the dialysis staff turnover is high, and patients won’t sit for the 4 hours required to reach adequate clearance levels. In the next round of Network contracts additional focus and considerations must be directed to towards innovative approaches to overcome barriers. CMS

8. National URRs >= .65 Data Source: National 5% CPM sample National sample trend line % CMS

10. Chapter 4 – Section III. ADEQUACY OF HEMODIALYSIS % of URRs >= .65 lower than expected as we approach 2002 Predictive model: 85% in 2002 Based on 5% annual CPM sample from all Networks 4% ‘off ‘ entering 2002 3% improvement in last 2 years – diminishing returns evident Actual results: 81% end of 2001 National sample trend line % A programmatic focus, and special consideration in planning and evaluation, for future adequacy work needs to be a priority. CMS

11. Questions: multiple choice with a ‘don’t know’ option Reminder: This isn’t the SAT exam – and yes, picking an answer other than ‘don’t know’ if you don’t know screws up this dumb game. Both groups raise hands at the same time … Foiled again…cheater

12. What Chapter was this… • Chapter 2 • Chapter 4 • Don’t know CMS

13. The national % of URRs >= 65% in the last couple of years was … • What was expected from the model • Higher than expected from the model • Lower than expected from the model • Don’t know

14. The predicted rate of the % of URRs >= .65% in 2002 was … • 85% • 83% • 81% • Don’t know

15. The difference entering 2002 between actual and predicted rates was… • 1% • 2% • 4% • Don’t know

16. Diminishing returns is evidenced by a ___ absolute improvement • in the last 2 years as opposed to rapid, large gains in prior 5 years… • 1% • 2% • 3% • Don’t know

17. A programmatic re-evaluation of the approach and evaluation of • future adequacy improvement work is suggested by this Chapter. • Strongly agree • Agree • Disagree • Strongly disagree • Not enough information

18. Were you reviewing a typical ‘display’ of URR adequacy rates… • Agree • Disagree • Neutral

19. Data • The selection and preparation of a graph affects the information, as perceived by the recipient of your graph (i.e., dialysis facilities, MRB, CMS, other stakeholders), and, ultimately, affects decisions. Analysis & Interpretation Information Encoding Flow diagram adapted from Cleveland & Fisher, The Quality Magazine, 1998, 7(4), 64-68 Picture of graph Interpretation … and then magic happens here! QI Decisions

20. FFBI communication and quality data coordination? CMS Industry LDOs (corporate) Patient/Professional-Advocates CKD Dialysis FFBI QIOs Nets Simple, clear, relevant, anticipated, motivating, meaningful, targeted, efficient. hospitals surgeons CKD-patients primary-care nephrologists facilities ESRD-patients SIMS CrownWeb? Claims Part A , B and D

21. AVF Placement Rate in Medicare Patients The ol’ “That’s not me!” argument…baseline accuracy”

22. AVF Placement Rate in Medicare Patients The ol’ “How can we help doc, we don’t care where you started, only that you change direction”. The starting address becomes Irrelevant and the direction, rate and distance become relevant.

23. Opportunity for Improvement ? (or… the curse of Powerpoint!) This is not a style issue, rather a statistical issue

24. (I’m in big trouble now, but this is my job here today ….)

25. The visual effect is the similar, and less likely to lead to Comments by your (picky) Project Officer ….

26. Comparisons to Averages May Not Motivate AVF Rates in Prevalent Patients: Facility, Network, National and CMS Goal Facility may conclude: We’re doing about as well as others

27. Can this Graph Aide In a Strategic Plan to Reach Target?

28. Address Attribution Through Interrupted Time Series

29. Central SW – High Yield Improvement Target! High AVF rates w/big or clustered bubbles on the left is the goal. Size of the bubble indicates relative size of the patient population affected. Data Source: CMS Vascular Access Database

30. The take away? You have license to be as creative and you want. There ARE a few rules and guiding principles, I just went over some. You get the jist. The FFBI crew wants to gather, explore, design, organize and perhaps share ‘effective’ display tools for the current and perhaps new audiences we seek. Test: in your mind, take two common ‘items’ that you have access to, and combine them into one that makes a completely different, yet clear, universally understood point that the separate items never could CMS

31. You are all Picasso !