Data for Decision-Making Processes: Linking Data to Quality Improvement Initiatives

1. Data for Decision-Making Processes: Linking Data to Quality Improvement Initiatives John Dow, M.S.W. South Florida Behavioral Health Network Bisma Sayed, M.S.W. University of Miami Department of Sociology

2. Objectives • Understand the value of utilizing data for decision-making • Determine what should be measured and what data elements should be used • Data analysis and interpretation • Recognize limitations • Validate findings using other data sources

3. The recent recession coupled with health care reform has had cascading consequences on behavioral health care service delivery in Florida. • Current funding limitations and budget cuts have increased the urgency for cost-effective and efficient delivery of behavioral health care services.

4. Behavioral Health Care Service Delivery • How can behavioral health care organizations lower cost, raise quality, and still offer accessible services to increasing numbers of consumers? • Meet standards • Coordinate • Demonstrate outcomes • Eliminate duplication • Produce reportable, effective, and sustainable results

5. Quality Improvement • Quality Improvement vs. Quality Assessment

6. Behavioral Health Care: Quality Improvement Initiatives • Quality Improvement Processes allow organizations to analyze current practices, identify strengths and weaknesses, set goals, and monitor progress • Quality in the behavioral health care setting may be defined as the ‘extent to which a health care service or product produces a desired outcome’

7. Quality Improvement Initiatives • Quality of care measures • Effective • Appropriate • Safe • Efficient • Responsive • Accessible • Continuous • Capable • Sustainable

8. Quality Improvement Processes • Examine current organizational functioning • Identify target problems • Identify quality of care measures • Identify goals (short term or long term) • Measure baseline performance on quality measures

9. Quality Improvement Processes • Develop and conduct interventions designed to affect the targeted measures • Repeat measurement of performance based on quality indicator • Document and disseminate results.

10. Quality Assessment • “If you do not measure it (or cannot measure it), it didn’t happen.” • How can we measure it?

11. DATA • Data provides the foundation for quality improvement initiatives • Timely • Transparent • Presented with humility • Based on past lessons learned • Accountability • Presented with compassion and understanding

12. The shift to evidence based care coupled with increased technological and statistical advances have resulted in an explosion of data. . .

13. . . . However, this knowledge remains to be harnessed

14. Behavioral Health Care: Data and Quality Improvement Initiatives • The influx of data has led organizations to report data, rather than analyze data. • Data Reporting Data Analysis

15. Quality Improvement and Data: A continuous relationship Data Information Knowledge Decision Action

16. The link between QI and Data • Quality Improvement • What is happening? • What factors affect delivery • How can we influence them • Reactive and Proactive • We need data to guide this. • “Data helps to push improvement (by identifying problems) and pull improvement (by identifying opportunities)”

17. What is data? • Facts and statistics collected together for reference or analysis • Surveys • Literature Reviews • Key informants • Surveillance data • Focus Groups

18. Why use Data for Decision Making? • Develop overall goal for improvement • Identify objectives using quality of care measures • Identify target populations • Identify data to be collected

19. How to use Data for Decision Making? • Determine data sources and/or collection method. • Determine data storage, management, and analysis techniques. • Analyze and Interpret Data • Utilize data for decision-making

20. Data Collection and Management

21. Data Collection and Management • Plan • Consider scope and purpose • Target Audience • Learn ( Do not reinvent the wheel) • Literature Reviews • Other sources of data • Test • Pilot-test on a smaller scale to identify challenges • Team work • Involve and Integrate

22. Types of Data • Internal Data • External Data • Administrative or Clinical • Regardless of source of data or type of data, it must be reliable and valid • What is reliability and validity?

23. Data collection techniques and tools • Process mapping: (Who? How long? Steps? Costs?) • Brainstorm • Quantitative or Qualitative • Nominal • Ordinal • Interval • Ratio

24. Data Collection • Surveys and questionnaires • Ethical Standards • Confidentiality and Anonymity • Response Rates • Existing Surveys • Guidance • Pilot test

25. Data Collection • What is your target population? • Consumers? Their families? Providers? Community?

26. Data Collection: Survey Questions • Clear and Understandable • Specific • Not loaded or leading • No double barreled question • No jargon or acronyms • Allow choice of only one option • Provide reasonable ranges of variation in the response options

27. Data Collection: Survey Questions • Social Desirability Bias • Target towards population • Appropriate for age, culture and literacy • Include adequate demographic information

28. Sampling • Why do we sample? • Sampling must be representative of your population • Selection bias

29. Data entry, checking, and cleaning • Important step that can cause significant error if not done properly • Identify inconsistencies • For example, the mean age of adolescents sampled across the nation is 23.5. The range is 13-56. • Why do we have a 56 year old adolescent?

30. Storing and Managing Data • Spreadsheet programs • Reporting, not analysis • Database programs • Database changes – Store data with reports • Reporting, not analysis • Statistical Programs • Analysis

31. Data Analysis

32. Analyzing Quantitative Data • Understand the variables • Categorical and numerical variables • Frequency Distribution • Median and Percentile • Counts and Sums • Measures of central tendency • Measures of variability

33. Data Analysis for Numerical Variables • Measures of Central Tendency • Mean • Median • Mode

34. Measures of Variability • Range • Standard Deviation • What does this tell you about your population?

35. Statistical Analyses • What is the goal of data analysis in QI? • Descriptive Analyses and Measures of Variation are useful, but. . . • Inferential statistics can add to the power of your conclusions. • Examine Relationship/Estimate size of difference • Confidence Intervals • Tests of statistical significance

36. Statistical Analyses • Correlation Analysis • Correlation Coefficient: Pearson Product Moment Correlation Coefficient (r) • Scatter plots • Linear Relationships • Non-Linear Relationships • Correlation does not equal causation

37. Statistical Analyses • Nominal Level Data: Non-Parametric Tests • Chi Square • Cramer’s V/ Contingency Coefficient/Others • Numerical Data: Parametric Tests • T-tests (independent or dependent) • ANOVA • Regressions • Confidence Intervals • What are they? • How can they be used? • Sample size matters

38. Statistical Analyses • When you combine your sample value with the margin of error, you obtain a confidence interval. • The confidence interval is the level of confidence that the sample value represents the true value as seen in the overall population.

39. For example, the waiting time for appointments for clients referred to your clinic might be expressed as a mean of 13.5 weeks with a 95% confidence interval of 11.6 to 15.3 weeks (95% CI 11.6-15.3). • This means that you expect your population on average would wait between 11.6 and 15.3 weeks for an appointment.

40. Data Analysis: Statistical Significance • The p value is the probability that the difference you have observed in your study samples could be due to chance. • Smaller p value = lowered probability that results are due to chance • Statistical Significance

41. Data Analysis: Statistical Significance • The size of the p value depends on the size of the sample, so be aware of possible mistakes that can occur in interpreting these values. • Statistical significance does not mean clinical significance.

42. Presenting Data: Tabulating Data • Keep it simple • Consistent units • Decimal Points • Include raw numbers and percentages • Always include n • Identify missing data • Group data appropriately

43. Presenting Data: Graphs • Keep it simple • Avoid complexity • Clear headings • Scale Carefully • Raw numbers and percentages • Always include n • Group data appropriately

44. Presenting Data: Graphs • Basic population characteristics: Pie chart; bar graph • Measures of magnitude including comparisons: Bar chart or box plot

45. Presenting Data: Graphs • Frequency: Pie chart; bar chart • Trends over time: Line graph • Distribution of Data: Histogram; Scatter plot • Relationship between two things: scatter diagram

46. Data Analysis and Interpretation

47. Data Analysis and Interpretation • Whether you are collecting your own data or relying on external sources, there is a difference between compiling/reporting data and analyzing data • Data : petabytes • Reports : terabytes • Excel : gigabytes • PowerPoint : megabytes • Insights : bytes • One business decision based on actual data: Priceless1

48. Data Analysis and Interpretation • What is the problem? • What can you improve? • How can you improve? • Have you achieved improvement? • Have we sustained improvement?

49. Data Analysis and Interpretation • State and national datasets provide important information about key health indicators and can serve as basis for comparison. • However, we must be careful in interpreting and analyzing this data. • Understand limitations • Understand how data is presented • Mean, Median, Mode • Raw sums or percentages

50. Data Analysis and Interpretation • Level of variables • Individual • Community • State • State level data can help guide decisions, but you must examine individual data in your community to determine if the problem exists at a local level.