Using Data to Drive Daily Practice and Program Success

1. Using Data to Drive Daily Practice and Program Success Gabrielle Bargerstock, MPH Business Development Manager Nurse-Family Partnership National Service Office Florida Association of Infant Mental Health/MIECHV Conference Many Paths to Enhancing Parent Child Relationships April 25, 2014

2. Presentation Overview • Brief discussion of data as a concept • Data types and usage • Context: MIECHV Program & Benchmarks • Data Analysis Tools and Strategies • Activities • Resources

3. What is Data? • Facts or information used usually to calculate, analyze, or plan something • Factual information (as measurements or statistics) used as a basis for reasoning, discussion, or calculation • Information output …that includes both useful and irrelevant or redundant information and must be processed to be meaningful.

4. ‘Knowing where you are going is the first step in getting there.’

5. What is Data? According to Russell Ackoff: the content of the human mind can be classified into five categories: • Data: symbols • Information: data that are processed to be useful; provides answers to "who", "what", "where", and "when" questions • Knowledge: application of data and information; answers "how" questions • Understanding: appreciation of "why“ • Wisdom: evaluated understanding.

6. Challenges in Prevention The work requires: • Substantial knowledge and skill • Entails many steps (ex: assessing need, setting priorities, planning/delivering programs, monitoring, and evaluation.) • Implementation in varied settings and communities • Requiring tailoring • Adherence to model fidelity

7. Other Complicating Factors • Getting enough data vs. being a burden • Credible numbers vs. “we know things are better” • Determining the right focus at the right time • process vs. outcomes • quantitative vs. qualitative

8. Types of Data Process • Process evaluation looks at a program’s implementation and establishes whether quantifiable targets have been achieve and strategies implemented as planned. • Can be useful determining whether a program should be refined continued, expanded, or eliminated. • Process data includes information such as the total program participants, demographics, how many homevisits have been completed per client, etc. Outcome • Outcome evaluation measures the change that has occurred as a result of a program. • An outcome evaluation would tell you how many program participants demonstrated changed behaviors as a result of programmatic activities i.e. initiated breastfeeding, quit using drugs or alcohol, improved test scores, returned to school or work, etc.

9. Types of Data: VS Quantitative • Data that can be accurately represented numerically. • Data that can be measured. • Mathematic measurements, frequency of events, or categorical surveys (Yes/No, or rate a response on a 1-5 scale, etc). • Length, height, area, volume, weight, speed, time, temperature, humidity, sound levels, cost, members, ages, etc Qualitative: • Descriptions • Data can be observed but not measured. • Colors, textures, smells, tastes, appearance, beauty, etc. • Includes things like verbal interviews or surveys that ask questions that are open ended

10. Maternal, Infant, &Early Childhood Home Visiting (MIECHV) Program • Facilitates collaboration and partnership to improve health and development outcomes for at-risk children through evidence-based home visiting programs. • Program Purposes: • Strengthen and improve the programs and activities carried out under Title V of the Social Security Act; • Improve coordination of services for at-risk communities; and • Identify and provide comprehensive services to improve outcomes for families who reside in at-risk communities. • MIECHV includes grants to states and six jurisdictions; as well as grants to tribes and tribal organizations.

11. Maternal, Infant, & Early Childhood Home Visiting (MIECHV) Program • MIECHV is an evidence-based policy initiative. • Requires that the majority of grant funds (75% or more) to be spent on programs to implement evidence-based home visiting models.  • Up to 25 percent may be spent on promising approaches that must be rigorously evaluated • Currently, 14 home visiting models meet the HHS criteria for evidence-based home visiting. • Florida Association of Healthy Start Coalitions oversees in Florida. Ten sites funded.

12. Florida’s Benchmark Plan • Legislation authorizing MIECHV funding requires that all programs collect data to evaluate program performance and outcomes across six benchmark areas for all enrolled families: • Maternal and child health; • Childhood injuries and abuse and neglect; • School readiness; • Domestic violence; • Family economic self-sufficiency; and • Coordination of services. • Florida’s online data collection system – Florida Home Visiting Information System (FLOHVIS) • For more information contact Virginia Holland, Data Manager (vholland@fahsc.org).

13. Data In Daily Practice • Using data on a day to day basis doesn’t have to be complicated or difficult to understand • You do it already, many times a day • Goal: Move from informal usage of data, to conscious focus on analysis • In other words: Ongoing Quality Improvement or CQI

14. Types of Analyses • Root Cause Analysis (RCA) is the formal search for an individual or group of true causes of a problem and not just the symptoms. • more than just one root cause to a problem often exists and they can interact with each other. • RCA can capture both the big-picture perspective and the details. It can also be pointed at any problem, both simple and complex. • Simplest technique: Ask ‘Why’ five times…

15. Types of Analyses The Plan-Do-Study-Act (PDSA) cycle is a simple yet powerful tool for accelerating quality improvement. The PDSA cycle is shorthand for testing a change—by planning it, trying it, observing the results, and acting on what is learned. This is the scientific method, used for action-oriented learning. The steps in the PDSA cycle are: • Step 1: Plan--Plan the test or observation, including a data collection plan • Step 2: Do -- Try out the test on a small scale • Step 3: Study--Set aside time to analyze data and study the results • Step 4: Act--Refine the change, based on what was learned from the test

16. Exercise

17. Real Life Example: • Home visiting staff weren’t entering data into the program and state’s reporting systems in a timely fashion, resulting in negative actions by the state oversight agency and payment penalties. • Core Reason Why: • Staff didn’t adequately understand or have comfort with operating the computers and subsequently the database system. • Response: Training AND modification of hiring practices

18. Organizational Example - NFP Continuous Improvement Quality Framework

19. Quality Improvement Process Determine Interventions Problem Identification Implement Measure, Analyze & Evaluate

20. Activities to Consider • Have a two hour meeting to review current work (no more, no less) • Identify all key areas • Reflect on data, resources, and results • Chose 1 or 2 initiatives to stop and/or start • Set ‘by when’ • Identify three points in time where you can report out on your efforts • Ex: Team Meeting, Staff event, Board Presentation, Community Meetings

21. Activities to Consider • Make daily operations more efficient, easier, rewarding…. • Survey employees, clients, etc. for processes/activities that are duplicative, time-consuming, nonsensical. • Identify one activity (or more) you could eliminate immediately and then stop doing it. • For activities that are necessary are there areas for potential streamlining?

22. Activities to Consider • Elevator speeches and key statistics statements • If you don’t have them, create them as a team or ask individual team members to each develop one then discuss • Set a ‘What by when’ goal or create a 100 day challenge

23. Resources/Tools

34. Additional Questions?