1 / 62

Process Measures

Process Measures. Session 3: Collaborative Learning Project. Today We Will…. Describe process measures relevant to healthy weight program (HWP) evaluations Identify valid, reliable, and generalizable tools and/or methods to collect process measures  

resendez
Download Presentation

Process Measures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Process Measures Session 3: Collaborative Learning Project

  2. Today We Will… • Describe process measuresrelevant to healthy weight program (HWP) evaluations • Identify valid, reliable, and generalizable tools and/or methods to collect process measures   • Prioritize process measures to identify core measures to recommend for inclusion across programs 

  3. Process Measures Documenting & Enhancing Engagement • Nancy Sherwood, PhD • University of Minnesota

  4. What Are Process Measures? • Process measures are measures that examine how your program activities are delivered. • Examples of process measures include: • Measures of participation • Readiness to change • Mastery of skills

  5. Core Process Measures • Program Enrollment • Are you engaging the target audience? • Are enrollment goals being met? • Program Attendance • How many sessions are people attending? • What percent of families who start the program, finish the program? • Program Activities: In-Session & Between Session • What percent of planned session activities are completed? • What percent of “homework” is completed (e.g., food/activity logs)?

  6. Core Process Measures • Program Enrollment • Are you engaging the target audience? YES • Are enrollment goals being met? YES • Program Attendance • How many sessions are people attending? 100% • What percent of families who start the program, finish the program? 100% • Program Activities: In-Session & Between Session • What percent of session activities are completed? 100% • What percent of “homework” is completed? 100%

  7. Using Process Data to Enhance Engagement: Example Source: French SA, Sherwood NE, Veblen-Mortenson S, et al. Multicomponent Obesity Prevention Intervention in Low-Income Preschoolers: Primary and Subgroup Analyses of the NET-Works Randomized Clinical Trial, 2012-2017. Am J Public Health. 2018 Dec;108(12):1695-1706.

  8. Pediatric Primary Care Annual Well Child Visit Pediatric Primary Care Annual Well Child Visit Child Physical Activity & Screen Time Home Environment & Parenting Practices Food, Physical Activity, Media, Parenting Parent/ Primary Caregiver Family Connector Home Visits 12 per year & Check-Ins 4 per year Child BMI Child Dietary Intake Parenting Classes 12 per year Neighborhood Physical Activity & Nutrition Resources

  9. 3 Year NET-Works Intervention • Goal: Keep parents actively engaged in all components of the intervention for a three year period. • Two key process measures documented in study database • Dose delivered, or amount and length of intervention sessions offered • Dose received, or participant attendance and completion of the offered sessions • Define “At Risk” families • 30 days, 45 days, 60 days, 90+ days without participating in an intervention activity • Implement re-engagement protocols

  10. Summary • Develop tracking systems and use process evaluation data to optimize engagement and participation • Flexibility and responsiveness to the unique needs of families can enhance engagement • Tracking systems & re-engagement strategies need to be tailored to specific contexts

  11. A Path to Sustainability Creating and Demonstrating Value • Chris Bolling, MD • American Academy of Pediatrics

  12. What Do Stakeholders Want?

  13. What Do Stakeholders Want? Before you figure what they want, there are some things to know: • Figure out who they are • Ask what they want • Know that there are different ways to ask • Be aware that the answers will surprise • You will need different messages for different stakeholders, BUT

  14. What Do Stakeholders Want? Be true to your mission keeping your big picture in sight

  15. What Do Stakeholders Want? • Return on Investment • Financial • Workforce productivity • Healthy community • Improved quality of life • Something you didn’t even think of!

  16. What Do Stakeholders Want? And you thought BMI was hard to change

  17. What Do Stakeholders Want? Outcomes can look very different to them • Do not underestimate the power of the subjective, especially early in the process • Reasonable deliverables • Patient satisfaction • A timeline for when things will happen • The overarching goal clearly stated • Ongoing follow-up with stakeholders

  18. What Does YOUR Community Want?

  19. What Does YOUR Community Want? As with stakeholders, ask first • What they tell you, may surprise as well • Who are the key players? • Who are the ones who really “get it” and are prepared for the long haul?

  20. What Does YOUR Community Want? • Time expectations • Again, keeping the big picture in view • Being clear about roles • Keeping commitments • Being prepared to be the driver Things to consider with your community

  21. Making the Cost-Benefit Case • Difficult to make on a purely monetary basis • Health care systems present special challenges • The depressing lesson of tertiary obesity care centers

  22. Making the Cost-Benefit Case • You have to think globally • The value added is system and community wide

  23. Making the Cost-Benefit Case • The goal may be “as revenue neutral as possible” • Think as infrastructure • Value to other entities • Speaks to community health where it is recognized as a public health crisis • Breakdown occurs at patient level • Weight bias at work (sorry to editorialize!)

  24. Making the Cost-Benefit Case • The importance of being a trusted expert • Picking your battles • Finding your “sugar daddy” • Using your voice • Prioritizing your efforts

  25. Process Measures • Measuring Provider Communication Style and Skill • Ken Resnicow, PhDUniversity of Michigan

  26. Motivational Interviewing Treatment Integrity Coding Manual 4.2.1Moyers TB, Manuel JK, & Ernst D. Motivational Interviewing Treatment Integrity Coding Manual 4.1. Unpublished manual, 2014.

  27. MI Specific Universal

  28. Manual for Motivational Interviewing Skill Code (MISC)Version 2.1Miller WR, Moyers TB, Ernst D, & Amrhein P. Manual for Motivational Interviewing Skill CodeVersion 2.1. University of New Mexico, 2008.

  29. MISC: Coding Client Speech

  30. One Pass • McMaster F & Resnicow K. Validation of the one pass measure for motivational interviewing competence. Patient Educ Couns. 2015 Apr;98(4):499-505.

  31. Behaviour Change Counseling Index (BECCI)Lane C, et al. Behaviour Change Counselling Index (BECCI). University of Wales College of Medicine, 2002.

  32. MI Coach Rating ScaleNaar, S. & Safren, S. (2017). Motivational Interviewing and Cognitive-Behavioral Interventions. Guilford Press, New York, NY.

  33. MI Coach Rating Scale

  34. MI Coach Rating Scale: Scoring Rubric

  35. The Roter Interaction Analysis System (RIAS): Universal

  36. RIAS: Universal • Forty exclusive behavior categories: • Socioemotional communication • Positive, negative, emotional, partnership building, and social exchanges • Task-focused communication • Asking questions, giving instruction and direction, and giving information • Grouped into four primary functions of the medical visit • Data gathering • Patient education and counseling • Responding to patient emotions (rapport) • Partnership building

  37. Source: https://doi.org/10.1371/journal.pone.0203970.t002

  38. ComOn CheckRadziej K, Loechner J, Engerer C, et al. 2017. How to assess communication skills? Development of the rating scale ComOn Check. Med Educ Online. 22:1.

  39. Cognitive Behavioral TherapyHepner KA, Howard S, Paddock SM, Hunter SB, Chan Osilla K, Watkins KE. A Fidelity Coding Guide for a Group Cognitive Behavioral Therapy for Depression. Santa Monica, CA: RAND Corporation, 2011. Sponsored by the National Institute on Alcohol Abuse and Alcoholism and the National Institute on Drug Abuse

  40. Measure of CBT Treatment Fidelity

  41. CBT Fidelity Checklist Source: Johnson C, L. Handen B, Butter E, et al. Development of a parent training program for children with pervasive developmental disorders. Vol 222007.

More Related