620 likes | 624 Views
Process Measures. Session 3: Collaborative Learning Project. Today We Will…. Describe process measures relevant to healthy weight program (HWP) evaluations Identify valid, reliable, and generalizable tools and/or methods to collect process measures
E N D
Process Measures Session 3: Collaborative Learning Project
Today We Will… • Describe process measuresrelevant to healthy weight program (HWP) evaluations • Identify valid, reliable, and generalizable tools and/or methods to collect process measures • Prioritize process measures to identify core measures to recommend for inclusion across programs
Process Measures Documenting & Enhancing Engagement • Nancy Sherwood, PhD • University of Minnesota
What Are Process Measures? • Process measures are measures that examine how your program activities are delivered. • Examples of process measures include: • Measures of participation • Readiness to change • Mastery of skills
Core Process Measures • Program Enrollment • Are you engaging the target audience? • Are enrollment goals being met? • Program Attendance • How many sessions are people attending? • What percent of families who start the program, finish the program? • Program Activities: In-Session & Between Session • What percent of planned session activities are completed? • What percent of “homework” is completed (e.g., food/activity logs)?
Core Process Measures • Program Enrollment • Are you engaging the target audience? YES • Are enrollment goals being met? YES • Program Attendance • How many sessions are people attending? 100% • What percent of families who start the program, finish the program? 100% • Program Activities: In-Session & Between Session • What percent of session activities are completed? 100% • What percent of “homework” is completed? 100%
Using Process Data to Enhance Engagement: Example Source: French SA, Sherwood NE, Veblen-Mortenson S, et al. Multicomponent Obesity Prevention Intervention in Low-Income Preschoolers: Primary and Subgroup Analyses of the NET-Works Randomized Clinical Trial, 2012-2017. Am J Public Health. 2018 Dec;108(12):1695-1706.
Pediatric Primary Care Annual Well Child Visit Pediatric Primary Care Annual Well Child Visit Child Physical Activity & Screen Time Home Environment & Parenting Practices Food, Physical Activity, Media, Parenting Parent/ Primary Caregiver Family Connector Home Visits 12 per year & Check-Ins 4 per year Child BMI Child Dietary Intake Parenting Classes 12 per year Neighborhood Physical Activity & Nutrition Resources
3 Year NET-Works Intervention • Goal: Keep parents actively engaged in all components of the intervention for a three year period. • Two key process measures documented in study database • Dose delivered, or amount and length of intervention sessions offered • Dose received, or participant attendance and completion of the offered sessions • Define “At Risk” families • 30 days, 45 days, 60 days, 90+ days without participating in an intervention activity • Implement re-engagement protocols
Summary • Develop tracking systems and use process evaluation data to optimize engagement and participation • Flexibility and responsiveness to the unique needs of families can enhance engagement • Tracking systems & re-engagement strategies need to be tailored to specific contexts
A Path to Sustainability Creating and Demonstrating Value • Chris Bolling, MD • American Academy of Pediatrics
What Do Stakeholders Want? Before you figure what they want, there are some things to know: • Figure out who they are • Ask what they want • Know that there are different ways to ask • Be aware that the answers will surprise • You will need different messages for different stakeholders, BUT
What Do Stakeholders Want? Be true to your mission keeping your big picture in sight
What Do Stakeholders Want? • Return on Investment • Financial • Workforce productivity • Healthy community • Improved quality of life • Something you didn’t even think of!
What Do Stakeholders Want? And you thought BMI was hard to change
What Do Stakeholders Want? Outcomes can look very different to them • Do not underestimate the power of the subjective, especially early in the process • Reasonable deliverables • Patient satisfaction • A timeline for when things will happen • The overarching goal clearly stated • Ongoing follow-up with stakeholders
What Does YOUR Community Want? As with stakeholders, ask first • What they tell you, may surprise as well • Who are the key players? • Who are the ones who really “get it” and are prepared for the long haul?
What Does YOUR Community Want? • Time expectations • Again, keeping the big picture in view • Being clear about roles • Keeping commitments • Being prepared to be the driver Things to consider with your community
Making the Cost-Benefit Case • Difficult to make on a purely monetary basis • Health care systems present special challenges • The depressing lesson of tertiary obesity care centers
Making the Cost-Benefit Case • You have to think globally • The value added is system and community wide
Making the Cost-Benefit Case • The goal may be “as revenue neutral as possible” • Think as infrastructure • Value to other entities • Speaks to community health where it is recognized as a public health crisis • Breakdown occurs at patient level • Weight bias at work (sorry to editorialize!)
Making the Cost-Benefit Case • The importance of being a trusted expert • Picking your battles • Finding your “sugar daddy” • Using your voice • Prioritizing your efforts
Process Measures • Measuring Provider Communication Style and Skill • Ken Resnicow, PhDUniversity of Michigan
Motivational Interviewing Treatment Integrity Coding Manual 4.2.1Moyers TB, Manuel JK, & Ernst D. Motivational Interviewing Treatment Integrity Coding Manual 4.1. Unpublished manual, 2014.
MI Specific Universal
Manual for Motivational Interviewing Skill Code (MISC)Version 2.1Miller WR, Moyers TB, Ernst D, & Amrhein P. Manual for Motivational Interviewing Skill CodeVersion 2.1. University of New Mexico, 2008.
One Pass • McMaster F & Resnicow K. Validation of the one pass measure for motivational interviewing competence. Patient Educ Couns. 2015 Apr;98(4):499-505.
Behaviour Change Counseling Index (BECCI)Lane C, et al. Behaviour Change Counselling Index (BECCI). University of Wales College of Medicine, 2002.
MI Coach Rating ScaleNaar, S. & Safren, S. (2017). Motivational Interviewing and Cognitive-Behavioral Interventions. Guilford Press, New York, NY.
RIAS: Universal • Forty exclusive behavior categories: • Socioemotional communication • Positive, negative, emotional, partnership building, and social exchanges • Task-focused communication • Asking questions, giving instruction and direction, and giving information • Grouped into four primary functions of the medical visit • Data gathering • Patient education and counseling • Responding to patient emotions (rapport) • Partnership building
ComOn CheckRadziej K, Loechner J, Engerer C, et al. 2017. How to assess communication skills? Development of the rating scale ComOn Check. Med Educ Online. 22:1.
Cognitive Behavioral TherapyHepner KA, Howard S, Paddock SM, Hunter SB, Chan Osilla K, Watkins KE. A Fidelity Coding Guide for a Group Cognitive Behavioral Therapy for Depression. Santa Monica, CA: RAND Corporation, 2011. Sponsored by the National Institute on Alcohol Abuse and Alcoholism and the National Institute on Drug Abuse
CBT Fidelity Checklist Source: Johnson C, L. Handen B, Butter E, et al. Development of a parent training program for children with pervasive developmental disorders. Vol 222007.