Evaluation Planning with Tobacco Prevention & Education Networks: A Community-Based Participatory Approach

1. Evaluation Planning with Tobacco Prevention & Education Networks: A Community-Based Participatory Approach NWHF Community-Based Participatory Research Conference Portland, OR Friday, July 20, 2007 This project was funded by the Oregon DHS Public Health Division www.npcresearch.com

2. Panel Members • R.E. Szego, OR LGBTQ Coalition Against Tobacco • Jennifer Kue, Asian & Pacific Islander TPEP Coordinator • Luci Longoria, Disparities Liaison for the Oregon Tobacco Prevention & Education Program • Carrie Furrer & Scott Burrus, NPC Research NPC Research

3. NPC Project Team • Beth Green, Ph.D., Principal Investigator • Carrie Furrer, Ph.D., Project Director • Scott Burrus, Ph.D., Project Coordinator • Jelani Greenidge, BA Research Assistant NPC Research

4. Acknowledgements • TPEN Program Coordinators • Evaluation Teams at each of the 5 TPENs • Oregon DHS Public Health Division NPC Research

5. Overview of Oregon TPEP • Tobacco Prevention & Education Program • Comprehensive, statewide public health effort aimed at reducing the toll of tobacco-related death and disease • TPEP-funded programs work to: • Create smoke-free and tobacco-free environments through public policy • Counter pro-tobacco influences • Promote quitting NPC Research

6. What is a TPEN? • Tobacco Prevention & Education Network • Focus on groups facing tobacco disparities: • African American • Asian and Pacific Islander • Hispanic/Latino • Urban American Indian • Lesbian/Gay/Bisexual/Transgender/Questioning (LGBTQ) • Develop and implement culturally-reflective best practice strategies NPC Research

7. Purpose of Project • Assist TPENs in the development of community-driven evaluation plans and tools • Build capacity for evaluation planning and eventual evaluation activities NPC Research

8. What is CBPR? • Community-Based Participatory Research • Engages program stakeholders • Shared learning and power • Evaluators as “coaches” • Values community expertise, traditions, priorities NPC Research

9. CBPR Evaluation Planning • Community “owns” the evaluation process • Articulates priorities • Crafts questions • Involvement in tool development • Defines success • Culturally mindful planning process • Culturally appropriate products NPC Research

10. CBPR Framework • Step 1: Information Gathering • Step 2: Information Synthesis • Step 3: Information Sharing & Feedback • Step 4: Develop Goals • Step 5: Complete Evaluation Plan • Step 6: Review Progress NPC Research

11. Evaluation Planning Activities • Create community buy-in for evaluation • Identify community needs • Identify TPEN practices & whether they address community needs • Develop logic model for TPEN practices • Craft evaluation questions & data sources for each step of the logic model • Design tools to answer evaluation questions NPC Research

12. Steps 1 & 2: Information Gathering and Synthesis • Interview DHS key stakeholders • Interview each TPEN Program Coordinator • Review each TPEN’s work plan • Review CDC Best Practices & related literature • Review other state Tobacco Control Plans NPC Research

13. Steps 1 & 2: TPEN Program Coordinator Questions • What was your initial reaction to the idea of evaluation planning? • Did you have any concerns? • Was there anything about evaluation that you were particularly interested in? NPC Research

14. Steps 1 & 2: DHS Project Officer Questions • Why was CBPR important to this process? • What did you hope that TPEN Program Coordinators would get out of the evaluation planning process? NPC Research

15. Step 3: Information Sharing & Feedback • Pre-planning meeting • Statement of purpose • Logistics • Review preliminary logic model • Recruitment NPC Research

16. Step 3: Information Sharing & Feedback • Initial Evaluation Team Meeting • Form collaborative partnership • Introduce basic evaluation principles • Establish structure to guide collaboration • Identify tobacco-related community needs • Select core TPEN activity • Develop logic model for selected activity NPC Research

17. Step 3: TPEN Program Coordinator Questions • How prepared were you for what was covered in your initial Evaluation Team Meeting? • What was the most important thing that you learned from your initial Evaluation Team Meeting? • What do you wish you would have learned but did not? NPC Research

18. Step 4: Develop Goals • Initial & Second Evaluation Team Meetings • Thinking through short-term and long-term goals • Identifying what is important for the community to know about the work • Linking evaluation questions to logic model • Authenticating different ways of knowing NPC Research

19. Step 4: TPEN Program Coordinator Questions • What challenges did you face going into your second Evaluation Team Meeting? • What was the most important thing that you learned from your second Evaluation Team Meeting? • What do you wish you would have learned but did not? NPC Research

20. Step 5: Complete Evaluation Plan • Second & Third Evaluation Team Meetings • Identify key evaluation questions • Revise preliminary logic model • Identify data sources, measures and data collection methods • Revise current data collection plans (if appropriate) • Develop evaluation tools NPC Research

21. Step 5: TPEN Program Coordinator Questions • Do you believe that the tools developed (logic models, evaluation questions, tracking forms, assessments, etc.) reflect the needs, values, traditions, and priorities of your community? • Do you now have the capacity to: • revise your current evaluation plan • execute evaluation activities, and • develop an evaluation plan for other activities? NPC Research

22. Step 5: DHS Project Officer Questions • What was the most beneficial technical assistance that you provided during this process? • What other kinds of technical assistance would have been useful? • What were the top 2 accomplishments of the evaluation planning process? • What do you wish would have been accomplished but was not? NPC Research

23. Step 6: Review Progress • Iterative process • Opportunities for feedback • Review at each community meeting • One-on-one meetings with Program Coordinator • Readiness for change NPC Research

24. Step 6: TPEN Program Coordinator Questions • Do you think that you were provided enough opportunities to provide feedback? • What kind of technical assistance was most helpful to you? • What else in the way of technical assistance would you have liked to receive? • How responsive was the process to your/your community’s needs? NPC Research

25. Accomplishments • Program advocacy • Build evaluation capacity • Design culturally-appropriate tools • Program documentation • Define community-specific processes • Develop a community knowledge base NPC Research

26. Lessons Learned • TPEN readiness for evaluation planning • Community readiness for evaluation • Community stakeholder recruitment • Versatile evaluation planning team • Flexible evaluation planning process • Need for clear next steps NPC Research

27. More Information • Carrie Furrer, NPC Research furrer@npcresearch.com • Full report online at: www.npcresearch.com • Oregon DHS Public Health Division: www.oregon.gov/DHS/ph/tobacco/index.shtml • R.E. Szego, OR LGBTQ Coalition Against Tobacco reszego@cascadiabhc.org • Jennifer Kue, API TPEN Coordinator jenniferk@mail.irco.org NPC Research