Continuous Improvement

1. Continuous Improvement Collecting, Analyzing, and Sharing Data

2. At the end of this presentation, you will know: • All the wonderful ways that you can use data for important purposes • What “formative evaluation” is, and how it relates to continuous improvement • How to organize yourselves for formative evaluation • How to collect data and use it to fuel continuous improvement • Different approaches to analyzing, reporting on, reflecting on, and sharing your data • How to work with C-PAD to accomplish data-related tasks

3. The Joys of Data!

4. Data are not just for grant requirements anymore!

5. By using data for continuous improvement, you will increase: • Participation • Satisfaction • Tailoring • Effectiveness

6. Using Formative Evaluation for Continuous Improvement

7. What is formative evaluation? • Evaluation = using data for assessment and learning • Formative = for the purposes of creating, developing, and improving • Formative (program) evaluation = Using data to inform how you can continually pursue program excellence

8. Formative Evaluation as a Tool for Continuous Improvement • Bullet Points

9. We are busy delivering a program – how can we be evaluators too? • Forget: “evaluation” • Remember: “leveraging data for program excellence!”

10. Story from the Field:Leveraging data for program excellence • Julie Seeley, Spoon River College

11. The Formative Evaluation Toolkit: A Guided Tour

12. What you will find in the FET: • Many pages (but not to worry!) • Step-by-step guidance through the phases of evaluation • Tools and templates for conducting your evaluation • Templates for working meetings to accomplish evaluation tasks • Checklists at the end of each chapter to keep you on track • Blank pages for notes, questions, and insights • Support for: • Completing deliverables due to Champion Colleges • Fulfilling grantee reporting requirements

13. Step-by-step guidance for: • Data collection (chapters 3 & 4) • Organizing for evaluation (chapter 2) • Reporting on findings (chapter 5) • Reflecting on findings (chapter 5)

14. Data Collection (Chapters 3 and 4)

15. Data Types • Process data • Outcome data • Stakeholder feedback

16. Process Data(Chapter 3) • Counts of program participants • Students completing workforce program courses • Students participating in support services • Information about program components, describing: • Workforce programs and courses • Math, English, and computer courses

17. Outcome Data(Chapter 3) • Number of students who have: • Received credits for prior learning • Received a degree • Received a certificate • Received a non-credit certificate • Become employed

18. Important Considerations for Collecting Process & Outcome Data • “Extractable” v. “Real-Time” Data • Make a data collection plan • Identify the data you need to collect • Figure out how you’ll collect it • Work with an IR Partner • Plan early for real-time data collection • Stay organized with data storage

19. FET Tools for Process and Outcome Data • Guidance for a working meeting to develop process and outcome data collection plans • Tools: • Tool 2: Data Choices Table (p. 42) • Tool 3: Process Data Collection Plan Worksheet (p. 45) • Tool 4: Outcome Data Collection Plan Worksheet (p. 54) • Tool 5: Options for Collecting Employment Data (p. 57) • Data Storage Excel Workbook (URL on p. 17)

20. Stakeholder Feedback(Chapter 4) • Hearing from students, internal partners, and community partners • Looking for feedback that will help you to: • Attract more students to the program • Tailor services better • Raise satisfaction levels • Support students better for completion and employment

21. Important Considerations for Collecting Stakeholder Feedback • Decide what you want to learn • Make a data collection plan • Decide which stakeholders you want to hear from • Identify the data collection methods you want to use • Identify people responsible for collecting feedback • Make data collection instruments • Don’t collect too much data!

22. FET Tools for Stakeholder Feedback • Guidance for two working meetings to develop: • A stakeholder feedback data collection plan • Data collection instruments • Tools: • Tool 6: Stakeholder Feedback Evaluation Questions Worksheet (p. 61) • Tool 7: Stakeholder Feedback Data Collection Plan Worksheet (p. 63) • Tools 8 & 9: Survey, Interview, and Focus Group Protocol Templates (pp. 66, 82) • Tools 10-12: Guidance for Conducting Interviews and Focus Groups (pp. 88, 90, 91)

23. Organizing for Evaluation(Chapter 2)

24. Assembling an Evaluation Team • Evaluation Team Manager • IR Partner • Process and Outcome (P&O) Data Lead • Stakeholder Feedback Lead

25. Convene the Team for an Evaluation Launch Meeting • Review how the Plus 50 data tasks fit together • Customize your evaluation timeline

26. Reflection and Reporting(Chapter 5)

27. Analysis, reflection, and reporting as an iterative process • Preliminary reporting: pulling together data with initial analysis • Reflecting on findings • Writing up memos: • Findings/Learnings • Recommendations (based on learnings and reflection)

28. FET Tools for Reflecting on and Writing up Evaluation Findings • Guidance for working meetings to reflect: • With the Plus 50 Team on key findings from the process and outcome data • With the Plus 50 Team on key findings from the stakeholder feedback • With the Plus 50 Advisory Committee on all key evaluation findings • Tools: • Tool 13: Evaluation Questions Tool (p. 92) • Tool 14: Qualitative Data Analysis Template (p. 95) • Tool 15: Memo Write-up Tool (including an memo outline) (p. 99)

29. Working with C-PAD to Support Continuous Improvement

30. Phase 5 • Collect Process and Outcome Data • Gather Feedback from Program Participants and Partners • Share Results with Key Stakeholders • Plan for Program Improvement

31. Feeding two birds with one seed:Process and Outcome Data in C-PAD

32. Questions?