Program Evaluation More Than Monitoring Performance Data

1. Program Evaluation More Than Monitoring Performance Data Phyllis Dower, MPH Susan Yurasevecz, MS Connecticut WISEWOMAN Program Public Health Initiatives Branch Comprehensive Cancer Program Connecticut Department of Public Health

2. Objective: We will be presenting examples of two different evaluation initiatives that focus on other types of evaluations besides measuring the number or frequency of Minimum Data Elements (MDEs). These two evaluations include: Assessing program data that identify effective strategies for increasing inreach rates (Example 1) and Assessing outreach activities (Example 2) using data gathered from the Connecticut Quarterly Progress Report.

3. CDC WISEWOMAN Program Definition of Evaluation Systematic assessment that provides useful information to: document the value of the program, guide program implementation and management, demonstrate accountability of program resources, lead to greater learning opportunities, and provide a solid base for decision making that leads to a stronger and more effective program.

4. WISEWOMAN Program Guidance and Resource Document Programs MUST evaluate their program to assess progress toward meeting all stated work plan objectives. Programs should use the CDC WISEWOMAN Evaluation Overview guidance when developing their evaluation plan. Programs should look beyond the required minimum data elements and consider questions that might be asked to help identify which activities are working well and which ones need improvement.

5. So�Which questions might be asked to help identify those activities working well and those activities in need of improvement? Some suggestions: What are my providers doing? Are they all doing it the same way? Where is it working better than others? Why is it working better than others? SusanSusan

6. Sample Evaluation Questions Everybody seems to be doing XYZ�why are Provider B and Provider F exceeding their screening numbers? (effectiveness study) Provider D spends the most money but sees the fewest number of women. What are they doing differently and are the results any more effective for reducing risk? (cost effective study) Providers F and G are not reaching their goal to enroll eligible populations when everyone else is. What do they need to do differently to improve performance? (process and outcome evaluation)

7. There is a relationship between Minimum Data ElementsCore Performance IndicatorsPerformance Measures and Evaluation Initiatives. Any of these data types can drive an evaluation initiative, but the initiative should go beyond the data itself to ultimately serve as a catalyst for driving programmatic change.

8. CDC�s Evaluation FrameworkDeveloping an Evaluation Plan The Process 1) What is the activity you plan to evaluate? 2) What are the program components to be evaluated? 3) What is the evaluation question to be asked? 4) What is the focus of the evaluation design? 5) What is your communication/dissemination plan? 6) What changes are anticipated as a result of the evaluation? CDC provides us with this evaluation framework that consists of the wee 6 component question.CDC provides us with this evaluation framework that consists of the wee 6 component question.

9. CDC Evaluation Framework Example 1 -Assessing Provider Inreach- To establish inreach rates Connecticut uses the MDE data file to determine: Of the population that came in for a breast or cervical cancer screening service, how many were eligible for a WISEWOMAN screening service? Of that eligible population, how many were actually given a WISEWOMAN screening service?