Understanding Cost-Effectiveness in Program Evaluation

1. Chapter 10:Cost-Effectiveness and Cost Analysis Designs

2. Why ‘Cost’ is an evaluative criterion • Programs must balance effectiveness with affordability • Social services are normally government funded • Funding is not always guaranteed • Legislature may opt not to fund program • Eliminate funding due to political climate • Agency directors must decide which program provide the most affordable, favorable outcome • Inexpensive program may not be as effective as one slightly higher in cost • Expensive program may provide a favorable outcome but only slightly better than one a fraction of the cost • Managed care doctrine requires care givers consider cost and be keenly aware of the economic side of intervention Cost-Effectiveness/Cost-Benefit Analysis - Measuring efficiency of intervention

3. Example Cost Effectiveness Comparison of Two Job Training Programs • Background data on the Programs: • Both work programs are located in large metropolitan areas • Primary responsibility is to help adults who have never experienced full time employment • Clientele is composed of adults who are dropouts or individuals with criminal histories • Both program teach work preparedness skills and provide an internship • Work Now’s program is less intense than Job Prep's program and can be completed one month earlier

4. Cost Effectiveness Analysis • Job Now requires 20% more budget to operate • Work Now has significant higher graduation rate (24%) • Work now has higher employment statistics after one year than Work Now • Cost per full-time employee is $1000.00 less per graduate Which program is more cost effective? Data appears conflicting!!!!! • Evaluator must consider the purpose of the of the program in determining the cost effectiveness • Job Now is the most cost effective because the purpose of the program is to help hard-core unemployed to become employed • Must consider expenditures in relation to outcomes

5. Step-by-Step Guide Cost Effectiveness Study • Define the program and its Outcome Indicators • Must fully understand all components and features of a program • When comparing multiple programs, must use the same operational definitions for successful outcomes • Compute Costs • Personnel - Salaries and benefits of program employees including volunteer • Facilities – Rent for office space and building overhead (power, sewage, insurance, etc.) • Equipment – Office supplies, computers, etc. • Other Expenses – Travel, training, in-direct employee contribution (secretaries, director, janitor) • Collect Program Outcome Data • In relationship to operational definitions • Compute Program Outcomes • Documenting the programs successes or failures; May be computed using multiple outcome indicators • Compute the Cost-Effectiveness Ratio • Divide cost of program by outcome indicators to detrmine relationship between resources and outcome • Perform Sensitivity Analysis • Test recommendations that develop from analysis • Estimates of success may influence data and it is necessary to determine if outside influences could have lead the evaluator to an incorrect conclusion

6. Whose Point of View? • Worth of program are viewed differently by different groups or individuals • Society • Providers • Consumers • Bill Payers • Perspectives will clash • Cost associated with a needed intervention may be very important to one group but not very important to another • Insurance company (Bill Payer) may not wish to pay for expensive but successful treatment when lower cost alternative is available • Cost-Effectiveness Analysis may not be comparable across interventions • Recommendation – CEAs must be undertaken from a societal perspective to account for all costs and effects and ensure that the interests of one group is not considered more important than a different group

7. Cost-Benefit Analysis • Cost Benefits can be either direct or indirect • Benefits can be either monetary or non-monetary Non-monetary and intangible benefits are more difficult to quantify; therefore cost benefit analysis may not be the best evaluation approach

8. Defining Benefits • Many Benefits cannot be easily converted to cost • Benefit analysis may consider pre and post-standardized measurements • Satisfaction with life • Satisfaction with intervention • When Benefits are non-monetary, cost benefit analysis will be indistinguishable from cost-effectiveness studies When examining costs and program outcomes, call your analysis “cost-effectiveness “ if the outcome variables are not measured monetarily

9. Recap • Policy makers, agency directors and managers want efficient programs • Cost of programs must be evaluated relative to clear outcome measurements • Cost-effectiveness studies are important tools used by decision makers to allocate resources • “It is imperative that we develop the skills necessary to demonstrate the benefit and cost savings when one treatment is chosen over another” Cost-effectiveness studies establishes a foundation for an agency to adopt the most efficient programs

10. A Useful Resource! The January 2006 issue of the journal Research on Social Work Practice contains a number of articles describing cost-benefits studies in social work practice. See www.sagepub.comto examine this issue. You may be able to download these articles for free by accessing this journal through your university library!