what do programs want
Download
Skip this Video
Download Presentation
What Do Programs Want?

Loading in 2 Seconds...

play fullscreen
1 / 23

What Do Programs Want? - PowerPoint PPT Presentation


  • 103 Views
  • Uploaded on

What Do Programs Want?. Jeanette Treiber, PhD Project Director Center for Program Evaluation and Research UC Davis School of Medicine. Objective. Determine local agencies’ evaluation capacity building needs. Background.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' What Do Programs Want?' - kent


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
what do programs want

What Do Programs Want?

Jeanette Treiber, PhD

Project Director

Center for

Program Evaluation and Research

UC Davis School of Medicine

objective
Objective

Determine local agencies’ evaluation capacity building needs

background
Background
  • California Department of Public Health (CDPH)’s Tobacco Control Program is carried out by
    • 61 Public Health Departments
    • Approximately 40 Competetive Grantees
more background
CDPH Strategy

Norm Change

Community base

Community Assessments

3yr work plans

Evaluation

More Background
evaluation components
Evaluation Components

Local Program Evaluation requirements

  • Comprehensive Evaluation Plan
  • Process and Outcome Evaluation
  • Internal or external evaluators
  • Progress Reports
  • Final Evaluation Reports
tobacco control evaluation center tcec
Tobacco Control Evaluation Center (TCEC)

TCEC provides Evaluation Capacity Building services to California TC programs through:

  • TA
  • Training
  • Tools
  • FER scoring

and feedback

rationale for tcec services
Rationale for TCEC Services

“…pressures have been accompanied by increasing awareness (and often frustration) on the part of funders as they confront the limited ability of the organizations they fund to meet the challenge of evaluating programs in ways that are likely to yield either genuine outcome-oriented accountability or useful information for formative improvement programs”Stevenson et.al. 2002

evaluation capacity building
Evaluation Capacity Building
  • Observed need and response in past decade
  • Various approaches to CB
  • Various definitions
  • Various approaches to

measuring effect of CB

  • Still little research
method
Method
  • A 21 item electronic survey (using the online service surveymonkey.com) to a list of 170 project directors and evaluators.
  • Analysis of access database for technical assistance from 2004 to 2009 (approx. 4.5 yrs)
results survey
Results (survey)

Need for training or materialsHigh orModerate

Need

Using culturally competent evaluation methods ……………….79.4 %

Writing evaluation plans ……………………………………….59.9 %

Adapting/developing data collection instruments ……………...57.1 %

Interpreting evaluation results…………………………………..54.1 %

As a reference during evaluation activities…………...................51.5 %

Writing evaluation reports……………………………………….50 %

Collecting evaluation data……………………………………….42.9 %

projected use of tcec survey
Projected Use of TCEC (survey)

Projected useResponse Percent

Finding, developing,

or adapting data collection

Instruments………………………………………..68.6

Writing an evaluation plan ……………………….57.1

As a reference during

evaluation activities ………………………………48.6

To build overall evaluation

capacity of the project …………………………….42.9

preferred format survey
Preferred Format (survey)

Webinar 68 %

“How-to” evaluation

guides on website 57.1%

Individual consultation

by phone or e-mail 45.7%

Website strategy

exchange 42.9%

requests for assistance ta log
Requests for Assistance (TA Log)

Total number of TA requests between 11/2/04 and 7/16/09

512

Average: 11 requests/month

Up to 27 requests/agency total

ta demand
TA Demand

(79%)

(71%)

fer score and ta requests
FER Score and TA requests

Final Evaluation report scores for Local Lead Agencies were correlated with frequency of TA request.

No statistically

significant correlation

was found.

summary conclusion
Summary/Conclusion
  • Majority of local health departments AND community organizations in CA TC require evaluation assistance
  • Greatest need is with DCIs, developing Eval plans, and help conducting evaluation with diverse populations.
  • Most DCI dev. need is with surveys
  • Webinars are the most desired mode of training delivery
  • Individual assistance requests vary greatly in frequency, scope, and content
limitations
Limitations
  • Response rate to survey was only 20%
  • TA assistance was not consistently entered into log in the same manner
  • Requests for TA are somewhat dependent on marketing of TA
  • TCEC team has changed personnel throughout the 4.5 years
what do programs want1
What do Programs Want?
  • Flexible,
  • Varied
  • Evaluation
  • Assistance;
  • Convenient
  • Delivery
references
References

California Department of Public Health Services, Tobacco Control Program. http://www.cdph.ca.gov/programs/tobacco/Pages/default.aspx

Innovation Network – Transforming Evaluation for Social Change. http://www.innonet.org/

Preskill, Hallie. (2005). Building Evaluation Capacity. 72 Activities for Teaching and Training. Thousand Oaks (Sage).

more resources
More Resources

Stevenson, John, Paul Florin, Dana Scott Mills, and Marco Andrade. (2002). Building Evaluation Capacity in Human Service Organizations: A Case Study. Evaluation and Program Planning 25, 233-243.

Tobacco Control Evaluation Center (TCEC). Center for Program Evaluation and Research, UC Davis. http://programeval.ucdavis.edu

University of Wisconsin Extension. Program Development and Evaluation. http://www.uwex.edu/ces/pdande/evaluation/index.html

contact
Contact

Jeanette Treiber, PhD

Project Director

Center for Program Evaluation and Research

Tobacco Control Evaluation Center

UC Davis School of Medicine

[email protected]

http://programeval.ucdavis.edu

1616 Da Vinci Court

Davis, CA 95618

ad