1 / 17

Cost Effective Evaluation Strategies

Cost Effective Evaluation Strategies. Robin Kipke Tobacco Control Evaluation Center rakipke@ucdavis.edu November 19, 2009. Topics We’ll Cover. Adapting existing instruments Saving time in data collection Tapping into volunteer labor Using TCEC resources. Caveat.

paiva
Download Presentation

Cost Effective Evaluation Strategies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cost Effective Evaluation Strategies Robin Kipke Tobacco Control Evaluation Center rakipke@ucdavis.edu November 19, 2009

  2. Topics We’ll Cover • Adapting existing instruments • Saving time in data collection • Tapping into volunteer labor • Using TCEC resources

  3. Caveat • TCEC is not advocating for cutting evaluation corners that result in a decline of rigor or quality of data! • Cost-cutting strategies should only be used as the context of other factors allow • Always think first about what data will be convincing to your target audience

  4. Adapt Existing Instruments • Not always necessary to reinvent the wheel • Adapt other data collection instruments for your project’s purposes • Start with high quality instrument • Add questions you need data on • Delete unnecessary questions • Make sure it will collect what you need

  5. Instrument Sources • TCEC website has model instruments on limited number of objectives • Contact TCEC to access repository of instruments created/used by other projects • Ask other projects working on similar objectives to share their instruments

  6. Statewide Sites Download instruments from statewide sites • CA Health Interview Survey – http://www.chis.ucla.edu/questionnaires.html • CA Healthy Kids Survey – http://www.wested.org/cs/chks/view/chks_s/17?x-layout=surveys • CDC – http://cdc.gov/tobacco/data_statistics/surveys/ • YTPS on STORE Website – http://www.tcsstore.org/stages/1_document/t_yps.pdf

  7. Saving Time on Data Collection • Ask to have your questions added to data collection activities being conducted by other departments • Pay attention to what’s going on within your county or agency • Find out when other projects/divisions will be collecting data • Learn which population groups will be targeted by data collection and see how that fits your needs

  8. Adding Your Questions • Obtain a copy of the instrument to see what questions are already being asked • Insert your most important questions • Add transitions as needed to preface your question topics • Examples: • County’s annual Healthy Kids survey • Maternal & Child Health survey (every 5 years) • Needs assessments (use of Prop. 63 funds, adult health survey)

  9. Some Limitations • Think about data sources – who will be reached/excluded? • Will not be a pre-/post measure of same people, so cannot measure intervention effect • Will be limited to space allowed • Access to analysis – may need raw data

  10. Saving Time on Data Collection • Distribute questionnaires/materials via high traffic locations • Back-to-school packets • Church service programs • Grocery store flyers • Classroom teachers • Entertainment tickets (e.g. movie theaters) • Organizational newsletters

  11. Other Distribution Avenues • Online surveys (Survey Monkey) • Invite participation via email, social networking media (Facebook, Twitter) • Your organization’s website (and those of coalition members, agencies that serve target population)

  12. Other Distribution – Limitations • With each distribution outlet, think about who will it reach/not reach? • How to address representativeness • Use screening questions to avoid duplication • Loss of control over administration • Potentially lower response rate

  13. Saving Time on Data Collection • When low population density makes collecting a sufficient POS sample difficult, a series of focus groups (with purposive samples to ensure representativeness) may suffice • Caveat: First find out if the data will be convincing enough for your purposes

  14. Tapping into Volunteer Labor • Develop relationships with local universities, high schools, programs or clubs • Recruit students looking for research experience to do data collection • Offer training, internships, course credit, other incentives

  15. Tapping into Volunteer Labor • Make use of community volunteers and coalition members • To field test instruments/materials for appropriateness to data sources • To translate instruments/materials into other languages • To collect data • To provide access to other distribution points

  16. Utilize TCEC Resources • Online coalition satisfaction survey administration and analysis service • Additional model data collection instruments on a variety of topics will be developed in coming months • Model instruments will be translated into other languages (Spanish, Vietnamese, Hmong and Russian) depending on need • We’re working with other statewide projects to develop tools that will help you use and share your data more effectively for your policy work

  17. Contact TCEC For individualized assistance with: • evaluation planning • development of data collection instruments • advice on analyzing & interpreting data • guidelines for reporting & sharing info

More Related