Using What Works: Adapting Evidence-Based Programs to Fit Your Needs

1. Using What Works:Adapting Evidence-Based Programs to Fit Your Needs Introducing an Important New Tool Carol Ann M. Covington, MS, NCI’s Cancer Information Service June 15, 2009

2. Pre-Test and Introduction • Evidence Based Cancer Control is an emerging area – addressing program planning, development and evaluation • An evolving discipline with many perspectives and resources • “Designing for Dissemination”

3. Objectives • Describe evidence-based programs and the value of using them • List resources for finding evidence-based programs • Give an overview of Using What Works Modules

4. Question What do you think of when you hear the term “evidence-based”?

5. Answer An evidence-based program has been: • Implemented with a group • Evaluated • Shown to be effective and has published results • Also referred to as “research-tested programs”

6. What is Evidence-Based Cancer Control? “...the development, implementation, and evaluation of effective cancer education and screening programs through systematic uses of data and research information, and appropriate use of theory-based program planning models.” Adapted from Brownson et al., J Public Health Management Practice 1999,5:86-97

7. OBJECTIVE (fact) SUBJECTIVE (opinion) What Is Evidence? • Surveillance Data • Systematic Reviews of Multiple Intervention Studies • An Intervention Research Study • Program Evaluation • Word of Mouth • Personal Experience

8. Your Experience • What has your experience been with evidence-based programs? • Where have you heard of them before? • Have any of you used these programs in the past?

9. Why use Evidence-based Interventions? • Saves time • May save money • Proven effective in the study populations • Reduces the time it takes to research a community • Helps narrow the evaluation.

10. Why Use Evidence-based Interventions? • More Federal funders are requiring program planners to use evidence-based programs. • Some Federal funders consider evidence that is proven through research (explicit). • Some Federal funders consider evidence that is derived from experience or practice (tacit). • The best evidence may be a combination of research and practice.

11. More Advantages of Evidence-Based Programs • Adds value to a grant application • Increased certainty of success • Saves time and resources during planning and implementation

12. And Even More Advantages of Evidence-Based Programs • The organization is successful sooner • Creates a clear vision of how best to use resources • Presents new ideas for consideration that could work well

13. Barriers • Using evidence-based programs limits my creativity. • Evidence-based programs take too much time and/or money. • Evidence-based programs are too scientific. • I do not know what evidence-based programsare or where to find them.

14. Evidence-Based Program Barriers • My community is unique. An evidence-based program will not be as appropriate as if I developed the program myself.

15. Evidence-based Programs have been Assessed for Different Levels of Evidence • Level 1 has the strongest level of evidence • Level 5 has acceptable evidence • Level 1 may be more effective than Level 5, but level 5 may serve as a useful model

16. Levels of Evidence in Programs • Can choose any one of these levels to adapt • The choice is based on resources and time available • Remember: Using programs with less evidence (e.g., Level 4 or 5) requires more rigorous evaluation measures

17. http://www.thecommunityguide.org/

18. Two Primary Objectives for the Community Guide • As a standard reference for effectiveness information on population based interventions – • “what does the scientific evidence tells us what works?” • Supporting prevention research • “where there is insufficient evidence of effectiveness or when we want to develop localized version of generally effective interventions”

21. Finding an Evidence-Based Program Excerpts from Module 3

22. Where Can I Find Evidence-Based Programs? • Cancer Control PLANET—http://cancercontrolplanet.cancer.gov • Step 4: Research-tested Intervention Programs (RTIPs) • Program summaries include a list of journal articles about the program. For Level 1 Programs

23. http://cancercontrolplanet.cancer.gov

28. Other Sources for Evidence-Based Programs • Journals that report on evidence-based programs, such as Evidence-based Healthcare and Public Health http://www.harcourt-international.com/journals/ebhc/

29. Where Can I Find Evidence-Based Programs? • PubMed: from US National Library of Medicinehttp://pubmed.gov • Educational Resource Information Center (ERIC):“…sponsored by the Institute of Education Sciences (IES) of the US Dept of Education… premier database of journal and non-journal education literature.”www.eric.ed.gov Databases for peer-reviewed journals:

30. Where Can I Find Evidence-Based Programs? • PubMed Central: NIH “…free digital archive of biomedical and life sciences journal literature.”www.pubmedcentral.nih.gov *full-text articles. • EBSCO’s Electronic Journals Service: “…containing millions of articles from hundreds of different publishers, all at one web site.”http://ejournals.ebsco.com/home.aspNOTE: not a free service, but your university or library might be a subscriber.

31. Where Can I Find Evidence-Based Programs? • FirstSearch: “…a rich collection of reference databases.” Sponsored by the Online Computer LibraryCenter.http://www.oclc.org/firstsearch • Web of Science: “…information from approximately 8,700 of the most prestigious, high impact research journals in the world.” *Must access through institution. • http://portal.isiknowledge.com/portal.cgi?DestApp=WOS&Func=Frame Tutorial: http://scientific.thomson.com/tutorials/webofscience

32. Where Can I Find Evidence-Based Programs? • SAMHSA (Substance Abuse & Mental Health Services Administration) NREPP Program (National Registry of Evidence-based Programs and Practices).www.modelprograms.samhsa.gov • Promising Programs • Effective Programs • Model Programs • Note: NREPP programs are mostly focused on substance abuse

33. Where Can I Find Evidence-Based Programs? • The Cochrane Review (UK) http://www.cochrane.org/reviews/“The abstracts of the reviews are available free… Where available, plain language summaries are also provided…short synopses of the reviews' core findings, with a minimum of technical terms.” Other services available by subscription

34. Other Sources for Evidence-Based Programs Another site (not from Using What Works) Google Scholar http://scholar.google.com “…a simple way to broadly search for scholarly literature… peer-reviewed papers, theses, books, abstracts and articles, from academic publishers, professional societies, preprint repositories, universities and other scholarly organizations.”

35. Once you find evidence-based programs, what do you do with them? Become familiar with and understand the entire study: • objective, • methods, • results/conclusions, and • limitations.

36. Once you find evidence-based programs, what do you do with them? • Refer to your needs assessment and compare it to the program(s) you found: • Was the program conducted with people who had similar: • Socioeconomic status • Resources • Ethnicity • Traditions • Priorities • Community structure and values. • Age

37. Criteria for Selecting a Program Pay attention to the strategies the programs used to achieve their objectives. • Giving information • Enhancing skills • Improving services or support systems • Addressing barriers • Promoting access • policy changes. Consider your resources

38. Talking With the Principal Investigator Whenever you can, speak with the team that developed the program or product in which you are interested. They can share information about the program that may be helpful.

39. Needs Assessment: Getting To Know Your Audiences Better Module 2

40. Objectives • Know how to assess your audience—know who they are, what they value, and what factors influence their health. • Be able to find resources and methods for collecting data from first-hand (primary) and second-hand (secondary) sources. • Understand the value of using good data sources to drive your program. • Be able to find other groups and agencies that could assist—or that may compete—with your program efforts.

41. Making the Evidence-Based Program Fit Your Needs: Adaptation and YourProgram Summary Module 4

42. Objectives • Know how to use program planning and brainstorming activities to find out what needs to be changed in your evidence-based program. • Know what can be changed and what cannot be changed in your program. • Know ways of shaping an evidence-based program.

43. Print Materials and Culture In addition to reading level, you should ask yourself: • Is the language appropriate for the culture? • Are there different meanings for words? Could the words be misinterpreted? • Do the materials fit with my audience’s culture? If you answer these questions, it may help you find other needed text changes.

44. FoCaS Program Flyer

45. Team Up Greene County

46. Does It Work? Evaluating Your Program Module 5

47. Objectives • Know how to use different methods of evaluation. • Be able to match your evaluation methods with your program objectives. • Know where to apply methods for evaluation in the different stages of your program.

48. Using What Works:Adapting Evidence-Based Programs to Fit Your Needs To order: Call1-800-4-CANCER (1-800-422-6237) Order online: http://cancer.gov/publications Download: http://cancercontrol.cancer.gov/use_what_works/start.htm