slide1
Download
Skip this Video
Download Presentation
Evidence-Informed Program Improvement: Using principles of effectiveness to enhance the quality and impact of youth and family programs

Loading in 2 Seconds...

play fullscreen
1 / 42

Evidence-Informed Program Improvement: Using principles of effectiveness to enhance the quality and impact of youth and - PowerPoint PPT Presentation


  • 267 Views
  • Uploaded on

Evidence-Informed Program Improvement: Using principles of effectiveness to enhance the quality and impact of youth and family programs. STEPHEN SMALL Professor of Human Development & Family Studies, University of Wisconsin-Madison and

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Evidence-Informed Program Improvement: Using principles of effectiveness to enhance the quality and impact of youth and' - adamdaniel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1
Evidence-Informed Program Improvement:Using principles of effectiveness to enhance the quality and impact of youth and family programs

STEPHEN SMALL

Professor of Human Development & Family Studies,

University of Wisconsin-Madison

and

Extension Human Development & Family Relations Specialist

University of Wisconsin-Extension

Iowa State Webinar

November 17, 2009

agenda
Agenda
  • What are evidence-based programs, practices, and organizations?
  • What are some common principles of effective evidence-based youth and family programs?
  • How can Evidence-Based Program Improvement (EIPI) be used to improve existing programs?
evidence based programs and practices
Evidence-based Programs and Practices

Years of research have demonstrated that specific approaches, practices and strategies can reduce problem behaviors and enhance positive developmental outcomes

The most effective programs and practices are termed “evidence-based” – these are the “gold standard”

why the growing interest in evidence based programs and practices
Why the growing interest in evidence-based programs and practices?
  • Accountability: Increasingly required by funders within political climate
  • Efficiency: Reduces development costs associated with creating a new program
  • Availability of good science: Body of scientific evidence has reached a critical mass
  • Cost-effectiveness: Increased likelihood that limited resources will be used wisely
  • Marketability: Documented evidence can help sell and justify program
  • Ethics: Responsibility to use what is known to be effective
the evidence based programs and practices continuum
The Evidence-Based Programs and Practices Continuum

Evidence-based Programs

Evidence-based Kernels and core elements

Evidence-based Practices

Evidence-based Principles

Evidence-based Organizational

Systems

evidence based programs
Evidence-Based Programs

A new class of programs that:

  • Are based on a sound scientific evidence and theory
  • Have been carefully implemented, evaluated and replicated using rigorous scientific methods
  • Have evaluation findings that have been subjected to critical review
  • Often “endorsed” as evidence-based by a federal agencies or respected research organizations (e.g., Campbell Collaboration, SAMHSA, CDC)
terminology
Terminology

Evidence-based Research-based

Evidence-based = Research-based + Rigorously tested

examples of evidence based programs
Examples of Evidence-based Programs

Family/Parent Education Programs

  • Nurse Family Partnership: 15-year follow-up showed ↓arrests (54%) among adolescents; $3.59 return on the dollar
  • Strengthening Families Program for Parents and Youth 10-14:↓ aggression, alcohol, tobacco, marijuana, meth; $7.82 return
examples of evidence based programs1
Examples of Evidence-based Programs

School-based programs

    • Life Skills Training: 6-year follow-up showed ↓alcohol, tobacco, marijuana, narcotics, hallucinogens; $25.61 return on the dollar
  • Olweus Bullying Prevention Program:↓in bullying and victimization, vandalism, fighting, truancy; ↑ in school attachment
examples of evidence based programs2
Examples of Evidence-based Programs

Mentoring

Big Brothers Big Sisters:↓ in violence, alcohol and drug use; ↑ in school performance; $1.01 return

the effect of the strengthening families 10 14 program on teen aggressive and hostile behaviors
The effect of the Strengthening Families 10-14 Program on teen aggressive and hostile behaviors

Aggression and Hostility Index

SOURCE: Spoth, R., Redmond, C., & Shin, C. (2000) Reducing adolescents\' aggressive and hostile behaviors: Randomized trial effects of a brief family intervention four years past baseline. Archives of Pediatrics and Adolescent Medicine 154, 1248-1257

what are some limitations of evidence based programs
What are some limitations of evidence-based programs?
  • Not always easy to find
  • Can be costly to implement in both time & $
  • May not address particular issues or audiences
  • Tend to be problem focused
  • May downplay local knowledge and community ownership
  • Overlooks value of promising grassroots programs
  • Often difficult to transport to new settings
  • Other?
the evidence based programs and practices continuum1
The Evidence-Based Programs and Practices Continuum

Evidence-based Programs

Evidence-based Kernels

and core elements

Evidence-based Practices

Evidence-based Principles

Evidence-based Organizational

Systems

evidence based kernels
Evidence-Based Kernels
  • Kernels = units of behavioral influence and practice that have been shown through experimental evaluation to produce reliable effects on behavior
  • Kernels =Active ingredients of effective programs
the evidence based programs and practices continuum2
The Evidence-Based Programs and Practices Continuum

Evidence-based Programs

Evidence-based Kernels

Evidence-based Practices

Evidence-based Principles

Evidence-based Organizational

Systems

definitions of evidence based practices
Definitions of Evidence-based practices
  • “The integration of the best available research with clinical expertise in the context of patient characteristics, culture and preferences.” (American Psychological Association)
  • “The integration of best-researched evidence and clinical expertise with patient values.” (Institute of Medicine of the National Academies)
evidence based practice
Evidence-Based Practice

Practitioner

Expertise & Experience

Evidence-Based

Practices

Research Evidence

the evidence based programs and practices continuum3
The Evidence-Based Programs and Practices Continuum

Evidence-based Programs

Evidence-based Kernels

Evidence-based Practices

Evidence-based Principles

Evidence-based Organizational

Systems

slide20

EVIDENCE-INFORMED PROGRAM IMPROVEMENT:

Using principles of effectiveness to enhance the quality

and impact of family-based prevention programs

February 2009, Family Relations, 58, 1-13.

http://whatworks.uwex.edu

principles of evidence based youth and family programs
Principles of evidence-based youth and family programs
  • Design & Content
  • Relevance
  • Delivery/Implementation
  • Assessment & Quality Assurance
program design and content
Program design and content

Effective programs…

  • Target relevant assets and/or risk and protective factors
  • Are theory-driven
  • Are of sufficient dosage and intensity
  • Are comprehensive
  • Use active learning approaches & appropriate delivery formats
theory driven
Theory-driven
  • Based on strong, scientific theory
  • Target relevant risk, protective factors and assets to accomplish objectives
  • Logical, well-developed program theory

A well thought out recipeis essential

theory driven example
Theory driven - Example

Nurse-Family Partnership (early childhood/parent ed)- grounded in theories of human ecology (Bronfenbrenner, 1979, 1992), self-efficacy (Bandura, 1977, 1982), and human attachment (Bowlby, 1969)

theory driven1
Theory-driven
  • Based on strong, scientific theory
  • Target relevant risk, protective factors and assets to accomplish objectives
  • Logical, well-developed program theory

A well thought out recipeis essential

have sufficient dosage and intensity
Have sufficient dosage and intensity

Enduring change takes effort

thinking about sufficient dosage and intensity
Thinking about sufficient dosage and intensity…

For a mentoring program to be effective, the relationship on average has to exist for at least…

2 weeks

1 month

6 months

9 months

1 year

comprehensive
Comprehensive

Simple solutions rarely work

program relevance
Program relevance

Effective programs…

  • Are developmentally appropriate
  • Are socio-culturally relevant
  • Are appropriately timed
  • Are sensitive to local context
  • Are responsive to individuals

One size does not fit all

program delivery
Program delivery

Effective programs…

Foster good relationships

Are delivered by well-trained and committed staff

Address environmental barriers to program delivery and behavior change

-Provide a safe setting for the intervention

Are embedded in a supportive organization infrastructure

well trained and committed staff examples
Well trained and committed staff - Examples

Big Brothers, Big Sisters (mentoring program) – Orientation, volunteer screening, youth assessments, matches, supervision

Promoting Alternative Thinking Strategies (PATHS, classroom program)- Teachers receive training in a two- to three-day workshop and in bi-weekly meetings with the curriculum consultant.

program assessment and quality assurance
Program assessment and quality assurance

Effective programs…

  • Are well-documented
  • Focus on evaluation and refinement
  • Provide opportunities for ongoing feedback
make sure the program is well documented
Make sure the program is well-documented

It’s important to understand the program in order to implement it well

Maintaining program fidelity requires that people know what the program is about

Program replication and evaluation is difficult if the program isn’t well documented

focus on evaluation refinement
Focus onevaluation & refinement

See evaluation as your friend

Evaluation takes many forms

Evaluation isn’t always easy

Consider EIPI as a first step

evidence informed program improvement
Evidence Informed Program Improvement

http://whatworks.uwex.edu/attachment/whatworks_manual.pdf

summary principles of effective youth and family prevention programs
SUMMARYPrinciples of Effective Youth and Family Prevention Programs

Program design and content

Program relevance

Program delivery

Program assessment and quality assurance

the evidence based programs and practices continuum4
The Evidence-Based Programs and Practices Continuum

Evidence-based Programs

Evidence-based Kernels

Evidence-based Practices

Evidence-based Principles

Evidence-based Organizational

Systems

evidence based organization
Evidence-based organization
  • Promotes an organizational culture that values science and evidence-based approaches
  • Creates an infrastructure that supports effective practices and programs:
    • Uses existing evidence-based programs when available and appropriate
    • Employs a strong theoretical/science approach to new program design
    • Provides regular opportunities to reflect on programs and effective practices
    • Uses scientific tools to assess needs/assets, understand local context and adapt programs
    • Provides ongoing technical support for selecting, implementing and evaluating effective programs and practices
    • Builds organizational and community capacity to identify relevant goals and implement appropriate and effective approaches to address these goals
ad