Improving Prevention Effectiveness
Download
1 / 49

William B. Hansen, Ph.D. Tanglewood Research, Inc. Greensboro, NC - PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on

Improving Prevention Effectiveness The Maryland Alcohol and Drug Abuse Administration Annual Management Conference October 5, 2006. William B. Hansen, Ph.D. Tanglewood Research, Inc. Greensboro, NC. Profile population needs, resources, and readiness to address needs and gaps.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' William B. Hansen, Ph.D. Tanglewood Research, Inc. Greensboro, NC' - illiana-spencer


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Improving Prevention EffectivenessThe Maryland Alcohol and Drug Abuse Administration Annual Management ConferenceOctober 5, 2006

William B. Hansen, Ph.D.

Tanglewood Research, Inc.

Greensboro, NC


Samhsa s strategic prevention framework

Profile population needs, resources, and readiness to address needs and gaps

Monitor, evaluate, sustain, and improve or replace those that fail

Mobilize and/or build capacity to address needs

Implement evidence-based prevention programs and activities

Develop a Comprehensive Strategic Plan

SAMHSA’s Strategic Prevention Framework

Assessment

Evaluation

Capacity

Planning

Implementation


Samhsa s strategic prevention framework1
SAMHSA’s address needs and gapsStrategic Prevention Framework

Assess

Evaluate

Select a Strategy

Implement

Develop Capacity


What do programs want to do
What Do Programs Want To Do? address needs and gaps

Assess

Evaluate

Select a Strategy

Implement

Develop Capacity


What do programs want to do when they are required to evaluate
What Do Programs Want To Do address needs and gapsWhen They Are Required to Evaluate?

Assess

Evaluate

Select a Strategy

  • The quality of delivery.

2. The effects achieved.

Implement

Develop Capacity


What do we mean quality of delivery
What Do We Mean: Quality Of Delivery address needs and gaps?

  • Dosage

    • How much?

    • How often?

  • Adherence

    • Was the program delivered as intended?

    • Was new content added?

    • Was important content deleted or modified?

  • Relevance

    • Was the program engaging to participants?

    • Did the program meet participants’ needs?


Dosage 1
Dosage 1 address needs and gaps

  • Meta-analysis of 25 SAMHSA model programs.

  • Programs that were delivered more frequently, generally had larger effects.


Dosage 2
Dosage 2 address needs and gaps

  • Same 25 SAMHSA model programs

  • Programs that had more opportunities for contact were generally more effective.


All stars dosage tracking
All Stars Dosage Tracking address needs and gaps


All stars dosage tracking1
All Stars Dosage Tracking address needs and gaps


All stars dosage tracking2
All Stars Dosage Tracking address needs and gaps


Adherence
Adherence address needs and gaps

  • An evaluation of Life Skills Training

  • Observers rated the percent of objectives met and lesson points covered

  • High-fidelity classes (>60% adherence) did best.


Adherence1
Adherence address needs and gaps

  • An evaluation of Life Skills Training

  • Observers rated the percent of objectives met and lesson points covered

  • High-fidelity classes (>60% adherence) did best.


Adherence2
Adherence address needs and gaps

  • An evaluation of Life Skills Training

  • Observers rated the percent of objectives met and lesson points covered

  • High-fidelity classes (>60% adherence) did best.


Local adherence
Local Adherence address needs and gaps

  • Drug Strategies assessed adherence of Life Skills Training implemented in Baltimore.

  • Teachers re-taught lessons they had previously delivered.

  • Observers rated adherence.

  • Teachers implemented 65% of objectives (Range = 45-100%).

  • Teachers implemented 58% of main points (Range = 38-93%).


Local adaptation
Local Adaptation address needs and gaps

  • All teachers made adaptations

  • 3.5 definable adaptations, on average, per observed session (range 1 to 7)

  • Overall, 63% of adaptations were judged to be negative


Helpful adaptations
Helpful Adaptations address needs and gaps

  • The addition of reading material, videos, and testimonials

  • Changes in methods to make them more interactive

  • Inclusion of examples for cultural relevance or interest


Important correlates of adherence
Important Correlates of Adherence address needs and gaps

  • Teacher’s Understanding of LST

    (r=.784; p<.01)

  • Quality of Process

    (r=.663; p=.03)

  • Level of Experience

    (r=.756; p<.01)


All stars approach to assessing adherence
All Stars Approach to address needs and gapsAssessing Adherence


Relevance
Relevance address needs and gaps

  • As part of the SAMHSA program meta-analysis, we coded relevance.

  • Three aspects of relevance were significantly correlated with outcomes:


All stars approach to assessing student engagement
All Stars Approach to address needs and gapsAssessing Student Engagement


Quality of implementation summary
Quality of Implementation Summary address needs and gaps

Data are needed to assess:

  • Whether a sufficient dose has been delivered

  • How closely delivery adhered to design

  • The relevance of implementation for participants

    Gathering and reporting data will improve quality of implementation.


Outcome evaluation
Outcome Evaluation address needs and gaps

  • Everybody is afraid of outcome evaluation.

  • Why?

    • No one likes to fail.

    • It is perceived to be mysterious, complex, and expensive.

    • Outcomes are not controllable.

    • It shows you are normal.


What are prevention goals
What Are Prevention Goals? address needs and gaps

The goal of prevention is not behavior change but either:

  • Non-behavior maintenance

  • Delay in onset

  • Reduce the intensity of use


How do programs work
How Do Programs Work? address needs and gaps

  • All Programs are based on a logic model.

  • The program changes a mediator:

    • Characteristics of the participant (skill, motivation)

    • Characteristics of the social environment

    • Characteristics of the physical environment

  • Characteristics targeted for change affect behavior.


All stars logic model example
All Stars Logic Model Example address needs and gaps

All Stars Core targets:

  • Lifestyle incongruence (idealism)

  • Normative beliefs

  • Commitment

  • Bonding to school

  • Positive parental attentiveness


Mediators in prevention

Motivation address needs and gaps

Attitudes

Bonding

Beliefs consequences

Commitment

Normative beliefs

Lifestyle incongruence

Personal Competencies

Academic skills

Decision-making skills

Emotional self-regulation

Goal setting skills

Self-esteem

Social Competencies

Resistance skills

Media literacy

Communication skills

Social problem solving skills

Social skills

Environment

Availability, access, enforcement

Alternatives

Classroom management

Family management

Monitoring and supervision

Positive peer affiliations

Support and involvement

Mediators in Prevention


The easy part of outcome evaluation
The Easy Part of address needs and gapsOutcome Evaluation

  • Collecting survey data is easy.

  • After over 30 years development, there are:

    • Many measures for assessing alcohol, tobacco, drug use, and consequences of use.

    • Many measures for measuring mediators (risk and protective factors targeted for change).


Sample outcome results all stars in a community setting
Sample Outcome Results address needs and gapsAll Stars in a Community Setting


Sample outcome results all stars in a community setting1
Sample Outcome Results address needs and gapsAll Stars in a Community Setting


Sample outcome results all stars in a community setting2
Sample Outcome Results address needs and gapsAll Stars in a Community Setting


What do when programs succeed
What Do When Programs Succeed? address needs and gaps

Assess

Evaluate

Select a Strategy

Implement

Develop Capacity


Results looking at mediators
Results Looking at Mediators address needs and gaps


Results of the all stars community trial
Results of the address needs and gapsAll Stars Community Trial


Sample local evaluation results from a community program in mn
Sample Local Evaluation Results address needs and gapsfrom a Community Program in MN


What do you do when you have not yet succeeded
What Do You Do When You address needs and gapsHave Not Yet Succeeded?

Assess

Evaluate

Select a Strategy

Implement

Develop Capacity


Working backwards
Working Backwards address needs and gaps

Evaluate

Implement

Develop Capacity


How to develop capacity
How To Develop Capacity? address needs and gaps

  • Two predictors of quality implementation:

    • Experience

    • Training


Experience counts
Experience Counts! address needs and gaps

  • Teachers with more experience were:

    • Most adherent when the taught LST

      (r = .630)

    • More likely to meet objectives

      (r = .590)

    • More likely to cover major points

      (r = .756)

    • More likely to make positive adaptations

      (r = .577)


Developing skill
Developing Skill address needs and gaps


Training counts too
Training Counts, Too! address needs and gaps

  • Teachers’ understanding of LST was strongly correlated with adherence (r = .784).


Improving understanding
Improving Understanding address needs and gaps

  • Program-specific training

  • Coaching and feedback

  • Independent study


Program specific training
Program-Specific Training address needs and gaps

  • Introductory training

    • Basics of theory and methods

  • Technical assistance

    • Help with specific issues and adaptations

  • Booster training

    • When important questions are asked

  • Certification of Mastery

    • A process of demonstration and certification


Does training matter
Does Training Matter? address needs and gaps

  • Video Training Project

  • Two conditions

    • 3-hour course without video

    • 3-hour course with video

  • Topic: Norm Setting

  • Knowledge pretest-posttest survey

http://www.PreventionABCs.com


All stars certification of mastery
All Stars Certification of Mastery address needs and gaps

  • Basic All Stars facilitator training

  • Implement one cycle of All Stars

  • Videotape implementation with feedback about mechanisms of delivery

  • Videotape implementation with feedback about interactivity

  • Implement Strategies for Success

  • Implement Parent Intervention

  • Complete online course (Prevention ABCs)

  • Videotape to demonstrate expert delivery

  • Improved student outcomes


Just in time support
Just-In-Time Support address needs and gaps

  • New project

  • Life Skills Training

  • Emailed helpful hints just before you teach

  • Links to a streaming video demonstration

  • Recruiting test schools

    [email protected]

    1-888-692-8412


Conclusion
Conclusion address needs and gaps

  • I was asked to answer two questions:

    • How can prevention programs use data to improve program effectiveness?

    • How can programs become data-driven?


Improving effectiveness
Improving Effectiveness address needs and gaps

  • How can prevention programs use data to improve program effectiveness?

    • Quality of implementation data

    • Behavioral outcome data

    • Targeted mediating variable data

  • All improve the potential of a program to be implemented with greater rigor


Improving effectiveness1
Improving Effectiveness address needs and gaps

  • How can programs become data-driven?

    • Collect data

    • Look at the data you have collected

    • Start with modest expectations

    • Find meaning in the data

    • Find alternatives in the data


ad