slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Some Lessons Learned in Conducting Scientific Research on Behavior Disorders within Applied School Contexts PowerPoint Presentation
Download Presentation
Some Lessons Learned in Conducting Scientific Research on Behavior Disorders within Applied School Contexts

Loading in 2 Seconds...

play fullscreen
1 / 42

Some Lessons Learned in Conducting Scientific Research on Behavior Disorders within Applied School Contexts - PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on

Some Lessons Learned in Conducting Scientific Research on Behavior Disorders within Applied School Contexts. By Hill M. Walker Ph.D. Institute on Violence and Destructive Behavior College of Education, University of Oregon a nd T he School Behavior Disorders Group Oregon Research Institute

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Some Lessons Learned in Conducting Scientific Research on Behavior Disorders within Applied School Contexts' - kyne


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Some Lessons Learned in Conducting Scientific Research on Behavior Disorders within Applied School Contexts

By

Hill M. Walker Ph.D.

Institute on Violence and Destructive Behavior

College of Education, University of Oregon

and The School Behavior Disorders Group

Oregon Research Institute

Eugene, Oregon

The Fifth Annual Institute of Education Sciences (IES)

Research Conference

June, 2010 – National Harbor, MD

topics to cover
Topics to Cover
  • Pressures Driving Demands for Scientific Research Outcomes in School Settings
  • Single-Case vs. Randomized Controlled Trials as Vehicles for Conducting Scientific Research in Schools
  • Integrating Single Subject and RCT Designs in a 3-Stage Research and Development Process
  • Some Conditions Necessary for a Randomized Control Trial to be Successful in Schools Contexts
slide4
Factors Driving Interest in EBPs
  • National Legislation
    • No Child Left Behind
    • Individuals with Disabilities Education Act (2004 Reauthorization)
  • Court Mandates
  • Threats to School Security
factors continued
Factors (continued)
  • Public Demand for Return on Investments in Federal Research
  • Growing Interest in Prevention
    • Especially as an Alternative to Specialized Accommodation and Special Education
  • Development of Quality Standards by Professional Organizations
slide6
Development of innovations in better serving at-risk students has ramped up substantially in the past decade
  • 3-tiered public health prevention model applied to school contexts
  • Advent in use of response to intervention approaches for screening, identification and treatment
  • Strong interest by psychologists in conducting school-based research on conduct disorders
  • Priority of adapting promising programs for routine usage in school practices
where are schools re new evidence based practices
Where are Schools Re: New Evidence-Based Practices?
  • Until the past decade, schools have been slow to adopt proven or promising EBPs.

Impact

    • Vast majority of K-12 students have not accessed effective interventions.
    • Emerging pressures on educators to adopt and implement best and preferred practices that have a solid evidence base.
bridging the gap between research and practice
Bridging the Gap Between Research and Practice
  • Time lag between the availability of an EBP and its adoption and effective use on a broad scale within routine contexts can be vast.

-Cure for scurvy (Rogers, 1995).

-Time lag in mental health is estimated to be 20 years.

-Time lag is at least this long in K-12 education.

  • Typical barriers to adoption

-Cost

-Difficulty accessing

-Philosophical objections

-Resistance to change

-Innovation takes too much time and effort

changed landscape of prevention science and public demand for reliable evidence on what works
Changed Landscape of Prevention Science and Public Demand for Reliable Evidence on What Works
  • Increasing demand for large-scale prevention studies that demonstrate positive outcomes.
    • Typically, RCTs are the preferred method of demonstrating such effects.
    • Other types of methods and evidence are regarded as less rigorous.
  • Prevention approaches and intervention studies that meet this standard are highly regarded and lead to many positive benefits.
  • Blueprint Programs Example
continuing challenges for advocates of single subject vs randomized control research designs
Continuing Challenges for Advocates ofSingle Subject vs. Randomized Control Research Designs
  • Need to demonstrate positive impact on individuals while also addressing population-based questions relating to public safety, health, education, social policy and so forth.

Sidman (2006) notes that:

  • Influencing large groups is not where applied behavior analysis has made its greatest impact and progress.
  • Not all problems are subject to the same kinds of solutions and approaches.
  • Need to address the challenge of achieving widespread behavior change simultaneously.
slide11
Sidman calls for:
  • Alternative approaches that can accomplish broad, population-based changes and adoption of new innovations.
  • Examples are:
    • Epidemiology
    • Actuarial-based decision-making
    • Scaling up promising interventions to evaluate their efficacy and effectiveness
    • Large-scale studies of policy decisions
referencing outcomes in behavioral disorders to macro societal issues and problems
Referencing Outcomes in Behavioral Disorders to Macro-Societal Issues and Problems

(1) Can the BD field more directly target issues and problems of urgent societal concern?

  • School safety
  • Bullying, mean-spirited teasing, sexual harassment
  • Delinquency
  • Gang membership
  • School dropout
  • Youth violence
  • Drug abuse
referencing outcomes in behavioral disorders to macro societal issues and problems continued
Referencing Outcomes in Behavioral Disorders to Macro-Societal Issues and Problems (continued)

(2) Can the BD field implement outcome measures of these problems as part of documenting intervention effects?

  • Examples of Longitudinal Initiatives that have done so:
    • Perry Preschool Program
    • Hawkins & Catalano 12- to 20-year intervention follow-up study
    • Fast Track
    • Olds Nurse Home Visitation Model
randomized control trials and critical errors
Randomized Control Trials and Critical Errors
  • RCTs defined--Randomized Controlled Trials are studies that randomly assign individuals to an intervention group or to a control group, in order to measure effects of the intervention.  
  • RCTs are considered the surest way to avoid making serious errors on heath, education and policy issues that can affect the public health and safety.
    • Pre/post only and control-comparison studies often produce erroneous conclusions. 
    • Random assignment allows one to evaluate whether the intervention itself, or some other set of factors, causes the observed outcomes.
aldous huxley
Aldous Huxley
  • “The Single Greatest Tragedy of Science is the Cold-Blooded Slaying of a Beautiful Theory by an Ugly Fact.”
    • Randomized Control Trials are often the Cruel Means by Which this Tragedy Occurs.
seminal rct derived medical and educational findings
Seminal RCT-Derived Medical and Educational Findings

Medical:

  • vaccines for polio, measles, and hepatitis B
  • interventions for hypertension and high cholesterol
  • cancer (Hodgkins, leukemia, multiple myeloma)

Educational:

  • reduced class sizes in grades K-3
  • reading improvement and phonics
  • high quality early childhood development programs
use of rcts in educational contexts
Use of RCTs in Educational Contexts
  • Researchers have had considerable difficulty in selling RCTs to school administrators.
    • As a rule, educational decision-makers will insist on use of wait-list control group designs in which:

     (1) usual care controls receive the same intervention within one school year.

    (2) long-term followup assessments are thus not possible.

    • When parents and/or teachers discover they are in a control group condition, participant dropout is much more likely.
    • Teachers in control conditions are often motivated to enhance options for target participants to compensate for their not having access to an intervention perceived as beneficial.
school context
School Context
  • Factors that Influence Educator Adoption of New Practices:
    • Fits seamlessly into ongoing school routines
    • Consistent with school and educator values
    • Universal versus targeted interventions
    • Solves a high priority problem or issue
    • Time and effort costs are reasonable
    • Teacher perceives s/he has the skills and resources to apply practice effectively
single case vs randomized control trials in demonstrating efficacy effectiveness of interventions
Single Case Methodology

Analysis of level and slope changes between phases

Target single individuals

Continuous recording of measures

Randomized Control Trials

Comparison of group averages and variability

Random assignment to treatment & control groups

Pre, Post, Follow-up administration of measures

Single Case vs. Randomized Control Trials in Demonstrating Efficacy-Effectiveness of Interventions
slide21
Single Case Methodology

-Allows daily inspection of co-variation of Independent and Dependent Variables

-Visual inspection of intervention effects across phases

-Strong internal validity

Randomized Control Trials

-Allows subgroup analyses of variations in subject group performance

-Allows calculation of effect size estimates of magnitude of treatment effect

-Strong external validity

slide24

Example of the Sensitivity and Valueof Using a Single Subject Design inUnderstanding Teacher FS ImplementationFirst Step Fidelity Study(Rob Horner and Colleagues)

slide25
po

Teacher Fidelity of Implementation and Student Problem Behavior during First Step and First Step Plus Coach Feedback

Star shows coach present, feedback after observation

Percent of Intervals with Problem Behavior & Percent Fidelity

three stage research and development model
Three-Stage Research and Development Model
  • Research Settings
    • Experimental Classroom
    • Regular Classroom and Playground Settings
    • Field Test Settings
  • Target Behavior Disorders
    • Acting Out
    • Peer Aggression
    • Social Withdrawal
    • Academic Survival Skills
each completed behavior management package contains
Each Completed Behavior Management Package Contains:

(1) Program Materials

(2) Consultant Manual

(3) Teacher Manual and/or

(4) Playground Supervisor Manual

intervention delivery process
Intervention Delivery Process
  • Local Program Coordinator—Masters package content and procedures, recruits behavioral coaches to implement program, maintains liaison-coordinator role.

  • Behavioral Coach—Screens and recruits target participants, teachers and classrooms, sets up intervention program, transfers control to teacher or playground supervisor.

  • Classroom Teacher-Playground Supervisor—Assumes control of program’s operation after behavioral coach transfers it, works with coach to trouble shoot program.

  • Target Student—Participates in and cooperates with program during implementation.
  • Parents—provide support and home rewards throughout program’s operation.
recess
RECESS
  • Reprogramming Environmental Contingencies for Effective Social Skills: An Early Intervention for Aggressive Young Children
  • Age-Grade Range: K-3
  • Key Components:
    • Social Skills Training
    • Praise, Points and Feedback
    • Group Rewards at School – Individual Rewards at Home
    • Response Cost
    • Daily Debriefing and Coaching
  • Duration: 3 months
  • Target Setting: Playground
key lessons learned for conducting successful rcts in school settings
Key Lessons Learned for Conducting Successful RCTs in School Settings

For an RCT to be successful in today’s schools, we believe certain conditions should be in place as listed below:

(1) The host school district(s) should be at a point where it is receptive to the innovation being tested and is open to a collaborative partnership in a trial test of it.

(2) Key educational decision makers need to be on board in supporting the study and be able to clearly see its benefits.

key lessons learned for conducting successful rcts in school settings continued
Key Lessons Learned for Conducting Successful RCTs in School Settings (continued)

(3) There has to be a local champion or champions who are in a powerful position to support and advocate for the intervention.

(4) The host site and the investigative team need to be willing to negotiate the terms of the RCT so that its strength and integrity are not compromised.

(5) The intervention approach or strategy needs to be well manualized to facilitate training of key staff and to allow a high quality implementation.

key lessons learned for conducting successful rcts in school settings continued41
Key Lessons Learned for Conducting Successful RCTs in School Settings (continued)

(6) It is essential that staff from the host school districts be hired, whenever possible, in conducting the study so as to motivate district investment in and ownership of the study.

(7) It is of the utmost importance that all RCT study staff display good citizenship and maintain excellent communications and working relationships with host district decision makers and key staff.

key lessons learned for conducting successful rcts in school settings continued42
Key Lessons Learned for Conducting Successful RCTs in School Settings (continued)

(8) It may become increasingly important to compare the intervention with another active intervention to enable policy-makers, both inside and outside the host district, to make informed decisions about its adoption potential at the conclusion of the study.