first step to success an evidence based secondary level intervention n.
Skip this Video
Loading SlideShow in 5 Seconds..
First Step to Success: An Evidence-based Secondary Level Intervention PowerPoint Presentation
Download Presentation
First Step to Success: An Evidence-based Secondary Level Intervention

Loading in 2 Seconds...

play fullscreen
1 / 71

First Step to Success: An Evidence-based Secondary Level Intervention - PowerPoint PPT Presentation

  • Uploaded on

First Step to Success: An Evidence-based Secondary Level Intervention. W. Carl Sumi, Ph.D. and Darcey Edwards, Ph.D. SRI International Northwest PBIS Conference March 8, 2010 Corvallis, OR.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'First Step to Success: An Evidence-based Secondary Level Intervention' - herbst

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
first step to success an evidence based secondary level intervention

First Step to Success: An Evidence-based Secondary Level Intervention

W. Carl Sumi, Ph.D. and Darcey Edwards, Ph.D.

SRI International

Northwest PBIS Conference

March 8, 2010Corvallis, OR


National Effectiveness Study of First Step to SuccessNational Center for Special Education Research (NCSER)Institute of Education Sciences (IES)U.S. Department of Education

NBRCC Staff: SRI International

W. Carl Sumi, Ph.D. Mary Wagner, Ph.D.

Frances Bergland, Hal Javitz, Ph.D.

Michelle Woodbridge, Ph.D. Patrick Thornton, Ph.D.

NCSER Project Officers

Jacquelyn A. Buckley, PhD

outline of presentation
Outline of Presentation
  • Provide description of First Step to Success.
  • Describe National Effectiveness Study of First Step and present preliminary outcome data.
  • Introduce and discuss Practice Guide: “Reducing Behavior Problems in the Elementary School Classroom.”

The information presented here does not necessarily reflect the views or policies of the U.S. Department of Education or Institute of Education Sciences nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. government.

social skill ratings of elementary middle school students
Social skill ratings of elementary/ middle school students

Percentage with low social skills rating by parents

Sources: Wave 1 SEELS parent interviews, 2000 and 2001, Social Skills Rating Scale (Gresham and Elliott, 1990) national norms.

Statistical significance: *p < .05.

school suspensions and expulsions
School suspensions and expulsions

Percentage ever suspended or expelled

Sources: Wave 1 SEELS and NLTS2 parent interviews, 2000 and 2001; National Household Education Survey, 1999.

Statistical significance: * p <.05.

examples of problem behaviors
Examples of problem behaviors
  • Arguing
  • Stealing
  • Disturbing others
  • Hyperactive, inattentive
  • Tantrums
  • Physical aggression- hitting, kicking, biting, scratching, spitting
  • Impulsive, attention problems
  • Uncontrollable crying
  • Not complying with teacher instructions or directives
  • Oppositional behaviors
consequences of problem behaviors
Consequences of problem behaviors
  • Less “time-on-task”
  • Poor grades, poor achievement
  • Increased negative interactions with peers, family members, teachers
  • Peer rejection
  • Delinquency
  • Substance abuse
  • Negative cycle of poor achievement and problem behavior
typical responses to problem behaviors
Typical responses to problem behaviors
  • Punishment
  • Exclusion/isolation
  • Lack of positive interactions
  • Ineffective, inconsistent consequences
  • Negative consequences
  • Inadvertent reinforcing of problem behaviors
where do we go from here
Where do we go from here?
  • Since a large percentage of children in our schools exhibit behavior problems to a varying degree what should teachers, administrators, families, school staff, and other professionals do?

=> Implement evidence-based interventions and programs that have proven positive effects for school-aged students!

first step to success
First Step to Success
  • A short-term (approximately 3 months), secondary level early intervention that targets children exhibiting problem behaviors (e.g., aggression, antisocial, oppositional, impulsive, hyperactive, etc).
  • Goals of First Step are to enhance children’s social competence skills and school engagement in an effort to decrease problem behaviors and prevent children from developing more serious antisocial conditions.
  • First Step relies on parents, teachers, and children in order to modify and, in turn, reward behavior both at school and at home.
first step to success1
First Step to Success

Major components of the program include:

  • A proactive, class-wide screening to nominate a student for the program.
  • CLASS: A classroom-based curriculum used to teach and reinforce appropriate behaviors.
  • A family-based component called “homeBase,” a brief child-focused program for the caregiver(s).
first step class wide screening
First Step: Class-wide Screening
  • Teachers complete the Systematic Screening for Behavior Disorders (SSBD).
  • Teachers are asked to evaluate all students on various measures of antisocial behavior.
  • Ultimately, teachers identify those children at risk for, or already exhibiting, internalizing or externalizing behavior problems.
first step class wide screening1
First Step: Class-wide Screening
  • SSBD Stage 1
    • Review class roster, identify five children in the class who exhibit externalizing behaviors, and rank order them from 1 to 5.
  • SSBD Stage 2: For the top 3 ranked students the teacher completes the:
    • Critical Events Index – 32 yes/no items (e.g., steals, sets fires, is physically aggressive).
    • Adaptive Behavior Index – Positive, prosocial behaviors rated on a 1-5 scale (e.g., follows rules, cooperates with peers).
    • Maladaptive Behavior Index – Antisocial behaviors rated on a 1-5 scale (e.g., manipulates others, refuses to participate in activities with other students at recess).
  • Implement First Step with child with highest Stage 2 score.
first step to success class
First Step to Success: CLASS
  • CLASS – Contingencies for Learning Academic and Social Skills
    • Intended to target and correct behavior problems.
    • One child per classroom can receive the intervention at a time.
    • Children learn how to:
      • Attend to the teacher.
      • Get along with others.
      • Participate in activities .
  • Three phases which last 30 program days:
    • Coach/consultant phase
    • Teacher phase
    • Maintenance phase
first step class coach phase
First Step: CLASS- Coach phase
  • The coach:
    • Can be school psychologist, social worker, counselor, behavior specialist.
    • Observes the focus child.
    • Meets with caregiver and teacher.
    • Provides the materials for the teacher and child.
    • Begins the program with the child.
    • Teaches acceptable behavior 1:1 with child.
    • Monitors student progress.
    • Phones parent regularly.
    • Implements homeBase starting Day 10.
first step class coach phase1
First Step: CLASS- Coach phase

The coach meets with the child and class

  • Conducts first meeting with the child.
    • Child agrees to play the Green/Red card “game.”
    • Child chooses appropriate and meaningful rewards
    • Role-play appropriate behaviors.
  • Introduces the Green/Red card game to the class.
    • Introduce “volunteer.”
    • Obtain cooperation from class (positive comments, model “doing the right thing”).
    • Class can participate in reward activity.
first step class coach phase2
First Step: CLASS- Coach phase

Days 1-5: The coach leads program

  • Coach operates Green/Red card in class.
    • Days 1-4 last 20 minutes and Day 5 is 30 minutes.
  • Coach awards points during game.
  • Child earns reward if 80% of points earned.
  • Coach announces reward to class.
    • Student must get opportunity for class reward immediately after session.
  • Child brings Green/Red card home.
  • Coach contacts caregiver each day first five days.
    • Caregiver rewards child for successful day.
first step class teacher phase
First Step: CLASS- Teacher phase

Days 6-20 the teacher takes the lead

  • Teacher operates the Green/Red card game.
  • Monitors student behavior and awards points.
    • The Green/Red card must be visible during the game.
  • Communicates with coach and caregiver regularly.
    • Sends the Green/Red card home daily.
  • Immediately delivers reward when earned.
  • Provides child and parent with encouragement and support.
  • “Catches” the child doing the right thing.
  • As the child progresses through the program the length of the Green/Red card game increases daily.
first step class maintenance
First Step: CLASS- Maintenance

Days 21-30 are considered the Maintenance phase

  • Teacher continues to lead program but time is dedicated to reducing dependence on the intervention and transferring involvement to families.
  • Rewards and privileges for appropriate behavior are used steadily less by teachers and adopted in the home environment.
  • Parents are encouraged to substitute praise for these privileges.
  • During the whole program a child must meet daily performance criteria in order to proceed to the next day of intervention.
first step class
First Step: CLASS

Additional information:

  • If the child fails to meet criteria one day, they must repeat that program day.
  • Most children take approximately two months to complete the school intervention.
first step to success homebase
First Step to Success: homeBase
  • Purpose is to help parents improve their child’s school adjustment, competence and performance.
  • Starts on Day 10 of CLASS program
  • Six lessons, approximately 1 hour each
    • Involve guide and parent-child activities.
    • Coaches emphasize ways in which parents can help their child with communication and sharing, cooperation, limits-setting, problem-solving, friendship skills and confidence.
  • Can meet at home, school, or comfortable location.
first step homebase parent role
First Step: homeBase- Parent role
  • Participate in homeBase lessons.
    • Play activity cards for 10 minutes each night with child.
  • Reward and acknowledge child when s/he brings home the Green/Red card
    • Do something fun with child when s/he earns reward at school
      • Read a book
      • Take a walk
      • Bake a cake
      • Play a game
      • Special snack/treat
      • Help mom/dad on project
first step to success homebase1
First Step to Success: homeBase
  • Week 1: Communication
    • Child practices giving information
    • Parent listens and gives encouragement

=>Information gives parents the ability to help children be successful outside the home.

  • Week 2: Cooperation
    • Parent & child learn strategies
    • Sticker chart or chart used at home

=>Being cooperative allows a child opportunities to avoid problems.

first step to success homebase2
First Step to Success: homeBase
  • Week 3: Limit setting
    • Giving effective directions and encouragement
    • Time-out procedures

=>Teaching children to follow limits at home leads to self-control and accepting limits.

  • Week 4: Problem solving
    • Stay calm and brain-storm
    • Parents help to guide, encourage and suggest steps to achieve goal

=>Children who see problems as opportunities rather than obstacles feel capable.

first step to success homebase3
First Step to Success: homeBase
  • Week 5: Friendship skills
    • Learning to initiate
    • Empathy and self-control
    • Cooperation

=>Learning friendship skills now provides a base for friendship throughout life.

  • Week 6: Confidence building

=>Self-confidence developed at home provides a foundation for success outside the home.

national effectiveness study of first step to success introduction
National Effectiveness Study of First Step to Success: Introduction
  • Goal 4 grant from the U.S. Department of Education, Institute of Education Sciences, National Center for Special Education Research
  • Well-developed evidence base for the efficacy of First Step
    • “Manualized” off-the-shelf intervention
    • Solid evaluation framework
  • Randomized controlled trial in 48 schools in 5 diverse elementary school districts across the country
  • Evaluators (SRI) independent of program developers (ORI)
study participants
Study participants
  • 8 to 10 schools in each district
    • Matched on basic demographics and randomly assigned
    • Half in the intervention condition receive First Step
    • Half in the usual-care condition receive typical services
      • Teachers trained in First Step at conclusion of data collection
  • 6 first- through third-grade students in each school
    • All students screened with SSBD
    • 1 student per class participates in each condition each year
    • In year 2, intervention teachers implement First Step again with another student

First Step to Success Effectiveness Study:

Core research questions


  • To what extent does First Step improve the behavior at school and the academic performance and participation of students with severe behavior problems?
  • For what kinds of students does First Step work best and less well?
  • In what contexts (classroom and school level) does First Step work best and less well?
  • How do variations in effectiveness relate to variations in implementation fidelity?



First Step to Success effectiveness study:

Core research questions


  • Are the effects of First Step sustained for 1 year? For 2 years?
  • Does maintenance of effects relate to variations in students or contexts?
student level information
Student-level information
  • Student Enrollment Survey
    • Basic demographics (gender, ethnicity, primary language, free or reduced-price lunch status)
  • Student Record Survey
    • School records information:
      • IEP/504 Plan status
      • Instructional settings (i.e., percentage of instructional time in general education classes)
      • Absences
      • Office Discipline Referrals (ODRs)


student level assessments
Student-level assessments
  • Social Skills Rating System (SSRS) –Teacher and Parent versions
    • Social skills, problem behaviors, and academic competence
  • Woodcock-Johnson III Letter-Word Identification Subtest
    • Reading skills
  • Oral Reading Fluency (ORF)
    • Ability to read aloud expressively
  • Academic Engaged Time (AET)
    • Active engagement in relevant academic material over two 15-minute observations
classroom level information
Classroom-level information
  • Classroom Atmosphere Rating Scale (CARS)
    • 30-minute observation of intervention classrooms (e.g., student compliance, cooperation, problem solving)
  • Classroom / Teacher Survey
    • Classroom characteristics (e.g., student enrollment)
    • Teacher characteristics (e.g., years experience, degrees)
    • Teacher support (e.g., training, classroom aides)
    • Teacher self-reported skills to work with students with behavior problems
school level information
School-level information
  • School Characteristics Survey
    • Student characteristics (e.g., mobility rate)
    • School climate (e.g., total number of ODRs)
    • Staff and program resources (e.g., number of FTEs)
  • NCES Common Core of Data (CCD)
    • Extracted data describing participating schools and districts (e.g., enrollment, teacher/student ratio)
implementation measures
Implementation measures
  • Fidelity
    • Integrity of program (monitored three times throughout intervention for each participant)
  • Social validity (teacher and parent’s perspective)
    • Acceptability–General support for intervention
    • Positive effects–for participating student(s) and classroom
  • Alliance
    • Strength of the relationship between coach and teacher
study timeline
Study timeline

~10 weeks

1 year

1 year


Follow-up 1

Follow-up 2



  • Screening: SSBD
  • Baseline: WJIII, ORF, AET, SSRS, Student Record Survey, Teacher Survey, CARS, School Survey
  • Posttest: WJIII, ORF, AET, SSRS, Satisfaction (parent), Social Validity, Alliance
  • Follow-up: WJIII, ORF, AET, SSRS, Student Record Survey, Teacher Survey, CARS, School Survey
analysis method
Analysis method
  • Repeated measures: significance reflects effect by time and group.
  • Effect size of the difference scores.
    • Effect sizes calculated by dividing the estimated treatment effect by the standard deviation of posttest measurements (using both comparison and intervention students).
preliminary behavior outcomes teacher reported
Preliminary behavior outcomes: Teacher reported

1 Intervention Group sample size ranged from 156 to 183; Comparison Group sample size ranged from 127 to 136.

* = p < .05.

SOURCE: Social Skills Rating System (Gresham and Elliott 1990), Woodcock-Johnson III.

preliminary behavior outcomes parent reported
Preliminary behavior outcomes: Parent reported

1 Intervention Group sample size = 138; Comparison Group sample size = 115.

2 Intervention Group sample size = 138; Comparison Group sample size = 115.

* = p < .05.

SOURCE: Social Skills Rating System (Gresham and Elliott 1990).

preliminary academic outcomes
Preliminary academic outcomes

1 Intervention Group sample size ranged from 150 to 183; Comparison Group sample size ranged from 123 to 136.

* = p < .05.

SOURCE: Social Skills Rating System (Gresham and Elliott 1990), Woodcock-Johnson III.

academic engaged time
Academic Engaged Time

p = .054

es = .22

oral reading fluency
Oral Reading Fluency

p < .05

es = .32

social validity and alliance summary1
Social Validity and Alliance Summary

Social Validity

  • Teachers: Mean rating of 3.56
    • Would recommend the intervention to other teachers.
    • Program had positive effect on child relationships.
  • Parents: Mean rating of 4.21
    • I would recommend the program to other parents.
    • I liked getting daily feedback from the green/red card.


  • Teachers: Mean rating of 4.58
    • “The coach is approachable.”
    • “The coach and I trust one another.”
  • Coaches: Mean rating of 4.53
    • “The teacher is approachable.”
    • “The teacher and I trust one another.”
next steps
Next Steps
  • Data collection will continue until spring 2011.
  • To follow the progress of the National Effectiveness Study of First Step to Success or to find out more information visit,

The What Works Clearinghouse

Practice Guide: “Reducing

Behavior Problems in the

Elementary School Classroom”

purpose of the what works clearinghouse
Purpose of The What Works Clearinghouse
  • The mission of the WWC is to “be a central and trusted source of scientific evidence for what works in education.”
  • WWC generates several different products:
    • Intervention reports
    • Topic reports
    • Quick reviews
    • Practice guides
practice guide purpose
Practice Guide: Purpose
  • Supply discrete recommendations that are intended to be actionable.
  • Provide a coherent approach to a multifaceted problem.
  • Explicitly connect each recommendation to the level of evidence supporting it (strong, moderate, or low).


practice guide purpose1
Practice Guide: Purpose
  • Use expertise and judgment of a panel to identify the most important research relevant to the recommendations.
  • Bring the best available evidence to bear on challenges that cannot currently be addressed by single interventions or programs.
practice guide scope
Practice Guide: Scope
  • Primary audiences include:
    • General education elementary school teachers who will implement practices
    • Elementary school/district administrators who will promote practices
  • Final product is “more like a consensus panel report than a meta-analysis” in terms of breadth and complexity of topic addressed.
reducing behavior problems in the elementary school classroom recommendations
Reducing Behavior Problems in the Elementary School Classroom Recommendations
  • Broader points about improving practice
    • Drawn from evidence about the effectiveness of specific programs and practices, including:
      • Suggestions for how to implement in school settings ( checklist)
        • Provide district-, school-, and teacher-level guidelines
      • Descriptions of potential roadblocks
        • Refute myths with evidence
        • Suggest solutions
recommendation 1
Recommendation 1
  • Identify specifics of problem behavior and conditions that prompt and reinforce it.
    • Level of Evidence: Moderate
    • Implementation Guidelines
      • Observe and record frequency of problem behavior.
      • Identify what prompts and reinforces behavior.
    • Potential Roadblocks and Solutions
      • Problem behaviors may persist even after intervening.
        • Interventions must be implemented with sufficient time and consistency.
        • A single behavior may stem from multiple triggers.
        • Intervene for at least one month, and continue to observe and record behavior. Try new approach if necessary.
recommendation 2
Recommendation 2
  • Modify the classroom learning environment to decrease problem behavior.
    • Level of Evidence: Strong
    • Implementation Guidelines
      • Reinforce classroom expectations.
      • Adapt/vary instruction to increase engagement.
    • Potential Roadblocks and Solutions
      • Teachers do not want to disrupt routines.
        • Time used to practice new routines will increase quality of instructional time in the end.
        • Prepare students well for change; ask students to model new behaviors as reward for appropriate behavior.
recommendation 3
Recommendation 3
  • Teach and reinforce new skills to increase appropriate behavior and preserve a positive classroom climate.
    • Level of Evidence: Strong
    • Implementation Guidelines
      • Teach replacement skills explicitly.
      • Reinforce appropriate behavior; withhold reinforcement for inappropriate behavior.
    • Potential Roadblocks and Solutions
      • Teachers fear extrinsic rewards undermine student motivation.
        • Tie reinforcement to student competence.
        • Reward students with behavior-specific praise.
recommendation 4
Recommendation 4
  • Draw on relationships with colleagues and families for guidance and support.
    • Level of Evidence: Moderate
    • Implementation Guidelines
      • Build collaborative professional partnerships.
      • Encourage families to participate in reinforcing appropriate behavior.
    • Potential Roadblocks and Solutions
      • Faculty meetings can be a waste of teachers’ time.
        • Administrators should encourage a culture of professional learning.
        • Use time together productively to joint problem-solve.
recommendation 5
Recommendation 5
  • Implement schoolwide strategies to reduce negative and foster positive interactions.
    • Level of Evidence: Moderate
    • Implementation Guidelines
      • Involve school improvement team in collecting data about school “hot spots.”
      • Adopt program that aligns with school climate.
    • Potential Roadblocks and Solutions
      • Packaged programs may be too costly.
        • Consider evidence-based programs that meet school needs.
        • If too costly, encourage school staff to observe patterns of problem behavior to assist in formulating an intervention.
  • Trusting and supportive relationships lay the foundation for positive behavior.
  • There is increased need for building cultural competence among school communities.
  • Collecting data is critical in targeting resources and changing strategies to improve behavior.
to download and print
To download and print

Go to:

or go to and find the Practice Guide

under the Publications link.


Q & A

Final Thoughts

and Wrap up