1 / 113

By James L. McDougal, Psy. D., Achilles N. Bardos, Ph.D., & Scott T. Meier, Ph.D.

TM. By James L. McDougal, Psy. D., Achilles N. Bardos, Ph.D., & Scott T. Meier, Ph.D. 3-tier Behavioral Progress Monitoring System for screening and assessing changes in response to behavior intervention follows the RTI model School-aged children and adolescents (5-18)

pierce
Download Presentation

By James L. McDougal, Psy. D., Achilles N. Bardos, Ph.D., & Scott T. Meier, Ph.D.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. TM By James L. McDougal, Psy. D., Achilles N. Bardos, Ph.D., & Scott T. Meier, Ph.D.

  2. 3-tier Behavioral Progress Monitoring System for screening and assessing changes in response to behavior intervention follows the RTI model School-aged children and adolescents (5-18) 2 main components: BIMAS Standard & BIMAS Flex Multi-informant tool with forms for teachers, parents, youth, and clinician web-based data management and reporting system. What is the BIMAS?

  3. Presentation Overview School Mental Health Services BIMAS: Rationale within an RTI context BIMAS: Theory and Structure BIMAS: Development and Psychometric Properties BIMAS: Applications in a School District BIMAS: Web-based Interface and Reports. Q and A

  4. Background & Development

  5. Why is Behavioral/Emotional Screening Crucial? • Students with behavioral/emotional problems: - lower grades - poorer reading skills - drop-out rate > 50% - worst social & academic outcomes of any disability group (Bradley, Dolittle, & Bartolotta, 2008) • Loss of instructional time: teacher intervention & student disruption (Arnold, 1997) • Early ID – prevents pervasive problems(Kern, Hilt-Panahon, & Sokol, 2009) • Systematic universal screening  ID children at-risk for E/B problems(Kamphaus & Reynolds, 2007)

  6. Early Identification Early identification and intervention with children who are at risk for EBD appear to be the “most powerful course of action for ameliorating life-long problems associated with children at risk for [EBD]” (p. 5). (Hester et al., 2004) Younger children are more likely to be responsive to and maintain the positive outcomes from early prevention and intervention programs (Bailey, Aytch, Odom, Symons, & Wolery, 1999)

  7. RTI and BehaviorRationale and need for Universal Behavior Screening Strong link between behavior/emotions and academic performance How do we define health? • Schools screen for vision, hearing, speech, and academic achievement • Behavior/Emotional screening occurs in less than 2% of districts across the U.S.

  8. RTI and BehaviorRationale and need for Universal Behavior Screening Screeners for children are mostly done in primary care settings; we miss many children for early identification and intervention (Pagano et al., 2000). Screening measures for children are frequently focused on one disorder to the exclusion of others, an approach that neglects large numbers of children who have problems other than the target condition (August et al., 1992; Taylor et al., 2000; Matthey & Petrovski, 2002).

  9. RTI and BehaviorRationale and need for Universal Behavior Screening Early problem identification allows for prevention & intervention Negative impact of problems can be minimized Only 15-20% of children with emotional/behavioral problems receive mental health services in the U.S (Ringel & Sturm, 2001)

  10. RtI and BehaviorRationale and need for Universal Behavior Screening Teachers accurately identify young children at high risk of academic and behavioral problems related to school adjustment with a great deal of accuracy(Taylor et al., 2000). Schools are the ideal setting for large-scale, broad based mental health screening of children and adolescents (Wu et al., 1999).

  11. Tier 1 Universal Level Students without serious problem behaviors (80-90%) Use BIMAS to Screen How to use BIMAS within the RtI Framework

  12. How to use BIMAS within the RtI Framework Tier 2 • Targeted Level • Students at risk for problem behaviors (5-15%) • Use BIMAS to monitor & assess response to intervention/treatment

  13. How to use BIMAS within the RtI Framework Tier 3 • Intensive Level • Students with chronic/intense problem behaviors(1-7%) • BIMAS Data for decision making: Treatment planning, Special education eligibility, IEP review, program evaluation

  14. Applications of BIMAS • Screening- To detect students in need of further assessment and to identify their respective areas of need. • Student Progress Monitoring- To provide feedback about the progress of individual students or clients. • Program Evaluation- To gather evidence that intervention services are effective.

  15. Instrument Structure The BIMAS consists of two levels • Standard • Flex

  16. BIMAS Standard • 34 change-sensitive items • Can be used as a brief screener, treatment monitoring or program evaluation tool • 4 forms: Teacher, Parent, Self (U.S. normed) & Clinician

  17. Intervention Item Selection Rules(IISRs; Meier 1997, 1998, 2000, 2004) • BIMAS: developed using empirically derived model for designing change-sensitive measures to assess RTI: IIRSs • “State” scale as opposed to a “Trait” scale • BIMAS developed using clinical & school samples in field settings (rare combo in psychotherapy research) • Identified constructs that change as a result of emotional and behavioral intervention

  18. IISRs (cont’d) • Be grounded in theory- to provide for interpretation and minimize the effects of chance; • Be aggregated across individuals- to reduce the effects of random error; • Not evidence ceiling and floor effect; • Evidence change after a psychosocial intervention; • Change in the theoretically expected direction; • Evidence change relative to control and comparison groups; • Show no difference at pre-intervention; • Have no relation to relevant systematic errors; and • Be subject to cross-validation studies. (Meier 1997, 1998, 2000, 2004)

  19. Rule 1. Items Based on Theory Review existing research and theory Select pool of items from a variety of theories Ensure large Item Pool Select those thought to be influenced by intervention.

  20. Rule 2. Aggregate Items Aggregation across individuals decreases error Increases chances of showing intervention effect Individual response used to inform theory

  21. Rule 3. Avoid Ceiling/Floor Effects Mean item scores should not be at top or bottom of possible range Drop items with means 2 or more SD’s out

  22. Rule 4. Detect Change Post-Intervention Significance vs. effect size Retain subset of items showing pre/post change Then filter those that demonstrate treatment group change

  23. Rule 5. Change in the Expected Direction Retain items showing pre/post TX change in expected direction Items changing in opposite direction: retained to investigate negative effects

  24. Rule 6. Change Relative to Comparison Group Compare to no TX control for: developmental and maturation effects, and TX effects Do items differentiate between TX groups? Retain items: pre/post difference in TX group Drop items with pre/post difference in control group

  25. Rule 7. No Difference at Pre-Test Drop items showing significant difference between samples drawn from the same population Random assignment best

  26. Rule 8. Remove Systematic Error Drop items reflecting systematic error (e.g., social desirability --Marlowe Crowne Social Desirability Scale)

  27. Rule 9. Cross Validate Repeat steps 3-8 with new samples from same population Emphasize theoretically predicted change items

  28. BIMAS Standard Behavioral Concern Scales: • Conduct— anger management problems, bullying behaviors, substance abuse, deviance • Negative Affect— anxiety, depression • Cognitive/Attention— attention, focus, memory, planning, organization Adaptive Scales: • Social — social functioning, friendship maintenance, communication • Academic Functioning— academic performance, attendance, ability to follow directions

  29. BIMAS Flex • List of specific behavioral items corresponding to each Standard item for progress monitoring • User can select items based on elevated Standard scale score for an individual student — customized treatment goals • Ability to make notes to describe specific behaviors, response to services, or to add other comments • Teacher, Parent, Self and Clinician forms

  30. Negatively worded: Argued with peers Argued with teachers Argued with parents Argued with siblings Talked back to parents Talked back to teachers Physically hurt peers Physically hurt parents Physically hurt teachers Physically hurt siblings Threatened peers Threatened teachers Threatened parents Threatened siblings Positively worded: Showed regret after a fight Was respectful to adults Walked away from a fight Prevented a fight Stopped an argument Found a positive outlet for frustration Avoided a verbal confrontation BIMAS Flex ExampleStandard Item:Fought with others (verbally, physically, or both) Or…custom create your own!

  31. Psychometric Properties

  32. Large Normative Sample Total Sample N = 4,855 Teacher N = 1,938 Parent N = 1,938 Self-Report N = 1,050 Normative N = 1,400 Clinical N = 538 Normative N = 1,400 Clinical N = 467 Normative N = 700 Clinical N = 350

  33. Age x Gender Distribution: Normative Sample

  34. Race/Ethnicity Distribution Highly comparable to the most recent U.S. Census (Weighted N’s)

  35. Geographic Region Distribution • Highly comparable to the most recent U.S. Census (Weighted N’s)

  36. Parental Education Level • Highly comparable to the most recent U.S. Census (Weighted N’s)

  37. Norm Groups Age groups based on: age group mean differences developmental settings (K-12) Combined-gender norms (♂ flagged as often as ♀) Gender-specific norms available as option

  38. Development of Standard Scores T-scores reported on every scale Raw scores  percentile rank to preserve the shape of original distribution (behavior: not normal curve but Original percentile scores smoothed by imposing empirical percentiles Final smoothed percentile scores converted to standard T-scores (M = 50; SD = 10)

  39. Interpretation of BIMAS Scores : Scale-level Descriptors Behavior Concern Scales (↑ scores = ) High Risk:T = 70+ Some Risk:T = 60-69 Low Risk: T < 60 Adaptive Scales (↑ scores = ) Strength: T = 60+ Typical: T = 41-59 Concern: T≤ 40

  40. Interpretation of BIMAS Scores: Item-Level Descriptors(Useful in Individualized Intervention Design) Likert Scale 0 = Never; 1 = Rarely; 2 = Sometimes; 3 = Often; 4 = Very Often Item Descriptors: Behavior Concern Scales No Concern Mild Concern = M + 1SD; ≥ 75th percentile Concern = >1 SD; ≥ 84th percentile Adaptive Scales Concern = ≤10th percentile Mild Concern = 10th - 20th percentile; M - 1SD Fair Positive = ≥ 75th percentile; M + .67SD

  41. Psychometric Properties Reliability • Internal Consistency • Test-Retest (stability) Validity • Content • Construct • Scale structure • Screening accuracy • Progress monitoring

  42. Reliability • Internal Consistency • The extent to which all items on the same scales measure the same content • Cronbach’s Alpha (ranges from 0.0 to 1.0; higher = more reliable) • Good range: α = .80 to .89 • Clinical cases added to the Normative Sample for reliability analyses (85% Normative; 15% Clinical) • To increase variability in the data • Most schools: mixture of students with & w/o diagnosis

  43. Internal Consistency Cronbach’s Alpha

  44. Reliability (cont’d) • Test-Retest Reliability • Refers to the stability of test scores when an assessment is administered on two or more occasions (without intervention) • Pearson’s Correlation (r) between Time 1 and Time 2 BIMAS scores • 2-to-4 week interval • Ranges from -1 to +1; higher = more reliable; good range: .7 or higher

  45. Test-Retest Reliability Coefficients All rs significant, p < .001.; A 2-4 week interval (non-clinical sample; no intervention in between)

  46. Across-Informant Correlations Correlation between parent & teacher ratings Correlation between self-report & parent/teacher Are the behaviors assessed by the BIMAS consistently detected by raters in different settings? (Diff informant: Diff observation context) Parent to Teacherr: range = .79 - .86 Parent to Self r: range = .59 - .69 Teacher to Selfr: range = .54 - .59

  47. Validity

  48. Validity The validity of a test refers to the quality of inferences that can be made by the test’s scores (i.e., how well does the test measures and how well are theclaims it makes for its use and applications supported by empirical evidence).

More Related