S. Andrew Garbacz, MA Susan M. Sheridan, PhD Michelle S. Swanger-Gagne, MA Amanda L. Witte, MA

1. Intervention Implementation Integrity Within Conjoint Behavioral Consultation: Strategies to Maximize Outcomes S. Andrew Garbacz, MA Susan M. Sheridan, PhD Michelle S. Swanger-Gagne, MA Amanda L. Witte, MA University of Nebraska-Lincoln The Nebraska Center for Research on Children, Youth, Families, and Schools (CYFS; http://cyfs.unl.edu) The project is funded by the Institute of Educational Sciences Grant Award Number: R305F050284 CBC in the Early Grades

2. School-Based Consultation • School-based behavioral consultation is an indirect form of service delivery wherein a specialist with expertise in social-behavioral and academic interventions works collaboratively with a treatment agent (i.e., consultee, such as a teacher or parent). • School-based consultation may focus on behavioral, socio-emotional, or academic concerns, which interfere with learning.

3. Conjoint Behavioral Consultation • Conjoint behavioral consultation (CBC; Sheridan, Kratochwill, & Bergan, 1996; Sheridan & Kratochwill, 2008) is an extension of traditional behavioral consultation services. • CBC is, by definition, an indirect model of service delivery intended to enhance the skills of natural treatment agents (i.e., teachers, parents) who assume responsibility for implementing intervention plans. • Goals of CBC focus on addressing the specific needs of the child, while working collaboratively with the child’s teacher and parent to strengthen family-school partnerships. - Including enhancing the skills of natural treatment agents (i.e., consultees).

4. Implementation Integrity Intervention implementation integrity refers to the extent to which an intervention plan is implemented as intended. An important consideration in CBC is the ability of individuals to deliver treatment plans accurately and efficiently. Thus, the issue of implementation integrity is central to the utility of CBC in addressing students’ behavioral needs.

5. Implementation Integrity • Little systematic attention is afforded to the specific strategies that consultants can use to ensure accurate and reliable implementation of treatment plan elements in natural settings (Sanetti & Kratochwill, 2008). • Strategies to support families in implementing interventions within a problem-solving framework have not received adequate attention.

6. Socioeconomic Conditions • A significant correlate with treatment dropout and poor outcomes following parent training interventions is socioeconomic hardship (Henggeler et al., 1992; Kazdin, 1990; McMahon et al., 1981). • Families living in low socioeconomic conditions face numerous stressors (e.g., poverty, lack of social support) that may interfere with their ability to implement parenting interventions, thereby compromising treatment effects.

7. Purpose • Describe the degree to which families implement interventions as designed in a consultation study to promote positive behavioral outcomes for children experiencing behavioral difficulties. • Describe and illustrate multiple methods of assessing integrity. • Discuss strategies used to maximize intervention implementation integrity of treatment plans by all families to increase positive outcomes. • Present methods used by consultants to support families with diverse challenges.

8. Methods - Participants • Child Participants: • 62 children • 75% Male • 67% White, non-Hispanic • Average age = 6.72 (range: 5 – 9) • All students were referred to the study by their teachers who indicated a need for further intervention due to behavioral concerns.

9. Methods - Participants • Parent Participants: • 62 Parents • 83% White, non-Hispanic • Average age = 35 • 88% Female • 78% received some college or post-secondary education • 26 parents met the criteria to be included in a demographically diverse sub-sample • Data used in this investigation represent part of a larger, four-year study evaluating the efficacy of CBC as an intervention that addresses concerns of kindergarten through third grade students whose disruptive behaviors place them at risk for academic failure.

10. Methods - Setting • Cases were conducted in 17 kindergarten, first, second, and third grade classrooms in a moderately-sized Midwestern city and surrounding area, including public and parochial schools. • Interventions were implemented across home and school settings for students who were the focus of consultation.

11. Methods - Selection • Students were selected via teacher ratings: • Systematic Screening for Behavior Disorders (SSBD) multiple-gate screening procedure (Walker et al., 1990). • Screening instrument with ratings for severity of externalizing behaviors; frequency of externalizing behaviors; and degree of need for additional intervention.

12. Methods – Data Procedures • Cases were separated into two groups: • General mainstream group (N=36) • Demographically diverse group (N=26) • Relative to mainstream group, demographically diverse was defined as low-income status, racially diverse, linguistically diverse, living in a single parent home, or less than high school diploma.

13. Consultation Procedures Consultants (i.e., trained graduate students) met with the teacher and two to three parents for approximately four to five conjoint consultation sessions over approximately eight weeks via three phases. 1) Needs Identification and Needs Analysis 2) Intervention Development and Implementation 3) Intervention Evaluation Behavioral interventions consisted of 3 components. 1) Communication component 2) Motivation component 3) Functional component

14. Implementation Support • Strategies to promote high integrity in the home setting were developed as part of the larger study • Sources: 1) Literature review that informed consultant training. 2) Narrative information reported by trained consultants • See Handout 1

15. Implementation Support General Strategies: Follow the partnership model during intervention development phase (Sheridan & Kratochwill, 2008). Provide a rationale for collecting integrity data. Script and package intervention implementation integrity forms (Watson, 2004). Train and educate the family on intervention delivery with various techniques. See handout 1

16. Implementation Support Additional strategies for families requiring more support: 1) Consider culturally-sensitive procedures (Sheridan & Kratochwill, 2008). 2) Focus on strategies that build trust between families and educators. 3) Enhance communication between consultant and families. 4) Implement family-centered approach throughout consultant process. 5) Gather information about the home setting and family system. 6) Facilitate regular contacts and provide collective support. 7) Adjust integrity data collection as to improve ease of collection. See handout 2

17. Case Study 6 year old female in 1st grade was the focus of consultation. Biological parents and 4 siblings moved to the United States from Sudan 7 years prior as refugees. The family currently meets the 2008 HHS criteria for a family living in poverty.

18. Case Study - Preconsultation Conducted an initial meeting to introduce consultation. Validated parent concerns (e.g., acculturating to the United States, feedback from classroom teachers). Met with a cultural liaison about roles, structure, and expectations. Conducted a home visit to discuss roles and expectations.

19. Case Study – Needs Identification and Analysis • Strengths: • Strong verbal skills • Independent • Concerns: • Reading skills • Ability to stay on-task; disruptive behaviors • Functional assessment revealed escape motivated behavior, and grade level academic skills.

20. Case Study – Needs Identificationand Analysis • Intervention development: • School: “chunking” reading work, appropriate escape, self-monitoring. • Home: structured homework routine that included entire family, home-school note, activity checklist with visual cues. • Family checklist to increase integrity. • Additional support (e.g., meeting reminders, frequent communication).

21. Measurement of Treatment Integrity • Parent Self-Report Form • Completed daily by parents and used to assess intervention implementation integrity at home. • Listed all the steps of the behavioral intervention plan clearly and objectively. • Example includes intervention plan steps that the parent in this case completed • See handout 3

23. Measurement of Treatment Integrity • Permanent Product Report Form • Completed by parents daily for the duration of intervention (i.e., at least 4 weeks of intervention). • Charts were collected from parents on which evidence (e.g., stickers, notes, marks, checks) demonstrating that s/he implemented steps of the intervention. • Trained research assistants and consultants reviewed permanent products and completed the permanent product report forms to reflect parents’ delivery of plan components as reported on permanent products. • See handouts 4 and 5

24. Activity Checklist

26. Case Study – Strategies to Maximize Integrity Integrity information was scripted and standardized Intervention steps were simple and consistent Opportunity to have forms translated to Arabic; use of visual cues. Home visits for modeling, observation, performance feedback, and discussion. The family reported adhering to 100% of intervention steps over 4 weeks of implementation.

27. Case Study – Intervention Evaluation • School • Increase from an average of 45% accuracy prior to the intervention to 80% accuracy after implementation. • Home • Decrease in the number of problem behaviors exhibited from an average of 4 to an average of 1. • Parents indicated improved behavior management strategies, communication with the school, and praising children for attaining goals.

28. Assessment of Implementation Table 3 Summary Statistics of Intervention Implementation Integrity across Sample a Participants completed self-report report measure, permanent product measure, or both measures

29. Discussion High levels of intervention implementation integrity were noted in the home setting across multiple sources (i.e., families via parent-report and coders of permanent product data) Families seem to be responsive to tactics used by consultants to maximize treatment integrity. Families also appear to adhere to the steps of behavioral interventions regardless of the integrity measure used, with families reporting slightly higher integrity when recording integrity on a permanent product.

30. Discussion • Permanent products are a feasible and useful way of measuring treatment integrity because families naturally use the products as they implement interventions (Sanetti & Kratochwill, 2008). • High integrity levels reported on permanent products may suggest that parents are not over-reporting on self-report measures. • Families “at-risk” reported implementing intervention with higher integrity than the mainstream group of families.

31. Implications To maximize outcomes, educators may find it helpful to use a collaborative consultation model to design and implement behavioral interventions with families. In CBC, training, supporting, and partnering with teachers and families provides a consistent and continuous approach to service delivery.

32. Future Research Directions • Examine integrity and the variables that may influence levels of treatment integrity experimentally. • Specifically, future research could attempt to measure the impact of disadvantaged social settings on the degree to which families follow a behavioral intervention protocol. • Explore resources that are needed and specific procedures that are critical to intervention implementation. The cost-effectiveness of such practices needs to be assessed in future studies.

33. Future Research Directions • Measure and evaluate treatment integrity in other ways. • For example, the dosage,quality of, and responsiveness to implementation may be important in understanding the quality of services students are receiving (Dane & Schneider, 1998). • Standardized measures of consultant strategies for maximizing integrity should be developed and used in future research.

34. Future Research Directions • Psychometric properties of future measures of integrity and strategies used by consultants to improve integrity should be examined (e.g., validity, reliability). • Similar issues with regard to measurement and evaluation should be explored at school. • The link between home and school treatment integrity and its effects on student performance requires examination.

35. Contact Information • Susan M. Sheridan: ssheridan2@unl.edu • Andy Garbacz: agarbacz2@unl.edu • Amanda Witte: awitte2@unl.edu • http://cyfs.unl.edu

36. References • Dane, A. V. & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23-45. • Henggeler, S. W., Melton, G. B., & Smith, L. A. (1992). Family preservation using Multisystemic Therapy: An effective alternative to incarcerating serious juvenile offenders. Journal of Consulting and Clinical Psychology, 60, 953-961. • Kazdin, A. E. (1990). Premature termination from treatment among children referred for antisocial behavior. Journal of Child Psychology and Psychiatry, 31, 415-425. • McMahon, R. J., Forehand, R., Griest, D. L., & Wells, K. C. (1981). Who drops out of treatment during parent behavioral training? Behavioral Counseling Quarterly, 1, 79-85. • Sanetti, L. M. H., & Kratochwill, T. R. (2008). Treatment integrity in behavioral consultation: Measurement, promotion, and outcomes. International Journal of Behavioral Consultation and Therapy, 4, 95-113.

37. References • Sheridan, S. M., & Kratochwill, T. R. (2008). Conjoint behavioral consultation: Promoting family-school connections and interventions (2nd ed.). New York, NY: Springer Publishing. • Sheridan, S. M., Kratochwill, T. R., & Bergan, J. R. (1996). Conjoint behavioral consultation: A procedural manual. New York: Plenum. • Walker, H.M. & Severson, H. H., Todis, B.J., Pedego, A.E. (1990). Systematic screening for behavior disorders (SSBD): Further validation, replication, and normative data. RASE: Remedial & Special Education, 11, 32-46. • Watson, T. S. (2004). Treatment integrity. In T. S. Watson & C. H. Skinner (Eds.), Encyclopedia of school psychology (pp. 356-358). New York, NY: Springer Publishing.