Reducing racial disproportionality in discipline using the MENDR assessment process

1. Reducing racial disproportionality in discipline using the MENDR assessment process Michael Pullmann Larissa Gaias University of Washington School Mental Health Assessment, Research and Training Center Special thanks to: Mylien Duong Semret Nicodimos Janine Jones Eric Anderson Samantha Bindman This project was funded by the U.S. Department of Education, Institute of Education Sciences (R305H150035).

2. “This past spring, the archivist for Seattle Public Schools unearthed a yellowed slip of paper that C.S. Barbo, former principal of Meany Junior High, had tucked into a time capsule in 1963.In three brief paragraphs, Barbo lamented what he called the “cultural, racial and economic disadvantages” that he believed slowed the learning of some of his students.“If this box is ever opened,” his letter reads, “I would assume the problems we face today in understanding the racial differences will have been resolved. Personally, I trust this will happen.””--Morton, N., The Seattle Times, Jan. 12, 2018

4. ODDS RATIOS | Odds of Being Suspended or Expelled DECREASED RISK INCREASED RISK STUDENT LEVEL 6.0 2.8 3.2 3.2 3.6 2.5 0.9 0.4 SCHOOL LEVEL 1.4 0.4 0.6 0.4 • Note: All predictors are significant p < .05 except Asian and High School. Students in 35 middle, high, and K-8 schools (6-8th graders only), 2015-2016 school year, Discipline data entered through PowerSchool, Approximately 26,000 secondary students, Some schools/programs excluded

5. Does anything work to reduce disproportionality? • Despite the longstanding and prevalent nature of disproportionality, very few intervention researchers ever focus on the matter • We reviewed nearly 100 rigorous education intervention studies • Only 18.6% analyzed potential effects on academic and socio-emotional disparities • Only 5% demonstrated potential to reduce disparities Cook, C., McIntosh, K., & Duong, M., (in press). Addressing discipline disparities for black male students: Linking malleable root causes to feasible and effective practices. School Psychology Review.

6. Does anything work to reduce disproportionality? • Three recent studies found positive impacts on discipline: • GREET-STOP-PROMPT • Interrupt implicit bias • Double Check (Classroom Check Up) • Culturally responsive classroom management professional development/coaching model • Threat assessment • Problem-solving approach to violence prevention, an alternative to zero-tolerance

7. Agenda • Overview of MENDR • MENDR Program Review • Feedback on Domains of Practice • MENDR Assessment of Practice Process • Feedback on MAP Process • Pilot Research on MENDR

8. Disclaimer • Race is a social construction • Race does not equal culture or socioeconomic status • While race is strongly associated with life experiences, no group is monolithic • Researchers often study disproportionality using race in a comparative framework—e.g. comparing all other races against white • We acknowledge this is problematic, as it reifies white hegemony

9. The challenge Most teachers, school leadership, and administrators feel deeply concerned about disproportionality BUT Few feel like they know what to do about it

10. Key strengths of MENDR theory of change • 1. Intentional & Logical—no random acts of intervention! Activities are designed to be: • Based on need as well as feasibility • Feasible—implementable, minimizes resistance and maximizes staff buy-in, focuses on bite-sized chunks • Logically connected to desired outcomes • 2. Evidence-based/evidence-informed • 3. Multicomponent—no quick solutions, easy answers, or “silver bullets”

11. MENDR Theory of Change Cultural responsive, culturally informed, and anti-racist 1. School-wide screening and selective supports for students with behavioral needs MENDR Reflection Worksheet: Determine which factor(s) to modify, based on need, readiness, and priority domains 2. Discipline policy and interpretation MENDR Assessment to Action Menu: Implement evidence-informed intervention, measure change, modify Assess schools’ strengths, needs, and readiness to address 6 evidence-based factors for disproportionality 3. Teacher-student-family relationships 4. Teacher’s use of effective proactive behavior management techniques 5. Teacher’s use of effective reactive behavior management techniques 6. Data-based, systematic decision making

12. Self-assessment School teams (Racial Equity Teams, PBIS teams, School Leadership teams) rate themselves on several items within each of the 6 factors (and their team) on a three point scale—1 = Not in place, 2 = Needs Improvement, 3 = In place Items were generated by our team and drawn from schoolwide assessment measures such as the SWPBIS Tiered Fidelity Inventory Teams develop consensus scores, a process of discussion and debate among team members

13. Screening & Selective Supports Teacher referral is the most common procedure for disciplinary action or recommendation for supports, but is vulnerable to subjective bias. Often, students of color are referred to punitive discipline while their white counterparts are referred to counselors and other supportive services at disproportionate rates. Consistent, race-neutral methods to identify students who are struggling with behavioral and emotional problems and refer them to supportive services. Example items: • The school screens or assesses social, behavioral, emotional, and academic needs of students using culturally relevant screens. • There are effective Tier 2 supports available to students with moderate emotional and behavioral needs such as: socioemotional groups/classes, mentoring programs (e.g., Check and Connect, Check In/Check Out). • Discipline and supports data are regularly reviewed to ensure that supports are offered and provided equitably

14. Discipline Policy Effective discipline policy considers the situation and the function or reason underlying a student’s behavior in order to develop an effective response. Zero-tolerance discipline policy and punitive procedures are often associated with racial and ethnic disproportionality. Example items: • Discipline procedures focus on alternatives to exclusion and emphasize teaching prosocial skills • Discipline policies reflect and respect the cultural values of the community • A racially diverse group of students is included as active participants in making decisions about school practices and policies

15. Teacher-Student-Family Relationships Relationships are at the heart of any learning environment, and the quality of relationships between teachers, students, and parents impacts how teachers respond to problem behaviors, how students behave in school. Similarly, the response that teachers have to problem behaviors impacts their relationships with students and parents. Although most teachers endorse the importance of positive educator-student relationships, many are unlikely to espouse an intentional approach to cultivating relationships. Example items: • School staff receive training on the cultural heritage of the students in the school, historic context of the school, and present-day issues specific to the school’s underserved populations • School staff can clearly describe the specific strategies they use to build relationships with students and families. • School staff can clearly describe how implicit bias expresses itself in their daily work

16. Proactive Behavior Management Proactive behavior management involves setting clear expectations and recognition systems so that students know what behavior is considered acceptable. It also involve designing engaging classroom structures that focus students on learning. Many or most teachers have not received substantive training in evidence-based practices for proactively managing student behavior and, thus, are vulnerable to using reactive practices. Example items: • Students’ cultures are considered when designing behavioral expectations and recognition systems • When behavioral expectations differ between home and school, staff teach the differences and the purpose for having the expectations at school • Teachers are able to clearly describe the proactive classroom management strategies they use

17. Reactive Behavior Management Reactive and punitive discipline strategies can undermine educator-student relationships and interfere with positive school climate, particularly for students from different cultural backgrounds. When teachers are mentally tired, reactive strategies are more likely to be used and more likely to be influenced by implicit bias. Example items: • School staff receive consistent support in how to manage their own emotional responses and reactions to student behavior. • School staff try to understand the root cause of a behavior and provide skills instruction, rather than relying solely on punishment. • School staff can confidently use effective reactive behavior management with students from diverse backgrounds

18. Data-based, systematic decision making Data about disciplinary actions are often collected in multiple ways, but may not always be used to inform decision making to reduce disproportionality. By disaggregating disciplinary data, schools can identify the places (e.g. cafeteria, classrooms), people, and events that are most contributing to disproportionality. Example items: • Discipline referral forms and data entry are standardized and consistent. • Discipline data reports are disaggregated by race • Data is used whenever making decisions about modifying or adopting discipline policy, discipline practice, or professional development approaches to discipline

19. Identify school strengths and needs for addressing disproportionality The MENDR Program Review is one option to assess strengths and needs

20. Reflect on the utility of this tool https://tinyurl.com/mendr COMPLETE SECTION 1 PAUSE AFTER COMPLETING THE FIRST PAGE

21. Needs improvement In place Not in place Note. Data collected from Racial Equity Team members representing 17 schools who attended a Saturday Institute held on March 4, 2017

22. Reflection • Discipline Policy • Screening and Selective Supports • Effective Proactive Behavior Management • Effective Reactive Behavior Management • Clean and Useful Data • Teacher-Student-Family Relationships • Break into groups of 2-5 people and discuss: • Are there important malleable factors that are missing from the MENDR framework? Do any of these factors not belong?

23. MENDR Reflection Worksheet

24. Reflect on the utility of this tool https://tinyurl.com/mendr COMPLETE SECTION 2 PAUSE AFTER COMPLETING THE SECOND PAGE

25. MENDR has a menu!

26. Reflect on the utility of this tool https://tinyurl.com/mendr

27. MENDR Assessment-to-Action Menu Strategies ProximalOutcomes Rationale Resources 1 2 3 4

28. Reflect on the utility of this tool https://tinyurl.com/mendr COMPLETE SECTION 3

29. Reflection & Discussion • Overall, of these three components, which do you believe is the most feasible to conduct? Why? • Of these three components, which do you believe is the least feasible to conduct? Why? • What other important considerations should be taken into accountfor the MENDR process?

30. Intervention testing • Readiness assessment with 8 schools • Washington Middle School welcomed MENDR’s involvement • Informal MPR and multiple meetings with WMS RET identified Teacher-Student-Family relationships as focal area • Teacher were trained to use a relationship-building practice with a growing evidence base, Engage-Maintain-Restore (EMR) during 4 total hours of training across the school year • Existing 16-minute protocol was modified to include EMR • Small group of teachers had two trainings to become “EMR Superusers” in order to provide peer-to-peer support on EMR

31. Establish-Maintain-Restore (EMR)Theory of Change Primary assumptions Logic Outcomes • Strong teacher-student relationships are one foundation for students’: • Academic motivation • School belonging • Engagement • Attendance • Behavior Through EMR training, teachers can learn to be more systematic, intentional, and structured in how they establish, maintain, and restore relationships • Teachers who have more positive relationships with students will be: • More likely to effectively communicate expectations • More likely to perceive challenging behaviors in a positive light • Less likely to use punitive disciplinary actions Trained and supported teachers will increase and intentionally focus effective relationship building strategies on students who seem hard to build relationships with Implicit bias impacts with whom and how people (including teachers) choose to build relationships • Students who have more positive relationships with teachers will be: • Have a better sense of belonging • More likely to live up to expectations • Less likely to perceive corrections as threatening Through more intensive practice with 1-2 students, relationship-building strategies will generalize to other students Students of color report the greatest mistrust and least connection with teachers Consulting with other teachers about relationship-building with students of concern will support EMR use Training a small group of teachers more intensively will diffuse EMR throughout the school

32. Primary student recipient of intentional relationship-building strategies

33. Use of EMR strategies and strength of relationshipsSelf-reported use within the prior 5 school days with one studentAll respondents: n=30 - 44 teachers at each timepoint

34. “How effective was EMR at building relationships?”

35. Number of office discipline referrals, SY 2016-2017 Third EMR training First EMR training Second EMR training School demographics (15-16) 55% White/Asian 45% Black, Hispanic, PI, Native, Multiracial

36. Climate Survey dataThe mean score for student self-report climate survey for belonging subscale items 2, 3, 4 and 5at three different time points (Time 1 = Spring 2016, Time 2= Fall 2016, Time 3 = Winter 2017)

37. Why or why wasn’t EMR effective? • “This has been my best year ever for relationship building” • “I thought I was good at building relationships before EMR, but the reality was that my relationships were suffering due to classroom management strategies and lack of 1:1 time with students. EMR helped me make more efficient use of the brief everyday interactions I have with individual students.” • “Establishing relationships makes my job easier… part of my job is very punitive, and if I have a relationship with the student they generally respond better as I enforce school rules.” • “There are so many outside influences this year that made my work too difficult to see any difference!” • “It just doesn’t seem to make a difference”

38. Why didn’t we find any effects on disproportionality in discipline? • At least four possible reasons: Faulty theories of change, faulty research approach, faulty “dosage”, other emerging school issues

39. Why didn’t we find any effects on disproportionality in discipline? • 1. Maybe the Theories of Change are wrong • Perhaps better relationships are not related to more positive behavior, less punitive discipline, and a decrease in implicit bias • Other research contradicts this • Maybe we picked the wrong factor • Perhaps relationship-building was not primary or should have been combined with another factor (such as effective behavior management strategies) • “Warm Demanders” requires both warmth (EMR) and structure (behavior management)

40. Why didn’t we find any effects on disproportionality in discipline? • 2. Perhaps our research approach did not detect changes that actually occurred • Teachers chose 1-2 students to focus on building relationships with, but ODR and climate data are from the entire school • Data on use of the EMR strategies was self-report and simple, and therefore may not be valid • Rates of participation in surveys, while consistent with other research, were lower than ideal

41. Why didn’t we find any effects on disproportionality in discipline? • 3. Perhaps “dosage” was not high enough • Only one year • Dosage for teachers • Few trainings (3), relatively short (1-2 hours), limited practice • Limited time for practice • Most common suggestions for sustainability: more reminders of EMR and strategies, more embeddedness of practice into school culture and language • Dosage for students • Teacher delivery of EMR was self-report and data focused on one student; it is unknown whether teachers generalized use to other students

42. Why didn’t we find any effects on disproportionality in discipline? • 4. Other emerging school issues • PBIS was beginning implementation during the same year • WMS was in the process of being split in order to provide students and staff for Meany middle school • Districtwide budget crises • Teachers and staff were not sure who would stay

43. Overall findings • Teachers, staff, and leadership are passionate about equity, but need evidence-based solutions and actionable recommendations. • The MENDR framework resonates with school staff as a practical and actionable approach. • There is a need to integrate skills-based (behavioral strategies) with attitude-based (cultural literacy, self-reflection) approaches at school and district levels • Train and hope doesn’t work. Essential aspects include: • Leadership buy-in and ongoing support • Implementation supports • Resources and time • Local cheerleaders and experts • Policy and practice change