They Said it Couldn t Be Done Data Driven System Change in a Large Urban School District

1. They Said it Couldn�t Be Done!Data Driven System Change in a Large Urban School District Kevin P. Dwyer, MA, NCSP David M. Osher, Ph.D.

2. Guiding Principles Complex problems can be scientifically identified and remedied using best-practices The change process requires sustained strategic planning, local ownership, and takes multiple years Ongoing Quality improvement with a focus on fidelity and results-based outcomes need to be in place Universal Prevention is foundational

3. Guiding Principles Universal social emotional learning should be integrated into academics Conditions for Learning are part of each school�s Annual Yearly Progress measures Early and intensive interventions developed & monitored help inform school-wide prevention activities

4. Background

5. Overview History Success Tech Response Hardware �Humanware� Approach Independent Audit Commissioned by CMSD and Mayor�s Office Gap Analysis Recommendation of Evidenced Based Programs and Practices to Adopt

6. Overview Ultimate purposes of audit: Enhanced Connectedness and School Bonding Enhanced Mental Wellness, Discipline, & Safety Reduced Emotional and Behavioral Problems, Disorder, and Risk of Violence Build Capacity to Efficiently : Intervene Early Provide Intensive Support Improved Academic Achievement Improved Community Health and Safety.

7. Methodology

8. Methodology Analyze Extent Data Reports Evaluations Archival Data Key informant interviews Stakeholder Outreach Case studies Stratified Random Selection of Schools Stratified Random Selection of Classes to Observe and Teachers, Students, Staff, and Families to Interview Expert Informants Observations Examination and Contextualization of School Archival Data.

9. Methodology Validation Activities Statistical Analyses Focus Groups Epidemiological Survey of Conditions for Learning Youth Risk Behavior Survey.

10. Findings

11. Problems: Overarching Themes Agency- and Adult- Driven Planning & Decision Making In Schools In Community Lack of Systemic Approaches to: Prevention and Treatment Targeting Resources Communication and Transparency Integrating Services in Schools Managing and Supporting Change Getting to Excellence.

12. Problems: Overarching Themes Lack of Monitoring and Accountability Lack of Effective Communication Between and Among Agencies, Police and Schools Cultural, Structural, and Historical Disconnects Between Agencies and Schools Fragmentation of Services Many School Staff Don�t Know How to Access Services Many Agency Staff Don�t Know How to Access Schools Lack of Communication Across Disciplines Need for Clear, Systematic Learning Support System Lack of Common Metrics and Data Systems.

13. Problems: Overarching Themes Weak Implementation of Innovative Efforts Failure to Systematize Change in Most

14. Problems: District Level Inconsistency of Services Across Schools Services May Not be Where the Need is Some Regions Have a lot of Services, Some Have Very Few Poor Data Systems Fragmentation of Pupil Services Weak Professional Development.

15. Recommendations and Examples of What Has Been Done

16. Reinforce Structural Supports Leadership Support & Collaboration Chief Executive Officer & Mayor Cleveland Teachers Union Community agencies, NGOs & foundations Internal Management Staff Distributed Leadership at a District Level Chief Academic Officer & Deputy Chief for Innovation Executive Team (Deputy Chief, Managers, Principals & CTU) Manager Leads � each activity Workgroups � select best-practice activities

17. Humanware Strategic Plan Improve capacity to plan and deploy Improve policies, procedures and practices Improve school climate Provide social emotional learning Develop early warning & response system Enhance school-agency collaboration Enhance family engagement Provide focused professional development Focus funding & resources Develop quality standards & ongoing improvement

18. Recommendations Implement Ten Strategies Implemented Developmentally Over 5 Years Employ 3-tiered Approach to Intervention Positive Behavioral Approaches, Social Emotional Learning, and Student Connectedness Build Structures to Support Change District Level Provide Appropriate Professional Development and Coaching Support Improve and Monitor Skill Set and Knowledge of ALL (Paraprofessionals) Target Resources That Go To Schools Enhance Collaboration between School Security and School Staff Monitor, Respond to, and Support the Use of Conditions for Learning Data Identify a Small Set of Programs and Strategies that the District will use

19. Work at Three Levels

20. Cleveland AcademicImprovementModel

21. Linking Student Support & School Improvement

22. Humanware ImplementationAccomplishmentsYear 3

23. Accomplishments PATHS being taught from pre-K thru Grade 5 in all schools � Coaches supporting instruction. All teachers trained & staff aware SEL program selected for grades 6 thru 8 for Sept. High schools planning possible class meetings & advisories Student Support Teams with administrator, counselor, or psychologist and selected teacher meeting weekly in all schools � self & external review for fidelity All staff trained in early warning signs & bullying prevention

24. Accomplishments Planning Center in every school (instead of in-school suspension) for positive behavioral and academic support � weekly consultation with school psychologist, social worker Collaboration of District and Teacher�s Union regarding Humanware All schools connected to clinical mental health services stressing best practices & program coordination and alignment Programs connected to families. PATHS parent workshops � training up family liaisons

25. Accomplishments Attendance Target 11 initiative targets interventions rather than punitive measures. Improved attendance. Check-and-Connect being explored Quality standards guidelines introduced to and used for all in-school & services connected to schools Electronic data systems track SST intervention effectiveness Schools periodically examine their quality improvement in implementation of PATHS, SSTs, Planning Centers & other activities and relate to Conditions for Learning data.

26. Challenges Consistently high quality implementation Penetration of Ideas Implementation Quality Culture Change (or lack thereof) around positive behavioral approaches Implementation timelines for change Focus on the impact of school climate

27. Supporting Conditions for Learning

28. Social and Emotional Conditions for Being Off Track

29. Safety and Statewide Tests

30. Impact of Safety on Math Achievement: National Surveys NAEP (National Assessment of Educational Progress) At all three grade levels students in schools reporting an above-average climate had higher mean NAEP mathematics scale scores than students in schools reporting average or below-average school climate on the same measure (Greenberg, Skidmore, Rhodes, & Nesbitt, 2001) NELS (The National Education Longitudinal Study of 1988) Adolescent perceptions of connections with teachers predicted academic growth in mathematics (Gregory & Weinstein, 2004)

31. Why Are Conditions for Learning Important? Maximizing the amount of time that students really attend to learning E.g., working memory (Davidson, 2002) Maximizing the opportunity for the teacher to: Concentrate and differentiate Teach in the Zone of Proximal Development (Vygotsky, 1978) Personalizing instruction Scaffolding learning and support

34. Social Emotional Learning

35. Evidence of Success with SEL 23% increase in skills 9% improvement in attitudes about self,others, and school 9% improvement in prosocial behavior 9% reduction in problem behaviors 10% reduction in emotional distress 11% increase in standardized achievement test scores (math and reading) . The most compelling findings come from a 2008 Meta-analysis (study of studies) conducted by CASEL and Loyola University. This review was the largest, most scientifically rigorous review of research ever done on interventions that promote the social and emotional development of students between the ages of 5 and 18. The results from the school-based study are based on 207 studies of programs involving 288,000 students from rural, suburban and urban areas. In this study, researchers used statistical techniques to summarize the findings across all the studies and found a broad range of benefits for students. Again we see that there is improvement in outcomes related to student attitudes, behaviors, and performance, as well as improvement in skills and emotional distress. 23% improvement in social and emotional skills, e.g., self-awareness, self-management, etc. 9% improvement in attitudes about self, others, and school, including higher academic motivation, stronger bonding with school and teachers, and more positive attitudes about school 9% improvement in prosocial school and classroom behavior, e.g., following classroom rules 9% decrease in conduct problems (behavior), such as classroom misbehavior and aggression 10% decrease in emotional distress, such as anxiety and depression 11 % improvement in academic performance, e.g. standardized achievement test scores There were also findings related to implementation. Students only achieved significant gains across all of the six outcome areas when: The program was implemented with fidelity to the program design. Teachers were the primary program deliverers. Programs were characterized as S.A.F.E. S- Sequenced set of activities that teach skills in a systematic way A- Active forms of learning, e.g. role play F- Focused attention on SEL, e.g. at least one component of the program was focused on SEL E- Explicit learning objectives related to specific social and emotional skills were included It is important to note that while SEL programs take time out of the school day, they do not detract from student academic performance. As these findings show, as academics were improving, so were feelings about self, other and school; classroom behavior; and emotional problems. The most compelling findings come from a 2008 Meta-analysis (study of studies) conducted by CASEL and Loyola University. This review was the largest, most scientifically rigorous review of research ever done on interventions that promote the social and emotional development of students between the ages of 5 and 18. The results from the school-based study are based on 207 studies of programs involving 288,000 students from rural, suburban and urban areas. In this study, researchers used statistical techniques to summarize the findings across all the studies and found a broad range of benefits for students. Again we see that there is improvement in outcomes related to student attitudes, behaviors, and performance, as well as improvement in skills and emotional distress. 23% improvement in social and emotional skills, e.g., self-awareness, self-management, etc. 9% improvement in attitudes about self, others, and school, including higher academic motivation, stronger bonding with school and teachers, and more positive attitudes about school 9% improvement in prosocial school and classroom behavior, e.g., following classroom rules 9% decrease in conduct problems (behavior), such as classroom misbehavior and aggression 10% decrease in emotional distress, such as anxiety and depression 11 % improvement in academic performance, e.g. standardized achievement test scores There were also findings related to implementation. Students only achieved significant gains across all of the six outcome areas when: The program was implemented with fidelity to the program design. Teachers were the primary program deliverers. Programs were characterized as S.A.F.E. S- Sequenced set of activities that teach skills in a systematic way A- Active forms of learning, e.g. role play F- Focused attention on SEL, e.g. at least one component of the program was focused on SEL E- Explicit learning objectives related to specific social and emotional skills were included It is important to note that while SEL programs take time out of the school day, they do not detract from student academic performance. As these findings show, as academics were improving, so were feelings about self, other and school; classroom behavior; and emotional problems.

36. What Works Clearinghouse

37. Logic Model for Safe and Successful Schools

38. More questions ? More ideas!