Discussion with the Project Compass Evaluation Team: Early Thoughts on Institutional Change

1. Discussion with the Project Compass Evaluation Team:Early Thoughts on Institutional Change Susan Lanspery and Tom Piñeros-Shields Center for Youth and Communities Heller School for Social Policy and Management Brandeis University

2. Overview of Presentation • Purpose: Assessing the Change Process in Project Compass • Our Approach • Early Thoughts about Organizational Change, Institutionalization, and Student Retention and Success • Discussion Questions

3. Assessing the Change Process in Project Compass • Organizational Change: What program models have institutions adopted? How effectively have the programs been implemented, and what elements have contributed to the implementation? • Institutionalization: To what extent is it likely that these programs and practices continue after the funding period (i.e., what is the level of institutionalization at the end of four years)? • Student Retention and Success: What is the impact of Project Compass on the practices of admissions, advising, and other student support services? What is the broader impact on campus climate? • Attributing Systems Change to Project Compass: What is the impact of the intermediary with regard to the implementation and sustainability of the Project?

4. Our Approach • We are following an asset-based or appreciative inquiry approach, seeking to cite positive changes and experiences at specific campuses while noting challenges in general terms. • Data collection methods include: • Interviews with Community of Practice Members at all four campuses. • Review of all logic models and reports. • Quarterly meetings and conference calls with evaluation research group. • Current interim report includes mini-case studies and a summary of themes that address four primary questions.

5. Early Thoughts About Organizational Change (1) What program models have institutions adopted? • BSU adopted a ‘non-targeted’ change strategy building upon structures and collaborative systems that were in place or developing before the project began, and relies upon the diffusion of language, practices, and pedagogies to improve support for FGLI students. • ECSU developed a change strategy based upon a campus-wide strategic plan that expanded and centralized student advising/ academic support, with special attention to a Targeted Advising Cohort. • LSC built campus-wide collaborative relationships and structures for communication, specific interventions for FFMI students, and community partnerships with K-12 systems. • UMPI emphasized cultural awareness of and renewed relationships with First Nation tribes/bands and a shift in campus systems/resources to better support student success.

6. Early Thoughts About Organizational Change (2) How effectively have the programs been implemented? • Cannot yet comment definitively about effectiveness …but • Early indicators suggest that programs are working well within the specific contexts of each campus. • In general, on all campuses we have seen: • Increased use – and appreciation – of data and evidence to guide decisions, leading towards a culture of evidence. • More collaboration and/or communication across the institution. • Identifying strategies for greater campus-wide focus on academic support for students (whether a targeted group or not).

7. Early Thoughts About Organizational Change (3) What elements have contributed to the implementation? • Leadership at the institution (formal). • Community of Practice or other PC initiative leadership (formal and informal). • Intentional communications strategies. • New or enhanced cross-institutional roles and conversations – “out of the silos.” • New external community partnerships. • Expanded use of institutional research to create a culture of evidence.

8. Early Thoughts About Institutionalization (or Sustainability) To what extent is it likely that these programs and practices continue after the funding period? • Explicit links between Compass activities and campus strategic plan. • Formally/permanently committing to new staff positions. • Leveraging opportunities/resources by building upon Project Compass work.

9. Early Thoughts About Student Retention and Success (1) What is the impact of Project Compass on the practices of admissions, advising, and other student support services? Campus self-reports assess the following : • Retention • Academic Performance • Awareness/Use of Services • Obstacles and Challenges to Students • Program Satisfaction

10. Early Thoughts About Student Retention and Success (2) What is the broader impact on campus climate? • Our assessment of impact on campus climate is ongoing, but… • UMPI and BSU are conducting systematic assessments (“Cultural Audits”) of their campuses. • ECSU and LSC share perceptions that increased collaboration among campus units positively influences campus climate.

11. Discussion Questions • Do these systems change themes resonate? What works, what doesn’t work? • What should we (the Brandeis team) focus on during this year’s site visits – numbers and stories – to understand more about … • Institutional change. • Sustainability for Project Compass efforts. • How each campus is assessing culture shifts/campus climate. • How each campus is assessing progress toward logic model outcomes. • Other ideas? • What would you tell a campus that is about to begin implementing a program like Project Compass?