The Chancellor's Office Research Agenda

1. The Chancellor's Office Research Agenda RP/CISOA Conference April 2009

2. Presenters • Willard Hom, Dean/Director • Alice van Ommeren, Research and Planning Staff • LeAnn Fong-Batkin, Research and Planning Staff

3. Objectives • To inform researchers and IT staff about specific future research efforts in the Chancellor’s Office • To gather comments related to the research agenda • Preview the research agenda prior to public release

4. Background • Board of Governors adopted the System Strategic Plan in 2006. http://strategicplan.cccco.edu/ • Research agenda addresses goal D3 of the Strategic Plan, Analytical Capacity for Measuring Success. • Prior work on organizing research topics was in the 1990s.

5. Goal of the Research Agenda • Research agenda helps prioritize projects. • Research agenda promotes coverage of topics that concern the wide array of system stakeholders. • Research agenda increases transparency.

6. Research Agendas • A research agenda structures a prioritization process so that an organization can systematically weigh factors in its consideration of research projects and activities. • Research agendas are used in a wide variety of disciplines

7. Creation of the Research Agenda • Two-day meeting in October 2008 • Used an external facilitator

8. Participants • Community college representatives, including chief executive officers, chief information systems officers, and researchers • Academic Senate • RP Group • Department of Finance • Legislative Analyst’s Office • Cal-PASS • Community College League of California • Foundation for California Community Colleges • Chancellor’s Office Staff

9. Process • Discussed value of creating a research agenda • Identified external opportunities and challenges • Identified internal strengths and weaknesses • Discussed potential research projects • Developed criteria to evaluate each project • Prioritized the projects

10. Value of Our Research Agenda • Main ideas guiding our discussion • Focus Resources via Prioritization • Manage Expectations • Provide Leadership

11. External Opportunities* • Culture of evidence • Economy • Changing role of researchers • Change in technology • Increased interest in CCs from external parties • New leadership and new relationships • Changing demographics • New research opportunities *as reported by participants

12. External Challenges* • Restrictions on access to data • Data quality, data coverage, and research methods • Research resources (expertise) • Dynamic environment *as reported by participants

13. Internal Strengths* • System alignment with certain entities • Collaborative approach to designing agenda • Availability of data elements • Ability to communicate and teach how to analyze and use the data *as reported by participants

14. Internal Weaknesses* • Leadership and staff turnover • Capacity • State and local “silos” • Funding pressures • Linking research to instruction • Too many areas to research • No consequences/incentives for bad data *as reported by participants

15. Potential Research Projects Considered: Strategy A • Strategy A: College Awareness and Access • Financial aid study (fees, role of fin. aid, practices) • Access study (Improve SEARS, GIS, time series study) • Program evaluations (EOPS, Financial Aid, EAP) • Distance education delivery model analysis • Classification Study • Other • Concurrent enrollment • Noncredit to Credit Transition • University of Phoenix phenomena

16. Potential Research Projects Considered: Strategy B • Strategy B: Student Success and Readiness • Transfer study (disaggregate transfer population, explore/identify transfers) • Analysis of course factors (distance ed, learning communities, scheduling) • Course placement recommendation collection (tie to CB 21 revision)

17. Potential Research Projects Considered: Strategy C • Strategy C: Partnerships for Economic and Workforce Development • Data integration (matching supply with demand) • Employment outcomes (longitudinal study) • Curriculum development (improve response time) • Partnership academies • CTE programs (measure costs and levels of success)

18. Potential Research Projects Considered: Strategy D • Strategy D: System Effectiveness • ARCC—integrate equity/national peering/benchmarking • Student learning objectives (impact on student success) • Swirl study • Study of professional development (what is being done and how)

19. Potential Research Projects Considered: Strategy E • Strategy E: Resource Development • Fee Policy (revenue vs. affordability; affect on student access and success; 50% law, distribution of funds, SB 361) • Cost index • Program analysis (break even costs for certain programs) • Develop inventory of existing and proposed studies

20. Potential Research Projects Considered: Others • Establish common guidelines for conducting studies • Performance-based funding • Expand research methods and reporting tools • Super model for forecasting

21. Selection Criteria • Is the project “doable”? Do we have the data? • Topic addresses multiple goals in the Strategic Plan • Will the study’s output move the system forward? • Will the study impact • Political leadership • Students • Economy/workforce • Can we use prior studies to leverage this study? • How much new information will the study provide?

22. Prioritization Process • We categorized the projects into: • Quick wins (high impact, short-term timeframe, 0-12 months) • Stars (high impact, long-term timeframe (12-24 months) • Building Blocks (low impact, short term timeframe, 0-12 months) • Back Burners (Low impact, long-term timeframe, 12-24 months) • Then, each participant voted for their choice of projects

23. Voting Process • Used the “facilitated decision making” method, also known as the “10-4” method. • Each participant received 10 dots. • The participant placed 4 dots on the project that has the highest priority for the individual; the remaining 6 dots are placed elsewhere.

24. Definition of Research Projects • Participants divided into three groups to discuss the following for the 9 projects that received the most votes: • Scope • Objectives • Benefits • Action Steps

25. And the winners are… • Project 1: Course Section Factors • Objectives: • Conduct program evaluations • Create infrastructure • Operationalize definitions and magnitude • Project 2: Course Placement Recommendations • Objectives: • Collect course placement recommendations and test scores

26. And the winners are… • Project 3: Inventory of Existing Studies • Objective: Create warehouse of existing studies, including program evaluations, financial aid, internal and external studies • Project 4: GIS Data Analysis • Objective: • Enrollment management • Program and service planning • Bond planning analysis

27. And the winners are… • Project 5: Employment Outcomes • Objectives: • Classification of programs • Expansion of outcome data (longer tracking) • Project 6: Evaluate CTE Programs • Objectives: • Start with evaluation of Nursing programs • Establish methodology for determining cost and performance indicators

28. And the winners are… • Project 7: Integrate equity data into ARCC • Objective: • Make colleges more aware of equity issues • Project 8: Expansion of Student Attributes (SEARS Survey) • Objective: • Conduct research using student attributes as related to success

29. And the winners are… • Project 9: Fees, Financial Aid, and Affordability • Objective: • Evaluate other states’ fees, revenue, financial aid, and participation • Information will inform state policy and budget discussions • Optimize student access and success

30. Caveats and Concerns • Agenda is system level research • Need literature reviews • Did not identify projects to build research capacity • Need to support classroom level research • Projects require technical assistance • Message and marketing of studies is important • Rigor and utility analysis in design

31. Retreat Evaluation • What went well: • Representation of a wide variety of CC and government organizations • Process and facilitation • Organization of event • Changes for next time: • Need small college representation • Need more rigorous guidelines and input about research unit’s priorities • Need a mechanism to connect with external research groups

32. Applications • Allows us to make decisions about how the Chancellor’s Office will allocate its scarce resources • Achieve additional efficiency in research-related activities • Help external stakeholders identify projects to pursue • Clarification of unmet research needs

33. Limitations of the Agenda • Narrow scope • Studies can be done with limited costs • Omits studies that require • Collection of new data • Use of field experiments • Large-scale studies • Extensive literature review not conducted; may have missed existing studies • Does not state how research capacity can be expanded

34. Future Action • We will attempt to do studies as time/resources permit • We will work with external researchers to see if they can partner with us to perform the study • We will periodically revisit the agenda • Specific features of research agenda need refinement

35. Questions? • Contact Willard Hom at (916) 327-5887 or whom@cccco.edu