Benchmarking in Community Colleges: Status of Two National Projects

1. Benchmarking in Community Colleges: Status of Two National Projects Jeffrey A. Seybert Director, Research, Evaluation, and Instructional Development Johnson County Community College George Malo Associate Vice Chancellor for Research and Assessment Tennessee Board of Regents John D. Porter Associate Provost The State University of New York

2. The Kansas Study Community College instructional costs and productivity Modeled on the Delaware Study Collects, analyzes, and reports data at the discipline level The National Community College Benchmark Project Involves a wide array of student outcomes, access, workforce development, faculty/staff, human resources, and finance variables Collects, analyzes, and reports data at the institutional level Two National Benchmarking Projects

3. The Kansas Study • Supported by a three-year, $282,000 grant from FIPSE (USDE). • Colleges will be able to analyze faculty workload and instructional cost at the academic discipline level of analysis.

4. Summer 2002 FIPSE project approval and grant award Fall 2002-Fall 2003 Advisory committee identifies data elements, designs processes, and conducts two pilot studies Fall 2004 Aggregate reports distributed; website opened for peer comparisons 2004 Year 1 project implementation – 50 institutions provided data Year 2 - 67 institutions participated 2006 Year 3 Kansas Study History

5. How Kansas Study Works • Data Collection • Excel Spreadsheets distributed electronically • Data Verification: • Missing data and logical errors • Partial Data OK (min. 10 disciplines) • Confidentiality assured • Annual Reports • National Norms and Institutional Data • Access to Kansas Study Website for Peer Comparisons

6. Kansas Study Timeline February 1 Data Collection Starts May 15 Data Verification Process Initiated June 18 Participant Institutional Data Due July 15 Data Verification Reports Sent July 5 Data Analyses Begin Early Fall Results Available; Database Opened for Peer Comparisons/ Benchmarking

7. Web Site • Kansas Study Website (www.kansasstudy.org) • Public Information • General Information • Enrollment Form • Sample Data Collection Template • Sample Report Tables • Advisory Committee • Participating Institutions • Information Available to Participants Only • Log In & Password • National Norms by Discipline • Peer Comparisons

8. Benchmarking Instructional Costs and Productivity: How a System and Campus Use the Kansas Cost Study George Malo Assistant Vice Chancellor for Research and Assessment Tennessee Board of Regents

9. Tennessee Board of Regents The Kansas Cost Model • Purpose • Colleges will be able to analyze faculty workload and instructional cost at the academic discipline level of analysis • Provides comparative data important to accountability processes and decision making at both the system and institution levels • Participants • Community Colleges • Context • Modeled in part on the Delaware cost model for universities

10. Tennessee Board of Regents TBR Context • Why do we participate? • Part of TBR and State initiatives • Defining Our Future • Strategic planning process • Accountability reporting • Performance funding • Cost model replaces Board’s former cost study • Opportunity for national comparisons as well as a management tool for campuses • Provides consistency across system institutions • Useful for system policy and management decisions

11. Tennessee Board of Regents Strategic Planning • Provides benchmarks • Annual monitoring of key program variables • Documentation of activities • Use in Decision-Making • Part of presidential evaluations

12. Tennessee Board of Regents Performance Funding • Five points awarded as part of assessment standard • All 13 community colleges in TBR system must participate • Report on 4 key indicators (to be discussed later in presentation) • Submit a report providing evidence of the usage of the Kansas model for institutional planning and improvement.

13. Tennessee Board of Regents Campus Uses of Kansas Study • Program Review or Academic Audit • Look at high risk or outlier programs • Look at Peer Costs • Staffing & tenure decisions • Problem–solving tool • SACS accreditation documentation

14. Tennessee Board of Regents Caveats to Avoid Misuse • Tool for institutional decision-making • Support credible case-making and informed decision-making • Must be used as trend data • Should not be used for inferences of an institution as a whole • Prerequisite for assessing the adequacy of accountability

15. Tennessee Board of Regents System Level Ad Hoc Committee • Institutional input through Academic Affairs committee for appropriate use of data • Adoption of key indicators as standard for framing instructional productivity and effectiveness reports • Development of common questions that would lead institutions to evaluate their decisions

16. Tennessee Board of Regents System Level Key Indicators • FTE students taught per FTE instructional faculty by discipline • Student credit hours per FTE faculty as a percentage of national norm by discipline • Percentage student credit hours taught by full-time faculty

17. Tennessee Board of Regents Comparison Group Selection • Each institution will construct its peer groups according to three standards, two for System use and the third for institutional use only • System reporting as an aggregate • System reporting per discipline • At the institution level, each discipline may select peers

18. Tennessee Board of Regents Evidence of Accountability/Productivity • For each key indicator, the TBR System adopted questions to guide institutions in the analysis of their own data • What significant changes can be detected over the last three years for the indicator? • How does this three-year profile for the indicator compare to that of institutional peers by CIP? • What factors have contributed to the changes at your institution around the key indicator? • Are you satisfied? Why, and if not, how do you plan to make any alterations to adjust key indicator?

19. Tennessee Board of Regents Case Making – A Central Goal • Data must be used for responsible decision making • Can the institution make a case, from its analysis of the allocation of faculty, that it is moving toward improvement in instructional management? • Can the institution make the case that it is effectively using its faculty ? • Can the institution make the case that it is moving toward improvement in contributions to the institution, system, state, students, or the public? • Do these contributions reflect a responsible use of resources ?

20. Tennessee Board of Regents System Wide Assistance • Programming for data collection • Programming for analysis of data • Templates for reporting of data • Revisions to/formulation of policies and guidelines

21. Tennessee Board of Regents Campus Uses of Kansas Study • Documenting accreditation compliance • Planning institutional change • Predicting academic/financial impact

22. Tennessee Board of Regents Accreditation • Institutional Effectiveness • Adequate Faculty • Sound financial base and adequate resources

23. Annual Program Documentation

24. Document Staffing by Program

25. Document Cost by Program

26. New Program Planning

27. The National Community College Benchmark Project • Involves a wide array of student outcomes, access, workforce development, faculty/staff, human resources, and finance variables • Collects, analyzes, and reports data at the institutional level

28. Purposes • To collect and report community college benchmark data on a national basis • To provide data for comparisons and benchmarks of instructional, workforce-development, and other community college activities

29. NCCBP History 2003 Project Designed and Piloted 2004 First year implementation; interactive project website designed and launched; 110 institutions participated - SUNY System (30 Colleges) - TN System (13 Colleges) 2005 Second year implementation; 113 institutions participated - SUNY System (30 Colleges) - TN System (13 Colleges) - PA Colleges (13 of 14) 2006 Third year implementation; enrollment opened in Feb. - SUNY System (30 Colleges) - TN System (13 Colleges) - PA Colleges (14 Colleges) - FL System (28 Colleges)

30. How NCCBP Works • Data collection • Excel spreadsheets distributed electronically • Data verification: Missing data and logical errors • Partial data OK; no peer comparison for missing data • Confidentiality assured • Cost: $1,000/year per institution • Annual reports • Aggregate data delivered electronically • Access to NCCBP Web site for peer comparisons • Website: www.NCCBP.org

31. Data-collection Form FORM 4: Credit Students Who Enrolled Next Term and Next Fall Column 1Enter unduplicated total credit students (including those who withdrew from all courses) at the end of the fall 2003 term. Do not I include high school students. Column 2Enter total students from Column 1 who graduated or completed certificates before the next (spring 2004) term. Column 3Enter total students from Column 1 who enrolled in the next (spring 2004) term. Column 4Column 3 / (Column 1 - Column 2) Column 5Enter total students from Column 1 who graduated or completed certificates before next fall (fall 2004) term. Include graduates and completers in Column 2. Column 6Enter total students from Column 1 who enrolled in the next fall (fall 2004) term. Column 7Column 6 / (Column 1 - Column 5)

33. Completion & Transfer Rates Persistence Rates Transfer Student Performance Student Satisfaction Student Performance Measures Career Preparation Academic Success Access & Participation Market Penetration Workforce Development Section Size, SF Ratio, Faculty Load Student Services Staff HR Statistics Instructional & Professional Development Costs Benchmark Categories

34. 2006 Timeline March Data collection begins. May Data-collection instruments are due. June Data confirmation reports are distributed. July Data updates are due. September Aggregate reports are distributed. Web site is opened for peer comparisons.

35. 2004 Participants

36. Campus Environment Institution Type Institutional Control Academic Calendar Credit Enrollment Minority Students Percent State Revenue Operating Budget Faculty Unionized Service Area Population Unemployment Rate Household Income Service Area Percent Minority College Characteristics

37. National Community College Benchmark Project: A System/State Perspective John D. Porter Associate Provost The State University of New York

38. NCCBP: A Valuable Resource for Systems & States • NY’s CCs are funded based on annual full-time student equivalents (FTE) • CCs need to benchmark their operations to maintain & expand state support • CC’s are as complex as research universities, which is not understood by most decision makers • NCCBP fills a critical void

39. SUNY’s Community Colleges • SUNY’s community colleges enroll 208,374 students • 50.3% of SUNY’s overall enrollment • Campuses range in size from 21,000 to 1,500 • Located throughout New York Stake, including New York City • One CC awards bachelor and master degrees (FIT) • These institutions have every conceivable governance/funding arrangement

40. SUNY Support of NCCBP • SUNY’s benchmarking has here-to-fore focused on “intra” measures • NCCBP offers the potential to benchmark against true peers and other states • For the past three years, SUNY has encouraged campus participation by paying the subscription fee • This year, all 30 community colleges will participate in NCBBP • SUNY’s hope is that other states and systems will see the value of this project and participate

41. NCCBP: A Valuable Resource for Systems & States • NCCBP has generally been conceived as a tool for campuses • Systems/States need this type of resource, since most community colleges are funded based on enrollment • SUNY requires CCs to plan enrollments 5 years into the future; also update the institutional mission every five years • NCCBP has the potential for developing reports tailored to the needs of Systems and States

42. Issues? • Participation needs to reach a critical mass – 300 institutions? • Gaining support for NCCBP on campus (some don’t want to be compared) • Funding – shifting cost to the campus at some point in the future • Accuracy/quality of data? How best to achieve? • Important that NCCBP keeps the cost of participating low

43. National Community College Benchmarking Project George Malo Associate Vice Chancellor for Research and Assessment Tennessee Board of Regents

44. Tennessee Board of Regents Uses of NCCBP • Strategic planning • Performance funding • Documenting accreditation • Policy development/analysis/evaluation

45. Where Does MCC Excel?Core Course Success

46. Example of Performance Funding Indicators

47. Outcomes of Educational ProgramsCareer Program Completers

48. Tennessee Board of Regents Policy Questions

49. Questions