Leveraging Comparative Analysis for Institutional Decision Making

1. Leveraging Comparative Analysis for Institutional Decision Making NJAIR Annual Conference April 17th, 2009 The College of New Jersey Robert Miller, Centenary College Chad May, The Richard Stockton College of NJ

2. Leveraging Comparative Analysis for Institutional Decision Making Benchmarking: What is it and who do we compare to?

3. What Forces are Driving the Need for Using Data for Institutional Decision Making • Internal • Finite Resources • Competition for Students • External • Increased Accountability • Increased Call for Transparency • Students as Consumers

4. What is Benchmarking? • Benchmarking is an ongoing, systematic process for measuring and comparing the work processes of one organization to those of another, by bringing an external focus to internal activities, functions, or operations (Kempner 1993). • Practitioners at colleges and universities have found that benchmarking helps overcome resistance to change, provides a structure for external evaluation, and creates new networks of communication between schools where valuable information and experiences can be shared (AACSB 1994). • Benchmarking is a positive process, and provides objective measurements for baselining (setting the initial values), goal-setting and improvement tracking, which can lead to dramatic innovations (Shafer & Coate 1992).

5. Why Benchmark? • Identification of “Best Practices” • Academic • Operational • Provide context for institutional data • Goal setting and measurement • Institutional Planning

6. How Do We Select the Institutions? Adapted from (Teeter & Brinkman 2003 in The Primer for Institutional Research, AIR)

7. Types of Peers • Definitional • Have similar identifiers expressing the essential nature of the institution • Informational • Hold practical knowledge of a desired process, outcome, accomplishment • Analytical • Provide realistic and practical benchmarks for internal and external review • Nonsensical • Have “…no meaning or [convey] no intelligible ideas”;”…absurd or contrary to good sense” *Webster’s Seventh New Collegiate Dictionary. G.&C. Merriam Company, Springfield, Massachusetts. 1967. Adapted from a presentation given at the NEAIR 2002 Conference

8. Reasons for PEER ANALYSIS ??? Compare Complain Assess Boast Improve Fund Evaluate

9. Some Common Peer Characteristics • Affiliation (Public vs. Private) • Carnegie Classification • Financials (endowment, tuition, assets, liabilities, expenses, revenue) • Enrollment and Staffing Levels • Selectivity (SAT, Acceptance rates) • Academic Programs (majors and degrees) • IPEDS PAS System can generate a comparison group automatically using the information above

10. Strategies of Developing Peer/Aspirant List • Data & Statistics & Judgment (Hybrid approach) • Data & Statistics (Cluster Analysis) • Data & Judgment (Threshold Approach) • Judgment (Panel Review) (Adapted from Teeter & Brinkman 2003 in The Primer for Institutional Research, AIR)

11. Sources of Comparison DataTo Help Identify Peers • Carnegie Foundation • National Student Clearinghouse- StudentTracker • Integrated Postsecondary Education Data System (IPEDS) Peer Analysis System, Dataset Cutting tool, Executive peer tool, etc. • http://nces.ed.gov/IPEDS/

12. Leveraging Comparative Analysis for Institutional Decision Making What Type of Comparison Data is Available?

13. Sources of Comparison DataRecruitment & Retention • Noel-Levitz National Enrollment Management Survey • Consortium for Student Retention Data Sharing • Data on retention rates & graduation rates • IPEDS Peer Analysis System • The College Board Admitted Student Questionnaire (ASQ and ASQ plus) • ACT, Inc

14. Sources of Comparison DataStudent Engagement • National Survey of Student Engagement (NSSE) • SPSS Syntax Files • UCLA’s Higher Education Research Institute Surveys • Cooperative Institutional Research Program (CIRP) • Your First College Year • College Senior Survey • Education Benchmarking Inc. (Resident Student Assessment, First Year Initiative Survey, etc.)

15. Sources of Comparison DataStudent Learning • Collegiate Learning Assessment (CLA) • ACT Collegiate Assessment of Academic Proficiency (CAAP) • ETS Measurement of Academic Proficiency and Progress

16. Sources of Comparison Data Financial Operations • NACUBO Endowment Study • NACUBO Tuition Discounting Study • Voluntary Support of Education • Fundraising results • IPEDS Finance Survey (Peer Analysis System) • Guidestar (990 data for non-profits)

17. Sources of Comparison DataSatisfaction • Student Satisfaction Surveys • Noel-Levitz Student Satisfaction Inventory • ACT Survey of Student Opinions • In-house surveys • Employee Satisfaction Surveys • HERI Faculty Survey • Harvard University (Collaborative On Academic Careers in Higher Education survey) • Alumni Surveys • ACT Alumni Survey and Alumni Outcomes Survey

18. Analytical Tools (Software/Services) • Proprietary Software • MS Excel • SPSS/SAS/STATA and other Stat packages • Rapid Insight Analytics / Data Integration • Tableau- Visual Analysis Software • Proprietary Services (Internet based applications) • AGB Benchmarking Service • Peer Analysis System (PAS) • Dataset Cutting Tool • Executive Peer Analysis Tool (create your own data feedback report) • CUPA- Data on Demand Services • Voluntary Support of Education (CAE)- Data tool • AAUP Faculty Compensation data published in Academe • JMA Associates

19. Big PictureInitiatives/Projects Available • Council for Independent Colleges • CIC KIT • This tool provides information relating to enrollment, staffing, admissions, and financial aid. • Key feature: allows you to conduct comparative analysis using schools with similar financial resources. • Sample of CIC KIT Tool • http://www.cic.edu/projects_services/infoservices/kit.asp • CIC FIT Tool • While the KIT tool provides traditional indicators such as acceptance rate, yield rate, and faculty counts, the FIT tool provides detailed financial comparisons • Ratio analysis for overall institutional health • Sample of CIC FIT Tool • http://www.cic.edu/projects_services/infoservices/fit/index.asp

20. Leveraging Comparative Analysis for Institutional Decision Making Pseudo Case Study

21. Comparative Data for Internal Analysis: Case Study Example • Using comparative data to answer institutional specific questions • Common Question for IR professionals • Who are students choosing over us and who are students choosing us over?(i.e. the win/loss question)

22. Using Admissions and FASFA Data • Admissions interview data • Extraction of enrolled and not enrolled students • Analysis of fields to identify what other institutions students sent their FAFSA data to- they can list up to six • Send batch files to the National Student Clearinghouse using the StudentTracker service • Return file from NSC shows enrollment history of your non-enrolling admitted students • Match NSC return file data to other institutional data

23. Internal Data Combined with Student Tracker

24. Example Output- “Win/Loss” Ratio

25. Hypothesis • Are institutional aid policies in line with other institutions? • Is there a significant difference in EFC of enrolling and non enrolling business students? • Internal analysis • Is there a significant difference in the institutional grant aid awarded to enrolling and non enrolling business students? • Internal analysis • How does grant aid compare between our institution and other institutions? • IPEDS PAS

26. Average Aid by Institution

27. Leveraging Comparative Analysis for Institutional Decision Making Integrating comparative Analysis with planning

28. Reporting Comparative Data • Standard comparative reports • Externally processed • Faculty Compensation Report (Academe) • IPEDS Feedback Report • University of Delaware Study of Instructional Costs and Productivity • NSSE, HERI, and other survey instruments • Internally Processed • Dashboards and/or report of Key Indicators report(s) • Competitors report and Tuition/Fee Comparison report • Other IR reports • Ad-hoc comparative reports • Retention- where are our students going? • Graduation Rate Study • Internal analysis of survey data (comparison of student satisfaction)

29. Yellow Bars- Represent Aspiration Institutions Dark Blue Bars- Represent Peer-Like Institutions Aqua Bars- Represent Peer- Below Institutions Orange line across represents the target institution

30. Example Institutional Dashboard Summary Dashboard Fall 2008 Total Gifts Gifts to Capital & Endwmnt Full-Time UG’s FR Applicants Endowment/Reserves $ $ 15,000 15,000 10,000 20,000 UG Alumni Participation Number $1,000 donors Part-Time UG’s % FR Acceptances Return on Endowment /Reserves Portfolio 65% 2,000 60% % % Graduate Students Faculty Gross Cost to raise $1 FY 2006 Yield (% Enrolled) Spending Rate UG Student/Faculty Ratio 1,000 (fall) 40% 37.0% % % UGs in-State H.S. Avg. Rank Student Aid Unrestricted Annual Fund Gifts (change) Full-Time Faculty 97.6% Discount Rate 74%ile 6-year Graduation Rate % Avg. SAT- Regular UG Class Size >=30 Inst. Financial Aid as % of Operating Budget 68% 60% Positive Variance 1017 1250 Diversity Enrollment % UG Class Size <10 Another Indicator 29% 21% Student Revenue Reliance % of FT Students w/ Financial Need First-year Retention Taught by FT Faculty % 60% 74.7% 81.6% Debt coverage ratio SR Stdnt Satisfaction Importance of Change: Green= better Red= worse Yellow= neutral KEY Change: Higher Lower None % FT Faculty W/ Term. Deg. 85% 85% Plant Reinvestment Rate (excludes current construction projects) % Resident Stds. (FT) % FT Faculty w/ Tenure 1.5-2% % Benchmark 65% Current 55% 50% 51.4%

31. Conclusion/Discussion • Comparative Analysis provides context for institutional data with respect to decision making/planning/and assessment. • There is a significant amount of data already available. Much of which is almost ready-made for dissemination. • If you do not do the comparative analysis someone else will. (students, government, parents, etc.)