2014 ARPD General & Pre-Professional Program Review

1. 2014 ARPD General & Pre-Professional Program Review Data Description ARPD Process Rev. 12Dec14

2. Covered in today’s session… • Quick review of Program and Unit Review website • Navigating the Annual Reports of Program Data (ARPD) website • What are we doing to improve our current program/unit review processes? • Process owners • Terminology used in the review • Data definitions • Joni and/or Joyce will be covering: Any templates used, use of PATH tool, process used on campus, comprehensive reviews, deadlines, timelines, etc.

3. Programs Covered in this Presentation • The following General and Pre‐Professional Education programs: • Liberal Arts Program • Hawaiian Studies Program • Natural Sciences Program

4. Reason for Program Review • The review of a program should be an on-going, year-round, reflective process.  • Program review processes assure educational quality and help programs to evaluate and improve their services. • Program review is an opportunity for self-study, self-renewal, and an opportunity to identify the need for improvement. • Your program review may be one of the few opportunities you have to showcase the accomplishments of your program. Take this opportunity to shine. • A robust program review process is one mechanism available to the college to improve student success.

5. How do we improve our program review process? Upon conclusion of every program/unit review cycle extra care is taken to ensure that we are improving our program/unit review process on campus. This is accomplished by sending out questionnaires (one for instruction and one for units), and then meeting with those groups across campus and collecting their feedback. The feedback is reviewed and an action plan is put into place, which is used in the planning phase for our next review cycle. Your suggestions, and the actions taken for improving this process, are then published to the Program Review website and have been linked here for your convenience. 2013 Program - Unit Review Process Improvement Summary Starting this year and going forward… “The CERC in conjunction with the Admin Team will ensure that the entire program-unit review process is evaluated every year, once the review process has ended. The evaluation will be prior to Summer break, and feedback from the campus will drive the plan for the following year’s review. IR will develop the survey and will ensure that a summary of this work is published every year.”

6. About the ARPD Web Submission Tool • If you have attended the training session in-person today you can skip the slides that deal with Navigating the ARPD site. I’ve included the slides here for those that may not have been able to join us today for the live demonstration. • The Annual Reports of Program Data Web Submission Tool (or ARPD for short), is a repository for Annual program and unit reviews for: Instruction, the Academic Support Unit, and Student Support Services. Much of the data that you will need to complete your review has been provided by the Office of the Vice President for Community Colleges. • The ARPD is a home-grown tool, developed in-house, and was developed specifically to meet the needs of the community college system.

7. ARPD Web Submission Tool Best Practices • Your programs data table is available on-line within the tool usually by August 15th every year. You should have everything you need to begin writing your review within the web submission tool. • Plan to save your work often—especially when switching between screens, and plan to do most of your formatting within the tool if you are copying and pasting in from Word. Also plan to spell check and save your review in Word before starting. • One of the nicest features about the ARPD is that you can go back and look at reviews from previous years. You can also enhance your own review by leveraging off similar reviews at other institutions.

8. Navigating the ARPD Web Submission Tool UHCC Annual Report of Program Data Web Submission Tool • Begin by clicking on the link above. To peruse the site select any review year, college, or program. To enter information on your review, click on the button on left of the screen called, “2014 Instructional”, then click on the “Web Submission” button. You will be asked to log in by typing in your UH username and password, in order to get to the web submission site. Now select your program. The default will take you to the “Status” tab, where you will be able to view whoever was the last user to modify the information in your review. • You can enter the information about your program in any order you wish, but moving from left to right across the tabs at the top of the screen follows the same logical path we used when we were filling out the templates in the past. Start by clicking on the “Analysis” tab. • On the Analysis screen you can either go in and just preview what has already been inputted by clicking on the “Preview” button, or you can go to the edit screen and begin entering or updating information.

9. Navigating the ARPD Web Submission Tool continued… • On the Analysis screen begin editing by going to the bottom of the page, under the data sheet. Click the edit button and scroll down. There are 3 sections for you to edit here: Analysis of your Program, Action Plan, and Resource Implications. Simply click on the “Edit” link and you will be taken to a screen very similar to MS Word, where you can begin typing in your analysis. (Note the Save button in the left hand corner!) • In the Action Plan section you will need to include your action plans for any of your Perkins Core Indicators, where the program did not meet the goal. Check your data sheet for this. • If you are requesting additional people, services, or equipment for your program, you will need to make the justification in the “Resource Implications” section. • Now click on the “Description tab” and input the year and web address of your last comprehensive review. You should be able to copy and paste the link right off the program-unit review website for the year of your last review. Finish this tab off by typing in a brief description of your program and mission. • Now, click on the “P-SLOs tab” and go to the next slide…

10. “P-SLO” tab in ARPD • Indicate which PLOs were assessed during the reporting year’s assessment(s). • Evidence of Industry Validation (if applicable) Provide documentation that the program has submitted evidence and achieved certification or accreditation from an organization granting certification in an industry or profession.  If the program/degree/certificate does not have a certifying body, the recommendations for, approval of, and/or participation in, assessment by the program’s advisory committee/board can be submitted. • Expected Level of Achievement Enter the benchmark for outcome(s) assessed.  What represented “excellent,” “good,” “fair,” or “poor” performance using a defined rubric and what percentages were set as goals for student success (for example: “85% of students will achieve good or excellent in the assessed activity.”) • Courses Assessed List the courses assessed during the reporting period. • Assessment Strategy/Instrument Describe what, why, where, when, and from whom assessment artifacts were collected. • Results of Program Assessment The % of students who met the outcome(s) and at what level they met the outcome(s). • Other Comments Include any information that will clarify the assessment process report. • Next Steps Describe what the program will do to improve the results.  "Next Steps" can include revision to syllabi, curriculum, teaching methods, student support, and other options.

11. “Cost Per SSH” tab in ARPD • There are 4 different values that need to be added on the Cost Per SSH screen in ARPD. Basically, all of the fund amounts that will be entered are used in the calculation for the cost per SSH for your program. The costs that need to be entered are: • General Funds • Federal Funds • Other Funds • Tuition and Fees • The definitions for what comprises each of these funds, are laid out in detail in the data definitions section of this presentation, so will not be duplicated here. • Joni will be uploading all of the budget information you need this year. Once the values are loaded and saved in the tool, the values for “Total Funds” and “Cost per SSH” will auto-calculate into your datasheet.

12. “External” tab in ARPD • The “External” screen is intended for programs that utilize external licensures. Currently, the only program at HawCC with external licensures is the Nursing program. • If you are not in the Nursing program you can skip this tab/screen altogether because the first question is, “Does this program utilize external licensures?” The radio button defaults to “No”. • If you are in the Nursing program please click the “Yes” radio button to answer the question and then enter the Number sitting for exam, and the Number passed. Do this for each program where you utilize external licensures. The percent passed will auto-calculate into your datasheet when you click the “Save External Data” button. • The Vice President of Community College’s office collects this data as part of our annual Graduate-Leaver reporting as well as the UH system, “Measuring our Progress” report.

13. “Capacity” tab in ARPD • The “Capacity” screen is only intended for programs that have an externally mandated capacity. • The following criteria is used as an alternate measure for the Student/Faculty Ratio measure within the Efficiency Health call: • “If your program has an externally mandated (e.g. professional accreditation or licensing) capacity of less than 16 students per faculty, the program may be eligible for the alternative efficiency health call calculation.” • If your program fits the criteria listed above, AND your Efficiency Health call is other than Healthy, AND your efficiency health call can be improved by using the alternate method… Please contact: Cheryl Chappell-LongDirector Academic Planning, Assessment, and Policy AnalysisPhone: 808-956-4561Email: cchappel@hawaii.edu

14. “Users” tab in ARPD • The users tab will provide a list of individuals have been identified by the College Vice Chancellor of Academic Affairs as having a specific program role in the analysis of Annual Report of Program Data. • If you need to make any changes to this list, contact the VCAA.

15. “Help” tab in ARPD • The “Help” screen is a very useful resource for some of the documentation needed to support your program within the review process. It contains all of the glossaries and health call scoring rubrics for each year we have used the online tool. • Be sure to use the corresponding glossary and health call scoring rubric for the review year you are interested in.

16. “Program Courses” tab in ARPD • The list of courses that are core to your program are listed here. Many of the data elements use this list to calculate values within your review. Over the course of the Summer every year, the VCAA’s on each campus are asked to submit the current listing of program courses, so that they can be programmed into the upcoming program review. • Take a moment to ensure that the courses listed here are correct for your program and be sure to report any discrepancies to the VCAA, so that they can be corrected.

17. Process for Completing the ARPD Instructional Program Review ANNUAL REVIEWS All Liberal Arts programs are required to submit an annual review. Please follow this simple 2-step process: Step 1:Complete your review using the HawCC annual instructional program review template or PATH tool, and follow the established Annual Review and Budget Process. Step 2: TheVCAA will have someone copy your information over from the template or PATH tool into the ARPD online tool. Note: The comprehensive review process, annual review and budget process, and their associated templates, are brought to you by the VCAA and Dean of CTE Programs. The template and instructions should be part of their presentation today.

18. Process Owners • The graphic below depicts who is responsible for each part of the program review process—from Assessment through CERC. For more information on the processes please use the contact list below. Contact List Assessment: Vacant 934-xxxx ARPD: Shawn Flood 934-2648 Annual/Comp Process: Joyce Hamasaki 934-2522 CERC: Joni Onishi 934-2514

19. Terminology / Timing • The Census freeze event is the fifth Friday after the first day of instruction. • The End of semester (EOS) freeze event is 10 weeks after the last day of instruction. • Degrees are conferred in the “Fiscal year”. The fiscal year value represents the ending of that fiscal year. For example, a FISCAL_YR_IRO value of 2013 indicates the fiscal year 2012-2013 (July 1, 2012 to June 30, 2013) which includes Summer 2012, Fall 2012, and Spring 2013 semesters. • “Home Institution” is the campus where the student was admitted. • “Program Year” listed on your data sheet for “13-14” represents your programs data in semesters: Summer 2013, Fall 2013, and Spring 2014. • Student information data this year comes exclusively from the Operational Data Store (ODS). Organizationally this means that all community colleges are getting the data from the same place and at the same snapshot in time (this is a good thing).

20. #1 Number of Majors • Count of program majors who are home‐institution at your college. • Count excludes students that have completely withdrawn from the semester at CENSUS. • This is an annual number. Programs receive a count of .5 for each term within the academic year that the student is a major. A maximum count of 1.0 (one) for each student. #1a Number of Majors Native Hawaiian Count of program majors who are Native Hawaiian and home‐institution at your college. Count excludes students that have completely withdrawn from the semester at CENSUS. This is an annual number. Programs receive a count of .5 for each term (fall and spring) within the academic year that the Native Hawaiian student is a major. A maximum count of 1.0 (one) for each student.

21. #1b Fall Full-Time • Percentage of majors (#1) enrolled in 12 or more credits in the college in the reporting Fall semester. #1c Fall Part-Time Percentage of majors (#1) enrolled in less than 12 credits in the college in the reporting Fall semester. #1d Fall Part-Time who are Full-Time in System Percentage of majors in #1c (enrolled in less than 12 credits in the reporting Fall semester in the institution) who are enrolled in credits in other UH institutions where their total number of credits enrolled in the UH System is equal to or greater than 12.

22. #1e Spring Full-Time #1g Spring Part-Time who are Full-Time in System Percentage of majors enrolled in 12 or more credits in the reporting Spring semester at the institution. #1f Spring Part-Time Percentage of majors enrolled in less than 12 credits at the institution in the reporting Spring semester. Percentage of majors in #1f (enrolled in less than 12 credits in the reporting Spring semester in the institution) who are enrolled in credits in other UH institutions where their total number of credits enrolled in the UH System is equal to or greater than 12.

23. #2 Percent Change Majors from Prior Year • In alignment with UHCC Strategic Planning Goals, General and Pre‐Professional Education programs are expected to grow by 3% per year. • Difference between the number of majors (#1) in the current year from the prior year, divided by the number of majors in the prior year. • For example, in the liberal arts program year (1112) there were 1306 majors. In the next year (1213) there were 1390 majors in the program. • ((1390-1306)/1306)*100 = about 6.4%

24. #3 SSH Program majors in Program Classes • The sum of Fall and Spring Student Semester Hours (SSH) taken by program majors in courses linked to the program. Captured at Census and excludes students who have already withdrawn (W) at this point. • Excludes Directed Studies (99 series). • Differs from MAPS because UHCC data includes Cooperative Education (93 series) as there is a resource cost to the program.

25. #4 SSH Non-Majors in Program Classes • The sum of Fall and Spring SSH taken by non‐program majors (not counted in #3) in courses linked to the program. Captured at Census and excludes students who have already withdrawn (W) at this point. • Excludes Directed Studies (99 series). • Differs from MAPS as UHCC data includes Cooperative Education (93 series) as there is a resource cost to the program.

26. #5 SSH in All Program Classes • The sum of Fall and Spring SSH taken by all students in classes linked to the program. Captured at Census and excludes students who have already withdrawn (W) at this point. • Excludes Directed Studies (99 series). • Differs from MAPS as UHCC data Includes Cooperative Education (93 series) as there is a resource cost to the program.

27. #6 FTE Enrollment in Program Classes • Sum of Student Semester Hours (SSH) taken by all students in classes linked to the program (#5) divided by 30. Undergraduate, lower division Full Time Equivalent (FTE) is calculated as 15 credits per term. • Captured at Census and excludes students who have already withdrawn (W) at this point.

28. #7 Total Number of Classes Taught • Total number of classes taught in Fall and Spring that are linked to the program. • Concurrent and Cross listed classes are only counted once for the primary class. • Excludes Directed Studies (99 series). • Differs from MAPS as UHCC data Includes Cooperative Education (93 series) as there is a resource cost to the program.

29. Liberal Arts Program Scoring Rubric Definitions Your program health is determined by 3 separate types of measures: Demand, Efficiency, and Effectiveness. This slide explains why these measures were chosen to determine program health. • Demand: A seeking or state of being sought after. i.e. your programs ability to attract new students every year based on your offering. • Efficiency: Acting or producing effectively with a minimum of waste, expense, or unnecessary effort. i.e. your programs ability to use its resources in the best possible way. • Effectiveness: Stresses the actual production of, or the power to produce an affect. i.e. your programs ability to produce the desired result.

30. Determination of Program’s Health • Again this year, the system office will calculate and report health calls for all instructional programs using the most recent program year data. • If you are interested in how the Liberal Arts Health Calls were determined, refer to the rubric link available on the ARPD website.

31. #8 Average Class Size • Total number of students actively registered in Fall and Spring program classes divided by classes taught (#7). Does not include students who have already withdrawn from the class by Census. • Excludes Directed Studies (99 series). • Differs from MAPS because UHCC data includes Cooperative Education (93 series) and there is a resource cost to the program. #9 Fill Rate Total active student registrations in program classes (number of seats filled) at Fall and Spring census divided by the maximum enrollment (number of seats offered). Captured at Census and excludes students who have already withdrawn (W) at this point.

32. #10 FTE BOR Appointed Faculty • Sum appointments (1.0, 0.5, etc.) of all BOR appointed program faculty (excludes lecturers and other non BOR appointees). • Uses the “hiring status” of the faculty member – not the teaching/work load. • Uses the Employing Agency Code (EAC) recorded in the Human Resources (HR) database to determine faculty’s program home. • Data provided by UH Human Resources Office as of 10/31/2013. • Faculty Teaching solely Remedial and/or Developmental Reading, Writing, or Mathematics are excluded from the counts in Liberal Arts and included in the counts for the Remedial/and or Developmental programs. Faculty with “split” assignments among Remedial/Developmental classes and Transfer level classes are reflected proportionally. • If your FTE BOR faculty count is off for your program, contact the VCAA. • Click here for the count of BOR Appointed Program Faculty in your program: ARPD Faculty to Program

33. #11 Majors to FTE BOR Appointed Faculty • Number of majors (#1) divided by sum appointments (#10) (1.0, 0.5, etc.) of all BOR appointed program faculty. • Data shows the number of student majors in the program for each faculty member (25 majors to 1 faculty shown as “25”)

34. #12 Majors to Analytic FTE Faculty • Number of majors (#1) divided by number of Analytic FTE faculty (#12a). #12a Analytical FTE Faculty (Workload) Calculated by sum of Semester Hours (not Student Semester Hours) taught in program classes divided by 27. Analytic FTE is useful as a comparison to FTE of BOR appointed faculty (#10). Used for analysis of program offerings covered by lecturers.

35. #13 OverallProgram Budget Allocation • The overall program budget allocation = General Funded Budget Allocations (13a) + Special/Federal Budget Allocations (13b) + Tuition and Fees (13c) + Other fees • The overall program budget allocation is automatically calculated when you enter your general funded budget allocation, special/federal budget allocation, other funds, and/or tuition and fees, into the online tool tab called, “Cost per SSH.” Note: Joni will have the budget data uploaded for you this year. • The overall program budget allocation is to be determined by the College using these guidelines from VCAA/DOI/ADOI and should include: Salaries (general funds, special funds, etc.), overload, lecturers, costs for all faculty and staff assigned to the program, supply and maintenance, amortized equipment, and tuition and fees.

36. #13a General Funded Budget Allocation • The general funded budget allocation = actual personnel costs + b budget expenditures #13b Special/Federal Budget Allocation The dollars from Federal grants #13c Tuition and Fees The amount collected for tuition and fees in the 2014 academic year.

37. #15 Number of Low Enrolled (<10) Classes #14 Cost per SSH Overall Program Budget Allocation (#13) divided by SSH in all program classes (#5) • Classes taught (#7) with 9 or fewer active students at Census. • Excludes students who have already withdrawn (W) at this point. • Excludes Directed Studies (99 series). • Includes Cooperative Education (93 series) as there is a resource cost to the program.

38. #16 Successful Completion (Equivalent C or higher) • Percentage of students actively enrolled in program classes at Fall and Spring census who at end of semester (EOS) have earned a grade equivalent to C or higher. #17 Withdrawals (grade = W) Number of students actively enrolled (at this point have not withdrawn) at Fall and Spring census who at end of semester have a grade of W.

39. #18 Persistence Fall to Spring • Count of students who are majors in program at fall census and at subsequent Spring semester census are enrolled and are still majors in the program. • Removed from the count (both numerator and denominator) are program major students to whom a program degree (or CA if highest credential awarded) has been conferred in the reporting Fall semester. • Example: 31 majors start in Fall 21 majors of the original 31 persist into Spring 21/31 = .6774 or 67.74%

40. #18a Persistence Fall to Fall • Count of students who are majors in program at fall census and at subsequent Fall semester census are enrolled and are still majors in the program. • Removed from the count (both numerator and denominator) are program major students to whom a program degree (or CA if highest credential awarded) has been conferred in the first Fall, Spring or Summer reporting term. • Example: 31 majors start in Fall 11 majors of the original 31 persist into Fall of the next year 11/31 = .3548 or 35.48%

41. #19 Unduplicated Degrees/Certificates Awarded • Unduplicated headcount of students in the fiscal year reported to whom a program degree or any certificate has been conferred. (Sum of 19a and 19b). • Uses most recent available freeze of fiscal year data. • Remember that we are actually counting the number of students that received a degree or certificate…if a student received a degree and a certificate , for example, they are counted once in the number awarded.

42. #19a Associate Degrees Awarded • Degrees conferred in the FISCAL_YEAR_IRO. • The count is of degrees and may show duplicate degrees received in the program by the same student. • Uses most recent available freeze of fiscal year data. • FISCAL_YEAR_IRO: “Fiscal year, where the value indicates the ending of the fiscal year. For example, a FISCAL_YR_IRO value of 2005 indicates the fiscal year 2004‐2005 (July 1, 2004 to June 30, 2005) which includes Summer 2004, Fall 2004, and Spring 2005 semesters…”

43. #19b Academic Subject Certificates Awarded • The count is of program Academic Subject Certificates and may show multiple Academic Subject Certificates in the same program received by the student. • Uses most recent available freeze of fiscal year data.

44. #19c Goal • The UH Community Colleges have established increasing the number of Associate Degrees awarded as a priority. • Beginning with the baseline of 2006 (see MAPS Degrees and Certificates Earned 2005‐2006) programs are expected to increase the number of Associate Degrees by 3% compounded annually. • The number in 19c reflects the goal for the appropriate year. • Example: 127 degrees was the goal for the liberal arts1112 program year, 130 was the goal for the 1213 program year (about a 3% increase).

45. #19d Difference Between Unduplicated Awarded and Goal • The percent difference between the number of Associate degrees awarded (#19a) and the goal (#19c). • General Pre Professional Programs are expected to increase Associate Degrees awarded by 3% compounded annually. • Uses most recent available freeze of fiscal year data. • Example: The liberal arts program’s goal for program year 1213 is 130 (19c). The actual number of Associate degrees awarded (19a) was 231. • ((231-130)/130)*100 = about 77.6% difference between the number of unduplicated degrees awarded and the goal for associate degrees

46. #20 Transfers to UH 4-yr • The number of students who are home‐institution at a UH System 4‐yr institution for the first time in Fall who were in the reporting program prior to that Fall. • In the event that a student has more than one major at the college, each program/major receives the count. • An individual student may be counted in more than one program. • UH Maui College is included when students transfer from any UHCC program to a UH Maui College four year programs. Number based on Fall semester only.

47. #20a Transfers with degree from program • Students included in #20 who have received a degree from the community college program prior to transfer. • Does not include any certificates. • Number based on Fall semester only. #20b Transfers without degree from program Students included in #20 who did not receive a degree from the community college program prior to transfer. Number based on Fall semester only.

48. #20c Increase by 3% Annual Transfers to UH 4-yr Goal • The UH Community Colleges have established increasing the number of students who transfer to UH 4‐year institutions as a priority. • Beginning with the baseline of 2006, General Pre Professional programs are expected to increase the number of students who transfer by 3% compounded annually. • The number in 20c reflects the goal for the appropriate year. • Example: The annual transfer goal for the liberal arts program year 1112 was 62, the goal for the 1213 program year is 64, about a 3% increase.

49. #20d Difference Between Transfers and Goal • The percent difference between the number of Transfers to UH 4‐year institutions (20) and the goal (#20c). • General Pre Professional Programs are expected to increase transfers to UH 4‐year institutions by 3% compounded annually. • Example: The liberal arts program’s goal in program year 1213 was 64 (20c). The actual number of transfers (20) is 137. • ((137-64)/64)*100 = about a 114% difference between the number of transfer students and the goal.

50. #21 Number of Distance Education Classes Taught • Measures the number of classes taught with the mode of delivery as “Distance Completely Online.” • In setting up the class, the college indicates the method of instruction used by the instructor in conducting the class. • If the method is Distance Education, and the college indicates the “mode” of distance delivery as “Distance Completely Online” the class will be included in this count. #22 Enrollments Distance Education Classes At the Fall and Spring census, the number of students actively enrolled in all classes owned by the program and identified as Distance Completely On‐Line (#21). Does not include students who at Census have already withdrawn from the class. The number is an unduplicated count of registrations but is a duplicated count of students. (e.g. If a student is enrolled in two DCO classes both are included in this count.)