ORGANIZING THE OPPORTUNITIES OF SURVEYS AND REPORTS

1. ORGANIZING THE OPPORTUNITIES OF SURVEYS AND REPORTS P. Shetty, C. Smith, V. Remennik, G. McLaughlin, N. Stott Office of Institutional Planning and Research DePaul University Chicago, IL

2. Outline • Introduction • Data Management • Physical Tools • Processes or Methodologies • Use & Dissemination of Information • Barriers • Conclusions

3. Introduction • Core competencies of IR office is to respond to Surveys and Reports (Saupe, Peterson, Dressel) • We are a data broker. Coordinate with the data source and negotiate with the user (Sheehan). • We must do important things for important people (Suslow)

4. Introduction • Functions of OIPR: • Institutional reporting • Program revision and measurement • Decision support • Education research • Planning

5. Introduction: DePaul University • Urban teaching Institution: 24,000 students, 9 colleges, 3 main campuses • Core business activity • Service oriented • Undergraduate teaching • Student body- besides traditional UG students, we have transfers, part-time, first-level grad students

6. Introduction: • Reporting takes up a lot of time • Inundated with data • How we store/manage our data is crucial • Data management process: • Streamlining using technology and tools • Restructuring methodologies • Efficient dissemination ofinformation

7. Introduction • Objective of our office-go beyond the reporting function: • streamlined our process • expanded our scope • used our contacts • reused the data (data exchange) • partner with other groups

8. Data Management: Using Technology and Tools

9. The Evolution of Data Management Mainframe Mainframe Mainframe Paper Output PC Database (SPSS/Access) PC Database (SPSS/Access) Excel /Paper (Manual Entry) Excel /Paper (Automated Entry) Excel /Paper (Automated Entry) Web Reports (Static) Web Reports/OLAP (Static/Dynamic) Time Line

10. Data Management: Present Staging DB (Access /SQL) ERP (PeopleSoft DB) Reporting DB (SPSS/Access/SQL) Excel /Paper (Automated Entry) Web Reports/OLAP (Static/Dynamic)

11. Data Management: Present • Different data sources on one system • Strong activity to standardize data • Daily refreshed data • User customized data mart • Repository of data –put census data in OLAP cubes and allow interacting users access to historical data • Web versions of university statistics • Rapid turnover of data request

12. Data Management: Progression Data Warehouse (Reporting Data) Reporting DB (SPSS/Access/SQL) Excel /Paper (Automated Entry) Web Reports/OLAP (Static/Dynamic)

13. Data Management: Progression • Data warehouse-integrate and improve quality of data • Customized Data Dictionary that will enable each department to define their meta data and add variants • Predictive RIS (Retention Information System): undergraduate data that can be queried for any statistically relevant information e.g. probability of a student being retained given a set of factors • Expand to local system similar to ANSWERS (NPEC) for measures recognition

14. Data Management: • PROCESSES / METHODOLOGIES

15. Data Management: Inflow chart HR Data Financial Aid Data Student Demographic/ Academic Data (IS) Enrollment Data (EMR) Student reporting & Operating Data Store (Updated 24 Hours) Student Admiss. Data (Student Affairs) OIPR Census reporting database

16. Data Management :Outflow Chart Census- Faculty/Staff (Frozen) Financial- Revenues/Expenditure Census – Student Data (Frozen) Surveys (Mandatory/In-house) External Requests/ Peer Institutions Within University Requests

17. Data Management • Our primary reporting responsibility is for the University • OIPR produces a number of internal reports that integrate students, financial and personnel activity-Fact File and Integrated Academic Information System (IAIS)

18. IAIS: Instructional Work Load & Cost Data Workload Measures Fulltime equivalent (FTE) Demographics, course hrs, Enroll Coll. (IS) Overhead costs HR-salaries • Reviewed by college reps • Feasible adjustments made IAIS IPEDS S, SA, EAS CUPA Delaware study WCAR Reports

19. Student Enrollment Files (e.g.) Census – Student Data (Frozen) OLAP cubes- Annual survey data Retention Database OLAP cubes: Course level Enrollment • SURVEYS: • IPEDS GRS • Fact File • US News • CSRDE • NCAA • External req. • STUDIES e.g.: • Transfer out rate and campus climate • Analysis of course grade patterns

20. Data Management: Process • OIPR project management: • Documenting the following information about internal/external survey: • Title • Topic • Agency-conducting the survey • Survey Frequency • Administrator • Location of Survey results • Contact person • Date the survey was received

21. Data Management: Categories of Reporting Private Groups Government Consortiums Annual Surveys Federal State State Own Institution • UG Satis. S • G/Law. Satis S • Suburban S S • Grad.Senior S • Fact File • BIS Report • IAIS • Alumni Employed • one year after grad survey • ISDE • NCAA • IPEDS Salaries • IPEDS Enroll. • IPEDS GRS • IPEDS Finance • NSSE • NSF • Peterson’s • NSF-LASMP • AAUP • NSoFaS • IBHE Enroll. • IBHE NCSE • CUPA • NSICP • AACTE • NSF-NIH • GRE-CGS • CSRDE Ret./Grad. • CSRDE SMET • CSRDE-Transfers • CDSX • US News • Delaware Study • IVC (illinois Virtual Campus)

22. Data Management: Process • Purpose: Set forth procedures and standards for obtaining consistency and uniformity in data collecting and reporting • Security and Access: Anyone who obtains sensitive data is required to comply by FERPA.OIPR will review FERPA regulations and implementation of data collection and dissemination and will provide interpretations of policy requirement

23. Data Management: Process • Distributed functions among personnel. • Reports and surveys answered by fact file and OLAP cubes • Others forwarded to other operating functions of the university • Survey log maintains records of when surveys received, whom they are forwarded to and information about completion

24. Data Management: Process • Developing a data element dictionary for the university • Provide university with congruent definitions of key terms related to research and management • Terms connected to items in OIPR fact file and University success factors • Definitions dynamically changing according to university research and data needs

25. DISSEMINATING INFORMATION

26. Leveraging outcomes by providing data • According to McLaughlin and Howard, 1999, to convert data into meaningful information, we need to: • Restructure data to reduce noise • Integrate data from multiple source to make interpretation with adequate confidence • Sufficient data available for making decisions

27. Leveraging outcomes by providing data • Supporting Administrative and Research purpose Partnership Outside Organization External Group Other University Offices Faculty & Student Research Projects • ISDE • National Student • Clearing House • ACT • Other Colleges • EMR • Academic Program • Review Committee

28. Leveraging outcomes by providing data • Analytical Tools that support use: • Peer Analysis System (IPEDS PAS) • Colleges Online (IPEDS COOL) • Using comparative groups or developing your own peer groups: • CUPA Salary Benchmark • Tuition and fees –competitive groups from ACT • Strategic planning: Four frequently mentioned Catholic groups

29. Leveraging outcomes by providing data • Partnerships with Organizations and external groups • expanded the scope of our data -analytics. E.g. ACT survey given to college entering students enabled us to look at standing of our transfer students • Gives credibility to collaborative research. • Gives additional information about our students

30. Leveraging outcomes by providing data • Educated and engaging our internal customers and delegated them some of our past functions • Do research as incremental activity • Provides interim results • Get on key committees to shape discussions • Make discussions about strategic progress • Coordination –not completion

31. How to just say “NO” • University importance • Relevance to Office • Difficulty to obtain data • Compatible with staff interest and capabilities • Supportiveness of other activities • Functionality to future projects

32. BARRIERS/ ISSUES

33. Barriers /Issues • No easy way to distribute reports or summary • Cleaning data: 80/20 rule • E.g. Workload by department requires population sample • General rules don’t apply for specific cases • Summer courses for school teachers -won’t apply to general rule • Fall to Fall retention analysis won’t apply to part time students • School of music and theatre workload-revise course equivalent • Data relevancy: e.g. Phased retirement, not in DB because not relevant to the Deans but important to us to compute instructional workload

34. Barriers/ Issues • Different Definitions, different requisites: Every definition has advantage and disadvantage to somebody (Competing perspectives): • different faculty definitions –tenure vs. instructor • People don’t use CIP codes –different codes not comparable to other universities • federal definitions of diversity-culture vs. ethnicity • Instructional workload- SNL vs. LA &S , IVC vs. IAIS –Distance learning, US News – class size- e.g. Art class • CIP code majors for education in other schools • Mandate in reporting requires timeliness not accuracy • Different levels of sophistication among users • Shifting time cycle of when reports are due

35. Summary and Conclusions • As “data stewards” DMS will improve data consistency and reliability • Availability of data has allowed for expansion of our function • Allowed for partnerships • Role of information providers to decision makers