Metrics and Money:

1. Metrics and Money: The Process and Politics of Accountability Stephen Daigle, Ph.D, California State University Michael Large, Ph.D, Social and Behavioral Research Institute, CSU San Marcos Patricia Cuocco, California State University

2. Overview • ITS and the Measures of Success (MOS) • MOS Research Agenda • Findings • Questions (and, we hope, answers)

3. Integrated Technology Strategy • Early 1990s – Presidential Interest in Using Technology As a Strategic Tool Driven By: • Immense Growth in Technology, • Antiquated Legacy Systems, • Increasing Demand for Access • Fiscal Constraints • Mid 1990s – Develop Integrated Technology Strategy Framework • Iterative Process, Constituent Input and Approval

4. ITS – The Process • 1996 Accepted by Board of Trustees • Living Framework – Not Written Plan • 2003 – Technology Decisions Still Governed by ITS

5. What Is ITS • Outcome Driven • Excellence in Learning and Teaching • Quality of the Student Experience • Administrative Quality and Productivity • Personal Productivity

6. ITS – the Icon

7. ITS – How It Fits • Personal Productivity - Attained Through Infrastructure Initiative – Minimum Baseline at All CSU Campuses • The Infrastructure Is Critical If ITS Benefits Are to Extend to All Students, Faculty and Staff • The Infrastructure Requires Money ($250M) = PROBLEM

8. Getting the Infrastructure • Legislature Challenged CSU to Be Creative • CSU Was Too Creative • Public/private Partnership Made Legislature Uncomfortable • Californians Passed a Bond • Presidents Decided Infrastructure Had Priority Over New Buildings

9. Getting to “Yes” * • Bond Expenditures Require Legislative Approval • Legislature Wanted to Tie Investment in Infrastructure to “Learning Outcomes” • Not What ITS Is About • Infrastructure Enables ITS Initiatives Which, in Turn, Enable Outcomes Apologies to Fisher, Ury and Patton

10. Negotiating Accountability • Worked With Legislature – Agreed on What Could Be Measured • Developed 10 Year Process for Measures of Success • Framework • Baseline • Changes to the Baseline

11. Types Of Data Collected • System • Campus • Individual (Student, Faculty, Staff) • External

12. Scope Of Data Collection • 23 Campuses; 1,000 Miles • 400,000 Students • 35,000 Faculty And Staff • 10 Year Commitment • Institutional Comparisons Across Time • S, F, S Comparisons Across Time

13. Technology User Survey Samples • Students: Campus, Class Level, Ethnicity; 23 X 5 X 9 Cells • Faculty: Campus, Rank, Discipline; 23 X 4 X 8 Cells • Staff: Campus, Job Classification; 23 X 7 Cells

14. CATI LOGISTICS(COMPUTER ASSISTED TELEPHONE INTERVIEWING) • No Self-select As With Mail • About 100 Questions; 20-30 Minutes • Importance Of Skip Facility • Instant Database Creation

15. CATI (Continued) • Standardized Context Provided (E.G., High Speed Network) • Trained Interview Staff, Monitoring, Evaluation • Refusals Less Than 2 Percent; Still Over Sample • $75 To $100k Per Survey, But Systemwide

16. Hardware Software Network Support Training ACCESS USE SATISFACTION User Outcomes

17. Examples of Metrics • Institutional • Library Cost Avoidance • Smart Classrooms • Data Center Savings • Training and Support Spending • Individual • Computer Ownership • Network Use • Help Desk Satisfaction • Use of Administrative Systems

18. Findings and Data

19. Findings

20. Hardware • Access • Students’ access to hardware • Use • Percent of Faculty Requiring Computer Use • Satisfaction • Faculty satisfaction with Teaching in Computer Labs

21. Software • Access • Faculty access to software • Use • Percent of Students Using Components of Their Student Information System • Satisfaction • Staff Satisfaction with Software

22. Network • Access • Students’ access to wireless networking • Use • Staff use of network from off-campus • Satisfaction • Staff satisfaction with e-mail

23. Student Access to a University-provided Computer

24. Percent of Faculty Requiring Students to Use a Computer

25. Faculty Satisfaction with Giving Instruction in a Computer Lab

26. Faculty Access to University-provided Software Needed

27. Percent of Students Using Their Campus Student Information System

28. Staff Satisfaction with University-provided Software Available

29. Percent of Students with Laptops and Wireless Capability

30. Staff Use of University Network from Off-campus

31. Staff Satisfaction with E-mail Services

32. Importance of Providing Electronic Access to Course Instruction Any Time and Place

33. Comparison between Faculty and Students in Perceived Importance of Providing “Any-time, Any-place” Instruction

34. Uses of the Data • Accountability • Description of the Population of Interest • Change Over Time • Subgroup Comparisons • Cross-group Comparisons

35. Conclusion • Expensive – but You Get What You Pay For: Valid • Reliable • High Level of Confidence • Negotiate – Don’t Be Passive – Be Proactive • Accountability Provides Cover • Infrastructure Is a Utility and a Prerequisite • Strategic Planning Is Dynamic – Change/add

36. Look For Yourselfhttp://its.calstate.edu/documents/Data_Collection/I_Reports_MOS/Measure_of_Success.shtml

37. Copies of the Presentation http://its.calstate.edu