Sharon Hamilton Policy and governance issues for electronic student portfolios

1. Building bridges to constituents: Communicating through performance indicators and electronic portfoliosTrudy Banta, Victor Borden, & Susan Kahn Sharon Hamilton Policy and governance issues for electronic student portfolios

2. Overview • Planning for improvement • PIs @ IUPUI • The tip of the assessment iceberg • iPort: The institutional portfolio • Placing PIs in context • ePort: The electronic student portfolio • Demonstrating student learning • Completing the cycle • Linking accountability to improvement

3. Planning for Improvement Improvement 1. Reporting to internal constituents 2. Demonstrating accountability to external stakeholders 3. Applying findings in campus improvement initiatives 4. Proposing improvement initiatives 5. Improving assessment methods · Web-based data · Electronic portfolios Assessable Outcomes Culture Of Evidence Application of Findings Instrumentation Data Collection Analysis

4. Planning and Budgeting • Mission, Vision, Goals developed • Unit goals aligned • Annual reports on web • Programs based on assessable goals, with performance indicators • Biennial planning/budgeting hearings conducted

5. Implementation Everyone on campus implements goals

6. Evaluation • Academic and administrative program reviews • Evaluation of process effectiveness • Assessment of learning outcomes • in major • in general education • Course evaluations • Student assessment • Constituent surveys • Management information and analysis • Program cost analysis • Web-based evaluation tools • Annual campus performance report • NCA accreditation

7. Improvement • Reporting to internal constituents • Demonstrating accountability to external stakeholders • Applying findings in campus improvement initiatives • Proposing improvement initiatives • Improving assessment methods • Web-based data • Electronic portfolios

8. Planning for Improvement Improvement 1. Reporting to internal constituents 2. Demonstrating accountability to external stakeholders 3. Applying findings in campus improvement initiatives 4. Proposing improvement initiatives 5. Improving assessment methods · Web-based data · Electronic portfolios Assessable Outcomes Culture Of Evidence Application of Findings Instrumentation Data Collection Analysis

9. PIs @ IUPUI Performance Indicators as the tip of the assessment iceberg

10. PIs @ IUPUI: Background • Status of PIs in the early 1990s • Kells (OECD) • Failure of PIs for national accountability in Europe and elsewhere. Movement toward quality assurance • Borden & Banta (NDIR 82) • Greater possibilities for PIs if they emerge from comprehensive planning efforts • Gaither, Nedweck, & Neal (ASHE ERIC) • Increasing demand for accountability and continuing inability of HEIs to communicate about performance to external audiences • The Rise of CSFs, KSFs, KPIs • Rockhart (1979) Critical Success Factors (CSFs) • NACUBO/ Peat Marwick – 67 Indicators • Dolence (1989) KPIs for enrollment management

11. Balanced Score Card (BSC) • Kaplan & Norton propose business model • Financial performance • Customer service and satisfaction • Process effectiveness and efficiency • Organizational learning • Ruben (1999) applies BSC to HEIs • Teaching/Learning - Programs/Courses, Student Outcomes • Service/Outreach - University, profession, alumns, state, prospective students, families employers • Scholarship/Research - Productivity/Impact • Workplace satisfaction - Faculty/staff • Financial - Revenues/expenditures

12. Problems with Current Systems • Despite Ruben’s challenge to focus on mission critical activities, most focus on administrative functions • There is a disconnect between performance indicators and core performance objectives • What is the organizational link between the accountability focus of campus executives and the performance of mission critical functions?

13. Making the Connection • Doing assessment right (Banta, 2002) • Participative planning • Assessment must be explicitly tied to institutional functions and be associated with goals that faculty and administrators value • Implementation • Assessment is most powerful at the unit level where changes will most immediately affect the faculty and students • Improving and sustaining • while the results of assessment may be used to respond to various external accrediting and stakeholder constituents, it will first and foremost provide the means by which the organization can continuously assess and improve itself

14. PIs @ IUPUI: Performance Objectives • Excellence in Teaching and Learning • Attract and support a better prepared, more diverse student population. • Support and enhance effective teaching • Enhance undergraduate student learning and success • Provide effective professional and graduate programs and support for graduate students and post-doctoral fellows. • Excellence in Research, Scholarship, and Creative Activity • Conduct world-class research and creative activities relevant to Indianapolis, the state, and beyond. • Provide support for scholarly activity and external funding. • Enhance infrastructure for research and creative activities. • Excellence in Civic Engagement, Locally, Nationally, and Globally • Enhance capacity for civic engagement. • Enhance civic activities, partnerships, and patient and client services. • Intensify commitment and accountability to Indianapolis, Central Indiana, and the state. Collaboration – Diversity – Best Practices

15. PI Development Process • Vested interested groups (handout) • Brainstorming session • What evidence would convince a (neighbor, prospective student, legislator) that you we were doing well? • What evidence do you (or could you) use in your unit to assess performance? • 3-4 performance indicators per objective • Three level rubric (green, yellow, red) • Adapted Nominal Group Technique for voting

17. Status and Results • In Handout • Summary of performance objectives and performance indicators, with measures shown for one set of indicators • Published in… • Campus annual performance report • Chancellors State of Diversity address • Institutional portfolio

18. iPort The Electronic Institutional Portfolio

19. “Institutional Portfolio” A focused selection of authentic institutional work, combined with narrative analysis, that demonstrates specific institutional achievements and shows learning and improvement over time—i.e., “institutional effectiveness”

20. IUPUI’s iPort • Begun in 1998 as part of national grant-funded project • Organized around main components of institutional mission and major strategic priorities • Intended for purposes of both internal improvement and accountability to multiple stakeholders • Used as platform for accreditation self-study in 2002 • Is on the Web—available to all

30. What do we gain from work onan electronic institutional portfolio? • Occasions for focused, campus-wide discussions of areas needing improvement • A means of demonstrating institution’s commitment to accountability and effectiveness • An easy, accessible medium for communicating with stakeholders about accomplishments and effectiveness

31. Next Steps • Develop new iteration/update as annual performance report • Update the design and technological infrastructure • Foreground links to other relevant campus Web sites • Incorporate more direct evidence of student learning through interface with student electronic portfolio

32. ePortfolio The Electronic Student Portfolio

33. Testing Digital Libraries Syllabus Schedule Federated Searching Ereserves In Touch Gradebooks Digital content Full Text articles Eportfolio OneStart Webservices “Unbundling” Oncourse Rich Learning Environment Authn/Authz Security Workflow Comm. Tools Storage

34. Learning Matrix Learning Profile Phase 1 Content Manager Advising Eportfolio Resume/Vita Builder Research Manager Knowledge Mapper Other Services Authn/Authz Security Workflow Comm Tools. Storage Other Services Webservices

35. ePortfolio Goals • Support students' learning of and engagement with the PULs over their entire undergraduate experience at IUPUI beginning in the freshman learning community and culminating in the capstone experience • Assist both faculty and students to reach a clearer, more coherent understanding of how various aspects of the curriculum support students' increasing mastery of the PULs • Contribute to assessment of student learning of the PULs at the levels of the individual student, the course, program, and institution

36. ePortfolio and the PULs The ePortfolio is organized around IUPUI's Principles of Undergraduate Learning • Core communication and quantitative skills • Critical thinking • Integration and application of knowledge • Intellectual depth, breadth, and adaptiveness • Understanding society and culture • Values and ethics

37. ePortfolio Levels of Competence • Introductory: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 26 credit hours • Intermediate: What all undergraduate students at IUPUI should know and be able to do in relation to the PULs within the first 56 credit hours • Advanced: What all baccalaureate students at IUPUI should know and be able to do in relation to PULs in their major or profession or academic program

38. Learning Matrix

39. Planning for Improvement Improvement 1. Reporting to internal constituents 2. Demonstrating accountability to external stakeholders 3. Applying findings in campus improvement initiatives 4. Proposing improvement initiatives 5. Improving assessment methods · Web-based data · Electronic portfolios Assessable Outcomes Culture Of Evidence Application of Findings Instrumentation Data Collection Analysis