Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

1. MeasuringInstitutional Success:An Electronic Portfolio forAccreditation and Assessment Outcomes Vivienne Carr Rosemary Skeele David Middleton

2. Context - Seton Hall University • Private, mid-sized, northeastern university • Technology accepted & promoted – mobile, wireless computing in place • IT courses for University located in College of Education University Mission Statement Seton Hall University is a major Catholic university. In a diverse and collaborative environment it focuses on academic and ethical development. Seton Hall students are prepared to be leaders in their professional and community lives in a global society and are challenged by outstanding faculty, an evolving technologically advanced setting and values-centered curricula.

3. What We Have Learned • A portfolio is not a container – It is a process • A portfolio is adaptable to any university program • “Every act of creation is an act of destruction” • The results of the process are revealing

4. Electronic Portfolio Institutional Initiative • History of Electronic Portfolios at Seton Hall • Curriculum Development Initiative (CDI): • Electronic Portfolio • 6 Branches of Project: • College of Education and Human Services • Psychology Department • English Department • Honors Program • Freshmen Studies • Outcomes Assessment Team

5. Electronic Portfolio in Higher Education - Considerations • Academic Freedom • New Role of the Teacher • Technological Equity • Resources

6. Faculty Evolution • Personal development – skills, comfort, needs, exploration – CBAM Model • Curriculum development – teaching, modeling, pedagogical reflection • Team development – assisting, supporting, collaborating

7. Outcomes Assessment Team (OAT) • Established by the Office of the Provost in 1991 • Examining methods of integration across University Community • Developing matrices of assessment in conjunction with SHU’s Core Curriculum Committee • Process: • Begin process at the department level • Department chairs to head the process • Devise a process • Construct syllabi with outcomes and related assessments • Develop a program matrix • Concurrently devise a university-wide content management system

8. Specific metrics developed for integration into at least 50% of a student’s courses • Conscious development of these habits of mind as part of coursework

9. Electronic Portfolios and the Core Curriculum • Database of courses certified as fulfilling the Core requirements • Ongoing checklist of student’s fulfilling Core requirements for advising • Depository of objects attesting to student’s skill development • Opportunity for student reflection on the learning process

11. College of Education and Human Services Educational Studies Department • History of paper portfolios • PT3 Grant in 1999 sparked commitment to technology- based student portfolios • Evaluation by professional societies impacts delivery decisions • Current leader in SHU: Electronic Portfolio Initiative

12. College of Education and Human Services • Required Flexibility in Portfolio Development • NCATE Influenced – Performance Evidence • Demonstrate & Motivate Student Learning • Reflection of the Student as a Teacher • Development of Student Professional Portfolios

13. Portfolio Product Evolution • Paper • Mini disks (floppy) • CD-ROM • Web-Based (Front Page, On-line free & low cost services) • Specialized product • Out of the Box product customized by SHU

14. CEHS Standards Electronic Portfolio • Vehicle for assessment in demonstrating learning outcomes linked to accreditation standards • Student products yield accomplished work in the field of education • All work is coupled with student reflections stating how products meet accreditation standards • This practice demonstrates the students’ knowledge and understanding of the standards

15. How We Met Our Goals: A Process Step 1: Determine what a portfolio means to your program. Define goal Step 2: Identify what we want students to learn. Set objectives • Align with mission • Look at institutional, state, and national standards • Research other programs Step 3: Describe content and tasks. Develop curriculum & activities • Map Curriculum 12 • Learn and integrate new teaching practices • Revise syllabi Step 4: Determine whether we met our objectives. Assess & revise learning • Develop evaluation instruments; Create rubrics • Analyze results • Revise curriculum based on results

16. Standards Building Block • College of Education and Human Services is NCATE Accredited. • Also meets INTASC (Interstate New Teacher Assessment and Support Consortium) and NJPS standards (New Jersey Professional Standards). Student’s Dilemma How do you keep track of standards as well as your course, program and institutional requirements?

17. Why Blackboard Electronic Portfolios? • Comfort level – students, faculty, staff • Training and support issues • Multiple uses for files • Easy to share files across courses, Electronic Portfolio, and people • Provides security not available on Web • Accepts variety of media • Let’s you build a homepage

18. Content Management Peer Review Student/Faculty Collaboration Assessment Resource Sharing Content ManagementSystem

19. Faculty Content Management System Blackboard Course Blackboard Course Blackboard Course Section I Section II Section III

20. Content Management System Course/Core Portfolio Professional Portfolio Faculty Career Center Advisor Employer Department Alumni

21. Students can access the Electronic Portfolio from their Main Page

22. Linking Student Products to Pertinent Standards

23. Products are connected to the Blackboard Content and Learning Management Systems

24. Adding/Editing Products and Reflections Products can be connected to multiple standards Our application for participation in the Center for Academic Transformation's Roadmap to Redesign Programs is an example of our understanding of the importance of effective utilization of Blackboard as a Mission Critical Application.

25. Automatically Generated Electronic Portfolio Combines Products and Reflections

26. Standards Data Reporting The system is built to allow for custom reporting according to needs.

27. Standards Data Reporting The data can be exported for analysis in either Excel or SPSS.

28. Standards Electronic Portfolio Version 2.0

29. Standards Electronic Portfolio Version 2.0Workflow • Faculty Member creates “Standards Based Assignment” • Assignment Appears as Typical Blackboard Assignment • Students Complete the Assignment by attaching the artifact • Entry in the grade book is the same as others. Faculty Grade this entry as normal • Grading UI is the same as typical Assignment • The Achievement of a specific level grade would generate the ePortfolios

30. Standards Electronic Portfolio Version 2.0Future Features • Standards Based Assignment • Rubric Maker & Grading Mechanism • Assessment Tools: • Ability to assess how students are doing across multiple sections and on specific subject matter - example: lesson planning • Filters – e.g. department/program; gender, race, etc; rubric portions; specific dates • Demographics • Track individual development Reporting will provide more granular data for Inter-rater reliability.

31. Plans for Institutional Change • Development and Implementation of the Standards Electronic Portfolio completed Summer 2005 • Used by over 50% of the faculty from College of Education and Human Services - Summer 2005 • Implementation and customization of the system across the Institution - Fall 2005 • Integration of the system in the assessment plans for the new University Core Curriculum

32. Outcomes • Demonstrates faculty excellence • Shows support from administration • Initiates a culture change • Provides Accreditation Reporting • Demonstrates Institutional-wide • Contributes to Assessment and Program Review • Meets Standards & Core Curriculum Proficiency Measures • Provides a bridge to the Outcomes Assessment Team, Institutional Research & Budget Offices • Provides Personal and Professional growth for students

33. Electronic PortfoliosMoving our Students into the Future