Developing an Effective Assessment Plan: From Mission to Outcomes - PowerPoint PPT Presentation

tevy
developing an effective assessment plan from mission to outcomes n.
Skip this Video
Loading SlideShow in 5 Seconds..
Developing an Effective Assessment Plan: From Mission to Outcomes PowerPoint Presentation
Download Presentation
Developing an Effective Assessment Plan: From Mission to Outcomes

play fullscreen
1 / 39
Download Presentation
Developing an Effective Assessment Plan: From Mission to Outcomes
145 Views
Download Presentation

Developing an Effective Assessment Plan: From Mission to Outcomes

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Developing an Effective Assessment Plan: From Mission to Outcomes Assessment Plan Workshop SUNY Oneonta March 22, 2012

  2. Presenter: Patty Francis Associate Provost for Institutional Assessment & Effectiveness

  3. Background • Development of College’s Action Plan for Planning and Assessment (Spring 2008) • Endorsement by College Senate (12/2008) • Approval by President’s Cabinet (Spring 2009) • Formation of Institutional Assessment Committee (IAC) (Spring 2009) • Development of assessment guidelines by IAC and approval by President’s Cabinet (11/2009) • Distribution of guidelines in 12/2009, with first plans due June 1, 2010 • Revision of guidelines by IAC in June 2011 • Submission of first assessment reports in June 2011

  4. Suggested Format for Unit Assessment Plans • All unit goals and objectives (i.e., Step 1) • Results of activity mapping exercise (i.e., Step 2) {optional} • Action plan summarizing those objectives units intend to assess (and how) for the next year (i.e., Step 3) • Brief description of how unit will “close the loop” once assessment results are attained (i.e., Step 4)

  5. Some Assessment Basics • Establishing congruence among institutional mission and goals, programmatic and unit objectives, unit activities, and assessments • Linking goals and objectives to outcomes through action plan • Assessment as an ongoing, iterative process • Using a variety of meaningful measures, both quantitative and qualitative, in search of convergence • Using existing data sources as much as possible

  6. Assessment’s Four Steps • Setting goals and objectives: “What you aspire to do and what you do” • Objective mapping: “How you do what you say you do” • Assessment: “How you know you are doing what you say you do” • “Closing the loop”: “What you do next based on results”

  7. Most Important • Done correctly, assessment: • Serves to align mission, goals, objectives, and assessments across all levels of the institution • Initiates a “never-ending” dialogue among staff members regarding programmatic priorities, objectives and effectiveness • Offers multiple, rich opportunities for professional interaction and development • Provides (mostly) affirming data in support of existing programs and services • Provides a systematic, focused direction for change and future activities

  8. Step I: Setting Objectives: “What are you attempting to do?”

  9. It All Starts With a Mission • Mission statements: • Summarize the unit’s status and major functions and objectives at present • Should be congruent with existing higher-level mission(s) (e.g., at the institutional/divisional levels) • Are intended primarily for internal stakeholders • Are optional in IAC guidelines but strongly encouraged

  10. Sample Mission Statement (University of Chicago Library) “The Library’s mission is to provide comprehensive resources and services in support of the research, teaching, and learning needs of the University community.”

  11. Sample Mission Statement(Creighton University Media Services) “To serve Creighton University and the community by exploring, designing, supporting and facilitating learning, teaching and research opportunities through the effective and creative use of technology and media in accordance with the mission of Creighton University.”

  12. Goals, Objectives, and Outcomes: What’s the Difference? • Goals ->Objectives ->Outcomes = More General (and Less Measurable)->More Specific (and More Measurable) • Goals: • Statements about general intentions/purposes that are broad and more long-range in scope and not directly measurable • May come directly out of unit mission statement • Usually developed at programmatic or divisional level and often are in the form of a “process” statement (i.e., begin with verbs like “establish,” “provide,” “enhance”)

  13. Goals, Objectives, and Outcomes: What’s the Difference? (cont.) • Objectives: • More specific than goals • Typically there are multiple objectives for each goal • Usually developed at the unit level to reflect “upper-level” goals

  14. Goals, Objectives, and Outcomes: What’s the Difference? (cont.) • Outcomes: • Very specific statements that “translate” into assessable measures • Process- vs. results-oriented statements • Two kinds, depending on assessment stage • “Expected outcome” refers to anticipated results of assessment – should include criterion to be used in determining success • “Actual outcome” refers to actual results of assessment

  15. Student Affairs Example Goal Enhance student learning through civic, professional and research engagement. Objective Students will learn to be responsible global citizens. Expected Outcome 80% of students will participate in a university-approved community service activity with an inter-cultural emphasis.

  16. Finance and Administration Example Goal Develop and implement processes that deliver value to the institution and campus community. Objective To identify and carry out strategies that result in the strategic allocation of resources. Expected Outcome During 2007-08, the total funds expended on institutional priorities will increase by 20%.

  17. Academic Affairs Example Goal Enhance public service and image throughout the region. Objective Promote and facilitate faculty involvement in the local public schools. Expected Outcome At least 40% of full-time faculty will participate in activities sponsored by schools in the community.

  18. Institutional Advancement Example Goal Create a culture of communication that ensures positive, supportive recognition of the school throughout all relevant constituencies. Objective Establish a research-based plan that defines university marketing and communications needs. Expected Outcome Plan is developed and approved by the President’s Cabinet.

  19. Establishing Goals and Objectives: Recommended Actions • Examine and review stated unit objectives for all constituencies served • Elicit and discuss staff members' perceptions of unit objectives • Analyze and compare unit objectives with college/divisional Mission Statement and planning documents, those at comparable institutions, and criteria/standards of certification agencies or national associations as appropriate • Develop or revise unit Mission Statement that reflects objectives and that staff members understand, agree with, and support through their actions

  20. Typically, Unit Objectives Reflect: • Institutional effectiveness performance indicators • Documentation of all services and programs offered • Tracking of use of services (and by whom) • Constituent satisfaction with services/programs • Direct impact of services/programs on constituents (including student learning if appropriate) • Adequate resources to support planned activities • Consultation with other campus units (when joint activities are required)

  21. And Remember: Don’t try to do everything at once – typically, 5-7 objectives for each assessment plan are plenty!

  22. Step II: Evaluating Activities: “How are you attempting to meet your objectives?”

  23. What is the Purpose of This Step? • This step is basically an “audit” of existing programs, services, and resources, in order to determine if unit goals and objectives are realistic • Involves a comprehensive review, being sure to “map” objectives to existing activities • Possible results • Redundancy – same objective being addressed by multiple programs • Gaps – specific objective not being addressed by any program; could lead to: • Identifying new\reallocating existing resources • Abandoning the objective, at least for the present

  24. Evaluating Activities: Example Note. Extent to which unit addresses objective is ranked on a scale of 1-5, with higher scores indicating more emphasis.

  25. Evaluating Activities: Recommended Actions • Determine the extent to which unit objectives are reflected in unit activities, with the aim of assuring that all objectives can be met through those activities • Review and analyze coherence and inter-relatedness of unit activities in order to assure the most efficient use of resources • Identify resources that are necessary to support activities aimed at realizing all unit objectives and make sure those resources are available

  26. Step III: Measuring Outcomes: “How successfully are you meeting your objectives?”

  27. What is the Purpose of This Step? • To collect data that clearly demonstrate whether or not the unit is meeting its objectives • Requires a priori identification of appropriate strategies\measures for each objective and statement of expected outcome (i.e., what is the unit aiming for?) • Utilization of services/programs (i.e., quantity) • Performance measures (i.e., quality) • Satisfaction surveys • Comparisons with other units (i.e., benchmarking)

  28. Measuring Outcomes: Recommended Actions • Develop a detailed action plan linking objectives to outcomes, specifying strategies/actions intended to accomplish each objective and, for each action, a timeline, person/persons responsible, resources required, measures to be used, expected outcomes, and actual outcomes once assessments have been conducted • Use a wide variety of information sources, including existing data as much as possible • Develop and administer satisfaction surveys to internal and external constituent groups • Establish criteria for unit effectiveness through comparisons with information provided by similar units at other institutions or other relevant sources (e.g., certification agencies, national organizations) • Units whose functions are evaluated through SUNY-wide measures (e.g., the Student Opinion Survey, the National Survey of Student Engagement) should include these measures as performance indicators in their assessment plan

  29. Examples: TV Services Goal To provide high quality and comprehensive television broadcasting experiences to our student staff members and interns. Objectives and Measures • To expand our formal internship program. • Number of students enrolled in internships • To enhance technical aspects of the program’s facilities. • Equipment upgrades that meet standards of benchmark institutions • To increase partnerships with academic programs, high schools, and community organizations. • Number of academic programs with co-curricular requirements • To establish minimal competencies for each of the program’s student learning outcomes. • Percentage of students who meet minimal competency levels

  30. Step IV: Using Outcomes to Plan: “How can you improve what you’re doing?”

  31. What is the Purpose of This Step? • To feed back the information obtained from the outcomes assessment into the unit’s objectives and activities • Will lead to the continuation of successful practices and to the discontinuation\revision of unsuccessful practices • May also lead to new objectives

  32. Using Outcomes to Plan: Recommended Actions • Reach overall conclusions regarding unit effectiveness, based on comparisons between expected and actual outcomes and with special consideration given to the different constituent groups served by the unit • Identify major strengths and weaknesses of unit operations revealed through assessment • Make major recommendations for changes in unit activities based on assessment outcomes • Analyze relationship between available resources and unit/program success • Revise objectives and identify new outcome measures as appropriate for next assessment round

  33. Linking Goals and Objectives to Outcomes The Action Plan

  34. Essential Components of Action Plan • Goals and Objectives for Unit • Strategies or Actions Intended to Accomplish Goals and Objectives • For Each Action: • Timeline • Person/Persons Responsible • Resources Required • Expected Outcomes • Actual Outcomes (Once Actions are Complete)

  35. Action Plan Example - IT Goal: Plan and deliver integrated information services to enable members of the campus community to access information when and where they need it.

  36. Assessment Planning vs. Annual Reports: What’s the Difference?

  37. Assessment Planning • Comprehensive process consisting of four steps described earlier in presentation (i.e., mission to outcomes), following common guidelines • A unit’s first assessment plan: • typically takes a broader, longer view of the unit’s functions and intentions • will provide information for each of the four steps • will delineate all goals and objectives for a unit, but action plan will specify sub-set of those to be assessed each year • In subsequent years, annual assessment plans will highlight changes in assessment approach and specify those objectives to be assessed during the coming year

  38. Annual Reports • Required of administrative units at the College for a number of years (although not in a consistent format across divisions) • Relatively short-term and even post hoc in nature, and not necessarily guided by unit mission and broader goals • Likely to include description of events and achievements not included in assessment plan

  39. All That Said……. • IAC intentionally developed assessment planning process that, ultimately, would easily “translate” into annual reporting format • Much correspondence exists between “action plan” components and annual report grid • Eventually, once assessment planning becomes routine, there may be little distinction between that process and annual reporting • Reasonable to suggest that assessment plan could be incorporated into the annual report