Assessment at the ttuhsc
This presentation is the property of its rightful owner.
Sponsored Links
1 / 54

Assessment at the TTUHSC PowerPoint PPT Presentation


  • 74 Views
  • Uploaded on
  • Presentation posted in: General

Assessment at the TTUHSC. Or….What does assessment have to do with me?. Assessment. Dialogue. Dialogue. Dialogue. ASSESS. PLAN. IMPROVE.

Download Presentation

Assessment at the TTUHSC

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Assessment at the ttuhsc

Assessment at the TTUHSC

Or….What does assessment have to do with me?


Assessment

Assessment

Dialogue

Dialogue

Dialogue

ASSESS

PLAN

IMPROVE

Process of collecting and reviewing evidence about the TTUHSC’s academic and administrative programs and services and using it to evaluate these programs and services to improve their quality


Institutional effectiveness

Institutional Effectiveness

The process by which the TTUHSC demonstrates how well it is succeeding in accomplishing its mission and meeting its goals


Sacs southern association of colleges and schools principles of accreditation

SACS (Southern Association of Colleges and Schools) Principles of Accreditation

2.5The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that incorporate a systematic review of programs and services that (a) results in continuing improvement and (b) demonstrates that the institution is effectively accomplishing its mission.

3.3.1The institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves those outcomes; and provides evidence of improvement based on analysis of those results.


Ttuhsc mission

TTUHSC Mission

To improve the health of people by providing educational opportunities for students and health care professionals, advancing knowledge through scholarship and research, and providing patient care and service


Ttuhsc goals

TTUHSC GOALS

  • Prepare competent health professionals and scientists

  • Increase externally funded, peer-reviewed research, especially NIH funded research, and research focused on aging, cancer, and rural health

  • Improve access to quality health care for the TTUHSC’s target populations

  • Prepare health professions students for an increasingly diverse workforce and patient population

  • Provide leadership in the development of partnerships and collaborations to improve community health

  • Operate TTUHSC as an efficient and effective institution


Relationship of types of planning at an institution

Relationship of Types of Planning at an Institution

Strategic Planning

is focused on

Process & Action

Institutional Effectiveness

is focused on

Results & Improvement

Institutional

Mission,

Goals & Strategies

Answers Question:

What actionsshould we take to support the mission & goals of the institution?

Answers Question:

How well are our students learning & our administrative services (AES) functioning?

Unit Mission/Functions

Academic Planning

Budget Planning

Administrative Planning

Facilities Planning

Means of Assessment

Use of Results

Assessment Results`

Outcomes

Inform the Planning Process

Adapted from Nichols, J.O & Karen W., A Road Map for Improvement of Student Learning and Support Services Through Assessment


What is weaveonline

What is WEAVEonline?

  • A data-based tool for tracking assessment at the unit or program level

  • Tracks both student learning outcomes for degree programs and outcomes for administrative & educational support units

  • Demonstrates links between units/programs and the TTUHSC mission, goals, strategies

  • Reporting capabilities


Weave

WEAVE


Weave1

WEAVE

Write expected outcomes

Establishcriteria for success

Assessperformance

Viewassessment results

Effect improvements


What s the assessment timeline

What’s the Assessment Timeline?

During our 2006-2007 “ramp up” year:

  • Assessment plans entered during the Fall Semester (ideally by Oct. 31, 2006)

  • Assessment activities Sept. 2006 through June 2007

  • Assessment reports due July 15, 2007

    • Includes “findings” or results

    • Includes a brief action plan on how findings will be used for improvement

  • Revised assessment plans due Sept. 2, 2007


Who will use weave

Who will use WEAVE?

  • All Academic degree programs (for student learning outcomes)

  • Administrative and educational support programs/units

    • Includes academic program units, i.e. Dept. of Pediatrics, Office of the Dean for SON, etc.

    • Administrative units, i.e. Traffic & Parking, IT-El Paso

    • Educational Support Units, i.e. Outreach Service, TTUHSC Libraries; SOAHS Office of Admissions & Student Affairs


Assessment at the ttuhsc

Help!

Office of Institutional Planning & Effectiveness (OIPE)

Add new office number

743-2918

http://www.ttuhsc.edu/admin/oipe/

Staff:

Sharon Kohout, Director

H.D. Stearman, Associate Director

Te’Ree Reese, Senior Business Assistant


Assessment 101

Assessment 101


Assessment1

Assessment

  • Assessment is a systematic process of gathering and interpreting information to discover if your program is meeting its outcomes and then using that information to enhance your program.

  • Assessment is a process designed to answer the question: “Are our efforts bringing forth the desired results?”


Who needs to do assessment

Who needs to “do” assessment?

  • All academic degree programs must conduct ongoing assessment related to student learning outcomes

  • Administrative & educational support programs or units

    • Possible criteria: Does your program or unit…..

      • have a purpose (or a mission) statement?

      • provide a unique service?

      • have a box on the organizational chart?

      • have a separate budget?

  • There are no “rules”

  • At what level does assessment makes sense if your focus is improvement of your program or services?


Where do you start

Where do you start?

  • Mission or Purpose Statement

    • A brief description of the unit or program’s mission and purpose stated in broad terms. Be sure to mention how the work of the unit supports some part of the University’s mission and/or goals.

    • Keep the mission statement to 2 – 3 sentences (for WEAVE).

    • Should be revisited annually in case services are eliminated or added.


Core functions services

Core Functions/Services

  • Four to seven functions/areas of responsibility that ensure that the mission is being accomplished

  • Helpful for determining outcomes/objectives and potential areas for improvement


Ask yourself these questions

Ask Yourself These Questions

  • What decision did you make about your unit or program last year?

  • What evidence did you use to inform that decision?

  • What was it that you were trying to influence or change about your unit/program when you made that decision?


Outcomes based assessment

Outcomes-Based Assessment

Most people…

  • Do capitalize on their innate intellectually curiosity to find out what works

  • Don’t articulate their intended end results (e.g., outcomes) ahead of time

  • Don’t document the decisions made based on their results

  • Don’t follow up later to see if their decisions made the intended improvement


The assessment cycle bresciani 2003

The Assessment Cycle(Bresciani, 2003)

Dialogue

Dialogue

Dialogue

  • The key questions…

    • What are we trying to do and why? or

    • What is my unit or program supposed to accomplish?

    • How well are we doing it?

    • How do we know?

    • How do we use the information to improve or celebrate successes?

    • Do the improvements we make work?

ASSESS

PLAN

IMPROVE


Assessment bresciani 2006

Assessment( Bresciani, 2006)

  • Most importantly, it should be:

    • Understood

    • Inclusive

    • Meaningful

    • Manageable

    • Flexible

    • Truth-seeking/objective/ethical

    • Systematic

  • Inform decisions for continuous improvement or provide evidence of proof

  • Promote a culture of accountability, of learning, and of improvement


The purpose of assessment

The Purpose of Assessment

  • Outcomes-Based assessment does not exist for assessment’s sake

  • It is taking what most of us already do, and making it systematic

  • It is NOT personnel evaluation

  • Its purpose is to reflect on the end result of doing—the outcome. Are we accomplishing that which we say we are?

Bresciani, 2002


Purpose of assessment continued

Purpose of Assessment (continued)

  • Reinforce or emphasize the mission of your unit

  • Improve programs and/or performance

  • Inform planning

  • Inform decision making

  • Evaluate programs, not personnel

Bresciani, 2002


Purpose of assessment continued1

Purpose of Assessment (continued)

  • Assist in the request for additional funds from the University and external community

  • Assist in the re-allocation of resources

  • Assist in meeting accreditation requirements, models of best practices, and national benchmarks

  • Celebrate successes

  • Create a culture of continuous improvement – a culture of accountability, of learning, and of improvement

Bresciani, 2002


Components of a ttuhsc assessment plan

Components of a TTUHSC Assessment Plan

  • Program/Unit Name

  • Program/Unit Mission or Purpose Statement

  • Outcomes

    • Student Learning or Administrative

    • Link to the TTUHSC Strategic Plan

  • Measure (Method of Assessment)

  • Target Level (Criteria for Success)

  • Findings (Assessment Results)

    • Summarize the results for each outcome

    • Summarize the process to verify/validate the results

  • Action Plan (Use of Results)

    • Summarize the decisions/recommendations made for each outcome


Mission statement

Mission Statement

  • A mission statement needs to communicate the essence or purpose of your unit or program.

  • Your mission statement answers the question:

    “What do we do and how do we support the TTUHSC mission?”


Outcomes

Outcomes

  • Outcomes are statements that describe results that you expect to see….

    • in the way you deliver your services

    • related to internal goals you might have set within your unit

    • as you carry out your functions or responsibilities

  • Outcomes are specifically about what you want the end results of your activities to be.


Guidelines for formulating outcomes

Guidelines for Formulating Outcomes

  • Must be related to something under the control of the unit

  • Should be worded in terms of what the unit will accomplish or what its clients should think, know, or do following the provision of services

  • Should lead to improved service

  • Must be linked to a TTUHSC strategy and/or goal (WEAVE will make the linking simple)

  • YOU decide which strategies and/or goals your unit best links to (can be more than one, but must link to a goal at the very least – see Goal 5)


Ideas for deciding on outcomes

Ideas for Deciding on Outcomes

  • Review your core services/ functions/responsibilities

  • Brainstorm with other staff about potential areas for improvement

  • Look at data/trends to determine issues or areas that need time and attention

  • What are areas in which you are held accountable by senior management or external audiences?

  • Remember: outcomes-based assessment is about improvement – not just “doing”

  • Start small – with just 2 or 3 outcomes


Examples of intended outcomes

Examples of Intended Outcomes

  • “All news media inquiries concerning the University will receive appropriate and timely responses.”

  • “An increasing number of undergraduate students will engage in research with faculty and graduate students.”

  • “Seminars and conferences offered by this program will provide enhanced opportunities for faculty to engage in interdisciplinary discussion and develop networks for future research collaborations.”

  • “Customers will receive prompt assistance in effectively resolving technical problems related to systems, networks, and desktop applications.”


Questions to ask yourself about outcomes

Questions to Ask Yourself About Outcomes

  • Is it measurable/identifiable?

  • Is it meaningful?

  • Is it manageable?

  • Who is the target audience of my outcome?

  • Who would know if my outcome has been met?

  • How will I know if it has been met?

  • Will it provide me with evidence that will lead me to make a decision for continuous improvement?


Before choosing a method or measure

Before Choosing A Method or Measure…

  • Think about what meeting the outcome looks like

    • For example:

    • “All news media inquiries concerning the University will receive appropriate and timely responses.”

      • What does it look like when your outcome has been met?

      • How do you currently deliver this outcome?

  • There may be clues in the delivery of the outcome that help you determine how to assess it


Before choosing a method or measure continued

Before Choosing a Method or Measure… (continued)

  • Think about collecting data

    • from different sources to make more meaningful and informed decisions for continuous improvement (e.g., surveys, observations, self-assessment)

    • that you believe will be useful in answering the important questions you have raised

    • that will appeal to your primary audience or to those with whom you are trying to influence


Measurement methods

Measurement Methods

  • Evidence of learning- basically two types

    • Direct-

      • methods of collecting information that require the customers/students to display their knowledge and skills

    • Indirect

      • methods that ask customers/students or some one else to reflect on the customer/student learning rather than to demonstrate it

(Palomba and Banta, 1999)


Some methods that provide direct evidence

Some Methods That Provide Direct Evidence

  • Observations of customer/student behavior

  • Tracking the use of a service (i.e. hits on a website)

  • Tracking processing time

  • Counts of participants/customers served

  • Tracking complaints and how they are resolved


Some methods that provide direct evidence continued

Some Methods That Provide Direct Evidence (continued)

  • Document analysis (e.g., meeting minutes, policies, handbooks)

  • Observations of work performance

  • Web tracking program analysis

  • Other document tracking analysis


Some methods that provide indirect evidence

Some Methods That Provide Indirect Evidence

  • Client satisfaction surveys (students, staff, faculty, alumni, other customers)

  • Interviews and focus groups with participants, customers, or other stakeholders

  • Benchmarks set by national, state, or peer organizations or institutions

  • Retention rates

  • Enrollment trends


Examples of measures

Examples of Measures

  • Surveys of customer satisfaction

  • Counts of program participants; growth in participation

  • Focus groups, individual interviews, phone surveys

  • Feedback from advisory groups or committees

  • Analysis of service usage


Choosing an instrument

Choosing an Instrument

  • What outcome(s) are you measuring?

  • Who is being assessed? How often do I have access to them? Do I know who they are?

  • What is my budget?

  • What is my timeline?

  • What type of data is most meaningful to me: direct/indirect and qualitative/quantitative

  • Who will have responsibility for data collection?


Target levels criteria for success

Target Levels (Criteria for Success)

  • What target will you set for yourself during the coming year in order to accomplish your outcome?

  • What level of accomplishment do you hope to see?

  • Target levels should specific, measurable, and attainable.

  • You are encouraged to set more than one target level or criteria for success for each outcome.


Examples outcomes with criteria for success target level

Examples: Outcomes with Criteria for Success/Target Level

  • Students will be satisfied with the health care provider on their campus.

    • Target: On Student Satisfaction Survey, 90% will respond that they are satisfied or very satisfied with health care provider on their campus.

  • Instructional support services will be provided to meet the needs of TechLink faculty.

    • Target: By Dec. 2006, faculty using TechLink will be surveyed to determine three most critical areas of need

    • Target: By June 2007, a workplan for addressing the major areas of need will be presented to IT leadership.


Conduct assessment activities

Conduct Assessment Activities

  • Be sure you have a plan for conducting the activities that will lead to accomplishment of your target level


Findings results of assessments conducted

Findings (Results of Assessments Conducted)

Summarize briefly the major findings from your assessments

  • “70% of respondents indicated that they had received a response to their request for service within 24 hours.”

  • “Customers expressed frustration with the wait time for help desk requests.”

  • “Feedback gathered from our advisory group indicated that department staff desire additional assistance with web design.”


Closing the loop

Closing the Loop

  • Briefly report method of assessment for each outcome

  • Document where the customers/students are meeting the intended outcome

  • Document where they are not meeting the outcome

  • Document decisions made to improve the program and assessment plan

  • Refine assessment method and repeat process after proper time for implementation


Action plans use of results for improvement

Action Plans (Use of Results for Improvement)

  • How do you plan to use the results of your assessments to improve the unit’s programs or services?

  • Briefly describe any improvements, changes, or plans that you will undertake

  • If you meet your target levels, review your outcomes to see if other improvements or activities are needed


Examples of improvement statements

Examples of Improvement Statements

  • “A new campaign emphasizing to new graduates the benefits of annual giving to the TTUHSC was developed and implemented in July 2004. We continue to monitor our progress in encouraging this group to contribute.”

  • “Since our service demands have increased, additional staff are needed to provide adequate coverage, and will be requested in the next fiscal year.”

  • “Our focus groups interviews with users indicated a preference for electronic reports, so the hard copy version was discontinued.”


Demonstrate document use of results for improvements

Demonstrate & Document Use of Results for Improvements

  • Most important step = data-based decision making

  • Data collected from doing assessment often will inform Strategic Planning


Relationship of types of planning at an institution1

Relationship of Types of Planning at an Institution

Strategic Planning

is focused on

Process & Action

Institutional Effectiveness

is focused on

Results & Improvement

Institutional

Mission,

Goals & Strategies

Answers Question:

What actionsshould we take to support the mission & goals of the institution?

Answers Question:

How well are our students learning & administrative services (AES) functioning?

Unit Mission/Functions

Academic Planning

Budget Planning

Administrative Planning

Facilities Planning

Means of Assessment

Use of Results

Assessment Results`

Outcomes

Inform the Planning Process

Adapted from Nichols, J.O & Karen W., A Road Map for Improvement of Student Learning and Support Services Through Assessment


Take home messages

Take-Home Messages

  • You do not have to assess everything you do every year.

  • You don’t have to do everything at once-start with a reasonable number of intended outcomes

  • Think baby steps

  • Be flexible

  • Acknowledge and use what you have already done.


Assessment at the ttuhsc

More Take-Home Messages

  • Assessment expertise is available to help you evaluate your program better – not to evaluate your program for you

  • Borrow examples from other institutions to modify as appropriate

  • Time for this must be re-allocated

  • We allocate time according to our values and priorities


Resources

Resources

  • Each Other

  • Web Resources

  • http://www2.acs.ncsu.edu/UPA/assmt/

  • http://www2.acs.ncsu.edu/UPA/assmt/resource.htm#hbooks

  • http://www.unt.edu/SACS/ie_nonacademic/index.htm


Assessment at the ttuhsc

Help!

Office of Institutional Planning & Effectiveness (OIPE)

743-2918

http://www.ttuhsc.edu/admin/oipe/

Staff:

Sharon Kohout, Director

H.D. Stearman, Associate Director

Te’Ree Reese, Senior Business Assistant


  • Login