1 / 54

Assessment at the TTUHSC

Assessment at the TTUHSC. Or….What does assessment have to do with me?. Assessment. Dialogue. Dialogue. Dialogue. ASSESS. PLAN. IMPROVE.

maxim
Download Presentation

Assessment at the TTUHSC

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessment at the TTUHSC Or….What does assessment have to do with me?

  2. Assessment Dialogue Dialogue Dialogue ASSESS PLAN IMPROVE Process of collecting and reviewing evidence about the TTUHSC’s academic and administrative programs and services and using it to evaluate these programs and services to improve their quality

  3. Institutional Effectiveness The process by which the TTUHSC demonstrates how well it is succeeding in accomplishing its mission and meeting its goals

  4. SACS (Southern Association of Colleges and Schools) Principles of Accreditation 2.5The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that incorporate a systematic review of programs and services that (a) results in continuing improvement and (b) demonstrates that the institution is effectively accomplishing its mission. 3.3.1 The institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves those outcomes; and provides evidence of improvement based on analysis of those results.

  5. TTUHSC Mission To improve the health of people by providing educational opportunities for students and health care professionals, advancing knowledge through scholarship and research, and providing patient care and service

  6. TTUHSC GOALS • Prepare competent health professionals and scientists • Increase externally funded, peer-reviewed research, especially NIH funded research, and research focused on aging, cancer, and rural health • Improve access to quality health care for the TTUHSC’s target populations • Prepare health professions students for an increasingly diverse workforce and patient population • Provide leadership in the development of partnerships and collaborations to improve community health • Operate TTUHSC as an efficient and effective institution

  7. Relationship of Types of Planning at an Institution Strategic Planning is focused on Process & Action Institutional Effectiveness is focused on Results & Improvement Institutional Mission, Goals & Strategies Answers Question: What actionsshould we take to support the mission & goals of the institution? Answers Question: How well are our students learning & our administrative services (AES) functioning? Unit Mission/Functions Academic Planning Budget Planning Administrative Planning Facilities Planning Means of Assessment Use of Results Assessment Results` Outcomes Inform the Planning Process Adapted from Nichols, J.O & Karen W., A Road Map for Improvement of Student Learning and Support Services Through Assessment

  8. What is WEAVEonline? • A data-based tool for tracking assessment at the unit or program level • Tracks both student learning outcomes for degree programs and outcomes for administrative & educational support units • Demonstrates links between units/programs and the TTUHSC mission, goals, strategies • Reporting capabilities

  9. WEAVE

  10. WEAVE Write expected outcomes Establishcriteria for success Assessperformance Viewassessment results Effect improvements

  11. What’s the Assessment Timeline? During our 2006-2007 “ramp up” year: • Assessment plans entered during the Fall Semester (ideally by Oct. 31, 2006) • Assessment activities Sept. 2006 through June 2007 • Assessment reports due July 15, 2007 • Includes “findings” or results • Includes a brief action plan on how findings will be used for improvement • Revised assessment plans due Sept. 2, 2007

  12. Who will use WEAVE? • All Academic degree programs (for student learning outcomes) • Administrative and educational support programs/units • Includes academic program units, i.e. Dept. of Pediatrics, Office of the Dean for SON, etc. • Administrative units, i.e. Traffic & Parking, IT-El Paso • Educational Support Units, i.e. Outreach Service, TTUHSC Libraries; SOAHS Office of Admissions & Student Affairs

  13. Help! Office of Institutional Planning & Effectiveness (OIPE) Add new office number 743-2918 http://www.ttuhsc.edu/admin/oipe/ Staff: Sharon Kohout, Director H.D. Stearman, Associate Director Te’Ree Reese, Senior Business Assistant

  14. Assessment 101

  15. Assessment • Assessment is a systematic process of gathering and interpreting information to discover if your program is meeting its outcomes and then using that information to enhance your program. • Assessment is a process designed to answer the question: “Are our efforts bringing forth the desired results?”

  16. Who needs to “do” assessment? • All academic degree programs must conduct ongoing assessment related to student learning outcomes • Administrative & educational support programs or units • Possible criteria: Does your program or unit….. • have a purpose (or a mission) statement? • provide a unique service? • have a box on the organizational chart? • have a separate budget? • There are no “rules” • At what level does assessment makes sense if your focus is improvement of your program or services?

  17. Where do you start? • Mission or Purpose Statement • A brief description of the unit or program’s mission and purpose stated in broad terms. Be sure to mention how the work of the unit supports some part of the University’s mission and/or goals. • Keep the mission statement to 2 – 3 sentences (for WEAVE). • Should be revisited annually in case services are eliminated or added.

  18. Core Functions/Services • Four to seven functions/areas of responsibility that ensure that the mission is being accomplished • Helpful for determining outcomes/objectives and potential areas for improvement

  19. Ask Yourself These Questions • What decision did you make about your unit or program last year? • What evidence did you use to inform that decision? • What was it that you were trying to influence or change about your unit/program when you made that decision?

  20. Outcomes-Based Assessment Most people… • Do capitalize on their innate intellectually curiosity to find out what works • Don’t articulate their intended end results (e.g., outcomes) ahead of time • Don’t document the decisions made based on their results • Don’t follow up later to see if their decisions made the intended improvement

  21. The Assessment Cycle(Bresciani, 2003) Dialogue Dialogue Dialogue • The key questions… • What are we trying to do and why? or • What is my unit or program supposed to accomplish? • How well are we doing it? • How do we know? • How do we use the information to improve or celebrate successes? • Do the improvements we make work? ASSESS PLAN IMPROVE

  22. Assessment( Bresciani, 2006) • Most importantly, it should be: • Understood • Inclusive • Meaningful • Manageable • Flexible • Truth-seeking/objective/ethical • Systematic • Inform decisions for continuous improvement or provide evidence of proof • Promote a culture of accountability, of learning, and of improvement

  23. The Purpose of Assessment • Outcomes-Based assessment does not exist for assessment’s sake • It is taking what most of us already do, and making it systematic • It is NOT personnel evaluation • Its purpose is to reflect on the end result of doing—the outcome. Are we accomplishing that which we say we are? Bresciani, 2002

  24. Purpose of Assessment (continued) • Reinforce or emphasize the mission of your unit • Improve programs and/or performance • Inform planning • Inform decision making • Evaluate programs, not personnel Bresciani, 2002

  25. Purpose of Assessment (continued) • Assist in the request for additional funds from the University and external community • Assist in the re-allocation of resources • Assist in meeting accreditation requirements, models of best practices, and national benchmarks • Celebrate successes • Create a culture of continuous improvement – a culture of accountability, of learning, and of improvement Bresciani, 2002

  26. Components of a TTUHSC Assessment Plan • Program/Unit Name • Program/Unit Mission or Purpose Statement • Outcomes • Student Learning or Administrative • Link to the TTUHSC Strategic Plan • Measure (Method of Assessment) • Target Level (Criteria for Success) • Findings (Assessment Results) • Summarize the results for each outcome • Summarize the process to verify/validate the results • Action Plan (Use of Results) • Summarize the decisions/recommendations made for each outcome

  27. Mission Statement • A mission statement needs to communicate the essence or purpose of your unit or program. • Your mission statement answers the question: “What do we do and how do we support the TTUHSC mission?”

  28. Outcomes • Outcomes are statements that describe results that you expect to see…. • in the way you deliver your services • related to internal goals you might have set within your unit • as you carry out your functions or responsibilities • Outcomes are specifically about what you want the end results of your activities to be.

  29. Guidelines for Formulating Outcomes • Must be related to something under the control of the unit • Should be worded in terms of what the unit will accomplish or what its clients should think, know, or do following the provision of services • Should lead to improved service • Must be linked to a TTUHSC strategy and/or goal (WEAVE will make the linking simple) • YOU decide which strategies and/or goals your unit best links to (can be more than one, but must link to a goal at the very least – see Goal 5)

  30. Ideas for Deciding on Outcomes • Review your core services/ functions/responsibilities • Brainstorm with other staff about potential areas for improvement • Look at data/trends to determine issues or areas that need time and attention • What are areas in which you are held accountable by senior management or external audiences? • Remember: outcomes-based assessment is about improvement – not just “doing” • Start small – with just 2 or 3 outcomes

  31. Examples of Intended Outcomes • “All news media inquiries concerning the University will receive appropriate and timely responses.” • “An increasing number of undergraduate students will engage in research with faculty and graduate students.” • “Seminars and conferences offered by this program will provide enhanced opportunities for faculty to engage in interdisciplinary discussion and develop networks for future research collaborations.” • “Customers will receive prompt assistance in effectively resolving technical problems related to systems, networks, and desktop applications.”

  32. Questions to Ask Yourself About Outcomes • Is it measurable/identifiable? • Is it meaningful? • Is it manageable? • Who is the target audience of my outcome? • Who would know if my outcome has been met? • How will I know if it has been met? • Will it provide me with evidence that will lead me to make a decision for continuous improvement?

  33. Before Choosing A Method or Measure… • Think about what meeting the outcome looks like • For example: • “All news media inquiries concerning the University will receive appropriate and timely responses.” • What does it look like when your outcome has been met? • How do you currently deliver this outcome? • There may be clues in the delivery of the outcome that help you determine how to assess it

  34. Before Choosing a Method or Measure… (continued) • Think about collecting data • from different sources to make more meaningful and informed decisions for continuous improvement (e.g., surveys, observations, self-assessment) • that you believe will be useful in answering the important questions you have raised • that will appeal to your primary audience or to those with whom you are trying to influence

  35. Measurement Methods • Evidence of learning- basically two types • Direct- • methods of collecting information that require the customers/students to display their knowledge and skills • Indirect • methods that ask customers/students or some one else to reflect on the customer/student learning rather than to demonstrate it (Palomba and Banta, 1999)

  36. Some Methods That Provide Direct Evidence • Observations of customer/student behavior • Tracking the use of a service (i.e. hits on a website) • Tracking processing time • Counts of participants/customers served • Tracking complaints and how they are resolved

  37. Some Methods That Provide Direct Evidence (continued) • Document analysis (e.g., meeting minutes, policies, handbooks) • Observations of work performance • Web tracking program analysis • Other document tracking analysis

  38. Some Methods That Provide Indirect Evidence • Client satisfaction surveys (students, staff, faculty, alumni, other customers) • Interviews and focus groups with participants, customers, or other stakeholders • Benchmarks set by national, state, or peer organizations or institutions • Retention rates • Enrollment trends

  39. Examples of Measures • Surveys of customer satisfaction • Counts of program participants; growth in participation • Focus groups, individual interviews, phone surveys • Feedback from advisory groups or committees • Analysis of service usage

  40. Choosing an Instrument • What outcome(s) are you measuring? • Who is being assessed? How often do I have access to them? Do I know who they are? • What is my budget? • What is my timeline? • What type of data is most meaningful to me: direct/indirect and qualitative/quantitative • Who will have responsibility for data collection?

  41. Target Levels (Criteria for Success) • What target will you set for yourself during the coming year in order to accomplish your outcome? • What level of accomplishment do you hope to see? • Target levels should specific, measurable, and attainable. • You are encouraged to set more than one target level or criteria for success for each outcome.

  42. Examples: Outcomes with Criteria for Success/Target Level • Students will be satisfied with the health care provider on their campus. • Target: On Student Satisfaction Survey, 90% will respond that they are satisfied or very satisfied with health care provider on their campus. • Instructional support services will be provided to meet the needs of TechLink faculty. • Target: By Dec. 2006, faculty using TechLink will be surveyed to determine three most critical areas of need • Target: By June 2007, a workplan for addressing the major areas of need will be presented to IT leadership.

  43. Conduct Assessment Activities • Be sure you have a plan for conducting the activities that will lead to accomplishment of your target level

  44. Findings (Results of Assessments Conducted) Summarize briefly the major findings from your assessments • “70% of respondents indicated that they had received a response to their request for service within 24 hours.” • “Customers expressed frustration with the wait time for help desk requests.” • “Feedback gathered from our advisory group indicated that department staff desire additional assistance with web design.”

  45. Closing the Loop • Briefly report method of assessment for each outcome • Document where the customers/students are meeting the intended outcome • Document where they are not meeting the outcome • Document decisions made to improve the program and assessment plan • Refine assessment method and repeat process after proper time for implementation

  46. Action Plans (Use of Results for Improvement) • How do you plan to use the results of your assessments to improve the unit’s programs or services? • Briefly describe any improvements, changes, or plans that you will undertake • If you meet your target levels, review your outcomes to see if other improvements or activities are needed

  47. Examples of Improvement Statements • “A new campaign emphasizing to new graduates the benefits of annual giving to the TTUHSC was developed and implemented in July 2004. We continue to monitor our progress in encouraging this group to contribute.” • “Since our service demands have increased, additional staff are needed to provide adequate coverage, and will be requested in the next fiscal year.” • “Our focus groups interviews with users indicated a preference for electronic reports, so the hard copy version was discontinued.”

  48. Demonstrate & Document Use of Results for Improvements • Most important step = data-based decision making • Data collected from doing assessment often will inform Strategic Planning

  49. Relationship of Types of Planning at an Institution Strategic Planning is focused on Process & Action Institutional Effectiveness is focused on Results & Improvement Institutional Mission, Goals & Strategies Answers Question: What actionsshould we take to support the mission & goals of the institution? Answers Question: How well are our students learning & administrative services (AES) functioning? Unit Mission/Functions Academic Planning Budget Planning Administrative Planning Facilities Planning Means of Assessment Use of Results Assessment Results` Outcomes Inform the Planning Process Adapted from Nichols, J.O & Karen W., A Road Map for Improvement of Student Learning and Support Services Through Assessment

More Related