Using cas standards and review process for improving student learning and program outcomes
This presentation is the property of its rightful owner.
Sponsored Links
1 / 44

Using CAS Standards and Review Process for Improving Student Learning and Program Outcomes PowerPoint PPT Presentation


  • 61 Views
  • Uploaded on
  • Presentation posted in: General

Using CAS Standards and Review Process for Improving Student Learning and Program Outcomes. Patricia Carretta George Mason University CAS Officer-at-Large NACE Representative to CAS NASPA 2008 International Assessment & Retention Conference. Purpose. CAS Standards and Review Process

Download Presentation

Using CAS Standards and Review Process for Improving Student Learning and Program Outcomes

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Using cas standards and review process for improving student learning and program outcomes

Using CAS Standards and Review Process for Improving Student Learning and Program Outcomes

Patricia Carretta

George Mason University

CAS Officer-at-Large

NACE Representative to CAS

NASPA 2008 International Assessment & Retention Conference


Purpose

Purpose

  • CAS Standards and Review Process

  • Integrating CAS into assessment of programs and services and student learning outcomes


Role and use of cas standards

Role and Use of CAS Standards:

  • Student Affairs / Student Services at the division level?

  • Student Affairs / Student Services at the departmental or program level?

  • Other?

  • Have looked at the CAS Standards for at least one functional area?

  • Have participated in a department/program review process using CAS standards?

  • Have used CAS standards for other purposes?


Participant outcomes

Participant Outcomes

  • Define the role of CAS self-assessment / review process within a comprehensive assessment plan

  • List the seven step process of conducting a CAS review

  • Appreciate the value of engaging a variety of institutional stakeholders in the review process

  • Strategize how to conduct a comprehensive assessment of programs, services and student learning using CAS Standards

  • Formulate an assessment plan

  • Understand the importance of assessing and reporting on input data as well as student learning and development outcomes as a strategies for improving learning and program quality


About cas

About CAS:

The mission of the Council for the Advancement of Standards in Higher Education (CAS) is to promote the improvement of programs and services to enhance the quality of student learning and development.

CAS is a consortium of professional associations who work collaboratively to develop and promulgate standards and guidelines and to encourage self-assessment.


Cas resources

CAS Resources

  • CAS Professional Standards, 2006 edition

  • Self-Assessment Guides

  • CAS Statement of Shared Ethical Principles

  • CAS Characteristics of Individual Excellence

  • Frameworks for Assessing Learning and Development Outcomes


Cas guiding principles

CAS Guiding Principles

  • Students and Their Institutions

  • Diversity and Multiculturalism

  • Organization, Leadership, and Human Resources

  • Health Engendering Environments

  • Ethical Considerations


Organization of cas standards

Mission

Program

Leadership

Human Resources

Ethics

Legal Responsibilities

Equity and Access

Diversity

Organization and Management

Campus and External Relations

Financial Resources

Technology

Facilities and Equipment

Assessment and Evaluation

Organization of CAS Standards


Student learning development

Student Learning & Development

  • Must promote student learning and development outcomes

  • Must identify relevant and desirable student learning & development outcomes

  • Must assess outcomes

  • Must provide evidence of their impact on student learning and development

  • Must articulate how they contribute to or support student learning and development


Cas student learning development outcomes

CAS: Student Learning & Development Outcomes

  • Knowledge acquisition, construction, integration and application

  • Cognitive complexity

  • Intrapersonal development

  • Interpersonal competence

  • Humanitarianism & civic engagement

  • Practical competence


Student learning development outcomes

Student Learning & Development Outcomes

  • Consistent with the institutional mission, programs and services must identify relevant and desirable student learning and development outcomes from among the six domains. When creating opportunities for student learning and development, programs and services must explore possibilities for collaboration with faculty members and other colleagues.

  • Programs and services must assess relevant and desirable student learning and development outcomes and provide evidence of their impact on student learning and development. Programs and services must articulate how they contribute to or support student learning and development in the domains not specifically assessed.


Essential elements in the cas approach

Essential Elementsin the CAS Approach

  • Culture that values involvement of all its members

  • Quality indicators

  • Use of standards and guidelines

  • Willingness/capacity:

    • To examine itself and assemble results

    • To report and use the results


Uses of cas standards

Uses of CAS Standards

  • Credibility, accountability, improvement:

    • Program & service improvement; measures of quality and effectiveness; measures of impact on learning

    • Design of new programs & services

    • Restructuring / reorganization

    • Change in leadership

    • Institutional self-studies

    • Preparation for accreditation

  • Staff development

  • Academic preparation


Cas standards and guidelines

CAS: Standards and Guidelines

STANDARD (BOLD TYPE; AUXILARY VERB MUST)

To support the institution’s mission and goals, Career services must provide faculty and staff and administrative units with information, guidance, and support on career development and employment issues and linkages with the broader community.

GUIDELINE (LIGHT-FACED TYPE; VERB SHOULD)

Career Services should support faculty and staff and administrative units by:

  • Identifying and disseminating information on employment trends, top employing organizations, and co-op and internship sponsors.


Comprehensive assessment plan

Comprehensive Assessment Plan

Includes:

  • Assessment of programs and services

  • Assessment of student outcomes, including student learning and development


Program evaluation

Program Evaluation

  • Is the program or service functioning effectively to achieve its mission?

  • What evidence is available to support the determination?

  • How is evidence used to make program decisions?


Student outcomes assessment

Student Outcomes Assessment

  • What is the effect of our work on students?

  • How are they different as a result of interacting with our programs and services?

  • How do we demonstrate their learning?

  • How do we know? Measure?


The cas sag a self assessment guide

The CAS SAG: A Self-Assessment Guide

  • Translates CAS standards into an effective workbook / evaluative format

  • Promotes program self-assessment and development

  • Informs on program strengths and weaknesses

  • Supports professional staff development

  • Leads to enhanced programs and services that benefit student learning and development


Cas self assessment process

CAS Self-Assessment Process

  • Establish and prepare the Self-Assessment Team

  • Initiate the Self Study

  • Compile and review Documentary and Evaluative Evidence

  • Identify Discrepancies

  • Determine appropriate Corrective Action

  • Recommend Steps for Program Enhancement

  • Prepare an Action Plan


The self assessment team

The Self-Assessment Team

Appoint and Train the Team:

  • Size of team

  • Composition: internal, external

  • Coordinator/Leader

  • Establish team ground rules

  • Discuss meaning of each standard

  • Establish team’s inter-rater reliability

  • Encourage team discussion; expect disagreements; commit to consensual resolution


Team actions

Team Actions

  • Decide whether to include guidelines or other measures that go beyond the standards

  • Gather and analyze quantitative and qualitative data

  • Individuals rate each and every criterion measure

  • Obtain additional documentary evidence if required to make an informed team decision

  • Complete the assessment, ratings & action plan worksheets


Evaluative evidence

Evaluative Evidence

STUDENTRECRUITMENT MATERIALS:

  • Brochures & other program information

  • Participation policies & procedures

    PROGRAM DOCUMENTS:

  • Mission statements; program purpose & philosophy statements

  • Catalogs & related materials

  • Staff & student manuals; policies & procedures statements

    ADMINISTRATIVE DOCUMENTS:

  • Organization charts; student & staff profiles

  • Financial resource statements & budgets

  • Annual reports

    STAFF ACTIVITY REPORTS:

  • Curriculum vitae & resumes; professional activity

  • Service to other programs, departments, or community

    STUDENT ACTIVITY REPORTS:

  • Portfolios, developmental transcripts, resumes

  • Reports of student service

    RESEARCH & EVALUATION DATA:

  • Needs assessments & self-studies

  • Program evaluation; graduate & follow-up studies


Quantitative data

Quantitative Data

  • Summary of objective responses on a questionnaire or program evaluation

  • Statistics about use by students and various population subgroups

  • Needs assessments, follow-up studies, and self-study reports

  • Institutional research reports and fact books

  • Comparative or benchmark data at local, regional or national level


Qualitative data

Qualitative Data

  • Focus group information

  • Written summaries of responses to open-ended questions in interviews and on evaluations

  • Client satisfaction surveys, self-reports, and written comments, both solicited and unsolicited


Using the sags

Using the SAGs

  • SAGs offer a ready format for evaluation

  • In most instances, there are multiple criterion statements for each standard

  • Each criterion measure focuses on a particular aspect of the standard, allowing raters to express more detailed and specific judgments


Criterion measure rating scale

Criterion Measure Rating Scale

ND1234NR

Not Done Not met Minimally Well Fully Not Rated

MetMetMet

ND1234NR

Not Done Not met Minimally Well Fully Not Rated

MetMetMet

  • Using this scale, consider each criterion statement and decide…

  • The extent to which each criterion measure has been met by the program or service


Assessment criteria example

Assessment Criteria Example

Part 6: FINANCIAL RESOURCES

Counseling Services (CS) must have adequate funding to accomplish its mission and goals. Funding priorities must be determined within the context of the stated mission, goals, objectives, and comprehensive analysis of the needs and capabilities of students and the availability of internal or external resources.

CS must demonstrate fiscal responsibility and cost effectiveness consistent with institutional protocols.

ND 1 2 3 4 NR

Not DoneNot Met Minimally Met Well MetFully MetNot Rated

PART 6. FINANCIAL RESOURCES (Criterion Measures) Rating ScaleNOTES

6.1 The program has adequate funding to accomplish its mission and goals. ND 1 2 3 4 NR

6.2 Funding priorities are determined within the context of program mission,

student needs, and available fiscal resources.ND 1 2 3 4 NR

6.3The program demonstrates fiscal responsibility and cost effectiveness

consistent with institutional protocols. ND 1 2 3 4 NR

Part 6: Financial Resources Overview Questions

A. What is the funding strategy for the program?

B. What evidence exists to confirm fiscal responsibility and cost-effectiveness?


Cas work form a assessment ratings and significant items

CAS: Work Form AAssessment, Ratings, and Significant Items

CAS

Work Form A

Assessment, Ratings, and Significant Items

INSTRUCTIONS:

This work form should be completed following individual ratings of the participants. For each of the 13 Parts, identify (circle) the criterion measure item number(s) in the column labeled for which there is a substantial rating discrepancy (two or more ratings apart). Items not circled should reflect consensus among judges that practice in that area is satisfactory. Items where judgment variance occurs need to be discussed thoroughly by team members. Follow this action by determining which practices (criterion measures) can be designated as “excellent” or “unsatisfactory” and record them in the Step One column. In Step Two, list the items requiring follow-up action including any criterion measure rated as being unsatisfactory by any reviewer.


Cas work form a continued

CAS: Work Form A continued


Cas work form b follow up actions

CAS Work Form B: Follow Up Actions

INSTRUCTIONS:

The purpose of this work form is to begin the planning for action to be taken on practices judged to merit follow-up (See Step 1, Work Form A). In Step Three, transfer short descriptions of the practices requiring follow-up and detail these items using the table format provided.

Step Three: Describe the current practice that requires change and actions to initiate the change


Cas work form c summary action plan

CAS Work Form C:Summary Action Plan

Step Four:

This form concludes the self-assessment process and calls for action to be taken as a consequence of study results. Write a brief action plan statement in the spaces below for each Part in which action is required.


Closing the loop

Closing the loop

  • Purpose & scope of study is defined

  • Self-study team is trained

  • Self-study is conducted

  • Evaluative evidence is collected

  • Discrepancies are identified

  • Appropriate actions are determined

  • Special actions for program enhancement are recommended

  • Action plan is developed and communicated


Student outcomes assessment1

Student Outcomes Assessment

  • What is the effect of our work on students?

  • How are they different as a result of interacting with our programs and services?

  • What have students learned?

  • How have they developed?


Why focus on student learning

Why Focus on Student Learning?

  • Accountability

  • Alignment with institution’s mission

  • Consistent with language of the academy

  • Quality improvement

  • More effective learning

  • Better defined and more effective planning and strategy development


Defining learning outcomes

Defining Learning Outcomes

“Outcomes usually identify growth in some dimension of knowing, being or doing.”

Susan Komives & Sarah Schoper, “Developing Learning Outcomes,”

Learning Reconsidered 2 (2006)

  • Knowledge (cognitive)

  • Attitude (affective)

  • Skill (psychomotor)


Cas student learning development outcomes1

CAS: Student Learning & Development Outcomes

  • Knowledge acquisition, construction, integration and application

  • Cognitive complexity

  • Intrapersonal development

  • Interpersonal competence

  • Humanitarianism & civic engagement

  • Practical competence


Plan for assessing student learning outcomes

Plan for Assessing Student Learning Outcomes

  • Review/revise mission

  • Identify major services, programs, activities

  • Specify desired student outcomes

  • Identify information needed to show student learning

  • Identify existing and new data needed

  • Develop process for measuring outcomes

  • Identify ways to use findings to change/improve


Writing learning outcomes

Writing Learning Outcomes

  • Identify words/phrases that describe what an ideal program participant or college graduate would be able to:

    • Know – Be – Do … as a result of engaging with your program, activity or service.

    • What would he/she learn from the experience?


Student outcomes matrix

Student Outcomes Matrix


Writing learning outcomes1

Writing Learning Outcomes

  • Describe the learner – characteristics of users/participants of programs and services. What knowledge, skills, experiences do they start with?

  • Determine acceptable levels of performance

  • Consider how you would provide evidence of achievement


Ways to measure learning outcomes

Ways to measure learning outcomes

  • Surveys, questionnaires, self-report (indirect)

  • Observations

  • Interviews

  • Focus groups

  • Reflective journals

  • Performance evaluations

  • Pre-test/post-test

  • Portfolios

  • Capstone projects

  • Review of project/performance by peers and internal/external reviewers


Tips suggestions

Tips, suggestions

  • Identify all your program outcomes

  • Rank outcomes based on importance

    • Activity outcome

    • Program outcome

    • Institutional or division outcome

  • Identify where learning opportunities are occurring (mapping)

  • Design and implement assessment cycle – one outcome at a time


Remember

Remember:

  • Start small

  • Start doable

  • Start narrow

  • Start targeted

  • Start!

    • One program, one workshop, one desired outcome, one measured tool


Additional cas resources

Additional CAS Resources

  • www.cas.edu

  • Standards book

    • Prologue & Context with “How to” information

  • FALDOs – book and CD formats as package

  • SAGs – CD or individually


  • Login