Juan avalos ph d dean student services and counseling cosumnes river college
Download
1 / 73

Juan Avalos, Ph.D. Dean Student Services and Counseling Cosumnes River College - PowerPoint PPT Presentation


  • 226 Views
  • Uploaded on

Beyond SLOs and SAOs: The Importance of Rubrics and Completing the Assessment Cycle CSSO Southern Drive-In Workshop Menifee Campus, Mt. San Jacinto College Friday, October 31, 2008. Juan Avalos, Ph.D. Dean Student Services and Counseling Cosumnes River College. A little bit about myself….

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Juan Avalos, Ph.D. Dean Student Services and Counseling Cosumnes River College' - fox


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Juan avalos ph d dean student services and counseling cosumnes river college l.jpg

Beyond SLOs and SAOs: The Importance of Rubrics and Completing the Assessment Cycle CSSO Southern Drive-In WorkshopMenifee Campus, Mt. San Jacinto CollegeFriday, October 31, 2008

Juan Avalos, Ph.D.

Dean Student Services and Counseling

Cosumnes River College


A little bit about myself l.jpg
A little bit about myself… Completing the Assessment Cycle

  • Formal training in Higher Education.

  • 10 years of direct experience with educational assessment and research(national research institutes, 4-yr private, and 4-yr public).

  • 12 years of management experience in higher education(4-yr private, 4-yr public, and 2-yr public).

  • Happily married w/ four beautiful kids and a black & white Chihuahua named “Oreo”.


Presentation deliverables l.jpg
Presentation Deliverables: Completing the Assessment Cycle

Concepts

Program Models

Sample Processes

Examples

Hands-on Exercise

Resources


Presentation goals l.jpg
Presentation Goals: Completing the Assessment Cycle

  • Everyone walks away with at least one new understanding or piece of information.

  • Everyone walks away with at least one process or concept that can be implemented or used on your campus next week.


Taking a quick pulse l.jpg
Taking a Quick Pulse… Completing the Assessment Cycle

“Everyone, Please Close Your Eyes…”

Thumbs Up, Down, or Side Ways?

  • I am confident in my ability to lead my Student Services division through the development of a robust “continuous improvement” system on my campus?

  • My understanding of assessment concepts and theoretical frameworks guides my day-to-day conversations with program leads in development of said system?

  • I am aware of specific assessment processes and am confident in my ability to lead my managers, faculty, and staff through said processes?


Slide6 l.jpg

Some Sort of Break…. Completing the Assessment Cycle


Main points l.jpg
Main Points Completing the Assessment Cycle

Understanding the core elements of continuous improvement models, having a research conceptual framework, and being able to gauge the relative strength of the evidence you collect is critical to keeping you from being overwhelmed by the complexity of the assessment process.


Main points cont l.jpg
Main Points… Completing the Assessment Cycle (‘cont)

Attention has been and will continue to be devoted to the articulation of SLOs and SAOs, don’t forget about the importance of rubrics and going through the entire assessment cycle.

…There will be time for further refinement, increased rigor, and broader scope with each iteration.


Core elements of continuous improvement models l.jpg
Core Elements of “Continuous Improvement” Models Completing the Assessment Cycle

  • Desired outcome(s)

  • Activities designed to address stated outcomes

  • Assessment of progress on said outcomes

  • Dialog & reflection of research/findings

  • Implementation of improvement plan based on previous four steps (a.k.a. “closing the loop”).

    …..then back around again.


Various business and education continuous improvement methodologies l.jpg
Various Business and Education “Continuous Improvement” Methodologies

More in-depth descriptions and additional information can be found at http://www.isixsigma.com/me/

Balanced Scorecard

Baldrige

Benchmarking

Business Process Reengineering

Deming

Document Control

DMADV

DMAIC

Financial Analysis/Cost Quality

ISO 9000

Lean

Plan, Do, Check, Act—PDCA

Process Management

Project Selection

Simulation

Six Sigma

Total Quality Management

Theory of Inventive Problem Solving






Common images of continuous improvement models15 l.jpg
Common Images of “Continuous Improvement” Models… Methodologies

Plan: Design or revise business process components to improve resultsDo: Implement the plan and measure its performanceCheck: Assess the measurements and report the results to decision makersAct: Decide on changes needed to improve the process


Common images of continuous improvement models16 l.jpg
Common Images of “Continuous Improvement” Models… Methodologies

Baldridge Framework Compared to ISO


Beware of l.jpg
BEWARE OF…. Methodologies

NOISE

Lots of terms, concepts, models, tools, vendors, and phrases tied to this process that can be tremendously overwhelming, be counter productive, and turn into NOISE.

One Example: Over 200 terms present in Instructional Assessment Resources website.

//www.utexas.edu/academic/diia/assessment/iar/glossary.php





One of our challenges l.jpg
One of our challenges… Methodologies

Overcoming a “Transactions” Framework

Do you teach?

If yes, you have SLOs.

If no, you are likely to only have SAOs.


One of our challenges cont l.jpg
One of our challenges… Methodologies(‘cont)

We are probably “ok” with Service Area Outcomes since Student Service areas are traditionally “used to” thinking about operational efficiencies, service to students, student satisfaction w/ services, etc.

…the quality of service we provide and it’s impact on the student (i.e. student satisfaction, responsiveness of service, indirect impact on other student success measures, etc.).


One of our challenges cont23 l.jpg
One of our challenges… Methodologies(‘cont)

Student Learning Outcomes will be more difficult as Student Service areas have not traditionally viewed themselves as contributing to Student Learning...viewed themselves primarily as administrative support units.

The necessary paradigm shift from “transactions” to “student learning” will require us to view what we do from a “do I teach?” lens. If “yes”…What do I teach? Where is my “classroom”? What are my “lesson plans”? How do I know students are learning what I’ve intended for them to learn? etc.



Walking the walk at crc l.jpg
Walking the Walk at CRC Methodologies

Prior to 2004 at CRC

Limited opportunity for Student Services faculty and staff to come together and dialog about organizational success.

Student Services participation in campus Program Review and Unit Planning processes were isolated and misaligned.

Noel-Levtiz SSI findings pointing to potential performance gaps, in key Student Services, over an eight-year span, viewed as “inaccurate” due to perceived concerns over rigor of survey instrument and proctoring process.

Focus of many departmental meetings on “updates” and reasons why improvements cannot be made without additional people and/or funding.


Walking the walk l.jpg
Walking the Walk… Methodologies

Over the Past Two Years at CRC

Increased opportunity for Student Services faculty and staff to come together and dialog on organizational success. Second Student Services Summit scheduled for October 24, 2008.

Dean’s counseling meeting only focuses on issues for dialog and reflection…need for “updates” handled via weekly email. Weekly email also includes a “Leadership Quote for the Week”, which provides an opportunity for reflection on important concepts connected to issues we are experiencing at the time.


Walking the walk27 l.jpg
Walking the Walk… Methodologies

Over the Past Two Years at CRC (‘cont)

Student Services participation in campus Program Review more inclusive, but product a bit disjointed. Unit Plan, however, more clear and precise in articulating department-wide issues, identified strategies, and necessary resources, where applicable.

Could have dwelled on the “imperfections” of our Performance Review document, but decided to accepted it as is, take from it as much insight as we could, and focus on production of our Unit Plan. Moving forward paid off.


Walking the walk28 l.jpg
Walking the Walk Methodologies


Walking the walk31 l.jpg
Walking the Walk… Methodologies

This Year…reaping the benefits of laying our foundation four years ago.

Reduction in resistance to accept that twelve-year trend of Noel-Levtiz SSI findings pointing to potential performance gaps may be “real”. Counselors willing to consider conversation on what actions we can take in response to concerns surfaced by SSI findings.

Counseling faculty engaged in Fall 08 Convocation SLO Dialog Exercise. Willing to “go off task” and entertain initial conversation on SLO competence levels, as a precursor to a broader discussion about appropriate assessment measures.


Walking the walk32 l.jpg
Walking the Walk… Methodologies

This Year…(‘cont)

Faculty willing to continue work started during Convocation. They will propose a method for going through the exercise. It may involve smaller groups of faculty initially (…producing draft work to share with Counseling core) and a process that is not as “boring”.

Next series of conversations likely to focus on how we will ensure that we provide opportunities for students to achieve our desired SLOs within the context of our regular counseling sessions (i.e. “what will you do, in sessions with students, to help them learn the skills, abilities, etc. we expect them to learn?”).


Slide33 l.jpg

What Methodologies we did at Convocation…


Getting to work l.jpg
Getting to Work Methodologies


Getting to work38 l.jpg
Getting to Work… Methodologies

  • Instructions:

  • You will be walking to Bldg. #800, Community Room.

  • You will be grouped by candy wrapper.

  • Exercise directions and worksheets are at your table.

  • I need you back in this Assembly Hall by 11:19 am.


Getting to work39 l.jpg
Getting to Work… Methodologies


Break out session debrief l.jpg
Break-Out Session Debrief Methodologies

Think about the PROCESS….

…and what you were feeling as you went through the process.

  • What were your conversations like?

  • What were you able to do with relative ease?

  • What did you need to M-u-d-d-l-e Through?



One research framework to consider l.jpg
One Research Framework to Consider Methodologies

Astin’s I-E-O Framework

ENVIRONMENT

OUTCOME

INPUTS



Slide44 l.jpg

A Guide to Using Evidence in the Accreditation Process: A Resource to Support Institutions and Evaluation Teams

Working Draft (January 2002)

WASC Senior Colleges & Universities

Primary Author: Peter Ewell


What is evidence from wasc evidence guide l.jpg
What is Evidence? Resource to Support Institutions and Evaluation Teams(from WASC Evidence Guide)

Evidence is the substance of what is advanced to support a claim that something is true.

Evidence…

  • Intentional and purposeful.

  • Entails interpretation and reflection.

  • Integrated and holistic.

  • Can be both quantitative and qualitative.

  • Can be either direct or indirect.


What is evidence cont from wasc evidence guide l.jpg
What is Evidence? Resource to Support Institutions and Evaluation Teams(‘cont)(from WASC Evidence Guide)

Syndromes to avoid when using evidence:

  • Trying to measure everything.

  • Trying to be too precise.

  • Premature closure.


What constitutes good evidence from wasc evidence guide l.jpg
What Constitutes Good Evidence? Resource to Support Institutions and Evaluation Teams(from WASC Evidence Guide)

Five principles to using good evidence:

  • Relevant…demonstratively related to the question being investigated.

  • Verifiable…is documentable and replicable.

  • Representative…typical, not an isolated case.

  • Cumulative…gains credibility as additional sources or methods are employed.

  • Actionable…reflectively analyzed and interpreted in way to provide guidance for action/improvement.


Slide48 l.jpg

Check for Understanding… Resource to Support Institutions and Evaluation Teams


Are you asking yourself l.jpg
Are you asking yourself… Resource to Support Institutions and Evaluation Teams

How do I ensure that my Student Service operations are effective?


Are you asking yourself cont l.jpg
Are you asking yourself… Resource to Support Institutions and Evaluation Teams (‘cont)

In addition to working closely with your Student Services faculty and staff in the identification of SAOs and SLOs, attention must be given to the following:

Moving your operations through the entire Assessment Cycle. It’s okay not to be perfect the first time around. There will be time for further refinement, increased rigor, and broader scope with each iteration….


Review l.jpg
Review… Resource to Support Institutions and Evaluation Teams

Outcomes

Strategies

Improvement

Analysis


Are you asking yourself cont52 l.jpg
Are you asking yourself… Resource to Support Institutions and Evaluation Teams (‘cont)

In addition to moving through the Assessment Cycle:

For SLOs particularly, focus immediate attention to articulating levels of competency attainment…since students will be at various skill levels prior to enrolling at our college or coming to us for services. This is a critical step towards accurately and intentionally connecting all elements of the Assessment Cycle.


Review53 l.jpg
Review… Resource to Support Institutions and Evaluation Teams

Astin’s I-E-O Framework

ENVIRONMENT

OUTCOME

INPUTS


Why bamboo l.jpg
Why Bamboo? Resource to Support Institutions and Evaluation Teams

Development of a continuous improvement system has been compared to the growth of Bamboo…

…it takes several years to get established (i.e. difficult to see growth initially), then will dramatically grow from month to month.

Be patient, you may be making progress even if you can’t see it…stay the course.


Slide55 l.jpg

Some Resources… Resource to Support Institutions and Evaluation Teams


Resources l.jpg
Resources… Resource to Support Institutions and Evaluation Teams

CRC Student Service Areas First Attempt at SLOs


Resources57 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams

CRC Student Service Areas First Attempt at SLOs


Resources58 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams

CRC Student Service Areas First Attempt at SLOs


Resources59 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams

CRC Student Service Areas First Attempt at SLOs


Resources60 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams

CRC Student Service Areas First Attempt at SLOs


Resources61 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams


Resources62 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams


Resources63 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams


Resources64 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams


Resources65 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams

Assessment Planning (ABET)

http://www.abet.org/assessment.shtml


Resources66 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams

Instructional Assessment Resources

http://www.utexas.edu/academic/diia/assessment/iar/


Resources67 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams

Bloom’s Taxonomy


Resources68 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams

Bloom’s Taxonomy (con’t)

Knowledge…Ability to memorize and recall facts. Action verbs: Cite, define, identify, label, list, match, name, recognize, reproduce, select, state.

Comprehension…Ability to read course content, understand and interpret important information, and put other’s ideas into own words.Action Verbs: Classify, convert, describe, distinguish between, explain, extend, illustrate, interpret, paraphrase, summarize, translate.

Application…Ability to take new concepts and apply them to another situation.Action Verbs: Apply, compute, construct, demonstrate, discover, modify, operate, produce, relate, show, solve, use.

Analysis…Ability to take new information and break it down into parts to differentiate between them.Action Verbs: Analyze, associate, classify, arrange, diagram, discriminate, distinguish, infer, separate.


Resources69 l.jpg
Resources… Resource to Support Institutions and Evaluation Teams

Bloom’s Taxonomy (con’t)

Synthesis…Ability to take various pieces of information and form a whole creating a pattern where one did not previously exist.Action Verbs: Combine, compile, compose, construct, create, design, develop, devise, formulate, integrate, modify, organize, plan, propose, rearrange, reorganize, revise, rewrite.

Evaluation…Ability to look at someone else’s ideas or principles and see the worth of the work and the value of the conclusions.Action Verbs: Appraise, assess, compare, conclude, contrast, criticize, evaluated, judge, justify, support, weigh, rank, grade, recommend, explain.


Taking a quick pulse70 l.jpg
Taking a Quick Pulse… Resource to Support Institutions and Evaluation Teams

“Everyone, Please Close Your Eyes…”

Thumbs Up, Down, or Side Ways?

  • I am confident in my ability to lead my Student Services division through the development of a robust “continuous improvement” system on my campus?

  • My understanding of assessment concepts and theoretical frameworks guides my day-to-day conversations with program leads in development of said system?

  • I am aware of specific assessment processes and am confident in my ability to lead my managers, faculty, and staff through said processes?


Open your eyes l.jpg
Open your eyes… Resource to Support Institutions and Evaluation Teams

I need a show of hands….

  • Who feels like they gained at least one new understanding or piece of information this morning?

  • Who feels like you are walking away with at least one process or concept that you can use on your campus next week?


Don t forget l.jpg
Don’t Forget… Resource to Support Institutions and Evaluation Teams


Slide73 l.jpg

Thank You! Resource to Support Institutions and Evaluation Teams


ad