slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
From Compliance to Improvement: Accountability and Assessment for California’s Community Colleges PowerPoint Presentation
Download Presentation
From Compliance to Improvement: Accountability and Assessment for California’s Community Colleges

Loading in 2 Seconds...

play fullscreen
1 / 83

From Compliance to Improvement: Accountability and Assessment for California’s Community Colleges - PowerPoint PPT Presentation


  • 153 Views
  • Uploaded on

From Compliance to Improvement: Accountability and Assessment for California’s Community Colleges. Norena Norton Badway, Ph.D. University of the Pacific Higher Education Evaluation and Research Group. Introduction: Choosing Improvement over Compliance. Compliance…Improvement.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'From Compliance to Improvement: Accountability and Assessment for California’s Community Colleges' - altsoba


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

From Compliance to Improvement: Accountability and Assessment for California’s Community Colleges

Norena Norton Badway, Ph.D.

University of the Pacific

Higher Education Evaluation and Research Group

slide2

Introduction:

Choosing

Improvement

over

Compliance

slide3

Compliance…Improvement

Compliance

Improvement

slide4

Improvement…Compliance

Compliance

Improvement

slide5

SayingYES to Assessment and Accountability

• acknowledges community colleges’ appropriate roles in equity, upgrade training, lifelong learning, and other unconventional missions

• gives faculty an appropriate voice in

running their institutions

• promotes a form of research in teaching and the creation of improvements in teaching

• provides a foundation for widespread institutional improvement

• become more effective learning environments

slide6

Accountability is NOT new in CA

California Community Colleges operate

under at least four accountability systems:

  • PFE uses system level goals
  • State Report Card assesses performance of all publicly funded workforce preparation programs
  • Federal Vocational and Technical Education Act
  • Workforce Investment Act
slide7

… and now WASC

  • WASC has been the last regional accreditation commission to require colleges to develop mechanisms of assessment and use of student learning outcomes

•••WASC lets LOCALITIES choose which aspects of SLOs to measure and how to measure them

slide8

Well developed system

of internal accountability

Ability to respond

to external

accountability requirements

slide9

Taking Stock

Of Existing

Resources

slide11

It is unlikely that any campus

does not have in place

— either informally or formally —

some aspects of a

Cycle of Inquiry

about Student Learning.

taking stock acknowledges existing practices
TAKING STOCK Acknowledges Existing Practices

Presenting

student learning outcomes

assessment cycles

as if it is a totally new process

ignores existing practices.

taking stock fosters faculty dialogue
TAKING STOCKFosters Faculty Dialogue

Dialogue

about a department’s philosophy about teaching and learning guides the choice of

student learning outcomes.

taking stock facilitates norming
TAKING STOCKFacilitates Norming

• Expectations for for what students know and can demonstrate upon completion of the course are collaboratively authored and collectively accepted.1

• Full time and adjunct faculty who teach a course come to consensus.

1Maki, P.L. (2004). Assessing for Learning. American Association for Higher Education, Sterling, VA: Stylus.

the benefit of alignment
The Benefit of Alignment

When these five elements

• student expectations,

• faculty expectations,

• curriculum content,

• institutional support,

• governance for assessment)

are in alignment, or in equilibrium, classrooms and student learning are likely to run smoothly.

the risks of non alignment
The Risks of Non-alignment
  • If professors disagree with the curriculum, they may undermine or embellish it, for good or bad.
  • If institutions fail to support faculty, they may undermine the possibility of being a “teaching institution”.
  • If professors and students disagree about the content or teaching methods, classes may be hostile, with little learning going on.
slide19
TAKING STOCK: STUDENTS

?

Students

• What do you know about

your students’ attitudes and beliefs about learning?

taking stock
Taking Stock
  • We have a systematic way of gathering information about student beliefs and values about learning.
slide21

Faculty usually know a great deal about students and their lives, and we try to be sympathetic to the “busied-up” conditions caused (often) by the need to work and maintain family responsibilities.

We may know much less about how students think about the purpose of college and the nature of learning.

slide22

In the conventional professorial model

of college, invisible disjunctions between

students’ and professors’ understanding

about teaching and learning become the

students’ responsibility.

In a more collaborative model of teaching, part of the professor’s

responsibility involves understanding how students perceive college, the curriculum, and the nature of learning.

activity students
ACTIVITY: STUDENTS
  • Into what key groups do you subcategorize your students?
activity students24
ACTIVITY: STUDENTS
  • How do you identify the changing needs of your student groups?
activity students25
ACTIVITY: STUDENTS
  • What are your students’ beliefs and values about learning?
activity students26
ACTIVITY: STUDENTS
  • How do you know?
  • What is the mechanism for gathering this information
  • What is the forum in which you discuss categories, changing needs, and attitudes of students?
activity students27
ACTIVITY: STUDENTS
  • Are your students credentialist, wanting credit/ credentials but not necessarily the learning the credential signifies (grades matter more than content)?
activity students28
ACTIVITY: STUDENTS
  • Are your students vocationalist, using college as a route to employment (relevancy matters more than intellectualism; students continuously make cost-benefit calculations)?
activity students29
ACTIVITY: STUDENTS
  • Do your students undermine their own learning outcomes by being fearful, afraid of being caught unprepared, isolated, intimidated by professors?
activity students30
ACTIVITY: STUDENTS
  • Do they manage their fear in unproductive ways by keeping quiet in class, by avoiding hard classes, by scaling down their ambitions, by failing to submit work even when it’s completed, or by dropping or stopping out?
activity students31
ACTIVITY: STUDENTS
  • Do your students define learning as the accumulation of facts?
discussion what are the results of mis alignment between faculty and student goals
Discussion: What are the results of mis-alignment between faculty and student goals?
  • Students rebel against professor efforts to expand students’ knowledge.
  • The counter-productive behavior of students can generate counter-productive reactions from faculty.
slide33

Changing the attitudes of students

may be difficult,

because we have to fight against larger social trends and pressures, but it is likely to be

better accomplished

through the collective actions

of all faculty,

than by the efforts of

professors one-by-one.

slide34
TAKING STOCK: INSTRUCTORS

?

Instructors

• What do you know about instructors’ attitudes, beliefs and knowledge about teaching and learning?

• Is teaching “community property”?

slide35
Higher education is unique in that

it generally denies employment

to those schooled in its craft:

teaching and learning.

Faculty autonomy, academic freedom, and professional discretion weigh against shared (normed) expectations, assessments and criteria.

The issue of inter-rater reliability rarely is raised in higher education.

taking stock36
Taking Stock
  • Everyone in our department agrees that we have common learning objectives for each course.
  • Everyone in our department agrees on a process for determining how we will assess student learning.
  • We regularly share what works and what doesn’t work with subgroups of students.
  • We have a formal process for using what we learn to improve what we do related to student learning.
activity faculty
ACTIVITY: FACULTY
  • What philosophies, principles, models of teaching, research on learning, or shared assumptions about teaching and learning underlie your curricular or co-curricular design, instructional design, or use of educational tools?
activity faculty38
ACTIVITY: FACULTY
  • What pedagogies or educational experiences develop the knowledge, understanding, habits of the mind, ways of knowing, and problem solving that this discipline, profession, program or institution values?
activity faculty39
ACTIVITY: FACULTY

• How do faculty and student support services build on each others’ courses and educational experiences to achieve departmental/ programmatic/ institutional learning priorities?

activity faculty40
ACTIVITY: FACULTY
  • What are faculty attitudes and knowledge about learning, teaching, assessment, “teaching as community property”, and continuous improvement?
activity faculty41
ACTIVITY: FACULTY
  • Is there a forum for discussing examples and reasons for student success or lack of success, teaching ideas and methods? How often are these the topics of discussion among faculty?
activity faculty42
ACTIVITY: FACULTY
  • How have faculty previously developed, shared and implemented student learning outcomes and assessments?
slide43
TAKING STOCK: CURRICULUM

?

Curriculum

• What are external influences on curriculum?

• What is the consistency of curriculum?

taking stock44
Taking Stock
  • Everyone in our department shares the same expectations of what students should know and be able to do at the end of each of our courses.
  • At the beginning of the semester, we share with students what is expected of them, the criteria by which they will be measured, and what standards are required for grades and course completion.
activity curriculum
ACTIVITY: CURRICULUM
  • Which of your critical curriculum is set by external agencies?
activity curriculum46
ACTIVITY: CURRICULUM
  • What is the consistency in expectations across sections of a course?
activity curriculum47
ACTIVITY: CURRICULUM
  • When/ where do instructors norm content and assessment?
activity curriculum48
ACTIVITY: CURRICULUM
  • Do instructors collaborate on and/or do peer review of learning outcomes for critical courses?
slide49
TAKING STOCK: INSTITUTIONAL

?

Institutional Support

• What regulations impact student learning outcomes?

• How do local practices and policies impact student learning outcomes and assessment?

taking stock50
Taking Stock
  • Our organizational practices and processes are designed to strengthen student learning outcomes.
  • Faculty time, professional development, as well as hiring, promotion, and tenure policies place highest priority on student learning outcomes.
activity institution
ACTIVITY: INSTITUTION
  • What federal, state, and/or professional-industry regulations impact student learning outcomes, their assessment, and improvement on your campus?
activity institution52
ACTIVITY: INSTITUTION
  • How much faculty time is devoted to meetings unrelated (or only peripherally related) to student learning outcomes?
  • Could that time be released for focusing on student learning outcomes?
activity institution53
ACTIVITY: INSTITUTION
  • Does professional development at your institution focus on norming student learning outcomes and assessments?
activity institution54
ACTIVITY: INSTITUTION
  • Where in your institution do faculty collectively set evidence-driven decisions related to student learning outcomes?
slide55
TAKING STOCK:

EXISTING ASSESSMENTS

?

Existing Assessments

• What assessments are in place now (placement tests, capstone projects, portfolios, paper-pencil tests, etc.?)

• How do those assessments contribute to improving student learning outcomes?

taking stock56
Taking Stock
  • Our entering assessments (placement tests) accurately and reliably assign students to the appropriate courses.
  • Our exit assessments gather evidence that is used for the review of program content and pedagogy.
slide58
TAKING STOCK:

GOVERNANCE of ASSESSMENT

?

Governance

• Who coordinates student learning outcomes, assessments, improvement strategies and continuous improvement?

slide59
If assessment is to be continuous, on-going and stable, then it must be overseen by a group that takes responsibility for all aspects of assessments.
slide60

In a self-reforming institution focused on instruction, the Assessment Committee would be the central committee in a college, so that concern over the nature and effectiveness of instruction drives all other aspects of a college.

In this way, the Assessment Committee should have responsibility not only for creating a series of assessments but also for overseeing the subsequent stages in the assessment system.

reporting out
Reporting Out

“When we take stock, we believe

we have strength from _________

and we want to build __________.”

• student attitudes & beliefs;

• instructor attitudes & knowledge;

• consistency of pedagogy & curriculum;

• local practices & policies;

• existing assessments;

• local governance of SLOACs

a primer
A Primer

Setting and Assessing Student Learning Outcomes

definition slo
Definition: SLO

“Robust”student learning outcomes incorporate

  • behavioral objective- what a student should know, value and be able to demonstrate/ perform
  • conditions under which performance will be assessed (simulation, lab, portfolio, writing task)
  • Criteria/ performance standards/ primary traits for assessing student performance
  • Rubric for scoring student performance

1Adapted from Scroggins, B. (2003, 2004). Targeting Student Learning. Modesto Junior College. <http:cai.cc.ca.us/workshops/SLOFocusOn Results.doc>

confusion terminology
Confusion: Terminology??

Course objectives versus student learning outcomes:

Generally, course objective states what student will demonstrate, represent or produce at end of course.

SLO also incorporates the conditions under which assessment will occur (test, portfolio, demonstration, etc) as well as evidence/criteria

slo sloa sloac
slo —SLOA—SLOAC

• Departmental/ institutional practitioners NORM

- objectives for student performance (what should students know, be able to do, value)

- conditions under which performance will be assessed (simulation, portfolio, lab experiment, writing assignment)

- traits/ criteria for assessing student performance and rubrics for scoring

• Implement assessment plan

• Compile and collectively analyze results from scoring student performance

• COLLECTIVELY SET IMPROVEMENT PLAN

• IMPLEMENT IMPROVEMENT PLAN

• CONTINUE CYCLE

norming 1
Norming1
  • “Nested”discussions, decisions and actions
  • Collaborativelyauthored andcollectivelyaccepted expectations for student learning and assessment
  • Norming does NOT mean identical learning activities, emphases, pedagogy — it means C&C

1Maki, P.L. (2004). Assessing for Learning. American Association for Higher Education, Sterling, VA: Stylus.

norming higher ed culture
Norming — Higher Ed Culture
  • SLOs, criteria/ primary traits, rubrics are set collaboratively with full and adjunct professors
  • Outcomes/ examples of student work are shared and peer-reviewed
  • Improvement alternatives are agreed upon
  • Autonomy/ Academic freedom/ Professional discretion and expertise
slide69

Validity:

Instrument/

procedure measures

what it is intended

to measure

Reliability:

Inter-rater or

test-retest

T

H

E

B

A

S

I

C

S

types of data
Types of Data
  • Quantitative
  • Qualitative

SYSTEMATIC

references norm criterion
References: Norm & Criterion
  • Norm-referenced assessments measure individual outcomes relative to the sample of people taking the test- grading on curve
  • Criterion-referenced assessments measure individual outcomes compared to certain norms or criteria -mastery, licensure

** Criterion-referenced assessments are appropriate for measuring improvement in SLOs.

direct vs indirect measures
Direct vs. Indirect Measures
  • Direct measures are reasonable replications of real world tasks; authentic assessment- DO IT
  • Indirect measures are proxies for demonstrated performance: grades, persistence, transfer (legislated measures are often proxies)
external vs internal accountability
External vs. Internal Accountability
  • External accountability is used to meet requirements of funding/ regulatory agencies
  • Internal accountability is used to improve student learning within courses, programs or degrees.
slide74

Well developed cycle of internal accountability

Ability to respond

to external

accountability requirements

slide75

A

S

S

E

S

S

M

E

N

T

F

O

R

M

S

O

F

assessments
Assessments

}

  • Capstone projects
  • Demonstration
  • Simulations
  • Portfolios

AUTHENTIC

ASSESSMENTS

  • Criterion-referenced tests (licensure exams)
  • Norm-referenced tests (curve)
embedding assessments
Embedding Assessments
  • Align with certificate, departmental, degree, institutional goals
  • Assessment is woven into existing courses
  • Identify SLO demonstration points
  • Retain and analyze results
levels of outcomes
Levels of Outcomes
  • Targeted population of students
  • Lesson/ unit of study
  • Program
    • Occupational certificate
    • Major
    • Department/ Division
  • Associate degree (A.A./A.S./A.A.S)
  • Institutional
targeted populations
Targeted Populations
  • For VTEA and some grants, a college may wish to focus on retention, persistence and/ or achievement for special populations of students
    • CalWorks
    • First generation
    • Limited English proficient
lesson unit level assessment
Lesson/ Unit-Level Assessment

Classroom assessment techniques

(Cross & Angelo)

– Systematic but informal, frequent gathering of information about content and pedagogy:

• What was hard to understand today?

• How did this teaching method work for you?

course level slos
Course Level SLOs
  • Most campuses are emphasizing course level assessments as part of program level SLOACs.1

1 Friedlander, J. & Serban, A. (2004). Meeting the Challenges of Assessing Student Learning Outcomes, in Friedlander, J. & Serban, A. [Ed.]Developing and Implementing Assessment of Student Learning Outcomes. New Directions for Community Colleges No. 126. San Francisco, Jossey Bass.

slide83

Norena Norton Badway, Ph.D.

Principal

Phone 209-951-7477 home office

209-946-2168 University office

209-601-7121

Email badway@aol.com

nbadway@pacific.edu