Assessment 101 the core curriculum
Download
1 / 87

- PowerPoint PPT Presentation


  • 253 Views
  • Updated On :

Assessment 101: The Core Curriculum. Susan Hatfield - Winona State UNC Wilmington November 2005 [email protected] Common Reactions to Assessment Initiatives. Ignoring it Bribing someone else to do it Complaining about it Losing sleep over it Sitting down and writing it.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '' - gin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Assessment 101 the core curriculum l.jpg

Assessment 101:The Core Curriculum

Susan Hatfield - Winona State

UNC Wilmington

November 2005

[email protected]


Common reactions to assessment initiatives l.jpg
Common Reactions to Assessment Initiatives

  • Ignoring it

  • Bribing someone else to do it

  • Complaining about it

  • Losing sleep over it

  • Sitting down and writing it


Slide3 l.jpg

Assessment is

-- first and foremost -- about student learning.



Slide5 l.jpg
Quiz

All faculty need to be actively engaged in assessment for a department or program to really be “doing assessment.”


Slide6 l.jpg
Quiz

All faculty need to be deeply committed to assessment for a department or program to really be “doing assessment.”


Attitudes toward assessment l.jpg
Attitudes toward Assessment

70%

Level of Commitment

15%

15%

Hostile

Accepting

Enthusiastic


Slide8 l.jpg
Quiz

The best way to build a campus culture of assessment is for top administration to be prescriptive in student learning outcomes, assessment measures and methods.


Slide9 l.jpg
Quiz

All departments and programs should be moving ahead on assessment at the same pace, meeting specific goals within specified time frames.


Slide10 l.jpg
Quiz

Effective assessment programs have eliminated indirect measures and measures of departmental effectiveness (satisfaction, etc.).


Slide11 l.jpg
Quiz

Assessment is a time-intensive add-on that will be a huge burden to faculty who are already overburdened.


Slide12 l.jpg
Quiz

Effective programmatic assessment plans have every faculty member assess every outcome in every course every semester.


Slide13 l.jpg
Quiz

Accrediting organizations expect to see fully realized assessment plans during site visits.


Slide14 l.jpg
Quiz

The most effective assessment programs are ones in which the students are not aware they are being assessed.


Uncertainty l.jpg
Uncertainty

  • Uncertainty related to questions of

    • How to do it

    • Why it needs to be done

    • What to do with the data

    • How the data will be used

    • How to find the time to implement it

    • What support is available


Reducing uncertainty l.jpg
Reducing Uncertainty

  • Clear

    • Understanding

    • Definitions

    • Processes

    • Rules

    • Resources

    • Understanding of pitfalls


The assessment core curriculum 10 things you should know about assessment l.jpg

The Assessment Core Curriculum:10 things you should know about assessment


Core curriculum l.jpg
Core Curriculum

  • Assessment terminology

  • Evolution of assessment initiatives

  • Difference between direct and indirect measures of learning

  • Assessment requires exertion and intention

    5. Writing student learning outcomes


Language of assessment l.jpg
Language of Assessment

  • A. General skill or knowledge category

    GOAL

  • B. Specific accomplishments to be achieved OUTCOME

  • C. Activities and Assignments to help students learn LEARNING EVENTS

  • D. The key elements related to the accomplishment of the outcome COMPONENTS


Language of assessment20 l.jpg
Language of Assessment

  • E. The objects of analysis OBJECTS

  • F. Data indicating degree of achievement CHARACTERISTICS

  • G. Combination of data indicating relative degree of achievement of the learning outcome INDICATORS



Goals22 l.jpg
Goals

Organizing Principle

Category or Topic Area

Subjects


Goals23 l.jpg
Goals

Composition

PE

Humanities

Fine Arts

Natural Sciences and Mathematical Sciences

Social and Behavioral Sciences

Interdisciplinary Perspectives



Student learning outcomes l.jpg
Student Learning Outcomes

Communication

Writing

Relating

Speaking

Listening

Teaming



Learning events27 l.jpg
Learning Events

  • Assignments (in class and out of class)

  • Feedback on practice

  • Self evaluation

  • Peer evaluation

  • Role Play

  • Pre Tests

  • Simulation



Student learning outcomes29 l.jpg
Student Learning Outcomes

Communication

Writing

Relating

Speaking

Listening

Teaming

Sales



Student learning outcomes31 l.jpg
Student Learning Outcomes

Goal

Outcome

Outcome

Outcome

Outcome

Outcome

Learning events

Object


Student learning outcomes32 l.jpg
Student Learning Outcomes

Communication

Writing

Relating

Speaking

Listening

Teaming

Verbal

Demonstration

Evaluative elements

Nonverbal

Organization



Student learning outcomes34 l.jpg
Student Learning Outcomes

Communication

Writing

Relating

Speaking

Listening

Teaming

Verbal

Demonstration

Nonverbal

Organization



Student learning outcomes36 l.jpg
Student Learning Outcomes

GOAL

Outcome

Outcome

Outcome

Outcome

Outcome

Degree to which outcome

is achieved

component

Object

component

indicator

component

component


Core curriculum37 l.jpg
Core Curriculum

  • Assessment terminology

  • Evolution of assessment initiatives

  • Difference between direct and indirect measures of learning

  • Assessment requires exertion and intention

    5. Writing student learning outcomes


Slide38 l.jpg

Maturing Assessment

BEGINNING

PROGRESS

MATURING

INSTITUTIONAL

RESPONSIBILITY

DEPARTMENT

RESPONSIBILITY


Slide39 l.jpg

Maturing Assessment

BEGINNING

PROGRESS

MATURING

INDIRECT

MEASURES

DIRECT

MEASURES


Slide40 l.jpg

Maturing Assessment

BEGINNING

PROGRESS

MATURING

PROCESS

MEASURES

OUTCOME

MEASURES


Slide41 l.jpg

Maturing Assessment

BEGINNING

PROGRESS

MATURING

CLASSROOM

ASSESSMENT

PROGRAM

ASSESSMENT


Slide42 l.jpg

Maturing Assessment

BEGINNING

PROGRESS

MATURING

INSTITUTIONAL

EFFECTIVENESS

STUDENT

LEARNING


Evolutionary trajectories l.jpg
Evolutionary Trajectories

98

99

00

01

02

03

04

05

MATURING

MAKINGPROGRESS

BEGINNING


Core curriculum44 l.jpg
Core Curriculum

  • Assessment terminology

  • Evolution of assessment initiatives

  • Difference between direct and indirect measures of learning

  • Assessment requires exertion and intention

    5. Writing student learning outcomes


Direct measures of learning l.jpg
Direct Measures of Learning

  • Capstone experience

  • Standardized tests

  • Performance on national licensure certification or professional exams

  • Locally developed tests

  • Essay questions blind scored by faculty

  • Juried review of senior projects

  • Externally reviewed exhibitions performances

  • Evaluation of internships based upon program learning outcomes


Indirect measures of learning l.jpg
Indirect Measures of Learning

  • Alumni, employer, and student surveys (including satisfaction surveys)

  • Exit interviews of graduates and focus groups graduate follow up studies

  • Retention and transfer studies

  • Length of time to degree

  • ACT scores

  • Graduation and transfer rates

  • Job placement rates


Non measures of student learning l.jpg
Non-Measures of Student Learning

  • Curriculum review reports

  • Program review reports from external evaluators

  • Faculty publications and recognition

  • Course enrollments and course profiles

  • Faculty / student ratios, percentage of students who study abroad

  • Enrollment trends

  • 5 year graduation rates

  • Diversity of the student body


Core curriculum48 l.jpg
Core Curriculum

  • Assessment terminology

  • Evolution of assessment initiatives

  • Difference between direct and indirect measures of learning

  • Assessment requires exertion and intention

    5. Writing student learning outcomes





Core curriculum52 l.jpg
Core Curriculum

  • Assessment terminology

  • Evolution of assessment initiatives

  • Difference between direct and indirect measures of learning

  • Assessment requires exertion and intention

    5. Writing student learning outcomes


Learning outcome format l.jpg
Learning Outcome Format

  • Students should be able to

    <<action verb>> <<something>>


Slide54 l.jpg

COMPREHENSION

EVALUATION

APPLICATION

ANALYSIS

SYNTHESIS

KNOWLEDGE

Associate

Classify

Compare

Compute

Contrast

Differentiate

Discuss

Distinguish

Estimate

Explain

Express

Extrapolate

Interpolate

Locate

Predict

Report

Restate

Review

Tell

Translate

Analyze

Appraise

Calculate

Categorize

Classify

Compare

Debate

Diagram

Differentiate

Distinguish

Examine

Experiment

Identify

Inspect

Inventory

Question

Separate

Summarize

Test

Arrange

Assemble

Collect

Compose

Construct

Create

Design

Formulate

Integrate

Manage

Organize

Plan

Prepare

Prescribe

ProducePropose

Specify

Synthesize

Write

Appraise

Assess

Choose

Compare

Criticize

Determine

Estimate

Evaluate

Grade

Judge

Measure

Rank

Rate

Recommend

Revise

Score

Select

Standardize

Test

Validate

Cite

Count

Define

Draw

Identify

List

Name

Point

Quote

Read

Recite

Record

Repeat

Select

State

Tabulate

Tell

Trace

Underline

Apply

Calculate

Classify

Demonstrate

Determine

Dramatize

Employ

Examine

Illustrate

Interpret

Locate

Operate

Order

Practice

Report

Restructure

Schedule

Sketch

Solve

Translate

Use

Write

Lower division course

outcomes


Slide55 l.jpg

COMPREHENSION

EVALUATION

APPLICATION

ANALYSIS

SYNTHESIS

KNOWLEDGE

Associate

Classify

Compare

Compute

Contrast

Differentiate

Discuss

Distinguish

Estimate

Explain

Express

Extrapolate

Interpolate

Locate

Predict

Report

Restate

Review

Tell

Translate

Analyze

Appraise

Calculate

Categorize

Classify

Compare

Debate

Diagram

Differentiate

Distinguish

Examine

Experiment

Identify

Inspect

Inventory

Question

Separate

Summarize

Test

Arrange

Assemble

Collect

Compose

Construct

Create

Design

Formulate

Integrate

Manage

Organize

Plan

Prepare

Prescribe

ProducePropose

Specify

Synthesize

Write

Appraise

Assess

Choose

Compare

Criticize

Determine

Estimate

Evaluate

Grade

Judge

Measure

Rank

Rate

Recommend

Revise

Score

Select

Standardize

Test

Validate

Cite

Count

Define

Draw

Identify

List

Name

Point

Quote

Read

Recite

Record

Repeat

Select

State

Tabulate

Tell

Trace

Underline

Apply

Calculate

Classify

Demonstrate

Determine

Dramatize

Employ

Examine

Illustrate

Interpret

Locate

Operate

Order

Practice

Report

Restructure

Schedule

Sketch

Solve

Translate

Use

Write

Upper division

Course / Program

outcomes


Learning outcome rules l.jpg
Learning Outcome Rules

  • Only one action verb

  • Identify single accomplishments

  • Focus on students, not faculty or curriculum


Example 1 l.jpg
Example #1

Gather factual information and apply it to a given problem in a manner that is relevant, clear, comprehensive, and conscious of possible bias in the information selected

of Bias


Example 2 l.jpg
Example #2

Imagine and seek out a variety of possible goals, assumptions, interpretations, or perspectives which can give alternative meanings or solutions to given situations or problems


Example 3 l.jpg
Example #3

Formulate and test hypotheses by performing laboratory, simulation, or field experiments in at least two of the natural science disciplines (one of these experimental components should develop, in greater depth, students’ laboratory experience in the collection of data, its statistical and graphical analysis, and an appreciation of its sources of error and uncertainty)


Core curriculum60 l.jpg
Core Curriculum

6. What an assessment plan looks like


Slide61 l.jpg

Write

Relate

Speak

Listen

Participate

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component

Component


Core curriculum62 l.jpg
Core Curriculum

  • What an assessment plan looks like

  • How assessment works


How assessment works l.jpg
How Assessment Works

Cycle 2

Cycle 3

Cycle 1

O

U

T

C

O

M

E

New / Revised

LEvent 1

New / Revised

LEvent 2

New / Revised

LEvent 3

New / Revised

LEvent 1

New / Revised

LEvent 2

New / Revised

LEvent 3

LEvent 1

LEvent 2

LEvent 3

Process

Reflection;

Compare

results

against

Benchmarks,

Standards,

Targets,Past

Performance

component

component

component

component

BASELINE

component

component

component

component

component

component

component

component


Core curriculum64 l.jpg
Core Curriculum

  • What an assessment plan looks like

  • How assessment works

  • Why you need to define your outcomes


Slide65 l.jpg

teacher4

teacher2

teacher1

teacher3

teacher5

Speaking

eye contact

gestures

volume

sources

transitions

style

rate

poise

examples

verbal variety

appearance

evidence

conclusion

organization

attention getter


Slide66 l.jpg

Can our students deliver

an effective Public Speech?

eye contact

gestures

volume

sources

transitions

style

rate

poise

examples

verbal variety

appearance

evidence

conclusion

organization

attention getter


Core curriculum67 l.jpg
Core Curriculum

  • What an assessment plan looks like

  • How assessment works

  • Why you need to define your outcomes

  • How to implement the plan


Slide68 l.jpg

Phase Four

Student

Learning

Outcomes

Course

1

Course

2

Course

3

Course

4

Course

5

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X


Core curriculum69 l.jpg
Core Curriculum

  • What an assessment plan looks like

  • How assessment works

  • Why you need to define your outcomes

  • How to implement the plan

    10. What to do with the data



Interpreting data l.jpg
Interpreting Data

  • Consistency - over time

  • Consensus - different populations

  • Distinctiveness - different situations/ variables / items


Consistency l.jpg
Consistency

  • Examines the same practice of and individual or group over time

  • Key question:

    • Has this person or group acted, felt, or performed this way in the past / over time?


Consistency73 l.jpg
Consistency

How well are students performing on the

departmental learning outcome measures?

High

performance

Low

performance

00

01

02

03

04

05


Consensus l.jpg
Consensus

  • Comparison to or among groups of students

    • Variation between disciplines, gender, other demographic variables

  • Key questions:

    • What is the general feeling, outcome, attitude, behavior?

    • Do other groups of people act, perform or feel this way?


Consensus75 l.jpg
Consensus

How well are students performing on the

departmental learning outcome measure?

High

performance

Low

performance

Females

Males

Transfers

OTA


Distinctiveness l.jpg
Distinctiveness

  • Examines individual or cohort perspectives across different outcomes

  • Key Question:

    • Does a person or group perform equally as well on different outcomes?


Distinctiveness77 l.jpg
Distinctiveness

How well are our students performing on

the learning outcomes?

S

P

E

A

K

I

N

G

High

Performance

A

N

A

L

Y

S

I

S

T

H

I

N

K

I

N

G

R

E

S

E

A

R

C

H

E

THICS

W

R

I

T

I

N

G

Low

Performance


Fundamental question 1 l.jpg
Fundamental Question #1

  • Do we have enough data from which to really draw conclusions?


Fundamental question 2 l.jpg
Fundamental Question #2

  • Does the data represent an identifiable trend in the level of activity / achievement / accomplishment?


Fundamental question 3 l.jpg
Fundamental Question #3

  • Does the data represent an acceptable level of activity / accomplishment / achievement given our mission and values?


Fundamental question 4 l.jpg
Fundamental Question #4

  • Are the differences in the sub-populations acceptable?


Fundamental question 5 l.jpg
Fundamental Question #5

  • What can we do about it?



Big mistakes in assessment l.jpg
Big Mistakes in Assessment

  • Assuming that it will go away

  • Allowing assessment planning to become gaseous

  • Assuming you got it right -- or expecting to get it right -- the first time

  • Not considering implementation issues when creating plans


Big mistakes in assessment85 l.jpg
Big Mistakes in Assessment

  • Borrowing plans and methods without acculturation

  • Setting the bar too low

  • Assuming that you’re done and everything’s OK, or rushing to “Close the Loop”

  • Doing it for accreditation instead of improvement


Big mistakes in assessment86 l.jpg
Big Mistakes in Assessment

  • Confusing program effectiveness with student learning

  • Making assessment the responsibility of one individual

  • Assuming collecting data is Doing Assessment


Assessment 101 the core curriculum87 l.jpg

Assessment 101:The Core Curriculum

Susan Hatfield - Winona State

UNC Wilmington

November 2005

[email protected]


ad