an assessment overview l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
An Assessment Overview PowerPoint Presentation
Download Presentation
An Assessment Overview

Loading in 2 Seconds...

play fullscreen
1 / 92

An Assessment Overview - PowerPoint PPT Presentation


  • 184 Views
  • Uploaded on

An Assessment Overview. VAG Workshop 2007 Presented by Dr. Barbara Boothe Liberty University. I. What Is The Assessment of Learning?. Assessment is a REFORM MOVEMENT!. History. Accountability movement began in 1970’s. 1973-1983—widespread dissatisfaction.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'An Assessment Overview' - dior


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
an assessment overview

An Assessment Overview

VAG Workshop 2007

Presented by Dr. Barbara Boothe

Liberty University

history
History
  • Accountability movement began in 1970’s
  • 1973-1983—widespread dissatisfaction
  • 1983-National Commission on Excellence in Education—”A Nation At Risk: The Imperative for Educational Reform”
  • Undergraduate Reform Reports of 1985-86
history5
History
  • 1986-National Governor’s Association—”Time for Results”
  • Early 1990’s—accreditation to forefront

Both the external audiences and internal participants of higher education are interested in answers to the same questions: Is college helping students? Is it increasing what they know and can do?”

Banta and Associates, 1993

history6
History
  • Higher Education Reauthorization Act--1998
  • President Bill Clinton—”Goals 2000”
  • President George Bush—”No Child Left Behind”
drivers of assessment
Drivers of Assessment
  • A true revolution in education: learning-centered paradigm
  • Federal requirements for regional accreditation
  • Disciplinary accreditation
  • Calls for accountability
  • Supports for faculty and students to improve their performance
what is assessment
What Is Assessment?

Assessment is the ongoing process of:

  • Establishing clear, measurable expected outcomes of student learning (or service).
  • Ensuring that students (or service users) have sufficient opportunities to achieve those outcomes.
what is assessment9
What Is Assessment?
  • Systematically gathering, analyzing, and interpreting evidence to determine how well student learning (or service) matches our expectations.
  • Using the resulting information to understand and improve student learning (or service).
good assessments
Good assessments…
  • Give us reasonably accurate, truthful information
  • Give us useful information
  • Are fair to all students
  • Are ethical and protect the privacy and dignity of those involved
  • Are systematized
  • Are cost effective, yielding value that justifies the time and expense we put into them
valuable discussion within departments
Valuable discussion within departments
  • What DO we want students to
    • Know?
    • Do?
    • Value?
shared decision making
Shared Decision-making
  • WE decided

“Counseling skills will be introduced in the sophomore year and reinforced in a senior course.”

faculty know what students are taught in which courses
Faculty Know What Students Are Taught in Which Courses
  • “Which course includes teaching students to do abstracts?”
    • 104?
    • 211?
minimize need to re teach the same material several times
Minimize Need to Re-teach the Same Material Several Times
  • “Are students applying what they learn in one class to the next?”
affirmation of effective teaching
Affirmation of Effective Teaching
  • “I know my students can calculate protein needs for surgical patients.”
more prompt meaningful feedback from learners
More Prompt, Meaningful Feedback from Learners
  • The minute paper
  • The ‘muddiest’ moment
enhanced teaching and learning
Enhanced Teaching and Learning
  • “Since I began using video-taping, students identify their errors and gain skill much more quickly.”
on the plus side
On the Plus Side:
  • Assessment yields evidence to confirm that students are learning.
as well as suggests areas for continued growth and improvement
…As Well As Suggests Areas for Continued Growth and Improvement
  • “This text doesn’t include needed relevant information. Let’s use a different book.”
areas for continued growth and improvement
Areas for Continued Growth and Improvement
  • “Students would learn even more if this course was taken earlier in the curriculum.”
areas for continued growth and improvement22
Areas for Continued Growth and Improvement
  • “Students would learn to communicate more clearly if we added oral presentations to that course.”
opportunities to market a successful program
Opportunities to Market a Successful Program
  • College Publications
    • Parents
    • Students
    • Alumni
    • Donors
  • Open houses
  • Websites
  • Recruiters
opportunities to improve teaching and learning
Opportunities to Improve Teaching and Learning
  • Better use of resources
  • More objectives achieved
  • Learning becomes more meaningful
opportunities to improve teaching and learning25
Opportunities to Improve Teaching and Learning
  • Students enjoy learning more
  • Students ask better questions
  • Students accomplish more in the same amount of time
assessment can be used to affirm quality
Assessment Can be Used to Affirm Quality

“This assignment did help students learn effective counseling skills.”

-They are able to collect far more relevant information from the clients.

assessment can be used to affirm quality27
Assessment Can Be Used to Affirm Quality

“This course does enable students to integrate theory with practice.”

assessment can be used to affirm quality28
Assessment Can be Used to Affirm Quality
  • Graduates of this program:
    • Pass the national exam
    • Continue their professional education
    • Find jobs
    • Express satisfaction with their career choice
one more way for faculty to spend time and energy
“One More Way for Faculty to Spend Time and Energy.”

“Is doing assessment really the best use of my time and energy?”

assessment is a passing fad
“Assessment Is a Passing Fad.”
  • “If I resist long enough, assessment will go away.”
i am already doing assessment
“I am Already Doing Assessment.”
  • “I assign grades…I am assessing my students.”
assessment is just the tip of the iceberg
“Assessment is just the tip of the iceberg…”
  • “Next…they are going to tell me what to teach.”
concerns include
Concerns Include:
  • The fear that less than stellar results will be used against programs—resulting in a loss of resources.
concerns include36
Concerns Include:
  • That assessment findings will be used to create program rankings and/or ratings.
concerns include37
Concerns Include:
  • The fear that assessment will result in mandated standardized tests.
responsibilities of faculty members
Responsibilities of Faculty Members

1. Develop measurable learning outcomes for courses

2. Develop measurable learning outcomes for programs

3. Create assignments and other strategies to assess learning

responsibilities of faculty members41
Responsibilities of Faculty Members

4. Analyze the results in light of learning as the whole

5. Initiate changes as indicated by the assessment

responsibilities of administration
Responsibilities of Administration

1. Transparent communication and decision processes

2. Base strategic planning and budget planning on assessment

3. Be consistent

responsibilities of assessment coordinator
Responsibilities of Assessment Coordinator

1. Assist and facilitate training of faculty in the assessment of learning

2. Summarize data as requested for strategic planning and budget planning

10 suggestions for more successful assessment
10 Suggestions for More Successful Assessment

1. Start with important questions

2. Focus on things you can improve

3. Plan, but be flexible and willing to adapt as you learn

4. Start small

10 suggestions for more successful assessment46
10 Suggestions for More Successful Assessment

5. Build in success from the start

6. Get students, faculty, and others actively involved

7. Set limits on time and effort you’ll invest

10 suggestions for more successful assessment47
10 Suggestions for More Successful Assessment

8. Collaborate with others who share your concerns

9. Remember that assessment may be new to many of your students and colleagues

10. Enjoy experimentation and risk taking, not success

-Peter Ewell—2005 Assessment Institute

slide49

To look is one thing.

  • To see what you understand is another.
  • To understand what you see is a third.
  • To learn from what you understand is still something else.
  • But to act on what you learn is all that
  • really matters.
problem suggestion
Problem & Suggestion

The Problem

There is still no standard language for assessment terms.

The Suggestion

Until there is more consensus, align with definitions from SACS, disciplinary accreditors, and your own institution.

basic definitions
Basic Definitions

GOAL: What an individual, project/program, or organization aims to achieve. Your destination, not your path.

Example: Help students write better

LEARNING OUTCOME: Learning outcomes are the knowledge, skills, attitudes, and habits of mind that students’gain from a learning experience.

Example: Write with clarity, unity, coherence,

and correctness.

basic definitions52
Basic Definitions

OBJECTIVE: The detailed aspects of goals; the tasks to be accomplished to achieve the goal, the means to the end; the process leading to the outcome.

Example: Explain why each planet except Earth cannot support human life.

BEHAVIORAL OBJECTIVE: A clear and unambiguous description of the educational expectations for students using concrete action words.

questions to ask before writing a learning outcome
Questions To Ask Before Writing a Learning Outcome
  • Do the students’ levels of awareness need to be raised?
  • Do they need a better understanding of the context in which the problem/issue exists?
  • Are there things they need to unlearn?
questions to ask before writing a learning outcome54
Questions To Ask Before Writing a Learning Outcome
  • What are the most essential things they need to know or be able to do?
  • Do they need a strong rationale to buy into the issue?
  • What specific skills or strategies do they need?
questions to ask before writing a learning outcome55
Questions To Ask Before Writing a Learning Outcome
  • How important is their level of confidence with this new learning?
  • What are the obstacles they face in the workplace using the new learning?
  • What are the most important things they need to be able to do when they finish?
what is a learning outcome
What Is A Learning Outcome?

It is a statement of the intended outcome of instruction

NOT the process of instruction itself.

a learning outcome has three parts
A Learning Outcome Has Three Parts

A verb that describes an observable action

A description of the conditions under which the student is expected to act

  • The level of acceptable performance—to what degree the student should be able to do it?
slide58
TIPS

Ask yourself, “What would a student have to do to convince me that he/she is where I want him/her to be at the end of this lesson, unit, or course?”

Keep goals in mind

  • To more easily sort your outcomes, write them on separate pieces of paper or index cards.
slide60
TIPS

Toss the outcomes that do not relate to goals

Use the stem, “Students will be able to. . .”

Use words that describe observable behaviors

Make sure you write outcomes at a variety of levels.

three major categories of the assessment of learning
Three Major Categories of the Assessment of Learning

“Real-World” Experiences

Internships

Service Learning

three major categories of the assessment of learning63
Three Major Categories of the Assessment of Learning

Course-Related/Course Connected or Embedded

Capstone Courses

Capstone Experiences

Portfolios

Competency Analysis

three major categories of the assessment of learning64
Three Major Categories of the Assessment of Learning

Testing

Standardized Testing

Locally Developed Test

Embedded Test Items

what are assessment strategies
What are Assessment Strategies?

Assessment strategies are the means by which a department will gather the evidence to determine whether the learning outcomes are being achieved.

types of assessment strategies
Objective

No professional judgment to score correctly – usually one correct answer

Multiple-choice test

Matching items

True-false questions

Subjective

Need professional judgment to score – many possible answers of varying quality

More assessments are of this type

Types of Assessment Strategies
types of assessment strategies67
Quantitative

Structured, pre-determined response options that can be summarized into meaningful numbers and analyzed statistically

Some audiences find quantitative results more convincing

Qualitative

Flexible, naturalistic methods that are usually analyzed by looking for recurring patterns and themes

Often underused, underappreciated

Can give fresh insight and help discover problems and solutions

Types of Assessment Strategies
types of assessment strategies68
Traditional

Have been around for a long time

Usually controlled, timed exam setting

Objective tests

“Blue book” essay questions

Oral examinations

Performance

Ask students to demonstrate skills

If “real life” tasks, are called authentic

Field experiences

Studio assignments

Projects

Term papers

Types of Assessment Strategies
types of evidence
Direct

Tangible, visible, self-explanatory

Scores and pass rates on licensure / certification exams

Portfolios of student work

Capstone experiences

Indirect

Signs, indicators, less convincing

Grades

Student self-ratings

Student / alumni satisfaction with learning

Honors, awards, and scholarships

Types of Evidence
grading vs assessment
Grading and assessment criteria appropriately differ (e.g., attendance)

Grading standards may be vague or inconsistent (or, at best, idiosyncratic)

Grades alone may give insufficient information on student strengths and weaknesses

Grades do not reflect all learning experiences (whole curriculum)

Grading vs. Assessment
research findings on testing and grading
Research Findings on Testing and Grading
  • Most tests ask for factual recognition or recall.
  • Tests rarely measure higher-order cognitive competencies.
  • Faculty tend to emphasize knowledge of facts and students write essays to meet this expectation.
research findings on testing and grading72
Research Findings on Testing and Grading
  • Less than 20% of faculty report using “problem-solving” items on essay tests.
  • Grades are used primarily for external reporting, not for feedback to students.
  • Low level of classroom assessment and the lack of validity and reliability call into question the results of most tests and the grades that are given
research findings on testing and grading73
Research Findings on Testing and Grading
  • Publisher-provided test items tend to focus primarily on recall items.
  • Nearly 50% of students report never or rarely having to write an essay examination or test.

Source: Redesigning Higher Education by Lion Gardiner

slide74

Grades

Don't

Count ?

slide75
Of course they do…

IF THE FACTORS THAT COUNT FOR SCORING ARE EXPLICIT FROM THE OUTSET

constructivist theories of learning
Constructivist Theories of Learning

Learner:

  • builds an internal representation of knowledge
  • a personal interpretation of experience
slide80

Students need not only to know HOW to perform, but also WHEN to perform and HOW TO CHANGE the performance to fit new and different situations.

what is authentic assessment
What is Authentic Assessment?
  • Methods that emphasize learning and thinking
  • Tasks that focus on students’ ability to produce a quality product or performance
  • Disciplined inquiry that integrates and produces knowledge rather than reproducing fragments of information that others have discovered.
what is authentic assessment82
What is Authentic Assessment?
  • Meaningful tasks at which students should learn to excel
  • Challenges that require knowledge in good use and good judgment
advantages of authentic assessment
Advantages of Authentic Assessment
  • Assess many important skills that objective tests cannot.
  • Assess skills directly.
  • Promote deep, lasting learning.
  • Allow for nuances in scoring.
  • Can capture a lot of information on a broad range of learning goals in a relatively short time.
specific techniques for developing authentic assessment activities
Specific Techniques for Developing Authentic Assessment Activities
  • Group activities encourage students to work together to develop a plan, carry it out, and communicate their findings to others.
  • Logs and journals provide an opportunity to brainstorm, to question, or to reflect on a problem.

3.Non-routine problems involve creative problem solving, critical thinking, and an innovative approach to the synthesis of ideas.

specific techniques for developing authentic assessment activities85
Specific Techniques for Developing Authentic Assessment Activities

4. Open-ended questions probe students’ ability to confront an unusual situation by applying a collection of strategies and ideas.

5. Student-generated questions are formulated and written for other students and the teacher to solve.

6. Performance tasks consist of real-world problems that employ useful, meaningful applications for students to tackle.

specific techniques for developing authentic assessment activities86
Specific Techniques for Developing Authentic Assessment Activities

7. Portfolios are a collection of student work over time used to demonstrate overall improvement.

8. Presentations, single or group, explain ideas and information to others.

9. Research projects require students to find information not readily available in the classroom and to draw their own conclusion about implications.

Source: Appalachia Educational Laboratory (1993). Alternative assessments I math and science: Moving toward a moving target. Charleston, WV: Author.

other terms you may hear
Other Terms You May Hear
  • Embedded– program assessments embedded into coursework – no need to convince students to participate
  • Competency-based (or criterion-referenced) – compare results to an established standard
other terms you may hear88
Other Terms You May Hear
  • Benchmarking – compare results to those of peers
  • Best-practice (or best-in-class) – compare results to the best of peers
  • Value-added (or pre-post) – compare results to performance at program (or course/experience) entry
other terms you may hear89
Other Terms You May Hear
  • Longitudinal – compare results of current students against peers in prior classes
  • Capability (or potential) – compares results against student’s capability
nine assertions about assessment of learning
Nine Assertions About Assessment of Learning
  • What students learn depends as much on your tests and methods of assessment as your teaching.
  • Don’t think of tests simply as a means for assigning grades.
  • Use some non-graded tests and assessments that provide feedback to the students and you.
nine assertions about assessment
Nine Assertions About Assessment

4. Check your assessment methods against your goals.

5.  Some goals (values, motivation, attitudes, some skills) may not be measurable by conventional tests.

6. Assessment is not synonymous with testing.

nine assertions about assessment92
Nine Assertions About Assessment
  • After the course is over, students will not be able to depend on you to assess the quality of their learning.
  • Don’t rely on one or two tests to determine grades.
  • Assessment is not simply an end-of-course exercise to determine student grades. Assessments can be learning experiences for students.

McKeachie’s Teaching Tips, by Wilbert J. McKeachie