Beyond student assessment
Download
1 / 35

Beyond Student Assessment - PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on

Beyond Student Assessment. Issues to consider in the assessment of Information Literacy. Objectives for this Workshop. By the end of this workshop, participants should be able to Identify possible assessments strategies that align with the Information Literacy objectives

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Beyond Student Assessment' - Anita


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Beyond student assessment

Beyond Student Assessment

Issues to consider in the assessment of Information Literacy


Objectives for this workshop
Objectives for this Workshop

  • By the end of this workshop, participants should be able to

    • Identify possible assessments strategies that align with the Information Literacy objectives

    • Identify instructional strategies and activities that align with Information Literacy objectives and support performance on Information Literacy assessments


Agenda
Agenda

  • Discuss course design triangle

  • Review Information Literacy objectives from ACRL

  • Describe issues to consider when planning assessment

  • Identify potential assessment options

  • Identify potential learning activities

  • Practice alignments


Assessment
Assessment

  • Where to begin?

    Objectives, objectives, objectives, objectives,

    objectives, objectives, objectives, objectives,

    objectives, objectives, objectives, objectives,

    objectives, objectives, objectives, objectives,

    objectives, objectives, objectives, objectives,

    objectives, objectives, objectives, objectives,

    objectives, objectives, objectives, objectives!


The education design triangle
The Education Design Triangle

Objectives

Descriptions of what students should be able to do

Assessment

Tasks that provide feedback on students’ knowledge and skills

Instructional Activities

Contexts and activities that foster students’ active engagement in learning


When a link is broken
When a Link is Broken…

Objectives

X

X

Assessment

X

Instructional Activities



Why focus on objectives
Why Focus on Objectives?

  • They communicate your intentions clearly to students and to colleagues.

  • They provide a framework for selecting and organizing content.

  • They guide you in decisions about assessment and evaluation methods.

  • They provide a framework for selecting appropriate teaching and learning activities.

  • They give students information for directing their learning efforts and monitoring their own progress.

    Based on A.H. Miller (1987), Course Design for University Lecturers. New York: Nichols Publishing.


Developing clear objectives
Developing Clear Objectives

  • Focus on students, e.g., “Students should be able to…”

  • Break down the task, focus on specific cognitive processes

  • Use action verbs, e.g.,

    • State theorems

    • Identify and describe trends…

    • Evaluate the arguments proposed…

    • Describe conditions under which techniques apply

  • Use measurable verbs

    Hard to measureEasier to measure

    Obtain a working knowledge Demonstrate

    Understand Describe, interpret

    Be aware List, describe

    Appreciate Identify value

    Begin to develop competency Perform


Cognitive and behavioral dimensions of objectives
Cognitive and Behavioral Dimensions of Objectives

  • Bloom’s taxonomy

    • Remember

    • Comprehend

    • Apply

    • Analyze

    • Synthesize

    • Evaluate

    • Create


Acrl objectives
ACRL Objectives

  • Information literate student…

    • Defines and articulates the need for information


An example to refine revise
An Example to Refine/Revise

  • Choose a learning objective(s) from your course.

    Is the objective…?

    Student centered (i.e., defined in terms of what students do)

    Broken down into component skills (not too big, not too small)

    Phrased with an action verb

    Measurable

    How well does it…

    communicate expectations to the student and guide students’ efforts through the course?

    help you prepare assessments and learning activities?


Your objectives
Your Objectives?

Global Education

Diverse Cultural/Historical Literacy?

International Awareness?

Global Competency?

CFA CIT Heinz HSS MCS SCS TSB

Unique objectives

Shared objectives





The basics
The Basics

  • Assessments must map onto objectives

  • Objectives need to be clearly articulated to students

    • What should they learn from the experience

    • What skills should they be developing

  • Students need to understand how they will be assessed and by who

    • Product and domain knowledge and skills

    • Process and group skills and strategies



Designing and conducting an assessment
Designing and conducting an assessment

  • Clarifying & operationalizing the objectives

  • Selecting a method

  • Selecting a measure

  • Interpreting the results

  • Reporting the results

  • Follow-up Actions


Question to ask before you begin
Question to ask before you begin

  • What change in students’ knowledge, skills, attitudes and/or behaviors are you interested in?

  • What kind(s) of assessment is necessary?

    • Knowledge-based assessment

    • Skill-based assessment

    • Product-based assessment

    • Group vs. individual

  • What is the timeframe of the assessment?

    • Short-term (within course life)

    • Longer-term (outside course life)?

      • Interest

      • Future course selection

      • Career path


How would you measure it
How would you measure it?

  • Method?

  • Measure?


Methodological issues
Methodological Issues

  • What type of information is most useful to YOU?

    • Observations, informal discussions, drafts are often used early in an assessment process

    • Surveys and tests, project-based deliverables can take considerable development to be informative, valid, and reliable

  • What level of Measurement is important to you?

    • Qualitative Quantitative

  • How defensible do you want to be?

    • Personally Institutionally Publicly Formally


Selecting methods

Qualitative

Student/Peer Comments

Classroom Observations

Logs and journals

Focus groups

Portfolios

Quantitative

Problem sets, quizzes, exams, exam questions

Performance/product

Qualitative & Quantitative

Projects

Surveys (self-reports)

Concept maps

Selecting Methods


What do you assess
What do you assess?

  • If application of skill and knowledge is the learning objective then assessment must include

    • Product

      • Quality (against some defined criteria)

        • Overall

        • By feature

      • Individual Knowledge (oral, written, visual, multi-modal)

        • Presentation

        • Exam Questions

        • Essay


What do you assess1
What do you assess?

  • If application of skills and teamwork are both learning objectives then assessment would include

    • Process

    • Product


Potential measures
Potential Measures

  • Process measures

  • Product measures

  • Concept inventories

  • Concept maps

  • Content inventories


Tools for assessing process
Tools for Assessing process

  • Team Charter Review

  • Meeting Minutes

  • Communication transcripts

  • Journals/logs

  • Early peer evaluation

  • Progress reports


Tools for assessing product
Tools for Assessing product

  • Client

    • Acceptance of product

    • Recommendations

      • Strength and weakness analysis

  • Performance Evaluation: Does it work?

    • Audience reaction (critic reviews)

    • Program/product execution: qualities (speed, efficiency)

  • Rubrics

    • Dimensions and qualities of product/design/outcome being evaluated



Inventories
Inventories

  • Concept Inventories

    • Goal: Identify how students reason about and apply critical concepts, principles, decision making strategies, etc.

      • Frequently used to uncover systematic but incorrect “theories”

    • Ideal Structure*

      • Multiple-choice format

        • Distractor items are common misconceptions or errors

      • Multiple questions address same concept in different contexts

        • Depth and breadth

    • Assessing Results

      • Pattern of responses across questions indicative of underlying “beliefs” or “theories”

      • Inconsistent responses across questions dealing with same concept reveal conceptual depth, contextualization, or fragile knowledge

        *Data driven from open-ended questions


Beyond student assessment

(Hestenes & Wells, 1992)

The diagram depicts a block sliding along a frictionless ramp. The eight numbered arrows in the diagram represent directions to be referred to when answering the questions.

4. The direction of the acceleration of the block, when in position I, is best represented by which of the arrows in the diagram?

1 B) 2 C) 4 D) 5

E) None of the arrows; the acceleration is zero.

5. The direction of the acceleration of the block, when in position II, is best represented by which of the arrows in the diagram?

1 B) 3 C) 5 D) 7

E) None of the arrows; the acceleration is zero.

6. The direction of the acceleration of the block, (after leaving the ramp) at position III, is best represented by which of the arrows in the diagram?

A) 2 B) 3 C) 5 D) 6

E) None of the arrows; the acceleration is zero.

1

8

2

7

3

6

4

5

III

I

II




Pragmatics what is practical
Pragmatics: What is practical?

  • Scope of Assessment

    • Class

    • Unit

    • Lecture

  • Time investment and resources

  • Timeframe for results


Recommendations
Recommendations

  • Start collecting and saving data now

    • Instruments & scores, student feedback, classroom videotaping, etc

  • Be as rigorous as is possible and practical

  • Use mixed methods & multiple measures

  • Defining clear operational goals and being pragmatic is key to good assessment and evaluation.