Jean hughes office of the vice president for learning innovation dcu
Download
1 / 37

Jean Hughes Office of the Vice President for Learning Innovation, DCU - PowerPoint PPT Presentation


  • 88 Views
  • Uploaded on

Jean Hughes Office of the Vice President for Learning Innovation, DCU. Overview. DCU Approach Framework Implementation Network Learning Outcomes General – Assessment of Learning Outcomes Common Questions. DCU. Already Modularised and Semesterised (1997)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Jean Hughes Office of the Vice President for Learning Innovation, DCU' - awena


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Jean hughes office of the vice president for learning innovation dcu

Jean Hughes

Office of the Vice President for Learning Innovation,

DCU


Overview
Overview

  • DCU Approach

  • Framework Implementation Network

  • Learning Outcomes

  • General – Assessment of Learning Outcomes

  • Common Questions


DCU

  • Already Modularised and Semesterised (1997)

  • Module Descriptors use ‘Learning Outcomes’

  • European Credit Transfer System (ECTS) well established

  • But

    • Mainly administrative exercise

  • Academic Framework for Innovation (AFI) – for Deep Curriculum Reform (3 year project)

    • Bologna Compliance

    • Demonstrable Alignment with NFQ

    • Use of Learning Outcomes – for real change in assessment practice

    • Flexibility – Designed in and through new Marks & Standards


Framework implementation network
Framework Implementation Network

  • Created by the IUA and the NQAI

  • Membership

    • 7 Universities

    • NUI Colleges

  • Purpose – to collectively address outstanding issues relating to Bologna and the National Framework of Qualifications

  • Three separate Working Groups

    • Award Titling

    • Disciplinary Learning Outcomes

    • Assessment of Learning Outcomes


Key issues
Key Issues

  • Ensuring that Assessment, Teaching and Learning are aligned

  • Moving from assessing content to assessing outcomes

  • Ensuring all outcomes are assessed and all assessment instruments actually assess outcomes

  • Avoiding over-assessment


Learning outcomes
Learning Outcomes

Clearly identify what a learner can demonstrate as a result of successfully completing a part of a learning programme

They describe student attainment which is demonstrable and assessable.


Benefits of learning outcomes
Benefits of Learning Outcomes

Guide students in their learning - what is expected of them - in turn helping them to succeed in their studies.

Help lecturers to focus on exactly what they want students to achieve in terms of both knowledge and skills.

Provide a useful guide to inform employers about the general knowledge and understanding that a graduate will possess.


Challenges
Challenges

  • Traditional view

    • inputs = quality of learning outcomes

  • Moving from content (Teaching) focus to Outcome (Learning) focus

  • Aligning teaching, learning and assessment


Award and module outcomes
Award and Module Outcomes

  • NFQ describes generic, high level outcomes for awards at each level under 3 strands and 8 substrands

  • How do we demonstrate achievement of award outcomes?

  • We don’t assess at award level we assess at module level

  • Need to ensure that module learning outcomes collectively deliver the award outcomes


Alignment of Programme and Module Outcomes

PO – Programme Outcome

Mod - Module



Constructive alignment
Constructive Alignment

is the underpinning concept behind the current requirements for programme specification, declarations of intended learning outcomes and assessment criteria, and the use of criterion based assessment.

(Biggs, 1999)


Example project management

Learning Outcome

(Mismatched)

Learning Outcome

(Matched)

Tasks

Students will understand how to plan a multi-disciplinary project

Students will demonstrate ability to plan a multi-disciplinary project

Assignment 1

Students will complete an in-class test on Project Management

Students will devise an appropriate project plan from a scenario/business case study

Assignment 2

Students will create a Project Plan using MS Project during labs

Students will create a Project Plan in MS Project from the plan in Assignment 1. They will be demonstrate appropriate use of Base Lines, Serial and Parallel Tasks, Variances, Resource Allocation and Critical Path, through an in-lab demonstration.

Final exam

1 Question on final exam

Case Study exam question requiring analysis of a Business Problem, design of appropriate project plan and discussion of variety of project planning issues.

Example – Project Management


Student perspective
Student Perspective

  • From our students’ point of view, assessment always defines the actual curriculum (Ramsden, 1992)

  • Biggs (2003)

    Teacher Perspectives:

    Objectives Learning Outcomes Teaching Activities Assessment

    Assessment Learning Activities Outcomes

    Student Perspectives

Alignment

If curriculum is reflected in the assessment, the teaching activities and the learner activities are directed towards the same goal – in preparing for the assessment students will be learning the curriculum


Assessment impact boud 1995
Assessment – Impact(Boud, 1995)

“There is probably more bad practice and ignorance of significant issues in the area of assessment than in any other aspect of higher education”.

“The effects of bad practice are far more potent than they are for any aspect of teaching. Students can, with difficulty, escape from the effects of poor teaching, they cannot (by definition if they want to graduate) escape the effects of poor assessment”.


Assessment terminology
Assessment - Terminology

  • Learning Outcomes

    • What a student should be able to demonstrate

    • (May imply the assessment criteria)

  • Assessment Criteria

    • The basis on which a judgement of the adequacy of the student work is made

  • Assessment Methods

    • The task(s) undertaken by the student – writing an essay, answering an exam question etc- that is subject to assessment.

    • Many assessment methods may be appropriate to the same outcomes – e.g. Written, Oral, Visual, Project etc

    • Must be valid and reliable – Fit for purpose


Over assessment
Over-Assessment

  • Typical with semesterisation and modularisation

  • High risk with learning outcomes

    • Tend to be quite granular and have several independent outcomes

  • To avoid

    • Try combining the assessment of more than one LO

    • LOs only need to be assessed once

    • Use assessments which inform/follow on from each other

  • Check what you are assessing


Checking assessment
Checking Assessment

Issues:

What does assignment 3 assess?

How is Learning Outcome 2 assessed?

Are Learning Outcomes 1 and 4 over-assessed?


Refining assessment
Refining Assessment

Perhaps design a new assessment 3 to re-balance?



Must all outcomes be assessed
Must all outcomes be assessed?

  • Yes, but

    • Not independently

    • Look at how you write the learning outcomes

      • Students are expected to be able to

        • Write a business report

        • Present orally

        • Design a visual presentation

          versus

      • Students are expected to be able to

        • Communicate effectively using a combination of written reports, oral presentations and visual media

      • This reduces the number of independent outcomes and also recognises the combination of communication mechanisms

  • BUT

    • For certain disciplines, for example languages, you may need these to be independently demonstrated


Example
Example

  • In a Business Module (1 outcome)

    • Students are expected to be able to communicate effectively in a business context using appropriate communication tools including Written, Oral and Visual

    • The outcome is communication, the tools are some means of demonstrating that outcome

  • In a languages Module (4 outcomes)

    • Students are expected to be able to demonstrate written proficiency in French

    • Students are expected to be able to translate a piece of French text

    • Students are expected to be able to comprehend a piece of French text

    • Students are expected to be able to deliver an oral presentation in French


Must all outcomes be passed
Must all outcomes be passed?

  • In theory, yes, but…

  • Threshold Achievement

    • Implies that ALL learning outcomes must be passed in order to pass a module

      • Very strict interpretation of LO approach

      • Leads to reductionist or ‘Tick Box’ approaches

  • Typical Achievement

    • Learning outcomes are written (and assessed) with the ‘typical’ or ‘’average’ student in mind

      • Grading criteria will allocate marks to degree of achievement


But we don t currently assess everything
But, we don’t currently assess everything….

  • Exams often require only 3 out of 5 questions

  • Therefore we don’t assess everything

  • But

    • Generally this refers to content

    • Outcomes should be about demonstrating knowledge/understanding aided by knowledge of content, not content in its own right

    • Assessment criteria and methods should satisfy you that students can demonstrate achievement of the outcomes

  • Worst-case scenario – no worse than current situation!


What if some outcomes are failed
What if some outcomes are failed?

  • Depends on the criticality of the outcome

    • If particular outcomes must be passed then passing other outcomes should not be able to compensate

    • Need to look at ‘Threshold’ and ‘Typical’ Outcomes

    • Need to write outcomes carefully

    • Need to ensure that supplemental/repeat assessment is also valid


What about repeats
What about repeats?

  • Did they actually fail?

    • “Students will present orally on 20th century politics”

      • If they don’t attend they fail….. Is this what you meant? or

    • “Students will effectively communicate on 21th century politics”

      • You may prioritise Oral assessment and allocate more marks, but this allows more choice of methods

  • If a student has to repeat the supplemental assessment must be valid if different from the original

    • Look at how you write the outcomes


Is it an outcome an assessment or an efficiency
Is it an Outcome, An Assessment or an Efficiency?

  • Often mix up pedagogical (or efficiency) approaches with learning outcomes

  • Example Group Working

    • Learning Outcome

      • ‘Students will demonstrate effective group working skills’

      • Group working needs to be ‘taught’ and assessed

      • Can’t assume that putting people in groups will automatically lead to effective group working!

      • How does an individual student repeat?

      • ‘Students will be able to evaluate group dynamics and effective group working’

        • Would this suffice but give an alternative assessment approach for repeat students?

    • Pedagogical Approach

      • Group working is used to facilitate learning of a particular outcome

      • Not assessing the group working – assessing the actual outcome

    • Efficiency

      • Often used to reduce the volume of marking

      • Need to be careful that it is pedagogically sound

      • Not assessing group working


Making the tacit explicit
Making the Tacit Explicit

  • Some disciplines rely on expertise of the individual lecturer

    • ‘I know a good piece of work when I see it’

    • ‘You can’t possibly express Creativity, Performance, Medieval English….. Using learning outcomes

  • NOT about subjectivity – expertise, tacit knowledge etc

    • How do students know what to aim for?

    • How do novice lecturers know about standards?

    • How can we demonstrate consistency, transparency etc?


Example1
Example

First, rewrite the outcome:

“By the end of this module students will understand the importance of post-renaissance European art in the context of art history”

Redesigned:

By the end of this module students will be able to:

Evaluate and criticise post-renaissance European art within its historical context

Date and identify key works of this specific period

Demonstrate an understanding of the particular art-historical debates that have taken place about this period

Show ability to argue why this particular place and time raise interesting issues about the discipline as a whole

University of Sussex


Then use a rubric
Then, use a rubric

  • Also called grading criteria or scoring guide

  • A set of guidelines for marking or scoring

  • Can be a list, chart, or guidelines

  • They state all dimensions being assessed

  • They contain a scale

  • They assist the marker on rating assessments on the scale


Rubrics 3 types
Rubrics - 3 Types

  • Checklists

    • Indicating list of criteria

  • Rating Scales

    • Checklist with rating scale added

    • Show degrees by which students meet criteria

  • Holistic Scoring Guide

    • Used where assignments can vary i.e. projects

    • Narrative descriptions of characteristics of different grades of work

    • Not list of criteria


Rubric design
Rubric Design

Achievement Levels

Criteria

Excellent

Good

Needs Work

Not acceptable

Objective 1

AcceptedMinorMajor Rejected

revision revision

Objective 2

Expert Advanced Intermediate Novice

Objective 3

6-5 4-3 2-1 0




This assignment asked students to take a position on a debateable issue regarding interpretation of literature they had studied. The lecturer has identified 3 major criteria – Position, Support and Acknowledgement of Alternative Points of View and has weighted them different levels of attainment of each criterion. Opposite is the detailed rubric for the Position criterion.


In summary
In Summary debateable issue regarding interpretation of literature they had studied. The lecturer has identified 3 major criteria –

  • Need to write learning outcomes so that they can be demonstrated and assessed

  • Need to identify assessment criteria and appropriate assessment methods

  • Need to be sure exactly what it is students need to demonstrate

  • Making the tacit explicit – Clearly written outcomes and the use of rubrics to articulate achievement


Questions

Questions? debateable issue regarding interpretation of literature they had studied. The lecturer has identified 3 major criteria –


ad