Implementing outcomes assessment an approach based on competencies
This presentation is the property of its rightful owner.
Sponsored Links
1 / 34

Implementing Outcomes Assessment: An Approach Based on Competencies PowerPoint PPT Presentation


  • 59 Views
  • Uploaded on
  • Presentation posted in: General

Implementing Outcomes Assessment: An Approach Based on Competencies. Steve Mickelson Agricultural and Biosystems Engineering. Agricultural & Biosystems Engineering. Agricultural & Biosystems Engineering (ABE). Two undergraduate majors Agricultural Engineering (~150 students)

Download Presentation

Implementing Outcomes Assessment: An Approach Based on Competencies

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Implementing outcomes assessment an approach based on competencies

Implementing Outcomes Assessment:An ApproachBased on Competencies

Steve Mickelson

Agricultural and Biosystems Engineering


Agricultural biosystems engineering abe

Agricultural & Biosystems Engineering

Agricultural & Biosystems Engineering (ABE)

  • Two undergraduate majors

    • Agricultural Engineering (~150 students)

    • Agricultural Systems Technology (~150 students)

    • Industrial Technology (~220 students)

  • Two graduate majors (~90)

  • In two colleges: Engineering and Agriculture

  • 33 faculty

    • 22 involved in undergraduate teaching

    • Faculty are academic advisors

    • No teaching assistants


Presentation road map

Agricultural & Biosystems Engineering

Presentation Road Map

  • Our path to competencies

  • Our outcomes assessment plan

  • Implications and implementation

  • Now what?


The path to competencies

Agricultural & Biosystems Engineering

The Path to Competencies


The buzz outcomes assessment

Agricultural & Biosystems Engineering

The Buzz: Outcomes Assessment!

  • ISU university-wide accreditation in 2005

    • Evidence of active outcomes assessment programs is required.

  • ISU College of Engineering

    • ABET in 2000 & 2006

  • ABE has been thinking and

    working a lot on this with

    mostly good faculty buy-in


Our outcomes via abet

Agricultural & Biosystems Engineering

Our Outcomes via ABET

  • an ability to apply knowledge of mathematics, science, engineering and technology

  • an ability to design and conduct experiments, as well as to analyze and interpret data

  • an ability to design a system, component, or process to meet desired needs

  • an ability to function on multi-disciplinary teams

  • an ability to identify, formulate, and solve engineering problems

  • an understanding of professional and ethical responsibility

  • an ability to communicate effectively

  • the broad education necessary to understand the impact of engineering solutions in a global and societal context

  • a recognition of the need for, and an ability to engage in life-long learning

  • a knowledge of contemporary issues

  • an ability to use the techniques, skills, and modern engineering tools necessary for engineering and technology practice

*

*

*

*

* Alverno “ABET” Outcomes

We don’t know how to measure an ability.


Outcomes abilities competencies

Agricultural & Biosystems Engineering

OutcomesAbilitiesCompetencies

  • A complex abilitycannot be observed directly; it must be inferred from performance.

  • Abilities are complex combinations of competencies.

  • Competencies are the application of behavior and motivation to knowledge, understanding and skill.

  • Key actions that demonstrate workplace competencies can be measured.


Identifying workplace competencies

Agricultural & Biosystems Engineering

Identifying Workplace Competencies

  • Cooperative effort with Development Dimensions International, Inc. (DDI)

  • Stakeholder dialogue, faculty analysis

    • “Critical incident” stories of workplace success

    • Stories identified and validated 14 competencies mapped back to outcomes

  • Each competency has Key Actions that are definable and measurable


Isu engineering competencies

Agricultural & Biosystems Engineering

ISU Engineering Competencies


Abet outcomes isu competency matrix

ABET Outcomes – ISU Competency Matrix


Competencies have meaning

Agricultural & Biosystems Engineering

Competencies Have Meaning

  • “Transparent” to all participants

  • A great match with:

    • employers’ goals

    • student and faculty understanding

  • Provides a map and tools for students to achieve their goals in a meaningful way


Our outcomes assessment plan

Agricultural & Biosystems Engineering

Our Outcomes Assessment Plan


How should we assess outcomes

Agricultural & Biosystems Engineering

How Should We Assess Outcomes?

  • Actual student work is the primary evidence.

  • Exit interviews, post-graduate surveys, etc. are opinions and are only supporting evidence

  • ABE will use:

    • ePortfolios

    • Internship evaluations

    • Employer evaluations of alumni

    • FE exam


What are eportfolios

Agricultural & Biosystems Engineering

What are ePortfolios?

  • Collection of artifacts for certain purposes displayed on the web

  • Our purpose: developing and demonstrating student competencies

  • It’s all about learning!

    • Reflection

    • Demonstration


Why eportfolios students

Agricultural & Biosystems Engineering

Why ePortfolios? (Students)

  • Student learning

    • Building an ePortfolio requires reflection

      Reflection → Learning!

    • Faculty can have access to student work for comment, review

  • Adds depth to resume for career search success

    • Highlight skills in ways not

      previously possible

    • Potential employers are interested


Why eportfolios faculty

Agricultural & Biosystems Engineering

Why ePortfolios? (Faculty)

  • Outcomes assessment

  • A tool to better manage, review, reflect, and comment on student work (class & advising)

  • Faculty learning

    • Building an ePortfolio requires reflection

      Reflection → Learning!

    • Adding access to student work to document teaching excellence


On line competency assessment

Agricultural & Biosystems Engineering

On-line Competency Assessment

  • OPAL by DDI, Inc.

    • On-line Performance and Learning

    • Web-based assessment and learning tool built on competencies

  • Multiple uses for us

    • Internship evaluations by employers

    • Alumni evaluations by employers

    • Classroom (self and 360º)


Implementing outcomes assessment an approach based on competencies

Ind Advisory Board

Curriculum

Committee

Strengths &

Weaknesses

DIRECT MEASURES

INDIRECT MEASURES

Recommendations

Faculty

(Individuals and as a Whole)

ABE Outcomes Assessment Map

Stakeholders

Mission, Goals

& Objectives

Ind Advisory Board

  • Co-op & internship evaluations

  • ePortfolios

  • Employer evaluation of 5- year grads

  • FE exam

  • Senior exit surveys

  • SEI

  • 5 year post grad survey

  • CSREES review

  • Advisor evaluations

  • Placement statistics

6 years

Faculty

Outcomes &

Competencies

ABET / ASAE

Classes mapped to outcomes & competencies

Annually

Key

Assignments

ePortfolios

Co-op &

Internships

Extra-curricular

activities


Ttyp turn to your partner

Agricultural & Biosystems Engineering

TTYP – Turn to Your Partner

  • What are the barriers to implementing an outcomes assessment plan?


Our barriers

Agricultural & Biosystems Engineering

Our Barriers

  • Faculty and administrative buy-in

  • Financial resources

  • Expertise

  • Clear vision

  • Time

    • Faculty development

    • Competing demands

  • Rewards

  • Champion(s)


Implications and implementation

Agricultural & Biosystems Engineering

Implications and Implementation


Stakeholders view of the probability of developing and demonstrating the communication competency

50%

Stakeholders’ View of the Probability of Developing and Demonstrating the Communication Competency


Competency based learning cbl

Agricultural & Biosystems Engineering

Competency-based Learning (CBL)

  • Learning outcomes in terms of competencies

    • curricula focus on competency development

  • “Transparent” to all participants

  • Great match with employers’ goals and student understanding

  • Provides a map and tools for students to achieve their goals in a meaningful way


Cbl demands change

Agricultural & Biosystems Engineering

CBL demands change

  • Faculty must think, teach and assess in terms of competencies.

  • Our curriculum must evolve.

    • It can no longer be a series of requirements (or hurdles).

    • Opportunity to integrate previously disparate parts of the undergraduate experience.


A cbl model

Agricultural & Biosystems Engineering

Assessment of

Performance

Demonstrations

Acquired Skills,

Abilities, and

Knowledge

Competencies

ASSESSMENT

Integrative Learning Experiences

Developed in the

Learning Process

Skills, Abilities and Knowledge

Learning Experiences

Traits and Characteristics

Foundation

A CBL Model

Voorhees et al, U.S. Dept. of Education, 2001


Implementing cbl

Agricultural & Biosystems Engineering

Implementing CBL

  • Avoid “extra” work for faculty

  • Classes already mapped to outcomes

    • Map competencies to classes

  • Identify key assignments for student portfolios

  • Help faculty learn the “language” of competencies

  • Provide common rubrics


Implementing cbl with technology

Agricultural & Biosystems Engineering

Implementing CBL with Technology

  • ePortfolios to develop and demonstrate competencies

  • On-line case simulations to simulate the workplace

  • On-line assessment for internships, alumni and in the classroom


Eportfolio implementation

Agricultural & Biosystems Engineering

ePortfolio Implementation

  • Freshman Learning Community

    • introduce competencies

    • start collecting and reflecting on artifacts

  • Sophomore and Juniors seminars

    • Start ePortfolios

    • Feedback from instructor and advisor

  • Senior seminar

    • Final ePortfolio required for graduation

    • Assessed by faculty and professionals


Abe eportfolio system

Agricultural & Biosystems Engineering

ABE ePortfolio System

  • Database for holding multi-modal artifacts, reflections, and assessments

  • Student-designed, student-owned interface to display artifacts


On line case simulations

Agricultural & Biosystems Engineering

On-line Case Simulations

  • Case Studies

    • Promotes competencies and reflection

  • English 314 (Technical Writing)

    • Communication - the weakest competency


One instance omega molecular

Agricultural & Biosystems Engineering

Omega Molecular

Feeding the world, better.

One Instance: Omega Molecular

  • Students are employees of consulting firm

  • Client: Omega Molecular

    • Engagement with virtual biotech organization

    • Analyzing and producing recommendations and communications


On line case simulation

Agricultural & Biosystems Engineering

On-line Case Simulation


Now what

Agricultural & Biosystems Engineering

Now What?

  • Our status today:

    • Students are collecting artifacts.

    • Pilot group is creating ePortfolios.

    • Faculty are designating key assignments and learning about CBL.

    • We’re assessing internships & alumni.

  • Just Completed:

    • Implement seminar classes (S’06)

    • Do a complete cycle (S’06)

    • Conducted ABET visit (F’06)


Implementing outcomes assessment an approach based on competencies

Agricultural & Biosystems Engineering


  • Login