Implementing outcomes assessment an approach based on competencies
Sponsored Links
This presentation is the property of its rightful owner.
1 / 34

Implementing Outcomes Assessment: An Approach Based on Competencies PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Implementing Outcomes Assessment: An Approach Based on Competencies. Steve Mickelson Agricultural and Biosystems Engineering. Agricultural & Biosystems Engineering. Agricultural & Biosystems Engineering (ABE). Two undergraduate majors Agricultural Engineering (~150 students)

Download Presentation

Implementing Outcomes Assessment: An Approach Based on Competencies

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Implementing Outcomes Assessment:An ApproachBased on Competencies

Steve Mickelson

Agricultural and Biosystems Engineering

Agricultural & Biosystems Engineering

Agricultural & Biosystems Engineering (ABE)

  • Two undergraduate majors

    • Agricultural Engineering (~150 students)

    • Agricultural Systems Technology (~150 students)

    • Industrial Technology (~220 students)

  • Two graduate majors (~90)

  • In two colleges: Engineering and Agriculture

  • 33 faculty

    • 22 involved in undergraduate teaching

    • Faculty are academic advisors

    • No teaching assistants

Agricultural & Biosystems Engineering

Presentation Road Map

  • Our path to competencies

  • Our outcomes assessment plan

  • Implications and implementation

  • Now what?

Agricultural & Biosystems Engineering

The Path to Competencies

Agricultural & Biosystems Engineering

The Buzz: Outcomes Assessment!

  • ISU university-wide accreditation in 2005

    • Evidence of active outcomes assessment programs is required.

  • ISU College of Engineering

    • ABET in 2000 & 2006

  • ABE has been thinking and

    working a lot on this with

    mostly good faculty buy-in

Agricultural & Biosystems Engineering

Our Outcomes via ABET

  • an ability to apply knowledge of mathematics, science, engineering and technology

  • an ability to design and conduct experiments, as well as to analyze and interpret data

  • an ability to design a system, component, or process to meet desired needs

  • an ability to function on multi-disciplinary teams

  • an ability to identify, formulate, and solve engineering problems

  • an understanding of professional and ethical responsibility

  • an ability to communicate effectively

  • the broad education necessary to understand the impact of engineering solutions in a global and societal context

  • a recognition of the need for, and an ability to engage in life-long learning

  • a knowledge of contemporary issues

  • an ability to use the techniques, skills, and modern engineering tools necessary for engineering and technology practice





* Alverno “ABET” Outcomes

We don’t know how to measure an ability.

Agricultural & Biosystems Engineering


  • A complex abilitycannot be observed directly; it must be inferred from performance.

  • Abilities are complex combinations of competencies.

  • Competencies are the application of behavior and motivation to knowledge, understanding and skill.

  • Key actions that demonstrate workplace competencies can be measured.

Agricultural & Biosystems Engineering

Identifying Workplace Competencies

  • Cooperative effort with Development Dimensions International, Inc. (DDI)

  • Stakeholder dialogue, faculty analysis

    • “Critical incident” stories of workplace success

    • Stories identified and validated 14 competencies mapped back to outcomes

  • Each competency has Key Actions that are definable and measurable

Agricultural & Biosystems Engineering

ISU Engineering Competencies

ABET Outcomes – ISU Competency Matrix

Agricultural & Biosystems Engineering

Competencies Have Meaning

  • “Transparent” to all participants

  • A great match with:

    • employers’ goals

    • student and faculty understanding

  • Provides a map and tools for students to achieve their goals in a meaningful way

Agricultural & Biosystems Engineering

Our Outcomes Assessment Plan

Agricultural & Biosystems Engineering

How Should We Assess Outcomes?

  • Actual student work is the primary evidence.

  • Exit interviews, post-graduate surveys, etc. are opinions and are only supporting evidence

  • ABE will use:

    • ePortfolios

    • Internship evaluations

    • Employer evaluations of alumni

    • FE exam

Agricultural & Biosystems Engineering

What are ePortfolios?

  • Collection of artifacts for certain purposes displayed on the web

  • Our purpose: developing and demonstrating student competencies

  • It’s all about learning!

    • Reflection

    • Demonstration

Agricultural & Biosystems Engineering

Why ePortfolios? (Students)

  • Student learning

    • Building an ePortfolio requires reflection

      Reflection → Learning!

    • Faculty can have access to student work for comment, review

  • Adds depth to resume for career search success

    • Highlight skills in ways not

      previously possible

    • Potential employers are interested

Agricultural & Biosystems Engineering

Why ePortfolios? (Faculty)

  • Outcomes assessment

  • A tool to better manage, review, reflect, and comment on student work (class & advising)

  • Faculty learning

    • Building an ePortfolio requires reflection

      Reflection → Learning!

    • Adding access to student work to document teaching excellence

Agricultural & Biosystems Engineering

On-line Competency Assessment

  • OPAL by DDI, Inc.

    • On-line Performance and Learning

    • Web-based assessment and learning tool built on competencies

  • Multiple uses for us

    • Internship evaluations by employers

    • Alumni evaluations by employers

    • Classroom (self and 360º)

Ind Advisory Board



Strengths &






(Individuals and as a Whole)

ABE Outcomes Assessment Map


Mission, Goals

& Objectives

Ind Advisory Board

  • Co-op & internship evaluations

  • ePortfolios

  • Employer evaluation of 5- year grads

  • FE exam

  • Senior exit surveys

  • SEI

  • 5 year post grad survey

  • CSREES review

  • Advisor evaluations

  • Placement statistics

6 years


Outcomes &



Classes mapped to outcomes & competencies





Co-op &




Agricultural & Biosystems Engineering

TTYP – Turn to Your Partner

  • What are the barriers to implementing an outcomes assessment plan?

Agricultural & Biosystems Engineering

Our Barriers

  • Faculty and administrative buy-in

  • Financial resources

  • Expertise

  • Clear vision

  • Time

    • Faculty development

    • Competing demands

  • Rewards

  • Champion(s)

Agricultural & Biosystems Engineering

Implications and Implementation


Stakeholders’ View of the Probability of Developing and Demonstrating the Communication Competency

Agricultural & Biosystems Engineering

Competency-based Learning (CBL)

  • Learning outcomes in terms of competencies

    • curricula focus on competency development

  • “Transparent” to all participants

  • Great match with employers’ goals and student understanding

  • Provides a map and tools for students to achieve their goals in a meaningful way

Agricultural & Biosystems Engineering

CBL demands change

  • Faculty must think, teach and assess in terms of competencies.

  • Our curriculum must evolve.

    • It can no longer be a series of requirements (or hurdles).

    • Opportunity to integrate previously disparate parts of the undergraduate experience.

Agricultural & Biosystems Engineering

Assessment of



Acquired Skills,

Abilities, and




Integrative Learning Experiences

Developed in the

Learning Process

Skills, Abilities and Knowledge

Learning Experiences

Traits and Characteristics


A CBL Model

Voorhees et al, U.S. Dept. of Education, 2001

Agricultural & Biosystems Engineering

Implementing CBL

  • Avoid “extra” work for faculty

  • Classes already mapped to outcomes

    • Map competencies to classes

  • Identify key assignments for student portfolios

  • Help faculty learn the “language” of competencies

  • Provide common rubrics

Agricultural & Biosystems Engineering

Implementing CBL with Technology

  • ePortfolios to develop and demonstrate competencies

  • On-line case simulations to simulate the workplace

  • On-line assessment for internships, alumni and in the classroom

Agricultural & Biosystems Engineering

ePortfolio Implementation

  • Freshman Learning Community

    • introduce competencies

    • start collecting and reflecting on artifacts

  • Sophomore and Juniors seminars

    • Start ePortfolios

    • Feedback from instructor and advisor

  • Senior seminar

    • Final ePortfolio required for graduation

    • Assessed by faculty and professionals

Agricultural & Biosystems Engineering

ABE ePortfolio System

  • Database for holding multi-modal artifacts, reflections, and assessments

  • Student-designed, student-owned interface to display artifacts

Agricultural & Biosystems Engineering

On-line Case Simulations

  • Case Studies

    • Promotes competencies and reflection

  • English 314 (Technical Writing)

    • Communication - the weakest competency

Agricultural & Biosystems Engineering

Omega Molecular

Feeding the world, better.

One Instance: Omega Molecular

  • Students are employees of consulting firm

  • Client: Omega Molecular

    • Engagement with virtual biotech organization

    • Analyzing and producing recommendations and communications

Agricultural & Biosystems Engineering

On-line Case Simulation

Agricultural & Biosystems Engineering

Now What?

  • Our status today:

    • Students are collecting artifacts.

    • Pilot group is creating ePortfolios.

    • Faculty are designating key assignments and learning about CBL.

    • We’re assessing internships & alumni.

  • Just Completed:

    • Implement seminar classes (S’06)

    • Do a complete cycle (S’06)

    • Conducted ABET visit (F’06)

Agricultural & Biosystems Engineering

  • Login