assurance of learning why what who when and how n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Assurance of Learning: Why, what, who, when and how? PowerPoint Presentation
Download Presentation
Assurance of Learning: Why, what, who, when and how?

Loading in 2 Seconds...

play fullscreen
1 / 69

Assurance of Learning: Why, what, who, when and how? - PowerPoint PPT Presentation


  • 102 Views
  • Uploaded on

Assurance of Learning: Why, what, who, when and how?. Marta Colón de Toro, SPHR Assessment Coordinator College of Business Administration UPR-Mayagüez. Today’s Learning Goals. After completing this workshop you should be able to:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Assurance of Learning: Why, what, who, when and how?' - claus


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
assurance of learning why what who when and how

Assurance of Learning:Why, what, who, when and how?

Marta Colón de Toro, SPHR

Assessment Coordinator

College of Business Administration

UPR-Mayagüez

today s learning goals
Today’s Learning Goals

After completing this workshop you should be able to:

  • Explain the importance and purpose of assessing student learning outcomes.
  • Recognize AACSB’s standards on assessment.
  • Describe the process of assessment and the challenges it represents.
  • Developa plan to conduct classroom assessment.
  • Identify and apply appropriate instruments to directly assess student learning in the classroom.
  • Create and maintain a course portfolio to document your classroom assessment.
slide3

Input Teaching-Learning Output

A Transformation Process

Curriculum

Seminars and Competitions

Internship/Coop

Student Associations

Extracurricular Experiences

Incoming Student

Completing Student

  • Demographics
  • Academics
  • Experience
  • Talents
  • Attitudes/Values
  • Knowledge
  • Skills
  • Abilities
  • Experience
  • Attitudes and Values
effective curricula is
Effective curricula is…

…pertinent

…coherent

…strategic

…dynamic …assessable

…intentionally designed!

objective 1
Objective #1
  • Explain the importance and purpose of assessing student learning outcomes
have you ever asked yourself

Objective #1

Have you ever asked yourself…?

Are my students learning?

How much are they learning?

How well are they learning?

What affects their learning?

assessment

Objective #1

Assessment

…is a way of describing student

learning to an identified audience

for clearly- articulated reasons.

…produces information useful for communication and decision-making.

purpose of assessment

Objective #1

Purpose of Assessment

Assessment

Reveals the Gap

Intended Outcomes

1. Act ethically

2. Quantitative Analysis

3. Apply models

4. Decision making skills

5. Communication skills

Actual Outcomes

1.

2.

3.

4.

5.

?

for example upr utuado

Objective #1

For example…UPR-Utuado

Misión

El Programa de Contabilidad de la Universidad de Puerto Rico en Utuado, reconoce que tiene ante si un reto y compromiso con la comunidad altamente industrializada y comercializada de nuestros tiempos. Su objetivo es capacitar al estudiantado con las herramientas necesarias: destrezas, conocimientos, aptitudes y actitudes que respondan a la demanda y expectativas del mundo laboral y en los principios técnicos necesarios en la Contabilidad General, en el análisis de información de transacciones financieras en forma sistemática y organizada.

La finalidad de esta tecnología es desarrollar el potencial individual de estudiantes para que puedan competir en una economía global, ampliar sus oportunidades de empleo y proveer a la comunidad de personal técnico capacitado.

another example arecibo

Objective #1

Another example…Arecibo

MISIÓN

El programa de Administración de Empresas tiene como misión lograr la formación humana, ética, intelectual y profesional del estudiante para que provea un servicio de excelencia en la administración de las empresas privadas y públicas, y en la comunidad. Además, el estudiante desarrollará las competencias necesarias para desempeñar posiciones de liderato y enfrentar retos con la disciplina y el grado de flexibilidad que exige el ambiente cambiante empresarial, y así contribuye a su éxito profesional.

another example arecibo1

Objective #1

Another example…Arecibo

Para lograr su misión, el Departamento tiene como metas:

  • Familiarizar al estudiante con los conceptos, teorías, y técnicas administrativas y fiscales de las empresas privadas y públicas.
  • Capacitar al estudiante para recopilar datos y analizar y presentar información de su área para que se pueda utilizar en el proceso decisional.
  • Fomentar en el estudiante la comprensión de su responsabilidad como ciudadano en el desarrollo de la organización, la economía, la sociedad y la cultura del país.
  • Capacitar al estudiante con las destrezas intelectuales, técnicas y profesionales que le permitan integrarse exitosamente a la constante innovación en la empresa y la sociedad.
  • Desarrollar en el estudiante la agudeza de percepción para reconocer los cambios del ambiente externo e integrar las competencias que faciliten identificar mecanismos administrativos que permitan la adaptación exitosa de la empresa.
assess learning to

Objective #1

Assess Learning to:
  • Provide feedback and guidance to individual students.
  • Assist the school and faculty members to improve courses and programs.
  • Assure external constituents that the institution meets its academicgoals.

AACSB

levels of assessment

Objective #1

Levels of Assessment

Institutional Learning Goals

Program Learning Goals

AACSB’s Focus

Course Goals

Today’s Focus

Lesson Objectives

when to assess

Objective #1

When to Assess
  • Entry
    • Demographics
    • Academic achievement
    • Previous Business KSAOs
  • Midpoint
    • Measure progress / added value
  • Exit
    • Measure attainment of expected learning outcomes
  • Follow-up
    • Further education
    • Workforce
    • Personal goals
effective assessment

Objective #1

Effective assessment…
  • Is goal directed
    • Ethics, writing, critical thinking, teamwork
  • Uses multiple measures
    • Tests, essays, projects, transcripts, surveys
  • Show student development overtime
    • Entry, midpoint, exit, follow-up
  • Uses multiple types of information
    • Demographics, behavioral, perceptual
challenges of assessment

Objective #1

Challenges of Assessment
  • Motivating faculty, staff, students, administration
  • Needs realistic goal setting
  • Data management
  • Assuring right purpose
  • Requires taking action
  • Could be costly
objective 2
Objective #2
  • Recognize AACSB’s standards on assessment
aacsb assurance of learning standards

Objective #2

AACSB Assurance of Learning Standards
  • AACSB only requires assessment at the program level.
    • Other levels of assessment bring support
  • Assessments using samples are allowed.
  • Assessment must use direct measures of achievement
    • Course-embedded exercises
    • Course-embedded examinations
    • Stand-alone examinations
  • Indirect measures may be used to supplement direct measures of achievement.
aacsb s learning expectations at the bachelor s level

Objective #2

AACSB’s Learning Expectations at the Bachelor’s Level
  • Educate students in a broad range of knowledge and skills as a basis for careers in business.
  • Build on the students’ pre-collegiate educations to prepare them to enter and sustain careers in the business world and contribute positively in the larger society.
  • Provide knowledge and skills for successful performance in a complex environment requiring intellectual ability to organize work, make and communicate sound decisions, and react successfully to unanticipated events.
  • Develop learning abilities suitable to continue higher-level intellectual development.
standard 15

Objective #2

Standard 15
  • The school uses well documented, systematicprocesses to develop, monitor, evaluate, and revise the substance and delivery of the curricula of degree programs and toassess the impact of the curricula on learning.
  • Curriculum management includes inputs from all appropriateconstituencies which may include faculty, staff, administrators, students, faculty from non-business disciplines, alumni, and the business community served by the school.
standard 16

Objective #2

Standard 16
  • Adapting expectations to the school’s mission and cultural circumstances, the school specifies learning goals and demonstrates achievement of learning goals for key general, management-specific, and/or appropriate discipline-specific knowledge and skills that its students achieve in each undergraduate degree program.
standard 17

Objective #2

Standard 17
  • The bachelor’s or undergraduate level degree programs must provide sufficient time, content coverage, student effort, and student-faculty interaction to assure that the learning goals are accomplished.
program level learning goals

Objective #2

Program-level learning goals
  • General, not specific.
    • general knowledge and skills
    • management-specific learning goals
  • State the broad educational expectations for each degree program.
  • Specify the intellectual and behavioral competencies a program is intended to instill.
  • Clarify how we intend for graduates to be different as a result of their completion of the program.
  • Need to be defined operationally in order to be assessable.
  • Normally, four to ten learning goals will be specified for each degree program.
objective 3
Objective #3
  • Describe the process of assessment.
the assessment process

Objective #3

The Assessment Process

External Reality Vision/Mission

Set Goals and Design Curriculum

Improve Continuously

Enable Learning

Assess

Learning

slide26

The Assessment Process

Objective #3

  • Establish learning goals
    • Consult appropriate constituencies
    • Outcomes-based
    • Specify level of learning
  • Identify/design enabling experiences
    • Courses and Extracurricular activities
  • Select instruments and indicators of goal attainment
  • Assess
  • Interpret and report results
  • Identify strategies for change
for example
Establish Learning goals: :

“The student will be able todiscuss the importance of assessing student learning.”

Identify or design enabling activities:

Brief presentation on topic

Brainstorming

Q&A

Select:

Indicator: “Participant will present 2 arguments pro/con of assessment w/o consulting notes.”

Instrument: Focused Listing

Assess:

List 2 arguments in favor and 2 arguments against doing assessment.

Measure, interpret and report results

_________________________

Introduce change

Revise plan

Objective #3

For example…

Assurance of Learning Standards

objective 4
Objective #4
  • Developa plan to conduct classroom assessment.
where to start plan

Objective #4

Where to start? PLAN!

Step#1 - Revisit your course learning goals (CLGs). (Also known as: learning outcomes, course objectives)

  • Definition

“…types of executions that the students can perform after

being taught in a way that they show that they have

learned what was expected.” Norman Gronlund

  • Purpose
    • Express expectations of what is to be learned in a course
    • Provide a context for what will be learned
learning outcomes

Objective #4

Learning Outcomes…
  • Should answer five questions:
    • Are they aligned with the Program Learning Goals?
    • What type of learning do they address?
    • Do they express the appropriate learning level?
      • Express the “ultimate goal”
    • Are they outcomes-based? Use action verbs!!
      • List what will the student will be able to doas a result of learning.
    • Are they student-centered?
    • Do they address one element of learning?
ejercicio
Ejercicio
  • Al finalizar el curso, los(as) estudiantes podrán:

Manejar la calculadora para calcular datos específicos como la media y la desviación …

Solucionar problemas estadísticos utilizando la calculadora científica.

hierarchy of learning goals
Institutional Goals

College Goals

Program Goals

Course Goals

Session Objectives

“…Apply scientific inquiry methods.”

“…Think logically and critically.”

“…basic and applied research.”

“…apply data collection techniques.”

“…list steps in the scientific research process.”

Objective #4

Hierarchy of Learning Goals
how to align goals

Objective #4

How to align goals

Complete the following for each course you teach: ADMI 3000

program goal matrix

Objective #4

Program Goal Matrix

Goal #2 - “The students will use the appropriate software packages and hardware as a working tool in the daily operations of a business”.

types of learning

Objective #4

Types of Learning
  • Cognitive: mental skills (Knowledge)
    • “…enumerate three major EEO laws…”
  • Affective: growth in feelings or emotional areas (Attitude)
    • “…will value diversity…”
  • Psychomotor: manual or physical skills (Skills)
    • “… correctly use the keyboard…”
you may want to use

Objective #4

You may want to use:
  • Compare & contrast,

critique, justify

Evaluate

  • Adapt, combine, compare, contrast, design, generate

Bloom’s Taxonomy (Cognitive Domain)

Synthesize

  • Correlate, diagram, distinguish, outline, infer

Analyze

  • Determine, develop, compute, utilize, conduct

Apply

  • Classify, explain, discuss, give examples, summarize

Comprehend

  • Define, describe, list, reproduce, enumerate

Know

dealing with understand

Objective #4

Dealing with “…understand”…
  • Always ask yourself…
    • How do you know the student understands?

“When he/she:

      • Gives examples of …
      • Determines the “correct” method to use…
      • Discusses pros and cons…
      • Identifies elements in a given case…etc.”
using measurable action verbs

Objective #4

Using Measurable Action Verbs

analyze compute classify collaborate

compare appreciate contrast define

demonstrate direct derive designate

discuss display evaluate know

identify infer integrate interpret

justify list understand organize

grasp report respond solicit

state synthesize name explain

course goals help
The Student

Clarify personal goals

Measure success

Reduce anxiety

Improve studying effectiveness

The Professor

Design relevant classroom material

Design relevant assignments

Design tests/projects

Objective #4

Course goals help…

Improve Teaching!

Improve Learning!

right focus the student s learning
Focused on Teaching

Focuses on the professor’s effort, intent

or

Focused on Learning

Focuses on the student’s achievement

Example:

“Expose the student to recent research in organizational behavior”.

Example:

“After completing this course the student should be able to evaluate recent research in organizational behavior”.

Objective #4

Right focus! The student’s learning.
where to start cont

Objective #4

Where to start? (cont.)

Step #2 - Identify/design activities to enable students to achieve each learning goal:

Learning Goals

1.

2.

3.

4.

5.

Teaching Methodologies

  • Lecture
  • Case analysis and discussion
  • Oral presentations
  • Essays/Reports
  • Team Projects
  • Experiential Learning

Extracurricular Activities

  • Seminars/forums
  • Field trips
objective 5
Objective #5
  • Identify and apply appropriate instruments to directly assess student learning at nd classroom levels.
slide44

Objective #5

Where to start? (cont.)

Step# 3 - Select or design classroom assessment instruments (CATs) and indicators

“Classroom assessment is a simple method faculty can use to collect feedback on how well their students are learningwhat they are being taught”

  • Angelo, T.A (1991)
slide45

Objective #5

Classroom Assessment Assumptions:

  • Quality of student learning is directly, although not exclusively, related to the quality of teaching.
  • Learning is goal oriented.
  • Students need to receive appropriate and focused feedback early and often
  • Classroom Assessment provides systematic inquiry and intellectual challenge.
  • Classroom Assessment does not require specialized training
  • With Classroom Assessment efforts, faculty (and students) enhance learning and personal satisfaction.
classroom assessment techniques

Objective #5

Classroom Assessment Techniques
  • Learner-Centered
  • Teacher-Directed
  • Mutually Beneficial
  • Formative
  • Context-Specific
  • Ongoing
  • Rooted in Good Teaching Practice
classroom assessment techniques1

Objective #5

Classroom Assessment Techniques

Direct Measures

  • 1 Minute paper
  • Muddiest Point
  • One-sentence summary
  • Direct paraphrasing
  • Application cards
  • Empty outlines
  • Pro-con grids
  • Class journals/Portfolios
  • What’s the principle?
  • Rubrics
  • Indirect Measures
      • Student evaluation of courses
      • Alumni Feedback
      • Internship/Coop
slide49

Objective #5

CATs
  • MINUTE PAPER
    • Ask them…
      • The most important thing learned
      • What remains unanswered
    • Used in lecture/discussion courses
slide50

Objective #5

CATs
  • MUDDIEST POINT
    • Ask them…
      • Identify what is least clear to you or the most confusing point
    • Used for large, lower level courses
slide51

Objective #5

CATs
  • ONE SENTENCE SUMMARY
    • Ask them…
      • Summarize today’s session.
slide52

Objective #5

CATs
  • DIRECTED PARAPHRASING
    • Ask them
      • Explain ______ in your own words.
slide53

Objective #5

CATs
  • APPLICATION CARDS
    • Ask them
      • Connect today’s topic with…
      • Mention two possible applications of today’s topic.
remember

Objective #5

REMEMBER…
  • CATs are suggestions to be adapted, not models to be adopted
  • Don’t ask if you don’t really want to know or are not going to do anything about it.
examples of measures of achievement

Objective #5

Examples of Measures of Achievement

Example 1

  • School A has defined a learning goal in ethical reasoning for each of its four undergraduate majors. Student achievement on this goal is relevant to demonstrating satisfaction of Standard 16. The school’s faculty has defined the goal.

Learning Goal

  • “Each student can recognize and analyze ethical problems and choose and defend resolutions for practical situations that occur in accounting, human resource management, and marketing.”

Demonstration of Achievement

  • The school uses course-embedded exercises in three required introductory-level courses. Faculty in the three disciplines have developed different methods for instructing and assessing achievement toward this learning goal.
slide56

Objective #5

Examples of Measures of Achievement

In Accounting

  • A two-week module near the end of the introductory course is devoted to “Ethical standards and fraud in accounting.”
  • A topic outline has been developed by faculty members to structure an exam on the materials of this module, and a standard set of expectations has been created for grading the exam.
  • In addition to this exam’s contribution to the course grade, it provides a pass/fail indication on the learning goal.
examples of measures of achievement1

Objective #5

Examples of Measures of Achievement

In Human Resource Management

  • Students must provide four written analyses of problem situations during the course. On three of these analyses (on the topics of selection, reward systems, and job design), students are asked to respond to ethical issues.
  • A standard scoring key on the ethical component provides evaluation toward the course grade and a pass/fail indication on the learning goal.
examples of measures of achievement2

Objective #5

Examples of Measures of Achievement

In Marketing

  • Each student must compose a term paper analyzing a current national or international marketing campaign.
  • The analysis must include a specified set of components, and ethical issues that have been presented in lectures are among the required components.
  • In addition to the overall grade of the paper, each student receives a pass/fail indicator on the ethics component.
examples of measures of achievement3

Objective #5

Examples of Measures of Achievement

In addition to reporting course grades

  • Each instructor of these three courses provides a checklist of all of those students who successfully completed the ethics expectation.
  • This information is a part of each student’s record and all three parts of the learning goal must be achieved before graduation.
  • Students who fail the ethics evaluation while passing the course repeat the evaluation exercise or ethics module until they are successful.
objective 6
Objective #6
  • Create and maintain a course portfolio to document your classroom assessment.
slide62
Step #5 Document results with course portfolios

“ The portfolio is to teaching what lists of publications, grants, and honors are to research and scholarship.”

Peter Seldin

The Teaching Portfolio (Anker, 1991)

uses of course portfolios

Objective #6

Uses of Course Portfolios
  • Document existing courses in order to:
    • Constitute an official record of courses syllabi
    • Facilitate the assessment of our students’ learning outcomes
    • Provide quantitative and qualitative information for continuous improvement
    • Serve as reference for professors that will teach the course in the future
    • Render accountability to stakeholders
proposed content

Objective #6

Proposed Content
  • Course Syllabus (In compliance with Certification #130 – UPR-BT)
  • Educational Goals Addressed in the Course
    • Institutional Learning Goals Matrix
    • Program Learning Goals Matrix
    • Concentration Learning Goals Matrix
  • Classroom Assessment Techniques – For each assessment instrument used please include the following documentation:
    • Brief description of each technique used
    • Graded Sample Instrument (High, average and low grades)
    • Outcomes
    • Analysis of Outcomes
    • Lessons Learned
    • Recommendations to improve teaching/learning
    • Specific Action(s) Taken or To Be Taken
  • Evaluation Instruments (Tests, Quizzes, Projects and Special Activities) - For each evaluation instrument used include the following documentation: (Same as above)
  • Recommendations and Proposed Action Plan
documenting cats

Objective #6

Documenting CATs

CAT’s

  • Diagnostic Tests
  • Pre-Post Test
  • Muddiest Point
  • Minute Paper
  • Application Cards
  • Direct Paraphrasing
  • Course Final General Assessment

Documentation

  • Brief Description or Sample copy
  • Sample of Grades (H,M,L)
  • Sample Answers
  • Analysis of Outcomes
  • Conclusions
  • Recommendations
  • Actions Taken
analysis of outcomes

Objective #6

Analysis of Outcomes
  • Quantitative
    • Descriptive statistics
      • Average, Median, Mode, Standard deviation
    • Question by question analysis
  • Qualitative
    • Look for
      • Misconceptions
      • Attitudes/prejudices
conclusions recommendations

Objective #6

Conclusions/Recommendations
  • Summarize lessons learned
  • List actions to be taken for improvement

Step #6 - TAKE ACTION!!!

    • Next time you teach the course read the list and apply recommendations
    • Refer pertinent recommendations to appropriate forum
stages of grieving
Stages of Grieving
  • Denial
  • Anger
  • Bargaining
  • Depression
  • Acceptance

On Death and Dying

by Elisabeth Kubler-Ross

slide69
Thank You!

colon-m@rigel.uprm.edu