Tcsg q s curriculum initiative
Download
1 / 135

TCSG: -Q-S Curriculum Initiative - PowerPoint PPT Presentation


  • 198 Views
  • Uploaded on

TCSG: -Q-S Curriculum Initiative. - Outcomes, Levels of Learning and Assessment - Curriculum Database Process – Dr. Tanya Gorman, VPAA DeKalb Technical College. Q-T Curriculum Initiative. I: Domains / Levels of Learning II: Learning Outcomes / Assessment

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'TCSG: -Q-S Curriculum Initiative' - mulan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Tcsg q s curriculum initiative l.jpg

TCSG: -Q-S Curriculum Initiative

- Outcomes, Levels of Learning and Assessment -

Curriculum Database Process –

Dr. Tanya Gorman, VPAA

DeKalb Technical College


Q t curriculum initiative l.jpg
Q-T Curriculum Initiative

I: Domains / Levels of Learning

II: Learning Outcomes / Assessment

III. Curriculum Database Tool


Slide3 l.jpg
SACS

  • Core Requirement 2.7.2

    • The Institution offers degree programs that embody a coherent course of study that is compatible with its’ stated purpose and is based upon fields of study appropriate to higher education


Slide4 l.jpg
SACS

  • Core Requirement 2.7.3

    • In each undergraduate degree program, the Institution requires the successful completion of a general education component at the collegiate level that (1) is a substantial component of each undergraduate degree; (2) ensures breadth of knowledge; and (3) is based on a coherent rationale…


Slide5 l.jpg
SACS

  • Comprehensive Standard 3.3.1:

    • The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas:(Institutional Effectiveness)

      • 3.3.1.1 educational programs, to include student learning outcomes


Slide6 l.jpg
SACS

  • Comprehensive Standard 3.5.1:

    • The institution identifies college-level general education competencies and the extent to which graduates have attained them.


Slide7 l.jpg
SACS

  • Federal Requirement 4.2:

    • The Institution’s curriculum is directly related and appropriate to the purpose and goals of the institution and the diplomas, certificates, or degrees awarded.


Student learning outcomes l.jpg
Student Learning Outcomes

Bad Practices

  • Doing assessment because ACCREDITORS say we have to

  • Confusing course assessment with program assessment

  • Insufficient concern about matching assessment tools to expected outcomes


Student learning outcomes9 l.jpg
Student Learning Outcomes

  • What constitutes as evidence of student learning?

    • Evidence can take many forms but must involve a direct examination of student levels of attainment:

      • Exams, capstone experiences, licensure exams, demonstrations, portfolios, assignments, etc.

    • Evidence such as surveys, self-reports, focus groups, interviews, student satisfaction surveys are useful, but represent indirect evidence and are not sufficient for documentation of learning outcomes


Student learning outcomes building blocks l.jpg
Student Learning Outcomes: Building Blocks

  • Student learning outcomes defined:

    A particular level of knowledge, skills and abilities (cognitive, psychomotor, affective) that a student has attained at the end (or as a result) of his/her engagement in a particular set of collegiate experiences


Student learning outcomes11 l.jpg
Student Learning Outcomes

  • The definition of SLO’s begins with the end in mind:

    • What does “student success” look like?

    • How would I know it when I see it?

    • What would students have to do, say, perform or behave like for me to determine they had mastered the competencies?


Student learning outcomes12 l.jpg
Student Learning Outcomes

  • Curriculum, and consequently, SLO’s must, therefore, derive from the “Big Picture” – the “end in mind” (Program Outcomes)

  • Curriculum cannot be in isolation: jigsaw puzzle analogy (corners and borders)

  • Be clear about outcomes you want to promote; if you don’t know where you’re going you’ll never know if you got there


Student learning outcomes construct model l.jpg
Student Learning Outcomes: Construct Model

  • Program: “when Joe finishes, what should he be able to do…?” (Generally 3-5)

  • Course: “what Competencies should Joe accomplish and demonstrate to achieve and answer the first question?” (Generally 5-10)

  • Unit: “what learning outcomes, experiences and content need to be embedded to ensure accomplishment of SLO’s and support Course Competencies?”

  • Unit: “what methods of assessment will demonstrate that Joe has learned what he needs to learn?”


Domains levels l.jpg
Domains & Levels

  • Basic understanding of Educational Methodologies (building blocks) is important in defining SLO’s

  • Can’t build a house without a foundation

    • Definition of learning desired  choice of assessment method


Domains levels15 l.jpg
Domains & Levels

  • Answer the Question: Students will…

    • Know, Think, Do

    • Cognitive, Affective, Psychomotor (Domains)

  • How do you know? How will you measure? What will it look like?

  • What instruments and benchmarks will you use?


Domains levels16 l.jpg
Domains & Levels

  • Assessment methods and tools must align with competency / learning outcomes

  • Assessment of outcomes must provide for comparative analysis across courses taught by multiple instructors

  • Course outcomes / General Education Core competencies and SLO’s must show logical progression in courses and programs of higher level outcomes and objectives


Domains levels17 l.jpg
Domains & Levels

  • Learning Outcomes are statements of performance expectations:

    • Cognitive

    • Affective

    • Psychomotor

  • Program / course completers must demonstrate performance in all three


Levels of learning l.jpg
Levels of Learning

  • Examples of dissonance between “knowing” and “doing” and “valuing”

    • Smoking

    • Overeating

    • Exercise

  • Unless knowledge is combined with a value system, behavior (learning) will not change


Domains levels19 l.jpg
Domains & Levels

  • Courses within a program of study must:

    • Be part of a package of

      • Interrelated

      • Coherent

        Experiences that result in student learning and

        lead to a functional and employable graduate

  • Faculty must structure experiences and define learning outcomes toward which ALL instruction, learning and assessment relate


The plan l.jpg

“THE PLAN”

One Possible Reason Why Things Aren’t Going According to the Plan…Is That There Never Was

A Plan!


Construct l.jpg
Construct

  • Example:

    • Program Learning Outcome: Provide competent and safe care in a healthcare environment

    • Course Competency: Demonstrate knowledge of the care of patients in renal failure

    • Student Learning Outcome:

      • Describe S/S of renal failure (cognitive)

      • Demonstrate proper care of dialysis catheter (psychomotor)

      • Protect patient’s privacy (affective)


Construct22 l.jpg
Construct

  • Another Example:

    • Program Learning Outcome: Function safely and competently in an automotive repair business

    • Course Competency: Demonstrate ability to diagnose, repair and replace transmissions

    • Student Learning Outcome:

      • Describe parts of a transmission (cognitive)

      • Disassemble a transmission (psychomotor)

      • Identify problems and defective parts of a transmission (cognitive)

      • Demonstrate respect for customer’s vehicle (affective)


Construct23 l.jpg
Construct

  • Another Example – General Studies Core:

    • Core Ability: Communicate effectively in writing using grammatically correct and appropriate language

    • Course Competency: Demonstrate ability to write an organized and defensible Theme Paper

    • Student Learning Outcome:

      • Prepare a topical outline (cognitive)

      • Produce a Theme Paper using MS Word (psychomotor)

      • Reflect sensitivity to opposing values and views (affective)


Construct24 l.jpg
Construct

  • Curriculum Mantra

    • Learning outcomes drive content

    • Assessment depends on defined learning outcomes

  • Program learning outcomes are broad

  • Course competencies are focused

  • Student learning outcomes support competencies

  • Unit content is structured based on course learning outcomes

  • Assessment methods must reflect learning outcomes


Nutshells l.jpg
Nutshells

Program Outcomes 

Course Competencies 

Student learning Outcomes 

Selection of Content 

Assessment / Evaluation Methods


Domains levels26 l.jpg
Domains & Levels

  • Good Learning Outcomes:

    • Identify a specific student behavior

    • Specify ONE learning outcome

      • Why is this important?

    • Are relevant (meaningful)

    • Are measurable / assessable


Domains levels27 l.jpg
Domains & Levels

  • Since learning occurs in all three Domains – cognitive, affective and psychomotor– learning outcomes, likewise, must be written for each domain and for varying levels within each course and program

  • There must be a trackable, observable and documentable progression of learning evidenced in courses and programs

  • Likewise, there must be an observable progression of assessment methods for higher levels within each domain


Levels of learning28 l.jpg
Levels of Learning

  • Domains: Learning (behavior change) occurs in three broad domains

    • Cognitive, Affective, Psychomotor (CAP)

  • Within each domain are levels of learning that drive assessment


Levels of learning29 l.jpg
Levels of Learning

  • Domain: Cognitive

    • Knowledge: recalling facts

    • Comprehension: seeing relationships

    • Application: using information in new ways

    • Analysis: breaking information into parts

    • Synthesis: forming new information

    • Evaluation: judging value


Levels of learning30 l.jpg
Levels of Learning

  • Domain:Psychomotor

    • Perception: awareness of need

    • Set: mental, physical emotional readiness to perform

    • Guided Response: skill performed by imitation, trial and error

    • Mechanism: habitual, skilled performance

    • Complex Response: smooth, efficient, automatic

    • Origination: adaptation to conditions


Levels of learning31 l.jpg
Levels of Learning

  • Domain:Affective

    • Receiving: willingness to hear

    • Responding: willingness to react

    • Valuing: demonstrating commitment

    • Organization: establishing pervasive values

    • Characterization: demonstrating characteristics of a unique individual


Levels of learning32 l.jpg
Levels of Learning

  • Breakdown of the levels using overriding framework of learning about computers


Levels of learning33 l.jpg
Levels of Learning

  • Cognitive:

    • Knowledge: define CPU; list software

    • Application: knowledge applied to selection

    • Evaluation: evaluate effectiveness / deficiencies


Levels of learning34 l.jpg
Levels of Learning

  • Psychomotor:

    • Perception: keyboarding / typing

    • Guided Response: practice, errors, correction

    • Complex Response: know where keys are

    • Origination: adjustments w/ broken finger


Levels of learning35 l.jpg
Levels of Learning

  • Affective:

    • Receiving: initial reaction – hesitation, fear

    • Valuing: what transpired to get past initial

      reaction

    • Characterization: purchase a personal PC


Levels of learning36 l.jpg
Levels of Learning

  • MAJOR POINTS:

    • Students must not only demonstrate learning in all domains but must demonstrate progression within the domains

    • You would never want a student to be “stuck” in lower levels and never progress


Levels of learning37 l.jpg
Levels of Learning

  • Reflection: How to structure and promote higher levels of learning for students?

    • Answer: By ensuring a planned and structured learning environment with identified outcomes that address varying levels and demonstrations of learning


Learning process l.jpg
Learning Process

  • Let’s play in the Domains vis-à-vis the writing of Learning Objectives:

    • Cognitive: Knowing

      • Knowledge: recalling facts

      • Comprehension: seeing relationships

      • Application: using knowledge

      • Analysis: breaking knowledge into parts

      • Synthesis: forming knowledge in new ways

      • Evaluation: judging knowledge’s value and appropriateness


Learning process measurable verbs knowledge l.jpg

Cite

Count

Define

Identify

Choose

Match

Recite

Repeat

Tell

Write

Select

Explain

Learning ProcessMeasurable Verbs: Knowledge


Learning process measurable verbs comprehension l.jpg

Associate

Compare

Describe

Discuss

Contrast

Outline

Predict

Report

Restate

Translate

Summarize

Learning ProcessMeasurable Verbs: Comprehension


Learning process measurable verbs application l.jpg

Apply

Classify

Determine

Illustrate

Choose

Interpret

Restructure

Solve

Use

Develop

Learning ProcessMeasurable Verbs: Application


Learning process measurable verbs analysis l.jpg

Analyze

Appraise

Differentiate

Distinguish

Categorize

Examine

Inspect

Question

Summarize

Infer

Learning ProcessMeasurable Verbs: Analysis


Learning process measurable verbs synthesis l.jpg

Assemble

Compose

Create

Formulate

Design

Integrate

Organize

Propose

Synthesize

Adapt

Learning ProcessMeasurable Verbs: Synthesis


Learning process measurable verbs evaluation l.jpg

Appraise

Assess

Critique

Evaluate

Conclude

Criticize

Judge

Defend

Revise

Validate

Prioritize

Disprove

Learning ProcessMeasurable Verbs: Evaluation


Learning process example cognitive l.jpg
Learning ProcessExample: Cognitive

  • Knowledge: List the components of a good outcome

  • Comprehension: Discuss the 3 Domains

  • Application: Determine which of 3 outcomes is appropriately stated

  • Analysis: Classify3 outcomes by domain level

  • Synthesis: Designoutcomes in all domains for a given learning scenario

  • Evaluation: Revisethese outcomes for alternate scenario


Learning process46 l.jpg
Learning Process

  • Let’s play in the Domains vis-à-vis the writing of Learning Objectives:

    • Psychomotor: Skills

      • Perception: awareness

      • Set: preparation to perform

      • Guided Response; demonstration trial/error

      • Mechanism: habitual skill development

      • Complex Response: smooth, efficient skill demo

      • Origination: adaptation of skill to conditions


Learning process measurable verbs perception set l.jpg

Perception:

Listen

Observe

Ask

Set:

Attend

Organize

Establish

Learning ProcessMeasurable Verbs: Perception / Set


Learning process measurable verbs guided response mechanism l.jpg

Guided Response:

Perform

Demonstrate

Establish

Mechanism:

Apply

Use

Implement

Learning ProcessMeasurable Verbs:Guided Response / Mechanism


Learning process measurable verbs complex response origination l.jpg

Complex Response:

Integrate

Activate

Construct

Origination

Adapt

Collaborate

Prioritize

Learning ProcessMeasurable Verbs: Complex Response / Origination


Learning process example psychomotor l.jpg
Learning ProcessExample: Psychomotor

  • Perception: Ask re: appropriate time to check BP

  • Set: Organize materials to check BP

  • Guided Response: Perform check with assistance of instructor

  • Mechanism: Implement routine to check 10 patients without assistance

  • Complex Response: Integrate BP checks w/ total assessment for a unit

  • Origination: Adapt BP check for pt. with bilateral arm fractures


Learning process51 l.jpg
Learning Process

  • Let’s play in the Domains vis-à-vis the writing of Learning Objectives:

    • Affective: Valuing

      • Receiving: willingness to hear

      • Responding: willingness to react

      • Valuing: commitment

      • Organization: establish pervasive values

      • Characterization: create “unique” person


Learning process measurable verbs receiving l.jpg

Listen

Perform

Comply

Assist

Care

Maintain

Learning ProcessMeasurable Verbs: Receiving


Learning process measurable verbs responding l.jpg

Adapt

Tolerate

Accept

Attend

Interact

Display

Learning ProcessMeasurable Verbs: Responding


Learning process measurable verbs valuing l.jpg

Cooperate

Anticipate

Consider

Seek

Influence

Follow

Learning ProcessMeasurable Verbs: Valuing


Learning process measurable verbs organization l.jpg

Respect

Resolve

Encourage

Offer

Establish

Relate

Learning ProcessMeasurable Verbs: Organization


Learning process measurable verbs characterization l.jpg

Promote

Share

Motivate

Initiate

Educate

Appreciate

Learning ProcessMeasurable Verbs: Characterization


Learning process example affective l.jpg
Learning ProcessExample: Affective

  • Receiving: Listen during class

  • Responding: Display interest in using new material

  • Valuing: Follow instruction re: new project

  • Organization: Offer evaluation of the project

  • Characterization: Promote behaviors consistent with new learning


Key questions l.jpg
Key Questions

  • Learning Outcomes:

    • What is actual, highest level learning expected ?

    • What will it “look like” when the student performs it?

    • What are essential elements of performance necessary to demonstrate learning?

    • What are the most effective ways for students to master these learning outcomes?

    • How will I know the student has achieved?

    • How can I assess it effectively?

    • How can I document learning outcomes?


Assessing learning outcomes l.jpg
Assessing Learning Outcomes

  • The assessment of learning outcomes requires a mindset change: movement from teaching to learning – and learner-centered assessment.


Assessment l.jpg
Assessment

  • Assessment is the ongoing process of:

    • Establishing clear, measurable expected outcomes of student learning

      • Learning Outcomes statements

    • Ensuring that students have opportunity to achieve the outcomes

      • Learning Opportunities - methodologies


Assessing learning outcomes61 l.jpg
Assessing Learning Outcomes

  • Differentiation between levels of Learning Outcomes and Assessments:

    • Institution

    • Program

    • Course

    • Student


Assessing learning outcomes62 l.jpg
Assessing Learning Outcomes

  • What is assessment of Student Learning Outcomes?

    • Formalized processes to collect and analyze learning outcome data that will be evaluated to derive meaning and purposeful use of the data for learning enhancement.

    • Assessment is meaningless unless evaluation occurs (BP example)


Assessing learning outcomes63 l.jpg
Assessing Learning Outcomes

  • Student learning outcomes defined:

    A particular level of knowledge, skills and abilities (cognitive, psychomotor, affective) that a student has attained at the end (or as a result) of his/her engagement in a particular set of collegiate experiences


Assessing learning outcomes64 l.jpg
Assessing Learning Outcomes

  • This, then, begs the questions:

    • How to make assessment /evaluation meaningful?

    • What outcome data should be assessed and evaluated?

  • O’Banion’s 2 questions related to teaching and learning are:

    • Does it (whatever happens in the classroom) enhance learning?

    • How do we know?


Assessing learning outcomes65 l.jpg
Assessing Learning Outcomes

Purpose and Principles

  • Purpose of Assessment (Cross Papers)

    • Faculty acquire information to enhance teaching

    • Students gain insight into effectiveness of their own learning processes

      • Provide support for student learning

      • Provide a means for assessing strengths/weaknesses

      • Evaluate effectiveness of teaching/learning strategies

      • Show evidence of progress or areas for improvement

      • Provide support for formative approach to improve


Assessing learning outcomes66 l.jpg
Assessing Learning Outcomes

  • Principles of Assessment (Cross Papers)

    • Learner-centered (learning vs. teaching)

    • Assess what matters: common expectations (taxonomies)

    • Assess for improvement vs. accountability

      • Banta: “…if we undertake assessment just for the purpose of demonstrating accountability, we waste our time. We must do assessment in a way that we are getting information that faculty can actually use to improve what they’re doing.”

      • Fear of use for evaluation; “teach to the test”


Assessing learning outcomes67 l.jpg
Assessing Learning Outcomes

  • Research validates that teaching for analytical, creative and practical thinking, as well as for memory, boosts achievement on assessment that measures achievement broadly – across subject matter areas; i.e. assessment focused on higher level processes positively effects achievement and learning outcomes in all areas of learning regardless of topic or particular assessment methodology.


Assessing learning outcomes68 l.jpg
Assessing Learning Outcomes

  • Foundations of Assessment

    • Bloom’s Taxonomy of Learning

    • “Backward Design”

    • Teaching to Standards

    • What / How of Assessment


Foundations of assessment l.jpg
Foundations of Assessment

  • MAJOR POINTS: Bloom’s Taxonomy

    • Students must not only demonstrate learning in all domains but must demonstrate progression within the domains

    • Assessment strategies must be compatible with the level of learning desired


Key questions70 l.jpg
Key Questions

  • Why is it critical to:

    • Assess the stage of learning at which the student is functioning?

    • Exhibit teaching strategies based on appropriate level?

    • Implement assessment methods based on appropriate levels of learning and expectations of performance?

      Example:

      • Course Competency: Write a Business Plan

        Assessment Method: Multiple choice test


Foundations of assessment71 l.jpg
Foundations of Assessment

  • “Backward Design” (Tomlinson)

    • Imperative to define what students must know, think and do and understand as a result of instruction

    • Teaching must match these expectations

    • To teach for success, assessment must be aligned with knowledge, understanding and skill identified as learning outcomes: assessment is an integral part of curriculum design – not an afterthought.


Foundations of assessment72 l.jpg
Foundations of Assessment

  • Teaching to Standards (Santoyo)

    • Faculty must achieve consensus on the standards (outcomes) and expectations of student learning outcomes (domains, levels of learning, etc.)

    • Faculty must achieve consensus on the assessment strategies and methods to be used to effectively elicit outcome data as evidence of student learning

    • Examples of teaching to standards when no clear standards / assessment identified:


Foundations of assessment73 l.jpg
Foundations of Assessment

  • Standard (outcome): The student will understand and use percents in a variety of situations

  • Variety of assessments used:

    • What is 50% of 20?

    • What is 67% of 81

    • Shawn got 7 correct answers out of 10 possible on a science test. What percentage of questions did he answer correctly?


Foundations of assessment74 l.jpg
Foundations of Assessment

  • Assessments, cont’d:

    • J.J. Redick was on pace to set a college basketball record in career free throw percentage. Going into the NCAA tournament in 2004, he had made 97 of 104 free throw attempts. What percentage of free throws had he made?

    • J.J. Redick was on pace to set an NCAA record in career free throw percentage. Going into the NCAA tournament in 2004, he had made 97 of 104 free throw attempts. In the first tournament game, Redick missed his first five free throws. How far did his percentage drop from right before the tournament game to right after missing those free throws?


Foundations of assessment75 l.jpg
Foundations of Assessment

  • Assessments cont’d:

    • J.J. Redick and Chris Paul were competing for the best free throw percentage. Redick made 94 percent of his first 103 shots, whereas Paul made 47 of 51 shots.

      • (a) Which one had a better shooting percentage?

      • (b) in the next game, Redick made only 2 of 10 shots and Paul made 7 of 10 shots. What are their new overall shooting percentages? Who is the better shooter?

      • (c) Jason argues that if J.J. and Chris each made their next 10 shots, their shooting percentages would go up the same amount. Is this true? Why or why not?

      • (d) Describe in detail how you arrived at your answers.


Foundations of assessment76 l.jpg
Foundations of Assessment

  • CENTRAL POINT re: relationship b/w standards (outcomes) and assessment: outcomes are meaningless until you define how you will assess them

  • Without dialogue, faculty may – and do – define standards (outcomes) according to personal level of expectations – which differ radically – and, therefore, teach to varying levels of mastery

  • Assessment without consensus  poor outcomes


Forms of assessment l.jpg
Forms of Assessment

  • Use of the appropriate assessment tool requires consideration of the purpose of assessment

    • Why are you assessing? (what purpose: development; feedback; mastery, etc.)

    • What do you want to assess?


Assessing learning outcomes78 l.jpg
Assessing Learning Outcomes

  • Assessment Types: (Earl and Tomlinson)

    • Formative: ongoing, periodic assessment of learning – “assessment for learning”

    • Summative: end-point assessment – “assessment of learning

    • Informative: “assessment as learning”


Forms of assessment79 l.jpg
Forms of Assessment

  • What and How of Assessment

    • Selection of the type of assessment depends on what faculty hope to learn from the assessment and what is being evaluated

      • Quizzes and exams: snapshots in time of recall ability and memory

      • Portfolios: longitudinal record of work and progress over time

      • Essays: ability to communicate in writing; analyze and solve problems; integrate ideas and information


Forms of assessment80 l.jpg
Forms of Assessment

  • Although educators may denigrate the validity, reliability and usefulness of tests/quizzes to effectively evaluate learning, it is, however, the most dominant form of assessment

  • Tests / exams / quizzes can be successful when questions focus and reflect learning outcomes and assess learning in different domains (Bloom)


Forms of assessment81 l.jpg
Forms of Assessment

  • Example:

    • If students know they will be tested on facts, they will memorize

    • If students know they will be assessed for comprehension and application, they will study and learn differently

    • Creation of a table (Cross) that reveals percentage of time spent on concepts at various domain levels helps define the nature of assessment methodologies


Assessing learning outcomes82 l.jpg
Assessing Learning Outcomes

Domain Questioning Strategy Concept 1 Concept 2 Concept 3

  • Knowledge Recall information and facts 25% 10% 15%

  • Comprehension Understand and interpret concepts 25% 35% 10%

  • Application Employ concepts in new situations 15% 15%

  • Analysis Classify information into components 20% 15%

  • Synthesis Produce structures /patterns: create new meaning 30% 30%

  • Evaluation Make judgments or valuations 30% 30%


Assessing learning outcomes83 l.jpg
Assessing Learning Outcomes

  • Good Assessment Practices:

    • Start with clear statements

    • Assess what is taught

    • Collect more than one kind of evidence

    • Make assignments and testing clear

    • Ensure that assignments and testing relate to learning objectives (matrices)

    • Score assignments fairly and consistently (rubrics)

    • Evaluate the outcomes of assessment efforts and revise strategies


Assessing learning outcomes84 l.jpg
Assessing Learning Outcomes

  • Assessment Data:

    • Quantitative: numbers indicate learning or achievement (points, grades)

    • Qualitative: quality differences vs. amount; judgment-based – observation of performance, interviews, etc.


Assessing learning outcomes85 l.jpg
Assessing Learning Outcomes

  • Assessment Score Interpretations:

    • Norm-referenced: comparison of student performance to other people in a referenced population; relative standing vs. specific mastery (grading on the curve)

    • Criterion-referenced: comparison to precise score that indicates mastery or competence (COMPASS, licensure scores, etc.)


Assessing learning outcomes86 l.jpg
Assessing Learning Outcomes

  • Assessment Characteristics:

    • Reliability: degree to which scores are consistent over repeated applications of the assessment

    • Validity: ability of the assessment to measure what it purports to measure – measurement aligned with learning outcomes and free of bias


Assessing learning outcomes87 l.jpg
Assessing Learning Outcomes

Formative Assessment:

“Assessment for learning” allows corrections to be made in the t/l process; evidence used to adapt teaching to meet learning needs


Assessing learning outcomes88 l.jpg
Assessing Learning Outcomes

  • Elements of Formative Assessment

    • Specific learning goals / outcomes clearly communicated to students

    • Specific criteria for meeting these goals / outcomes clearly communicated (e.g. rubric)

    • Frequent interaction b/w faculty and students for clarification and reinforcement of expectations

    • Timely feedback re: assessment results

    • Willingness of faculty to modify curriculum and t/l strategies based on results


Assessing learning outcomes89 l.jpg
Assessing Learning Outcomes

To be effective and provide maximum learning value, a variety of assessment techniques should be applied numerous times during a course.

Comprehensive assessment includes formative and summative strategies - with variety of methodologies. Students tend to excel at 1 or 2 methods – to the detriment of others who might excel at others. A variety ensures that all students have opportunity to demonstrate learning.


Assessing learning outcomes90 l.jpg
Assessing Learning Outcomes

  • Steps of Assessment: formative and/or summative

    • ID learning outcome to be assessed

    • Select assessment technique that will accurately and appropriately measure the outcome

    • Apply the assessment technique

    • Analyze the results of assessment

    • Share results with students and provide feedback

    • Respond to the results and effect changes


Assessing learning outcomes91 l.jpg
Assessing Learning Outcomes

  • Tools of Assessment:

    • Tests: quizzes, T/F, multiple choice, essays

    • Performance demonstrations

    • Portfolios

  • Rubrics: eliminates ambiguity re: criteria to meet learning goals, expectations of performance and how evaluation will occur

  • Qualitative and Quantitative rubrics acceptable


Assessing learning outcomes92 l.jpg
Assessing Learning Outcomes

  • Elements of Rubrics:

    • Identified levels of mastery (excellent, good, 5,4…)

    • Organizational groupings of skill sets: teamwork, problem-solving, etc.

    • Commentaries – description of elements to be identified in the work

    • Description of consequences





Assessing learning outcomes96 l.jpg
Assessing Learning Outcomes

  • Benefits of Assessment: Students

    • Clear expectations upon which assessment is based helps students focus

    • Assessment motivates students

    • Feedback helps understand strengths and weaknesses (must be timely)


Assessing learning outcomes97 l.jpg
Assessing Learning Outcomes

  • Benefits of Assessment: Faculty

    • Assessment encourages faculty discussion on issues: what taught; why; standards; expectations

    • Assessment links courses together to form coherent programs; how courses provide foundation for subsequent courses

    • Assessment results are evidence of quality T/L


Assessing learning outcomes98 l.jpg
Assessing Learning Outcomes

  • No one “right” or “best” way

  • Strategic Questions:

    • What do you want assessment to do?

    • What questions do you want it to answer?

    • What assessment method will answer these questions?


Reflection assessment l.jpg
Reflection: Assessment

  • Do outcomes determine what is assessed or does ease of assessment determine outcomes?

  • Are achievements inferred from process completion or affirmed by assessment?

  • Are assessment criteria and methods relevant and meaningful for outcomes being evaluated?

  • How valid and reliable is the assessment of outcomes?

  • Who assesses? Who evaluates? Are these assessors and evaluators selected for convenience or effectiveness?


Reflection assimilation l.jpg
Reflection: Assimilation

  • Does assessment occur on a regular, purposeful basis?

  • How are assessment findings used for planning and decision making?

  • How are assessment findings used to improve institutional, unit and program effectiveness?

  • How are assessment findings used to improve teaching effectiveness and student learning?


Reflection assimilation101 l.jpg
Reflection: Assimilation

  • Is higher order thinking assessed when evaluating student learning?

  • Does the assessment relate to and support course / program outcomes?

  • Does the assessment support the development of stated learning outcomes and competencies?

  • Should course outcomes, test questions, or assignments be revised?


Reflection assimilation102 l.jpg
Reflection: Assimilation

  • Do assessment tools accurately reflect and assess identified learning outcomes?

  • Are faculty in consensus as to assessment tools/methods and content so that learning outcomes for a given course can be assessed regardless of who has taught the course?

  • Is there a common final exam, standardized assessment tool, portfolio, presentation, etc. that could provide across-course common outcomes assessment?


Reflection assimilation103 l.jpg
Reflection: Assimilation

  • Evaluate Assessment Tools

    • Use outcome-measure comparison matrix to map learning outcomes to assessment instrument

    • If gaps, use outcome-test item comparison matrix to map assessment items to learning outcomes


The bridge l.jpg
The Bridge

Level Outcomes Assessment

College: College-wide Employment Rate

Programs that support workforce College-wide Retention Rate

development College-wide Graduation Rate

Program Outcomes: Automotive

1. Employed in field Program Placement Rate

2. Function safely, competently, ethically Employer Survey

3. Graduate ASE Certifications Graduate Success Rate on ASE Certifications

4. Self-Starter and Team Player Employer Survey


The bridge105 l.jpg
The Bridge

Level Outcomes Assessment

Program Outcome 1: Employed in field

Course: AUT 220 Automotive Technology Course Grade

Internship

Competencies:

(1) Demonstrate appropriate Employability Skills

Learning Outcome: Communicate effectively: oral

and written skills Supervisor Evaluation

(2) Acceptable Job Performance

Learning Outcome: Perform a thorough automotive

assessment, recommend and

perform necessary repairs Skills performance


The bridge106 l.jpg
The Bridge

Level Outcomes Assessment

Program Outcome 2: Function safely, competently, ethically

Course: AUT 120 Introduction to Automotive

Technology Course Theory Grade

Course Shop Grade

Competencies:

(1) Demonstrate safe practices in lab setting

Learning Outcome: Analyze hazardous conditions and

implement safety precautions Shop Performance: Safety Unit

(2) Summarize legal / ethical responsibilities

Learning Outcome: Describe legal and ethical

implications r/t billable hours Unit Exam


Wrap up l.jpg
Wrap-Up

  • Pat Cross “Basic Assumptions”

    • The quality of student learning is directly, although not exclusively, related to the quality of teaching. Therefore, one of the most promising ways to improve learning is to improve teaching.

    • To improve teaching, teachers need first to make their goals and objectives explicit and then to get specific, comprehensible feedback on the extent to which they are achieving those goals and objectives.

    • To improve learning, students need to receive appropriate and focused feedback early and often; they also need to learn how to assess their own learning.


Curriculum database process l.jpg
Curriculum Database: Process

Application of Theory to Curriculum Database Tool


Slide109 l.jpg

The Relationship between Programs, Courses and Competencies

  • Program:

  • Course:

  • Course Competency:

  • Learning Outcome:

    • Learning Domain:

    • Levels of Learning:

    • Curriculum Integration: (Core Ability)

  • Learning Outcome:

    • Learning Domain:

    • Levels of Learning:

    • Curriculum Integration:











  • Additional resources learning objectives l.jpg
    Additional Resources: Learning Objectives

    • Angelo, Thomas A. and Cross, K. Patricia. Classroom Assessment Techniques: A Handbook for College Teachers. 2nd Edition. Jossey-Bass Publishers. San Francisco, CA. 1993

    • Cross, K. Patricia. Learning is About Making Connections. The Cross Papers, Number 3. League for Innovation in the Community College: Educational Testing Service. 1999

    • Lowman, Joseph. Mastering the Techniques of Teaching, 2nd edition. Jossey-Bass Publishing, San Francisco, CA. 1995.

    • Popham, W. James; Baker, Eva L. Establishing Instructional Goals. Prentice-Hall, Inc., New Jersey. 1970

    • Suskie, Linda. Assessing Student Learning: A Common Sense Guide. Anker Publishing Company, Bolton, MA. 2004.

    • Wilson, Cynthia D., et.al,. Learning outcomes for the 21st Century: Report of a Community College Study. League for Innovation in the Community College: The Pew Charitable Trusts. 2000.


    Additional resources assessment and evaluation l.jpg
    Additional Resources: Assessment and Evaluation

    • Banta, T.W. et. al., (Eds.), (1996), Assessment in Practice: Putting Principles to Work on College Campuses. San Francisco: Jossey-Bass Publishers. ISBN: 0-7879-0134-2

    • Erwin, T. Dary, (1991), Assessing Student Learning and Development. San Francisco: Jossey-Bass Publishing. ISBN: 1-55542-325-6

    • Nichols, James O., (1995), A Practitioner’s Handbook for Institutional Effectiveness and Student Outcomes Assessment Implementation. New York: Agathon Press. ISBN: 0-87586-113-x

    • Nichols, James O., (1995), Assessment Case Studies: Common Issues in Implementation with Various Campus Approaches to Resolution. New York: Agathon Press. ISBN: 0-87586-112-1


    Additional resources assessment and evaluation135 l.jpg
    Additional Resources: Assessment and Evaluation

    • Palomba, C. A.. & Banta T. W., (Eds.), (1999), Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education. San Francisco: Jossey-Bass Publishing. ISBN: 0-7879-4180-8

    • Suskie, Linda, (2004), Assessing Student Learning: A Common Sense Guide.Bolton, MA: Anker Publishing Company. ISBN: 1-882982-71-1

    • Woolvard, Barbara E., (2004), Assessment Clear and Simple: A Practical Guide for Institutions, Departments, and General Education. San Francisco: Jossey-Bass Publishers. ISBN: 0-7879-7311-4


    ad