slide1
Download
Skip this Video
Download Presentation
Developed & Presented by Edward J. Caropreso, PhD Watson School of Education

Loading in 2 Seconds...

play fullscreen
1 / 73

Developed & Presented by Edward J. Caropreso, PhD Watson School of Education - PowerPoint PPT Presentation


  • 167 Views
  • Uploaded on

Developed & Presented by Edward J. Caropreso, PhD Watson School of Education University of North Carolina Wilmington [email protected] 910.962.7830 Director of Academics Marine Corps Combat Service Support Schools Camp Johnson, NC January, 2011.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Developed & Presented by Edward J. Caropreso, PhD Watson School of Education' - zachary-nelson


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Developed & Presented by

Edward J. Caropreso, PhD

Watson School of Education

University of North Carolina Wilmington

[email protected]

910.962.7830

Director of Academics

Marine Corps Combat Service Support Schools

Camp Johnson, NC

January, 2011

Developing Assessment Literacy:A workshop on the relationship between instruction, assessment & evaluation

developing assessment literacy learning outcomes
Developing Assessment Literacy: Learning Outcomes
  • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)

2. Recognize/value the relationship between professional vocabulary and practice

3. Recognize/value the use of professional vocabulary

  • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation

2. Recognize differences between objectives models

3. Write objectives using a specific model

4. Critique objectives using a specified evaluation system

  • 1. Recognize/value the role of appropriate information in decision-making

2. Recognize/value the relationship between types of assessment & resulting information

3. State the difference between objective & subjective (non-objective) assessment

  • 1. State the difference between traditional & alternative assessment models

2. State the difference between models of alternative assessment

3. State the purpose/s for rubrics in assessment

4. Identify appropriate applications of rubrics (in specific assessment settings/contexts)

  • 1. Describe formative assessment strategies

2. State appropriate applications for formative assessment/s

  • 1. Recognize/value the need for systematic planning for assessment

2. Evaluate item types for given applications/purposes

3. Construct items to meet specific Los in given learning domains

4. Evaluate items/tests as to purpose/use of assessment information

  • 1. Describe validity; reliability

2. Explain the relationship between high quality assessment and validity/reliability

  • 1. Recognize/value the role of instructors as student study supports

2. Recognize the relationship between test item types & specific study strategies

developing assessment literacy learning outcomes1
Developing Assessment Literacy: Learning Outcomes
  • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)

2. Recognize/value the relationship between professional vocabulary and practice

3. Recognize/value the use of professional vocabulary

  • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation

2. Recognize differences between objectives models

3. Write objectives using a specific model

4. Critique objectives using a specified evaluation system

  • 1. Recognize/value the role of appropriate information in decision-making

2. Recognize/value the relationship between types of assessment & resulting information

3. State the difference between objective & subjective (non-objective) assessment

  • 1. State the difference between traditional & alternative assessment models

2. State the difference between models of alternative assessment

3. State the purpose/s for rubrics in assessment

4. Identify appropriate applications of rubrics (in specific assessment settings/contexts)

developing assessment literacy learning outcomes2
Developing Assessment Literacy: Learning Outcomes
  • 1. Describe formative assessment strategies

2. State appropriate applications for formative assessment/s

  • 1. Recognize/value the need for systematic planning for assessment

2. Evaluate item types for given applications/purposes

3. Construct items to meet specific Los in given learning domains

4. Evaluate items/tests as to purpose/use of assessment information

  • 1. Describe validity; reliability

2. Explain the relationship between high quality assessment and validity/reliability

  • 1. Recognize/value the role of instructors as student study supports

2. Recognize the relationship between test item types & specific study strategies

workshop contents
Workshop Contents
  • Introduction & Context for Training
    • Knowledge-Practice Model
    • Professional Vocabulary & Concepts
  • Context for Instruction
    • Educational Goals & Objectives
    • Models of Objectives
    • Working with Objectives
  • Context for Assessment & Evaluation
    • Role of Information for Decision-making
      • Types of data related to types of decisions
    • Assessment Model
    • Types of Assessments
      • Objective vs Subjective, Etc.
  • Alternative Assessment: Performance Strategies Complementing Paper Assessment
  • Formative Assessment: Information about Teaching & Learning to Improve Instruction
  • Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work context
  • Assessment & Test Construction: Technical Features & “Quality” Tools
    • Validity
    • Reliability
  • Supporting Student Learning through Assessment
    • KSU Study Strategies; Test Preps
    • Improving Performance; MIT Test Strategies
workshop contents1
Workshop Contents
  • Introduction & Context for Training
    • Knowledge-Practice Model
    • Professional Vocabulary & Concepts
  • Context for Instruction
    • Educational Goals & Objectives
    • Models of Objectives
    • Working with Objectives
  • Context for Assessment & Evaluation
    • Role of Information for Decision-making
      • Types of data related to types of decisions
    • Assessment Model
    • Types of Assessments
      • Objective vs Subjective, Etc.
workshop contents2
Workshop Contents
  • Alternative Assessment: Performance Strategies Complementing Paper Assessment
  • Formative Assessment: Information about Teaching & Learning to Improve Instruction
  • Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work context
  • Assessment & Test Construction: Technical Features & “Quality” Tools
    • Validity
    • Reliability
  • Supporting Student Learning through Assessment
    • KSU Study Strategies; Test Preps
    • Improving Performance; MIT Test Strategies
workshop contents3
Workshop Contents
  • Introduction & Context for Training
    • Knowledge-Practice Model
    • Professional Vocabulary & Concepts
developing assessment literacy learning outcomes3
Developing Assessment Literacy: Learning Outcomes
  • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)

2. Recognize/value the relationship between professional vocabulary and practice

3. Recognize/value the use of professional vocabulary

1 introduction context for training
1. Introduction & Context for Training
  • Knowledge-Practice Model of Assessment & Evaluation
  • Professional Vocabulary for Concepts
knowledge practice model
Knowledge-Practice Model

Developing Professional Knowledge

Knowledge of Learner Achievement

Resulting from Instruction

Knowledge of Instructional Practice: Strategies & Activites (incl. assessment)

Dynamic Reciprocity

knowledge practice model1
Knowledge-Practice Model
  • Developing Professional Knowledge
knowledge practice model discussion activity
Knowledge-Practice Model(Discussion Activity)
  • What’s the current status of your professional knowledge?
  • How have you developed your knowledge of instruction, assessment and evaluation?
  • What are your expectations for developing your professional knowledge & practice?
  • What are your professional development alternatives?
slide14
Professional Vocabulary for ConceptsActivity: Review/id concepts you know well; compare/contrast with colleagues’/groups’

Resources & Databases

  • New Horizons for Learning
    • http://www.newhorizons.org/strategies/assess/terminology.htm
  • Nat’l. Center for Research on Evaluation, Standards, & Student Testing (CRESST @ UCLA)
    • http://www.cse.ucla.edu/products/glossary.html
  • Coalition of Essential Schools: Defining Assessment
    • http://www.essentialschools.org/cs/resources/view/ces_res/124
  • System of Adult Basic Education Supports
    • http://www.sabes.org/assessment/glossary.htm
  • Beyond Confusion: An Assessment Glossary, by Andrea Leskes
    • http://ctl.stanford.edu/Tomprof/postings/448.html
  • Continuous Assessment: Practical Guide for Teachers
professional vocabulary activity
Professional Vocabulary(Activity)
  • Review the glossaries
  • Identify the terms & concepts you know
    • Mark the ones you know & understand
  • Compare your list to colleagues’/groups’
    • What’s the same; what’s different?
    • Why are there differences?
  • What are the implications for professional practice given inconsistencies in the knowledge base?
workshop contents4
Workshop Contents
  • Context for Instruction
    • Educational Goals & Objectives
    • Models of Objectives
    • Working with Objectives
developing assessment literacy learning outcomes4
Developing Assessment Literacy: Learning Outcomes
  • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)

2. Recognize/value the relationship between professional vocabulary and practice

3. Recognize/value the use of professional vocabulary

  • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation

2. Recognize differences between objectives models

3. Write objectives using a specific model

4. Critique objectives using a specified evaluation system

2 context for instruction
2. Context for Instruction
  • Educational Goals & Objectives
  • Models of Objectives
  • Working with Objectives
educational goals objectives
Educational Goals & Objectives
  • Educational Goals:
    • “…those human activities which contribute to the functioning of a society (including the functioning of an individual in society), and which can be acquired through learning.” (Gagne, Briggs, & Wagner, 1988, p. 39; cited in Nitko, 2001, p. 23)
    • Goals are stated in broad terms that give direction and purpose to planning overall educational activities.
educational goals objectives1
Educational Goals & Objectives
  • Educational Goals (or global objectives) :
    • Broad, complex learning outcomes that require substantial time and instruction to accomplish
    • Typically encompassing many specific learning objectives which require learner success on most/all to reach the general goal

(from Airasian, 2001)

educational goals objectives2
Educational Goals & Objectives

Educational Goals

  • UT Examples of Goals vs Objectives
  • More Examples of Goals vs Objectives
  • Airasian’s 3-Level Model
models of objectives
Models of Objectives
  • Two Common Models: Mager & Gronlund
  • Mager Tips
  • Gagne & Briggs & ABCD Models
    • A Comparison of Models
  • Greenberg’s 6 Keys to Successful Performance Objectives
comparison of formats seels glasgow 1990 exercises in instructional design columbus ohio merrill
Comparison of Formats[Seels & Glasgow (1990). Exercises in Instructional Design. Columbus, Ohio: Merrill.]
working with objectives
Working with Objectives
  • Understanding the Context of Learning
    • How do you make sense of learning?
    • How do you identify/describe expectations?
  • Putting Bloom’s Taxonomy to Work
  • Three Types of Learning
  • What Drives Statements of Learning: Active Verbs
working with objectives activity
Working with Objectives: Activity
  • Identify & State 1-3 TLOs (goals; in your area)
  • For each TLO (goal), using the course materials, identify & state 1-2 knowledge & 1-2 performance objectives (ELOs) in at least two models we’ve reviewed
  • Compare your objectives with a colleague’s
    • Using Greenberg’s Criteria to review your objectives, how do they hold up?
    • What’s similar to your colleague?
    • What’s different?
  • Keep your objectives for later; we’ll use them again!
workshop contents5
Workshop Contents
  • Context for Assessment & Evaluation
    • Role of Information for Decision-making
      • Types of data related to types of decisions
    • Assessment Model
    • Types of Assessments
      • Objective vs Subjective, Etc.
developing assessment literacy learning outcomes5
Developing Assessment Literacy: Learning Outcomes
  • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)

2. Recognize/value the relationship between professional vocabulary and practice

3. Recognize/value the use of professional vocabulary

  • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation

2. Recognize differences between objectives models

3. Write objectives using a specific model

4. Critique objectives using a specified evaluation system

  • 1. Recognize/value the role of appropriate information in decision-making

2. Recognize/value the relationship between types of assessment & resulting information

3. State the difference between objective & subjective (non- objective) assessment

3 context for assessment evaluation
3. Context for Assessment & Evaluation
  • Establishing vocabulary & conceptual framework
  • Assessment and Evaluation:
    • What do they mean?
  • Relationship between assessment, evaluation and decision-making:
    • The role of time
  • What about instructional outcomes?
    • Instruction, assessment and evaluation
    • Developing Criteria
3 context for assessment evaluation1
3. Context for Assessment & Evaluation
  • Model of Assessment
    • Identifying & applying appropriate strategies to gather information for specific purposes &/or decisions
    • Determining appropriate assessment strategies
    • Appropriately applying strategies
3 context for assessment evaluation assessment model
3. Context for Assessment & Evaluation: Assessment Model

Purposeful Decision Making

Assess Instructional Practices:

Methods, Materials, Strategies & Activities

Assess Learner Achievement

Resulting from Instruction

Dynamic & Reciprocal

3 context for assessment evaluation2
3. Context for Assessment & Evaluation
  • Types of Assessments by Purpose
    • Bloom’s Taxonomy as Context
    • Aim High
  • Objective Assessment
    • Selected vs Constructed Response
  • Subjective (Non-Objective) Assessment
    • Writing/Generating Responses
    • Role & Control of Subjectivity in Assessment
  • Alternative Assessment
  • Authentic Assessment
  • Performance Assessment
3 context for assessment evaluation http vudat msu edu assessment
3. Context for Assessment & Evaluation [http://vudat.msu.edu/assessment/]
  • Objective Assessment
    • Objective assessments (usually multiple choice, true false, matching, short answer) have correct answers.
    • These are good for testing recall of facts and can be automated.
    • Objective tests assume that there are true answers and assume that all students should learn the same things.
3 context for assessment evaluation http vudat msu edu assessment1
3. Context for Assessment & Evaluation[http://vudat.msu.edu/assessment/]
  • Subjective (Non-Objective) Assessment
    • With subjective assessments, teacher\'s judgment determines the grade
    • These include essay tests.
    • Essay tests take longer to answer and they take longer to grade than objective questions and therefore only include a small number of questions, focusing on complex concepts.
  • Writing/Generating Responses
    • Role & Control of Subjectivity: Criteria &/or Rubrics
workshop contents6
Workshop Contents
  • Introduction & Context for Training
    • Knowledge-Practice Model
    • Professional Vocabulary & Concepts
  • Context for Instruction
    • Educational Goals & Objectives
    • Models of Objectives
    • Working with Objectives
  • Context for Assessment & Evaluation
    • Role of Information for Decision-making
      • Types of data related to types of decisions
    • Assessment Model
    • Types of Assessments
      • Objective vs Subjective, Etc.
  • Alternative Assessment: Performance Strategies Complementing Paper Assessment
developing assessment literacy learning outcomes6
Developing Assessment Literacy: Learning Outcomes
  • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)

2. Recognize/value the relationship between professional vocabulary and practice

3. Recognize/value the use of professional vocabulary

  • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation

2. Recognize differences between objectives models

3. Write objectives using a specific model

4. Critique objectives using a specified evaluation system

  • 1. Recognize/value the role of appropriate information in decision-making

2. Recognize/value the relationship between types of assessment & resulting information

3. State the difference between objective & subjective (non-objective) assessment

  • 1. State the difference between traditional & alternative assessment models

2. State the difference between models of alternative assessment

3. State the purpose/s for rubrics in assessment

4. Identify appropriate applications of rubrics (in specific assessment settings/contexts)

4 alternative assessment performance strategies complementing paper assessment
4. Alternative Assessment: Performance Strategies Complementing Paper Assessment
  • Alternative
    • To what?
    • Portfolio Assessment: An example
  • Authentic
    • With respect to what?
  • Performance
    • What types?
4 alternative assessment activity
4. Alternative Assessment : Activity
  • Alternative
    • Possible examples?
    • Portfolio Assessment: Possible examples?
  • Authentic
    • Possible examples?
  • Performance
    • Possible examples?
4 alternative assessment alternative methods to document learning
4. Alternative Assessment: Alternative Methods to Document Learning
  • 3 Formal Models of Assessment:
  • Holistic:
    • Applying all criteria as a single judgment
  • Modified Holistic:
    • Adding levels of judgment
  • Analytical:
    • Judgment by criteria; weighting factors; scaling
  • Valid decision-making:
    • Matching the “tool” and the instructional outcome
  • Examples of Rubrics: Rubric Primer
    • Information Skills
    • Oral Presentation
    • Creating Rubrics
workshop contents7
Workshop Contents
  • Introduction & Context for Training
    • Knowledge-Practice Model
    • Professional Vocabulary & Concepts
  • Context for Instruction
    • Educational Goals & Objectives
    • Models of Objectives
    • Working with Objectives
  • Context for Assessment & Evaluation
    • Role of Information for Decision-making
      • Types of data related to types of decisions
    • Assessment Model
    • Types of Assessments
      • Objective vs Subjective, Etc.
  • Alternative Assessment: Performance Strategies Complementing Paper Assessment
  • Formative Assessment: Information about Teaching & Learning to Improve Instruction
developing assessment literacy learning outcomes7
Developing Assessment Literacy: Learning Outcomes
  • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)

2. Recognize/value the relationship between professional vocabulary and practice

3. Recognize/value the use of professional vocabulary

  • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation

2. Recognize differences between objectives models

3. Write objectives using a specific model

4. Critique objectives using a specified evaluation system

  • 1. Recognize/value the role of appropriate information in decision-making

2. Recognize/value the relationship between types of assessment & resulting information

3. State the difference between objective & subjective (non-objective) assessment

  • 1. State the difference between traditional & alternative assessment models

2. State the difference between models of alternative assessment

3. State the purpose/s for rubrics in assessment

4. Identify appropriate applications of rubrics (in specific assessment settings/contexts)

  • 1. Describe formative assessment strategies

2. State appropriate applications for formative assessment/s

5 formative assessment information about teaching learning to improve instruction
5. Formative Assessment: Information about Teaching & Learning to Improve Instruction
  • Formative Assessment
  • Types of Formative Assessment
    • Objective assessments
      • Forms/versions of summative assessments that will be implemented for final evaluation of students
    • Alternative assessments
      • Forms/versions of summative assessments that could be implemented for final evaluation of students
    • Used to provide feedback on developing learning
      • Should not contribute to final grades
    • May or may not be “graded” assessments
      • Limited direct impact on final evaluation of student
5 formative assessment information about teaching learning to improve instruction1
5. Formative Assessment: Information about Teaching & Learning to Improve Instruction
  • Observation Strategies
    • Using explicit formal tools
    • Informal observations
  • Questioning Strategies
    • Taba Tables
  • Self & Peer Assessment Strategies
5 formative assessment information about teaching learning to improve instruction2
5. Formative Assessment: Information about Teaching & Learning to Improve Instruction
  • Alternative assessments
    • Used to provide feedback on developing learning
  • Examples:
  • Formative Evaluation Form
  • Summary Evaluation Form
  • Observation Record Form
  • Observation Guide Form
  • Observation Assessment Form
  • Open-ended Questions
workshop contents8
Workshop Contents
  • Introduction & Context for Training
    • Knowledge-Practice Model
    • Professional Vocabulary & Concepts
  • Context for Instruction
    • Educational Goals & Objectives
    • Models of Objectives
    • Working with Objectives
  • Context for Assessment & Evaluation
    • Role of Information for Decision-making
      • Types of data related to types of decisions
    • Assessment Model
    • Types of Assessments
      • Objective vs Subjective, Etc.
  • Alternative Assessment: Performance Strategies Complementing Paper Assessment
  • Formative Assessment: Information about Teaching & Learning to Improve Instruction
  • Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work context
developing assessment literacy learning outcomes8
Developing Assessment Literacy: Learning Outcomes
  • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)

2. Recognize/value the relationship between professional vocabulary and practice

3. Recognize/value the use of professional vocabulary

  • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation

2. Recognize differences between objectives models

3. Write objectives using a specific model

4. Critique objectives using a specified evaluation system

  • 1. Recognize/value the role of appropriate information in decision-making

2. Recognize/value the relationship between types of assessment & resulting information

3. State the difference between objective & subjective (non-objective) assessment

  • 1. State the difference between traditional & alternative assessment models

2. State the difference between models of alternative assessment

3. State the purpose/s for rubrics in assessment

4. Identify appropriate applications of rubrics (in specific assessment settings/contexts)

  • 1. Describe formative assessment strategies

2. State appropriate applications for formative assessment/s

  • 1. Recognize/value the need for systematic planning for assessment

2. Evaluate item types for given applications/purposes

3. Construct items to meet specific Los in given learning domains

4. Evaluate items/tests as to purpose/use of assessment information

6 working with assessment tools strategies within local work context
6. Working with Assessment Tools & Strategies (within local work context)
  • Writing assessment plans
    • Who’s responsible for developing plans?
    • What’s the basis for developing plans?
  • Developing assessment tools & items
    • Who develops summative assessments?
      • Who implements summative assessments?
      • Who interprets data from summative assessments?
    • Formative assessments
    • Who develops formative assessments?
      • Who implements formative assessments?
      • Who interprets data from formative assessments?
6 working with assessment tools strategies
6. Working with Assessment Tools & Strategies
  • Developing Objective Tests & Assessments
  • Planning Assessments
  • Types of Objective Assessments
  • Sample Table of Specifications
  • Judging Assessments
6 working with assessment tools strategies1
6. Working with Assessment Tools & Strategies
  • Developing Objective Tests & Assessments
  • Resources online @
  • Writing Test Items
  • Testing Primer
  • Matching Items to Objectives
6 working with assessment tools strategies2
6. Working with Assessment Tools & Strategies
  • Developing Objective Tests & Assessments
  • Test Construction Activity

(online @ http://people.uncw.edu/caropresoe/)

6 working with assessment tools strategies3
6. Working with Assessment Tools & Strategies
  • Developing Subjective (Non-objective) Tests & Assessments
    • See Test Primer materials

(online @ http://people.uncw.edu/caropresoe/)

6 working with assessment tools strategies4
6. Working with Assessment Tools & Strategies
  • Developing Alternative Assessments
    • See Test Primer materials

(online @ http://people.uncw.edu/caropresoe/)

workshop contents9
Workshop Contents
  • Introduction & Context for Training
    • Knowledge-Practice Model
    • Professional Vocabulary & Concepts
  • Context for Instruction
    • Educational Goals & Objectives
    • Models of Objectives
    • Working with Objectives
  • Context for Assessment & Evaluation
    • Role of Information for Decision-making
      • Types of data related to types of decisions
    • Assessment Model
    • Types of Assessments
      • Objective vs Subjective, Etc.
  • Alternative Assessment: Performance Strategies Complementing Paper Assessment
  • Formative Assessment: Information about Teaching & Learning to Improve Instruction
  • Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work context
  • Assessment & Test Construction: Technical Features & “Quality” Tools
    • Validity
    • Reliability
developing assessment literacy learning outcomes9
Developing Assessment Literacy: Learning Outcomes
  • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)

2. Recognize/value the relationship between professional vocabulary and practice

3. Recognize/value the use of professional vocabulary

  • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation

2. Recognize differences between objectives models

3. Write objectives using a specific model

4. Critique objectives using a specified evaluation system

  • 1. Recognize/value the role of appropriate information in decision-making

2. Recognize/value the relationship between types of assessment & resulting information

3. State the difference between objective & subjective (non-objective) assessment

  • 1. State the difference between traditional & alternative assessment models

2. State the difference between models of alternative assessment

3. State the purpose/s for rubrics in assessment

4. Identify appropriate applications of rubrics (in specific assessment settings/contexts)

  • 1. Describe formative assessment strategies

2. State appropriate applications for formative assessment/s

  • 1. Recognize/value the need for systematic planning for assessment

2. Evaluate item types for given applications/purposes

3. Construct items to meet specific Los in given learning domains

4. Evaluate items/tests as to purpose/use of assessment information

  • 1. Describe validity; reliability

2. Explain the relationship between high quality assessment and validity/reliability

7 assessment test construction technical features quality tools
7. Assessment & Test Construction: Technical Features & “Quality” Tools
  • Validity
  • Reliability
workshop contents10
Workshop Contents
  • Introduction & Context for Training
    • Knowledge-Practice Model
    • Professional Vocabulary & Concepts
  • Context for Instruction
    • Educational Goals & Objectives
    • Models of Objectives
    • Working with Objectives
  • Context for Assessment & Evaluation
    • Role of Information for Decision-making
      • Types of data related to types of decisions
    • Assessment Model
    • Types of Assessments
      • Objective vs Subjective, Etc.
  • Alternative Assessment: Performance Strategies Complementing Paper Assessment
  • Formative Assessment: Information about Teaching & Learning to Improve Instruction
  • Working with Assessment Tools/Strategies: Writing assessment plans; developing assessment items within local work context
  • Assessment & Test Construction: Technical Features & “Quality” Tools
    • Validity
    • Reliability
    • Difficulty & Discrimination
  • Supporting Student Learning through Assessment
      • KSU Study Strategies; Test Preps
      • Improving Performance; MIT Testing Strategies
developing assessment literacy learning outcomes10
Developing Assessment Literacy: Learning Outcomes
  • 1. Recognize/value the relationship between components of professional practice (Inst; Asmt; Eval)

2. Recognize/value the relationship between professional vocabulary and practice

3. Recognize/value the use of professional vocabulary

  • 1. Recognize/value the relationship between goals/objectives, instruction & assessment/evaluation

2. Recognize differences between objectives models

3. Write objectives using a specific model

4. Critique objectives using a specified evaluation system

  • 1. Recognize/value the role of appropriate information in decision-making

2. Recognize/value the relationship between types of assessment & resulting information

3. State the difference between objective & subjective (non-objective) assessment

  • 1. State the difference between traditional & alternative assessment models

2. State the difference between models of alternative assessment

3. State the purpose/s for rubrics in assessment

4. Identify appropriate applications of rubrics (in specific assessment settings/contexts)

  • 1. Describe formative assessment strategies

2. State appropriate applications for formative assessment/s

  • 1. Recognize/value the need for systematic planning for assessment

2. Evaluate item types for given applications/purposes

3. Construct items to meet specific Los in given learning domains

4. Evaluate items/tests as to purpose/use of assessment information

  • 1. Describe validity; reliability

2. Explain the relationship between high quality assessment and validity/reliability

  • 1. Recognize/value the role of instructors as student study supports

2. Recognize the relationship between test item types & specific study strategies

8 supporting student learning through assessment
8. Supporting Student Learning through Assessment
  • Improving Student Performance on Assessments
    • KSU Study Strategies
    • Test Preps
    • Improving Performance
    • MIT Testing Strategies
developing assessment literacy
Developing Assessment Literacy
  • The END!!!
  • Hopefully, a beginning of improved professional practice!!!
decision making for learning the role of assessment and evaluation

Edward J. Caropreso, PhD

Watson School of Education

University of North Carolina Wilmington

Assessment Workshop: Part 1 (June, 2007)

Revised

Decision-making for learning:The role of assessment and evaluation

a conversation with 3 components
A Conversation with 3 Components
  • A context for instructional decision-making
    • Establishing vocabulary and a conceptual framework
  • Performance assessment as a model
    • Developing and applying criteria
    • Exploring formal models of performance assessment:
      • Holistic; Modified Holistic; Analytical assessment
  • Planning: Instruction to assessment
    • Expectations, outcomes, tasks & assessments
    • Developing tasks and assessments
      • Review of current assignments, tasks, tools
      • Development: Revisions and new constructions
instructional decision making 1 establishing vocabulary and a conceptual framework
Instructional Decision-making 1: Establishing vocabulary and a conceptual framework
  • What about instructional outcomes?
    • Intent, expectation and instruction
    • Two models of instructional objectives
      • Gronlund: From the general to the specific
      • Mager: From the specific to the general
  • What’s the relationship between teaching and learning?
    • Instruction and assessment?
  • Some web sites of interest
    • http://www.ncrel.org/sdrs/areas/misc/glossary.htm
    • http://ctl.stanford.edu/Tomprof/postings/448.html
    • http://www.ericdigests.org/
    • http://www.ericdigests.org/pre-9213/terms.htm
    • http://www.ericdigests.org/pre-9213/
what about instructional outcomes
What about instructional outcomes?
  • Two models
    • Gronlund\'s General and Specific Objectives
    • Mager\'s Instructional Objectives
what about instructional outcomes1
What about instructional outcomes?
  • Gronlund\'s General and Specific Objectives
    • Gronlund’s claim: Performance-oriented objectives are effective with simple skills, but
    • Complex cognitive behaviors require another approach
    • Proposes that main objectives be expressed as general instructional outcomes
    • Each main objective should be elaborated in terms of specific objectives; linking general, abstract level with specific, concrete performances
what about instructional outcomes2
What about instructional outcomes?
  • Mager\'s Instructional Objectives
    • Must specify clearly what the learner should be able to do following instruction.
      • Uses behavioral terms ­ the actual, observable performance of the student
    • Specifies exactly what students must do to demonstrate that they have reached the course goal
    • Provides teacher with specific guidelines for determining whether course goals have been reached; Uses specific, behavioral terms as opposed to general terms (general or vague terms include: student will understand, appreciate, know, etc.)
more about instructional outcomes
More about instructional outcomes
  • Some useful websites on learning outcomes
    • http://web.bsu.edu/IRAA/AA/WB/chapter2.htm
    • http://chiron.valdosta.edu/whuitt/col/plan/plan.html
    • http://www.harding.edu/USER/dlee/WWW/objectives.doc
    • http://www.e-learningguru.com/articles/art3_4.htm
    • http://www.nbii.gov/datainfo/metadata/training/ttt/skills/pdf/objectives.pdf
instructional decision making 2 establishing vocabulary and a conceptual framework
Instructional Decision-making 2: Establishing vocabulary and a conceptual framework
  • Assessment and Evaluation:
    • What do they mean?
  • Relationship between assessment, evaluation and decision-making:
    • The role of time
  • What about instructional outcomes?
    • Instruction, assessment and evaluation
    • Developing Criteria
a working model 1 developing criteria
A Working Model 1: Developing Criteria
  • Using a T-chart:
    • Instructional intent/expectation vs.
    • Student performance (and its assessment)
  • The task and the assessment:
    • Not the same but necessarily related
  • Expanding the chart beyond the “T”:
    • Identifying assessment criteria
  • Assessment and Evidence of Learning
using a t chart
Using a T-chart
  • T-charts in education
    • http://www.enchantedlearning.com/graphicorganizers/tchart/
    • http://www.myread.org/organisation.htm
    • http://www97.intel.com/en/ProjectDesign/InstructionalStrategies/GraphicOrganizers/T_Charts.htm
  • T-charts in law
    • http://carbon.cudenver.edu/~econry/law2.doc
a working model 2 applying criteria
A Working Model 2: Applying Criteria
  • From criteria as evidence to a “tool” for assessment and evaluation:
    • Checklists vs. Rating Scales
  • Whose tool is it anyway?:
    • What teachers and students should know to ensure learning occurs
  • Evaluation decisions and “grading” student performance
performance assessment 3 formal models
Performance Assessment: 3 Formal Models
  • Holistic:
    • Applying all criteria as a single judgment
  • Modified Holistic:
    • Adding levels of judgment
  • Analytical:
    • Judgment by criteria; weighting factors; scaling
    • http://www.teachervision.fen.com/teaching-methods/rubrics/4524.html?detoured=1
  • Valid decision-making:
    • Matching the “tool” and the instructional outcome
  • Examples of Rubrics
    • http://www.teachervision.fen.com/teaching-methods/rubrics/4524.html
    • http://www.tcet.unt.edu/START/instruct/general/rubrics.htm
planning instruction to assessment
Planning: Instruction to assessment
  • Your expectations, outcomes, tasks & assessments
  • Developing tasks and assessments
    • Review: Current assignments, tasks, tools
    • Development:
      • Revisions and reconstructions
      • New constructions
examples of instruction and assessment tools
Examples of Instruction and Assessment Tools
  • Rubrics and other assessment tools
    • http://www.teachervision.fen.com/teaching-methods/rubrics/4521.html?detoured=1
    • http://www.tcet.unt.edu/START/instruct/general/rubrics.htm
    • http://www.tcet.unt.edu/START/instruct/general/rubrics.htm
    • http://www.curriculum.org/csc/library/profiles/10/html/ELDBOP5.htm
    • http://www.inmotionaame.org/education/lesson.cfm;jsessionid=f8303421051155499527848?migration=&id=6_003LP&bhcp=1
    • http://school.discovery.com/schrockguide/assess.html
    • http://www.ncsu.edu/midlink/ho.html
    • http://www.uwstout.edu/soe/profdev/rubrics.shtml
bloom s taxonomy
Bloom’s Taxonomy
  • Introduction & Major Categories
    • http://www.officeport.com/edu/blooms.htm
    • http://faculty.washington.edu/krumme/guides/bloom1.html
  • Applying Taxonomy (Cognitive Domain/Action Verbs)
    • http://www.coun.uvic.ca/learn/program/hndouts/bloom.html
    • http://chiron.valdosta.edu/whuitt/col/cogsys/bloom.html
    • http://www.teachers.ash.org.au/researchskills/dalton.htm
  • Across the Domains
    • http://www.nwlink.com/~donclark/hrd/bloom.html
    • http://www.tedi.uq.edu.au/downloads/Bloom.pd
  • Psychomotor Domain
    • http://www.olemiss.edu/depts/educ_school2/docs/stai_manual/manual10.htm
    • http://coe.sdsu.edu/eet/articles/BloomsLD/index.htm
    • http://academic.udayton.edu/health/syllabi/health/lesson01b.htm
ad