developing tests for departmental assessment n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Developing Tests for Departmental Assessment PowerPoint Presentation
Download Presentation
Developing Tests for Departmental Assessment

Loading in 2 Seconds...

play fullscreen
1 / 24

Developing Tests for Departmental Assessment - PowerPoint PPT Presentation


  • 76 Views
  • Uploaded on

Developing Tests for Departmental Assessment. Deborah Moore, Assessment Specialist Institutional Research, Planning, & Effectiveness University of Kentucky. Session Overview. Steps in Test Development and Test Administration Process for Evaluating Test and Items Common Difficulties

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Developing Tests for Departmental Assessment


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
developing tests for departmental assessment
Developing Tests for Departmental Assessment

Deborah Moore, Assessment Specialist

Institutional Research, Planning, & Effectiveness

University of Kentucky

session overview
Session Overview
  • Steps in Test Development and Test Administration
  • Process for Evaluating Test and Items
  • Common Difficulties
  • Strategies for Analysis and Reports
exercise 1 10 minutes
Exercise 1 (10 minutes)

Under What Conditions Do You Give a Test?

  • How do you decide you are going to give a test?
  • What do you do to develop a new test?
  • How do you decide what types of item formats to use?
  • How do you develop items?
  • How long does it take to put the test together?
using a test for classroom assessment
Using a Test for Classroom Assessment
  • Generally, instructors develop their own classroom tests, making all decisions about when and how to construct, administer, score, and report results of tests.
  • Construction is often done without formality or documentation.
  • The most frequent use of tests by instructors is to assign grades related to individual student learning.
a test is a good choice when
A Test Is a Good Choice When…
  • the student must demonstrate acquisition of knowledge or ability to process and use knowledge
  • the student's knowledge about a wide range of content is to be evaluated (e.g., survey and capstone courses)
  • multiple observations of the content-related knowledge are needed (e.g., math and foreign languages)
  • more resources are available for constructing the test than for scoring and reporting
  • a large group is being assessed
using a test for departmental assessment
Using a Test for Departmental Assessment
  • Planning, implementing, and using results become a group effort – a shared set of decisions and responsibilities. Consensus is emphasized.
  • Additional planning time, communication, and record keeping will be needed.
  • When used for program assessment, test performance is generally used along with other information to describe group achievement and is independent of grading.
advantages of selected response item formats
Advantages of Selected-Response Item Formats
  • Better content coverage
  • Higher reliability
  • Greater efficiency
  • Objectivity
  • Mechanical scoring
test development process
Test Development Process
  • Define purpose/use
  • Outline curriculum & consult goals/objectives
  • Create test plan (aka blueprint or specifications)
  • Create a pool of items
  • Critique and revise items
  • Pilot/field test (e.g. item analysis, reliability/validity studies)
  • Set guidelines for test administration, scoring procedures, interpretation of scores (e.g. develop norm tables and standard setting procedures)
exercise 2 10 minutes
Exercise 2 (10 minutes)

Review a Sample Departmental Packet

  • What role does the exam play in the assessment plan for this department?
  • What type of item formats should be used on the comprehensive exam?
administrative approaches
Administrative Approaches
  • Common exams
  • Course-embedded testing
  • Assessment Center/Assessment Days
sources for items
Research

Curriculum

Real life

Professional development materials & textbooks

Own final exam files

Comprehensive exams from other universities

Retired test/item banks

Sources for Items
common difficulties with test quality
Common Difficulties with Test Quality
  • Push for higher order thinking often weak
  • Insufficient consensus on test plan/blueprint
  • Discomfort with various item-writing approaches (writer’s block)
  • Insufficient editing
  • Little or no piloting of items
critiquing items
Critiquing Items
  • Match with test plan
  • Item value
  • Item quality
typical item review questions
Typical Item Review Questions
  • Has each item received a quality appraisal?
  • Has each item’s content been verified?
  • Has each item been classified?
  • Has the key been identified?
  • Have you edited the items?
  • Have items been checked for bias/insensitivity?
  • Have you field tested the items?
planning a departmental test
Planning A Departmental Test
  • Develop test
  • Plan administration
  • Analyses test data
  • Report and use results
  • Agree on leadership/support roles
exercise 3 30 minutes
Exercise 3 (30 minutes)

Developing a Planning Checklist

  • Review the headings of the checklist form
  • List additional headings needed
  • List activities that need to be completed under the various headings
  • Add a number to the activities to represent the order of completion
analyses
Analyses
  • Examine quality of your assessment tools and procedures
  • Describe the test results
    • By test plan features
    • By meaningful subgroups
  • Combine with other datasets
  • Execute and summarize planned comparisons linked to research design
reports
Reports
  • Determine stakeholders/constituents
    • Identify specific interest
  • Create a timeline and share
  • Include a vetting procedure
testing in the future
Testing in the Future
  • Computer based testing
    • Adaptive (CAT)
    • Self-Adaptive (SAT)
  • Computer assisted item generation
  • Automated essay scoring
thank you for your attention
Thank you for your attention.
  • Deborah Moore, Assessment Specialist
  • 101B Alumni GymOffice of Institutional Research, Planning, & Effectiveness
  • dlmoor2@email.uky.edu
  • 859/257-7086
  • http://www.uky.edu/LexCampus/