assessment professional learning module 2 activity 2 4b designing better written tests n.
Download
Skip this Video
Download Presentation
Assessment Professional Learning Module 2: Activity 2-4B Designing Better Written Tests

Loading in 2 Seconds...

play fullscreen
1 / 39

Assessment Professional Learning Module 2: Activity 2-4B Designing Better Written Tests - PowerPoint PPT Presentation


  • 195 Views
  • Uploaded on

Assessment Professional Learning Module 2: Activity 2-4B Designing Better Written Tests. Assessment OF learning: Assessment OF learning occurs when teachers use evidence of student learning to make judgements on student achievement against goals and standards.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Assessment Professional Learning Module 2: Activity 2-4B Designing Better Written Tests' - wood


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
assessment professional learning module 2 activity 2 4b designing better written tests

Assessment Professional LearningModule 2: Activity 2-4BDesigning Better Written Tests

slide2
Assessment OF learning:

Assessment OF learning occurs when teachers use evidence of student learning to make judgements on student achievement against goals and standards.

It is usually formal, frequently occurring at the end of units of work where it sums up student achievement at a particular point in time. It is often organised around themes or major projects and judgements may be based on student performance on multi-domain assessment tasks. It has a summative use, showing how students are progressing against the Standards, and a formative use, providing evidence to inform long term planning.

the teacher s role
The Teacher’s Role:
  • is to construct valid and reliable assessment tasks - including written tests - that fairly assess student achievement;
  • to moderate with colleagues to ensure fair and consistent evidence of learning;
  • to modify the learning and teaching program in light of the assessment evidence.
assessment of learning
Assessment OF Learning
  • Can involve extended integrative multi-domain tasks that are authentic, rich and focus on complex learning across domains.
  • Can be more focussed, shorter tasks possibly completed under exam conditions, simple tests conventional or “live” performances.
  • This presentation focuses ONLY on aspects of constructing better quality written tests.
30 second think
30 second think:

How would you feel if you were told there will be a test or exam: right NOW?

slide7

Rule for designing pencil and paper test items:The easier and quicker to design, the harder and longer to “mark” (assess)(and vice versa).

written tests open closed book timed individual
Written TestsOpen/closed book; timed; individual

• Multiple choice items

• Matching items

• True-False questions

• Completion items

• Short answer questions

• Extended response

or essay questions

Select a response

Supply or Construct a response

slide9
• Venn diagrams, concept maps,

drawings, etc

• Problem-solving, calculations, (closed/open)

• Creative/lateral thinking

responses

  • Teacher-designed tests tend to be predominantly low order thinking
  • (in Bloom terms).

Construct a response

objective test items
“Objective” test items

• Objective test items are assessment tasks for which there is general agreement about the “right” answer and a mark to be given to each item.

• However, in another sense, there is no such thing as a completely objective test item. Subjective judgements are involved in constructing the items and selecting them for inclusion in the test.

advantages of objective tests
Advantages of objective tests
  • They allow a wide sampling across a lot of content, because you can usually ask more questions in a given time.
  • They generally give reliable marks luck being less important because of the large number of questions.
  • They can be efficiently marked (even by computer). Marking is quicker and easier than essays & projects.
  • Assessment doesn’t depend on the assessor. On a good objective test, it is clear to all assessors what the correct answers are.
disadvantages of objective tests
Disadvantages of objective tests
  • They often provide an indirect assessment of learning (a ‘proxy’ for the real thing).
  • There are many worthwhile Standards and learning outcomes that cannot be assessed this way.
  • Constructing objective test items is very time-consuming.
  • Writing good objective items requires high level skills.
  • Some students don’t handle these questions well.
  • They can encourage guessing (but not successfully

if the test contains many questions).

multiple choice items
The longest river

in the world is the

A Amazon.

B Murray-Darling.

C* Nile.

D Mississippi.

• The stem should be complete, simple & meaningful.

• Each choice should state one idea, and plausibly flow from the stem.

• Put choices in sequence (especially numbers).

• Negative words should be avoided or capitalised.

•“All of the above”, “two or more of the above”, and “none of the above”, should not be used.

• Choices should be similar in length and detail.

Multiple Choice Items
common errors in the construction of multiple choice items
Common errors in the construction of multiple-choice items
  • Teachers may legitimately disagree about the best answer (sometimes more than one is correct).
  • The level of reading skill required can be too high.
  • Use of “textbook language” encourages rote learning.
  • The stem contains irrelevant material.
  • Negative terms are used unnecessarily, and are not emphasised.
  • Words are repeated in each option when they would be better in the stem.
  • The question provides clues to the correct answer:

e.g. by the grammar.

spot the design flaw
Spot the design flaw:

Who were the first explorers of Australia?

A. the Dutch.

B. Dampier.

C. the French.

D. British.

design flaws
Design flaws:
  • Dampier is singular (question implies plural)
  • “British” has no article (can’t be easily read with the stem)
  • brings it down to choice of only two
  • options not in alphabetical order
design flaw
Design flaw?

The Prime Minister of Australia is

A. elected directly by the electorate.

B. elected by the Members of Parliament in the governing party or parties.

C. elected by the Parliament.

D. appointed by the Governor-General.

flaw this item has more than one correct answer
Flaw:This item has more than one correct answer

The Prime Minister of Australia is

A. elected directly by the electorate.

B. elected by the Members of Parliament in the governing party or parties.

C. elected by the Parliament.

D. appointed by the Governor-General.

  • Answers B and D
design flaw1
Design flaw?

Newton’s third law states that

A. when a body is acted on by a force, the body will move in the direction of the force.

B. the work done by a force is the product of the force and the distance moved by the body.

C. for every action there is an equal and opposite reaction.

D. curriculum only changes if acted upon by a large force of assessment.

slide20

Flaw 1: Use of “textbook language” that encourages rote learning, rather than making meaning.Flaw 2: a nonsense distractor is easy to detect (it is Sherborne’s law of curriculum motion).

Newton’s third law states that

A. when a body is acted on by a force, the body will move in the direction of the force.

B. the work done by a force is the product of the force and the distance moved by the body.

C. for every action there is an equal and opposite reaction.

D. curriculum only changes if acted upon by a large force of assessment.

design flaw2
Design flaw?

Which of the following types of test item best measures outcomes at the level of recall?

A. Supply-type items.

B. Selection-type items.

C. Matching items.

D. Multiple-choice items.

flaw overlapping distracters more than one category b c or d
Flaw: Overlapping distracters - more than one category: B, C or D

Which of the following types of test item best measures outcomes at the level of recall?

A. Supply-type items.

B. Selection-type items.

C. Matching items.

D. Multiple-choice items.

true false items
True-False items

Advantages

  • Easy to write; quickly constructed
  • Quick to administer
  • Can reduce student anxiety at the start of a test

Disadvantages

  • Hard to write worthwhile statements that are unequivocally true or unequivocally false
  • Susceptible to guessing.
designing true false items
Sulfuric acid on a metal gives off hydrogen gas.

True* or False

• Only useful for facts, cause/effect, definitions,

but not for opinions.

• Keep language simple & clear.

• Avoid negatives and qualifiers like always, never, some, sometimes, may, often.

• Mix T/F answers, but not in a recognisable pattern.

• T/F questions do encourage guessing.

Designing True-False Items
design flaw3
Design Flaw?

John Marsden is Australia’s greatest author of fiction for young adults.

True of False?

slide26
Flaw:

Even experts might have good reasons for disagreeing about the right answer.

John Marsden is Australia’s greatest author of fiction for young adults.

design flaw4
Design Flaw?

Archimedes Principle states that when a body is wholly or partially submerged in a fluid, it is subject to an upthrust equal to the weight of fluid displaced.

slide28
Flaw:

Use of textbook language encourages rote learning.

Archimedes Principle states that when a body is wholly or partially submerged in a fluid, it is subject to an upthrust equal to the weight of fluid displaced.

designing matching items
Adelaide NSW

Brisbane Qld

Hobart SA

Melbourne Tas

Perth Vic

WA

• Only useful for facts, cause/effect, dates/events, people/events, symbols/concepts but not for opinions.

• Keep to one type of connection e.g. date/event.

• Keep language simple & clear.

• Make columns of unequal length to minimise guessing.

• Limit to 4-6 matches (keeps them plausible).

• Consistently use alphabetical or numerical order.

Designing Matching Items
completion cloze items
A verb is a …………… word and a …………… is a naming word.

• Easy to construct but take care to stick to facts not opinion.

• Keep language simple & clear.

• Make sure there is only one possible correct response for each blank.

• Place blanks in the middle or end of the sentence.

• Do not insert too many blanks - removes context.

• Make blanks the same length.

• Avoid ‘a’ or ‘an’ before the blank.

Completion (Cloze) Items
completion and short answer items
Completion and short-answer items

These can be in the form of:

Questions:

Who was the first wife of the Prince of Wales?

Incomplete sentences:

The Prime Minister of Australia is ______.

Simple computational problems:

The mean of 3, 4, 5 and 6 is _______.

Directions asking for information in response to cues:

Beside the name of each element, write its valence.

short answer
Briefly describe four major functions of the Supreme Court.

• Students must know something to respond (can’t choose a supplied option) & minimises guessing.

• Allows for flexibility in student responses.

• Allows for “it depends” responses.

• Easier to construct than multiple choice questions.

• More difficult to mark - lower ‘inter-rater reliability’ (see Module 5).

• Needs care to keep the item clear and direct.

Short Answer
problems with completion and short answer items
Problems with completion and short-answer items

Can only ask simple recall of facts, or simple computational tasks.

Completion items are often ambiguous and may leave students too much room to wriggle.

design flaw5
Design Flaw?

For example:

The capital city of South Australia is _________.

Australia was founded in ___________.

design flaw too much wiggle room
Design Flaw: too much “wiggle” room

For example:

The capital city of South Australia is _________.

full of churches.

populated by crazy Port and Crows fans.

Australia was founded in ___________.

the summer holidays.

error.

1901.

essay items
Compare and contrast the work of the Supreme Court and the High Court.

Give examples and evidence to support your views.

• Students must know something to respond (can’t select an option) and hence minimises guessing.

• Allows for flexibility and creativity in student responses.

• Allows for “it depends” responses.

• Very easy to construct.

• Much more difficult to mark - low ‘inter-rater reliability’.

• Needs care to keep the item clear and direct.

Essay items
advantages of essay items
Advantages of Essay items

For many performance objectives, the essay is a direct measure of students’ achievement of the intended learning outcome.

Only the essay items measures students’ ability to communicate ideas in writing.

Essay items require students to supply the answer, not just to recognise it.

(Oosterhof, 1999, pp. 71-72)

limitations of the essay items
Limitations of the essay items
  • Essay items often do not provide an adequate sampling of content.
  • Essay items are time-consuming to score.
  • The scoring of essays is notoriously unreliable.
  • “Bluffing” can have undue influence on students’ scores.

(Oosterhof, 1999, pp. 72-73)

slide39
What type of test item type do you perform best on?Do you tend to ask more of these type of questions?

You are now ready to take the online quiz about designing tests and quizzes.

Good luck!

Note:

It is private:

no one will know your result except you!