Week 4
Download
1 / 44

Week 4 - PowerPoint PPT Presentation


  • 131 Views
  • Updated On :

Week 4. 11/23/07. Five General Item-Writing Commandments. 1. Thou shall not provide opaque directions to students regarding how to respond to your assessment instruments. Five General Item-Writing Commandments. 2. Thou shall not employ ambiguous statements in your assessment items.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Week 4' - zander


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Week 4 l.jpg

Week 4

11/23/07


Five general item writing commandments l.jpg
Five General Item-Writing Commandments

  • 1. Thou shall not provide opaque directions to students regarding how to respond to your assessment instruments.


Five general item writing commandments3 l.jpg
Five General Item-Writing Commandments

  • 2. Thou shall not employ ambiguous statements in your assessment items.

  • Sentences in which pronouns are used can frequently be unclear about the individual or individuals to whom the pronouns refers.


Five general item writing commandments4 l.jpg
Five General Item-Writing Commandments

  • True or False:

  • Leaders of developing nations have tended to distrust leaders of developed nations due to their imperialistic tendencies.

  • The word their could refer to the leaders of developing or developed nations.


Five general item writing commandments5 l.jpg
Five General Item-Writing Commandments

  • 3. Thou shall not provide students with unintentional clues regarding appropriate responses.

  • Thus, we want you to intentionally avoid unintentional clues.


Five general item writing commandments6 l.jpg
Five General Item-Writing Commandments

  • Inexperienced item writers often tend to make the correct answer to a multiple-choice items twice as long as the incorrect answers.

  • Absolute qualifiers such as never and always are sometimes used for the false items in a true false test.

  • Even uninformed students know that there are few absolutes in this world.


Five general item writing commandments7 l.jpg
Five General Item-Writing Commandments

  • Here is an unintended clue:

  • The bird in the story was an…

  • A. Falcon

  • B. Hawk

  • C. Robin

  • D. Owl

  • Can you guess the correct answer, even though you did not read the story?


Five general item writing commandments8 l.jpg
Five General Item-Writing Commandments

  • 4. Thou shall not employ complex syntax in your assessment items.

  • Avoid complicated sentence constructions and, instead, use very simple sentences.


Five general item writing commandments9 l.jpg
Five General Item-Writing Commandments

  • 5. Thou shall not use vocabulary that is more advanced than required.

  • You should eschew obfuscative verbiage.


Binary choice items l.jpg
Binary-Choice Items

  • Two options from which to select.

  • (True-False, Yes-No, Correct-Incorrect, Fact-Opinion)

  • Good things…so terse, that students can answer many items in a short time. You can cover a lot of content.

  • Bad thing…50-50 chance.


Item writing guidelines for binary choice items l.jpg
Item-Writing Guidelines for Binary-Choice Items

  • 1. Phrase items so that a superficial analysis by the student suggests a wrong answer.

  • (Phrase the items to elicit thoughtfulness)


Item writing guidelines for binary choice items12 l.jpg
Item-Writing Guidelines for Binary-Choice Items

  • 2. Rarely use negative statements, and never use double negatives.

  • Example:

  • The League of Nations was not formed immediately after the conclusion of World Ware II.

  • Or

  • The League of Nations was formed immediately after the conclusion of World War II.


Item writing guidelines for binary choice items13 l.jpg
Item-Writing Guidelines for Binary-Choice Items

  • Example:

  • The League of Nations was not formed immediately after the conclusion of World Ware II. True

  • Or

  • The League of Nations was formed immediately after the conclusion of World War II. False


Item writing guidelines for binary choice items14 l.jpg
Item-Writing Guidelines for Binary-Choice Items

  • 3. Include only one concept in each statement.

  • This is common sense…one concept at a time please!


Item writing guidelines for binary choice items15 l.jpg
Item-Writing Guidelines for Binary-Choice Items

  • 4. Have an approximate equal number of of items representing the two categories being tested.


Item writing guidelines for binary choice items16 l.jpg
Item-Writing Guidelines for Binary-Choice Items

  • 5. Keep item length similar for both categories being tested.

  • Avoid systematic patterns wherein long statements are true and short statements are false.


Multiple binary choice items l.jpg
Multiple Binary-Choice Items

  • A multiple binary-choice item is one in which a cluster of items is presented to students requiring a binary response to each of the items in the cluster.


Multiple binary choice items18 l.jpg
Multiple Binary-Choice Items

  • Rules:

  • 1. Separate item clusters vividly from one cluster to another.

  • 2. Make certain that each item meshes well with the cluster’s stimulus material.


Mode median and mean l.jpg
Mode, Median and Mean

  • Mode: The mode is the most frequent piece of data in a frequency distribution.


Mode median and mean20 l.jpg
Mode, Median and Mean

  • MedianThe median measures the middle position of a frequency distribution for a group of data. In order to calculate the median we need to order our group of data from the lowest (the 1 st score in our order) to the highest (the last score in our order). How we calculate the median depends on whether there is an even or odd number in your group of data.


Mode median and mean21 l.jpg
Mode, Median and Mean

  • Median:

  • Even numbers: If the group of data contains an even number of scores; in other words, if your group of data contains 100 scores, for example, then we calculate the median by adding the two middle scores together and dividing them by two.


Mode median mean l.jpg
Mode, Median, Mean

  • Median:

  • As a result, we must take the middle two values, namely the 50th and 51st scores, add them together (for example…58 + 59 = 117) and divide them by 2 (117 ÷ 2 = 58.5). Hence, the median for our group of data is 58.5 out of 100.


Mode median mean23 l.jpg
Mode, Median, Mean

  • Median:

  • Odd numbers: If the group of data contains an odd number of scores; in other words, if your group of data contains 9 scores, for example, then the median will simply be the middle score, the score of the 5th student.


Mode median mean24 l.jpg
Mode, Median, Mean

  • The median is considered to be a good measure of central tendency because it describes the middle position of a frequency distribution for a group of data. However, it has one main weakness. Like the mode, the median fails to take into consideration all the values in the group of data.


Mode median mean25 l.jpg
Mode, Median, Mean

  • MeanThe mean, otherwise called the arithmetic mean or average, is the most commonly used measure of central tendency. In order to calculate the mean, we simply add up all the values in the group of data and divide them by the number of values.


Mode median mean26 l.jpg
Mode, Median, Mean

  • As a result, the mean is much more effective at measuring the central position in a frequency distribution because it takes into account not only the total number of scores but also their distance from the central position.


Range l.jpg
Range

  • In descriptive statistics, the range is the length of the smallest interval which contains all the data. It is calculated by subtracting the smallest observations from the greatest and provides an indication of statistical dispersion.

  • It is measured in the same units as the data. Since it only depends on two of the observations, it is a poor and weak measure of dispersion except when the sample size is large.


Multiple binary choice items28 l.jpg
Multiple Binary-Choice Items

  • … Suppose that a dozen of your students completed a 10 item multiple-choice test that earned the following number of correct scores:

  • 5,6,7,7,7,7,8,8,8,8,9,10

  • 9. The median of your students’ scores is 7.5 True or False

    • The mode for the set of scores is 8.0 True or False

    • The range of the students’ scores is 5.0 True or False

    • The median is different than the mean. True or False


Multiple choice items l.jpg
Multiple-Choice Items

  • The strength of multiple-choice questions is they can contain several answers that differ in their relative correctness, thus, the student can be called upon to make subtle distinctions among answer options.

  • A weakness is that students need only recognize the correct answer not generate the correct answer.


Multiple choice items30 l.jpg
Multiple-Choice Items

  • The first-part of a multiple-choice item is referred to as the item’s stem.

  • The potential answer options are described as item alternatives.

  • Incorrect alternatives are typically referred to as the item’s distracters.

    Item stems can be a direct question or an incomplete statement.

    With younger students the direct question approach is preferable. Using these stems, you can ask the student to select the best answer or the best answer.


Multiple choice items31 l.jpg
Multiple-Choice Items

  • 1. The stem should consist of a self-contained question or problem.

  • It should be loaded with as much content as possible so that the student knows exactly what they are to consider.


Multiple choice items32 l.jpg
Multiple-Choice Items

  • 2. Avoid negatively stated stems. If you use the word not in a multiple-choice stem, be sure to highlight it so that the student sees and pays attention to this negative.


Multiple choice items33 l.jpg
Multiple-Choice Items

Which one of the the following cities is located west of the Mississippi? San Diego, Pittsburgh, Boston or Atlanta

Which one of the following cities is not located in a state east of the Mississippi? San Diego, Pittsburgh, Boston or Atlanta


Multiple choice items34 l.jpg
Multiple-Choice Items

  • Do not let the length of alternatives supply unintended clues.

  • Keep it short and sweet or make all lengthy…you make the call.


Multiple choice items35 l.jpg
Multiple-Choice Items

  • Randomly assign correct answers to alternate positions.

  • As a rule, if you have four choices, make sure the correct answer is assigned to each position (a,b,c,or d,) 25% of the time.


Multiple choice items36 l.jpg
Multiple-Choice Items

5. Never use all-of-the-above alternative but do use none of the above alternatives to increase item difficulty.


Matching items l.jpg
Matching Items

A matching item consists of two parallel lists of words or phrases that require the student to match entries on one list with appropriate entries on the second list.

Entries in the list for which a match is sought are referred to as premises.

Entries in the list from which selections are made are referred to as responses.


Matching items38 l.jpg
Matching Items

Advantage:

Easily scored

Takes up little space and taps a good deal of information efficiently

Disadvantage:

Encourages memorization of low level factual information.


Matching items39 l.jpg
Matching Items

  • Matching Item-Writing Guidelines

  • 1. Employ homogeneous lists

  • 2. Use relatively brief lists, placing the shorter words or phrases at the right.

  • 3. Employ more responses than premises.

  • 4. Order the responses logically

  • 5. Describe the basis for matching and the number of times responses may be used


Constructed response items l.jpg

Constructed-Response Items

Short answer items and essays


Constructed response items41 l.jpg
Constructed-Response Items

  • Short-Answer Items:

  • Students are asked to supply a word, a phrase, or a sentence in response to either a direct question or an incomplete statement.

  • The major advantage is that the student needs to produce a correct response answer, not merely recognize it from a set of selected-response options.

  • The major disadvantage is that student responses are difficult to score. The longer the responses sought, the tougher it is to score them.


Five guidelines for writing short answer items l.jpg
Five Guidelines for Writing Short Answer Items

  • 1. Usually employ direct questions rather than incomplete statements, particularly for younger students.

  • 2. Structure the item so that a response should be concise.

  • 3. Place blanks in the margin for direct questions or near the end of incomplete statements.

  • 4. For incomplete statements, use only one or, at most, two blanks.

  • 5. Make sure blanks for all items are equal in length.


Essay items development l.jpg
Essay Items: Development

  • Essay items are particularly useful in gauging a student’s ability to synthesize, evaluate and compose.

  • For assessing certain types of complex learning outcomes, the essay item is hands-down the winner.

  • Disadvantages:

  • Essay items are more difficult to write properly--than is generally thought.

  • Scoring essays can be difficult also.


Essay items writing guidelines l.jpg
Essay Items: writing guidelines

  • 1. Convey to students a clear idea regarding the extensiveness of the response desired

  • 2. Construct items so that the student’s task is explicitly described.

  • 3. Provide students with the approximate time to be expended on each item as well as each item's value

  • 4. Do not employ optional items

  • 5. Precursively judge an item’s quality by composing, mentally or in writing, a possible response.