1 / 66

Construct Evaluations

Construct Evaluations. True-False. Meaningful true-false items are difficult to compose Many statements are neither completely true nor completely false Amount of guessing may be high Test takers have a 50 percent chance of being correct Tend to measure recognition only. True-False.

jadyn
Download Presentation

Construct Evaluations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Construct Evaluations

  2. True-False • Meaningful true-false items are difficult to compose • Many statements are neither completely true nor completely false • Amount of guessing may be high • Test takers have a 50 percent chance of being correct • Tend to measure recognition only

  3. True-False • Precautions • Provide accurate directions • Balance the true and the false items and distribute them so no pattern is evident • Keep true and false statements about equal in length • Used positive rather than negative statements • Avoid the use of none, never, no, or not and always

  4. True-False • Construct true-false items that are either true or false, not partially true or false • Provide a space to the left of each item where the student can insert the correct answer for an item which is false • This distinguishes an accidental correct guess from a correct response

  5. True-False • Example • Directions: read each of the following statements. If a statement is dramatically correct, circle the T. If the statement is incorrect, circles the F and change the underlined word to make this statement dramatically correct. Place the correct word in the blank beside the F. T F _____ 1. To whom should I give the umbrella? T F She__ 2. Her and I signed up for the trip

  6. Multiple Choice • Consists of an incomplete statement or a question followed by four alternatives • Three parts • The incomplete statement-the stem • The correct answer • 3 alternatives-distractors

  7. Multiple Choice • Example Directions: in the space provided beside each question, write the letter of the response which best completes the statement. Stem Correct Response Distractors

  8. Multiple Choice • 12 guidelines • Word each multiple-choice item as either a statement or a question • What is the period covered by the balance sheet? • Year • Month • Day • Week

  9. Multiple Choice • Make sure the stem contains a single clearly worded idea • Good: the rule of 72 is used to determine • Average annual rate of return • Number of years needed to double investment • Total return on investment • Daily rate of return • Poor: what is the chief purpose of using the rule of 72? • Average annual rate of return • Number of years needed to double investment • Total return on investment • Daily rate of return • The term purpose may confuse the reader • The alternatives are not phrased in a way as to answer the question

  10. Multiple Choice • Keep the correct response the same length as each distractor • If a negative statement is used, underline it, italicize it, or place it at the end of the stem or do both • Good: an employee whose only employment is in the factory is eligible to invest in all of the following except • An individual retirement account • Keogh plan • Stock option plan • Municipal bonds • Poor: An employed factory worker is not eligible to invest in which of the following?

  11. Multiple Choice • Make certain that the correct response is accurate or certainly the best of alternatives • Avoid any clues such as repeating keywords from the stem in the alternatives • Determine that all the distractors are believable • Vary the length of the correct response from item to item to avoid giving clues • Make certain that all alternatives are grammatically correct and written in parallel form

  12. Multiple Choice • Avoid alternatives like “none of the above” or “all of the above” • Vary good position of the correct response from question to question • In formatting the questions, put each alternative on a separate line to permit easy reading

  13. Matching • Parallel columns are usually provided for matching items • The items below column A are called premises and below column B responses

  14. Matching Directions: in this space provided beside column A, write a letter of the statement in column B that best completes each thought expressed. Each item in column B may be used more than once. Responses Premises

  15. Matching • Matching items are best suited to large units of subject matter which are sufficiently homogeneous to require discrimination on the part of the students

  16. Matching • Guidelines for designing matching questions • Provide clear, simply stated directions • Restrict the number of items in the stem column (column A) (no more than 10) • Include fewer or more alternatives (column B) than items in the stem column (column A) to prevent students from guessing

  17. Matching • Arrange items in the stem column in logical order or in ascending or descending order • Arrange alternatives in logical order • Keep all items in a matching question on one page

  18. Completion • Simply an incomplete statement • The examinee applies the correct word, amount, or date

  19. Completion • Directions: for each of the items below insert the correct response in the blank space at the end of the sentence • The maturity value for an $850, 120-day note at 14 percent is _?_. • The date of the census for which punch cards were first use is _?_.

  20. Completion • Use completion items only for those questions for which there is only one correct response • Some guidelines • Give explicit instructions • But the blank at the end of the sentence

  21. Completion • Keep all blank lines the same size but provide enough room for the longest answer • Provide no hints • First letter • Number of blank spaces • Length of the blank line • Avoid copying statements directly from the textbook • Only evaluates recognition

  22. Short Answer • The examinee is presented with a question and required to supply a word, name, date or amount

  23. Short Answer • Directions: Insert the correct response in the space to the left of the number of each test item

  24. Short Answer • Use test items for which there is only one correct response • Correct • What is the name of the COBOL program division which gives step-by-step commands for the computer to follow in processing the data? • Answer: procedure division • Incorrect • Explain the divisions of a COBOL program and the function of each • Suggests an essay, which is inappropriate for “understanding-level” outcomes • Not a short answer item

  25. Design of Mastery Tests • Chief purpose-discriminate between the student who has fully mastered a competency and one who has not • The progress test • The person identified as non master can seek additional learning experiences and later be retested • Each learner should eventually achieve the minimum level of performance required

  26. Design of Mastery Tests • Three critical decisions • Eliminate subjectivity from scoring • Scored each item either as correct (1 point) or incorrect (0 points) • No judgment is made; the answer is either right or wrong

  27. Design of Mastery Tests • Set a cutoff score • A student who achieves a 70 percent score and passes the test has not mastered 30 percent of the items • The cutoff score must be set high in order to increase the probability that it will differentiate masters from non masters • Experts suggest a cutoff of 85 to 90 percent

  28. Design of Mastery Tests • Decide on number of items per competency • A useful rules found is between 12 and 20 test items are needed for each enabling objectives that you wish to test • A mastery test is appropriate only for competencies which are essential • Nice-to-know competencies should not be tested by mastery test

  29. Design of Norm-Referenced Test • The various types of objective items used for mastery test are also appropriate for an norm-referenced test • The norm-referenced test is designed differently

  30. Design of Norm-Referenced Test • Four major guidelines • The purpose is to gather information to rank individuals within the group-only a sample of the content in the unit is selected for the test • Seek to avoid questions which all students are likely to answer correctly • A wide variation in scores is sought • Since several competencies are being measured, the norm-referenced test contains more questions than the mastery test

  31. Design of Norm-Referenced Test • No decision on mastery can be made from a norm-referenced test

  32. Performance Test • Includes essay questions, product performances, and process performances • It is essential that as much objectivity as possible be built into these tests

  33. Performance Test • Judgments made on performance test are more likely to be made objectively when the teacher uses • Rubrics • Rating scales • Checklist • Assessment instruments designed at the time to test is constructed

  34. Essay Questions • Appropriate for evaluating the competency when a student is required to • Select relevant information • Organize • Integrate • Analyze • Synthesize • Make decisions

  35. Essay Questions • The process of assigning points to each essay question is subjective and subject to human error • Essay questions fall into two categories • Restricted response • Extended response

  36. Essay Questions • Restricted response • Limits the content involved as well as the response • Example 1: describe two causes of inflation. For each cause describe two actions the government may take in controlling it • The student must simply recall two causes • Evaluates understanding of inflation and demands analysis of remedies

  37. Essay Questions • Example 2: analyze the two income statements attached and identify any problems the Best Company is experiencing. For each problem, explain a possible cause or causes and recommend one or more solutions • Involves analysis • Content is limited by the case, as is the response

  38. Essay Questions • Extended response • Gives the student full responsibility for • Selecting relevant information • Organizing • Analyzing • And presenting the analysis

  39. Essay Questions • Example: contrast Keynesian theory with supply-side theory in terms of objectives, applications, and possible effects on (a) a declining economy and (b) an inflationary economy • Forces a student to analyze and communicate effectively in writing

  40. Essay Questions • Guidelines • Use essay only for those learning outcomes which cannot be measured by objective items • Develop a classification system • Summarize prominent ideas • Analyze the task and formulate logical steps for accomplishing it • Create a classification • Explain a phenomenon • Decide which data are relevant • Conclude from information given • Defend decisions or actions • Predict trends or actions based on data or information

  41. Essay Questions • Design the question to elicit the behavior specified in the learning outcome • Example: if you want a student to analyze financial statements, “conclude from information given,” is appropriate • Essay question: from the financial statements provided for the Lexion Company, determine the financial condition of the company and present a complete analysis of the company’s financial strengths and weaknesses

  42. Essay Questions • State each question precisely so that the individual understands what is expected • Poor: described the impact of word processing systems on office equipment • Fails to provide a clue as to what is expected (impact on numbers of people, task performed, or organization, or all?) • Better: describe the impact of word processing in the office on productivity, work measurement, workflow, and career paths

  43. Essay Questions • Specify the maximum number of points to be awarded for a correct and complete answer beside each question • Aids the student in managing time • Prepare a key to use in evaluating responses at the time the questions are designed and written

  44. Product and Processed Performances • Subjective judgments sometimes interfere with the evaluation process • Teachers have many opportunities to observe students affective behaviors

  45. Product and Processed Performances • Teachers may structure observations using some form of assessment instrument or checklist, for example • Individual working independently—reading, performing the task, organizing for performance, or observing a peer • Individuals seeking assistance from peers • Individual giving assistance to peers • Individual working with one or more peers while interacting; observing: perhaps as part of a team; peers giving report or performing; organizing a project; or performing a task

  46. Product and Processed Performances • A teacher planning to observe the student’s oral presentation might decide to observe both cognitive and affective behaviors • Cognitive • Accuracy of content • Organization • Quality of summary • Use of correct grammar

  47. Product and Processed Performances • Affective • Eye contact and other nonverbal language • Attitude of the speaker when responding to questions • A rubric, rating scale, checklist, or assessment instrument can be used to reduce subjectivity • Such instruments are designed before the learning process begins and shared with the learner

  48. Product and Processed Performances • Rating scales • Sets of statements describing qualities of the product or process being judged • Descriptors placed on a scale guide the evaluator in determining where on the scale the product being judged falls Rating Scale for Oral Presentations.doc

  49. Product and Processed Performances • Students should be provided with copies before the learning process begins • When learners are aware of the criteria on which a evaluation will be based they can concentrate their energies on meeting them • A single performance does not necessarily represent later performances • Many performances are required

More Related