Designing the Questionnaire or Observation Form

1. Designing the Questionnaire or Observation Form Chapter 12

2. Procedure for Developing a Questionnaire Step 1 Specify what information will be sought Step 2 Determine type of questionnaire and method of administration Step 3 Determine content of individual questions Step 4 Determine form of response to each question Step 5 Determine wording of each question Step 6 Determine question sequence Step 7 Determine physical characteristics of questionnaire Reexamine steps 1-7 and revise if necessary Step 8 Step 9 Pretest questionnaire and revise if necessary SLIDE 12-1

3. Telescoping Error and Recall Loss TELESCOPING ERRORA type of error resulting from the fact that most people remember an event as having occurred more recently than it did. RECALL LOSS A type of error caused by a respondent’s forgetting that an event happened at all. SLIDE 12-2

4. Randomized Response Model An interviewing technique in which potentially embarrassing and relatively innocuous questions are paired, and the question the respondent answers is randomly determined but is unknown to the interviewer. SLIDE 12-3

5. Illustration of the Randomized Response Technique Interviewer: Items handed to respondent Please reach in the jar and pull out one of the colored balls, but do not show the ball to me. However, if the ball is blue, please answer question A on the card, whereas if it is red, please answer question B. Since I don’t know which colored ball you’ve drawn, I won’t know which question you’re answering. Respondent: Says yes. • Is your birthday in January? • Have you ever shoplifted? No Yes Questions: No Yes • Do we know if this respondent ever shoplifted? • Can we calculate the proportion of people in the sample who have shoplifted? SLIDE 12-4

6. Illustration of the Randomized Response Technique (Cont.) Suppose the proportion of people in the sample who answered yes was .20 and there were an even number of blue and red balls in the jar, say 50 of each. Let π = the total proportion of “yes” responses = .20 p = the probability that the sensitive question (the person has shoplifted) is being answered = .50 1-p = the profitability that the nonsensitive question (the person’s birthday is in January) is being answered = .50 πs = the proportion of “yes” responses to the sensitive question or the proportion of people who have shoplifted πNS = the proportion of “yes” responses to the nonsensitive question or the proportion of people whose birthday is in January; suppose census data indicate that πNS = .05 Now π= p πS + (1-p) πNS and .20 = .50 πS x + .50(.05) and πS = .35 SLIDE 12-5

7. Open-Ended Questions • How old are you? ______ • Do you think laws requiring passengers in motor vehicles to wear seat belts are needed? _____ • Can you name three sponsors of the Monday-night football games? _____ • Do you intend to purchase an automobile this year? _____ • Why did you purchase a Magnavox brand color television set? _____ • Do you own a VCR? _____ SLIDE 12-6

8. Multichotomous Questions • A fixed-alternative question in which respondents are asked to choose the alternative that most closely corresponds to their position on the subject. • Example • Age • How old are you? • Less than 20 • 20-29 • 30-39 • 40-49 • 50-59 • 60 or over SLIDE 12-7

9. Dichotomous Questions • A fixed-alternative question in which respondents are asked to indicate which of the two alternative responses most closely corresponds to their position on the subject. • Example • Age • Do you think laws requiring passengers in motor vehicles to wear seat belts are needed? • Yes • No SLIDE 12-8

10. Split Ballot A technique used to combat response bias in which one phrasing is used for a question in one-half of the questionnaires while an alternate phrasing is used in the other one-half of the questionnaires. Example Do you think gasoline will be more expensive or less expensive next year than it is now? More expensive Less expensive Do you think gasoline will be less expensive or more expensive next year than it is now? More expensive Less expensive SLIDE 12-9

11. Item Non-Response A source of non-sampling error that arises when a respondent agrees to an interview but refuses, or is unable, to answer specific questions. SLIDE 12-10

12. Leading Questions • A question framed so as to give the respondent a clue as to how he or she should answer. • Example • Do you feel that limiting taxes by law is an effective way to stop the government from picking your pocket every payday? • Yes • No • Undecided SLIDE 12-11

13. Implicit Alternative A problem that occurs when a question is not framed so as to explicitly state the consequences, and thus it elicits different responses from individuals who assume different consequences. Example Would you like to double the number of job offers you receive as a senior? Yes No Would you like to double the number of job offers you receive as a senior if that means devoting an additional 10 hours per week to studying so as to raise your grade point? Yes No SLIDE 12-12

14. Double-Barreled Question A question that calls for two responses and thereby creates confusion for the respondent. Example Are the food and service at the local Pizza Hut good? Yes No SLIDE 12-13

15. Guidelines for Question Wording • Use simple words and questions • Avoid ambiguous words and questions • Avoid leading questions • Avoid implicit alternatives • Avoid implicit assumptions • Avoid generalizations and estimates • Avoid double-barreled questions SLIDE 12-14

16. Guidelines for Question Sequencing • Use simple, interesting opening questions • Use the funnel approach, asking broadquestions first • Carefully design branching questions • Ask for classification information last • Place difficult or sensitive questions near the end SLIDE 12-15

17. Guidelines to Question Sequencing (Cont) Funnel ApproachAn approach to question sequencing that gets its name from its shape, starting with broad questions and progressively narrowing down the scope. Example How would you rate your last experience when eating at ABC restaurant? Extremely Extremely Poor Poor So-So Good Good How would you rate the quality of the service? Extremely Extremely Poor Poor So-So Good Good SLIDE 12-16

18. Guidelines to Question Sequencing (Cont) Branching QuestionA technique used to direct respondents to different places in a questionnaire, based on their response to the question at hand. SLIDE 12-17

19. Guidelines to Question Sequencing (Cont) • Branching QuestionExample • 1. Please answer the following question Yes or No. Did you, personally, give money to Canopy of Care last year? Y or N • {If Yes, go to #3.} • We are interested in why people do not contribute. The following is a list of answers others have given. Please tell me which, if any, apply to you. • a. ___ Someone else in my household had already contributed. • b. ___ I did not have the money at the time. • c. ___ I gave to other charities. • d. ___ I volunteered my services to Canopy of Care instead of contributing money. • e. ___ I volunteered my services to other charities instead of contributing to Canopy of Care. • f. ___ I did not give because Canopy of Care spends its money inefficiently. • g. ___ None of the above. • To how many different charities do you think Canopy of Care gives money? • a. ___ 0-20 d. 81-100 • b. ___ 21-40 e. More than 100 • c. ___ 41-80 f. Don’t know. SLIDE 12-18

20. Pretest Use of a questionnaire (or observation form) on a trial basis is a small pilot study to determine how well the questionnaire (or observation form) works. SLIDE 12-19