Goals for Effective Survey Questions • Focused • Concise • Clear
Source of Bias and Error • Instrumentation • The way the questions are written, asked, or set up • Response Bias • The way the respondent chooses to answer the question(s) asked
Ways to Avoid Bias • Basic ‘core’ vocabulary should be used • Avoid jargon, technical language, abbreviation • Are you cognizant of the variables influencing your decision? Vs. • What variables influenced your decision?
Ways to Avoid Bias • Stay in an “active” versus passive voice • Avoid ‘leading’ questions • Don’t you believe SUVs are good for off-road driving? vs. • Do you believe SUVs are good vehicles for off-road driving?
Ways to Avoid Bias • Avoid double barreled questions • What do you think about the image and advertising for the Pontiac Aztek? • Avoid ambiguity, be specific as necessary • How would you categorize your public television usage? • Sometimes, often, regular, irregular, infrequent
Ways to Avoid Bias • Avoid making assumptions about the respondent’s state of mind • Explain what you like about PBS… vs. • What, if anything, do you like about PBS?
Ways to Avoid Bias • Loaded questions • Increases the possibility for responses that makes self look more socially desirable • Loaded questions can also generate • Acquiescence • Yea-/Nay saying • Threat defense responses • Hostility responses • Extremity responses
Ways to Avoid Bias • Loaded words (over-dramatized, inflammatory language) • To what extent do you believe global warming affects the planet’s environment? Vs. • To what extent do you believe climate change affects the planet’s environment?
Ways to Avoid Bias • Burdensome Questions (overtaxing to the memory) • How often have you gone to a sporting event since the last Olympic Games? • How often did you watch sports on ESPN last week?
Ways to Avoid Bias • Avoid negatively phased questions • Do you agree that it is not the job of the government to make decisions about social security? Vs. • To what extent do you agree or disagree with the idea that the government should make decisions about social security?
Ways to Avoid Bias • Order bias • Responses influenced by the order of question presentation Solution • Funnel questions - Asking brand specific questions before general questions
Ways to Avoid Bias • Filter [or qualifying] questions • Maximizes the chance of asking appropriate questions for the respondent in question • Which Bank of America location has the best customer service? Vs. • Are you familiar with Bank of America’s customer service?
Ways to Avoid Bias • Alternatives should be exhaustive • Options should avoid response bias • Response alternatives should be appropriate [typical] for the behavior, attitude in question