Survey and questionnaire design
This presentation is the property of its rightful owner.
Sponsored Links
1 / 37

Survey and Questionnaire Design PowerPoint PPT Presentation


  • 152 Views
  • Uploaded on
  • Presentation posted in: General

Survey and Questionnaire Design. MAR 6648: Marketing Research January 20, 2010. Overview. How do you write survey questions? What are some common traps survey writers fall into? How do we avoid them? What about special circumstances? Embarrassing questions, etc.?. Survey Research.

Download Presentation

Survey and Questionnaire Design

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Survey and questionnaire design

Survey and Questionnaire Design

MAR 6648: Marketing Research

January 20, 2010


Overview

Overview

  • How do you write survey questions?

    • What are some common traps survey writers fall into?

    • How do we avoid them?

  • What about special circumstances?

    • Embarrassing questions, etc.?


Survey research

Survey Research

  • Descriptive research

  • Survey modes and methods:

    • Telephone surveys

    • Mail surveys

    • Face-to-face surveys

      • At home

      • Mall intercepts

    • Web-based surveys


Advantages of surveys

Advantages of Surveys

  • Can collect a great deal of data, including

    • Attitudes, interests, opinions, behaviors

    • Classification variables

  • Can be employed in virtually any setting

  • Can be inexpensive, depending on format


Disadvantages of surveys

Disadvantages of Surveys

  • Effective implementation requires considerable judgment

  • Social interaction of interview may lead to errors

  • Subject to demand and honesty concerns


First big point biases

First Big Point: Biases

  • Basic Tenet: Ask a question, and you’ll get a response!

  • Translation: Garbage in, garbage out!


Typical problems in wording questions

Typical problems in wording questions


How questions create answers

How questions create answers


How questions create answers1

How questions create answers


How questions create answers2

How questions create answers

  • A survey for Black Flag said:

    • “A roach disk… poisons a roach slowly. The dying roach returns to the nest and after it dies is eaten by other roaches. In turn, these roaches become poisoned and die. How effective do you think this type of product would be in killing roaches?”

    • Not surprisingly, 79% said effective!

  • A Gallup poll sponsored by the disposable-diaper industry asked:

    • “It is estimated that disposable diapers account for less than 2% of the trash in today’s landfills. In contrast, beverage containers, third-class mail and yard waste to account for about 21% of the trash in landfills. Given this, in your opinion, would it be fair to ban disposable diapers?”

    • 84% said no!


How questions shape answers

How questions shape answers

  • Some examples

    • Behavioral frequency questions

      • “How often do you…?”


Behavioral frequency questions

Behavioral Frequency Questions

  • How many hours of TV do you watch on an average week day? Please choose the category from the list below that best describes your behavior.

Marketers Question: What % of class watches more than 2.5 hrs of TV?


Behavioral frequency questions1

Behavioral Frequency Questions

  • Scale represents distribution in ‘real’ world

    • Values in middle reflect “average” / “usual” behaviors

    • Values at an end (or ends) reflect extreme behavior

  • Low Frequency Scale

    • Lower estimates of “average”

    • Avoid marking extreme high box

  • High Frequency Scale

    • Makes marking “>2.5 hours” more acceptable

  • “Response Scale Effect”


How questions shape answers1

How questions shape answers

  • How much money do you have in savings?

    • 1-7; 1 = $0, 7 = $500 and over

    • 1-7; 1 = $500 and under, 7 = $500,000 and over

  • How wealthy are you?


How questions shape answers2

How questions shape answers

  • Some examples

    • Behavioral frequency questions

    • Aided versus unaided questions


Aided and unaided questions

Aided and Unaided Questions

  • What do you consider to be the most important thing for children to prepare them for life?

    • 1. _______________

    • 2. _______________

    • 3. _______________

    • 4. _______________

  • What do you consider to be the most important thing for children to prepare them for life?

    • __Good education

    • __Inheritance

    • __Moral values

    • __To think for themselves

    • __Good friendships

    • __Others (please specify)


How questions shape answers3

How questions shape answers

  • Some examples

    • Behavioral frequency questions

    • Aided versus unaided questions

    • Framing of questions


Framing effects

Framing Effects

  • Question forms that lead respondents to give a certain response

    • Example: Allow versus forbid

Yes No

5%95%

81%19%

Q: Do you think the U.S. should forbid public speeches against democracy?

Q: Do you think the U.S. should allow public speeches against democracy?


Framing effects1

Framing Effects

  • Question form

  • Choice of example/source


Framing effects2

Framing Effects

  • Question form

  • Choice of example/source

    • Surveyor power: Mispredicting political outcomes (Bischoping & Schuman 1992)

    • Perhaps partly motivational?

      • People who feel good about themselves show less of this bias (Cohen 2003)


How questions shape answers4

How questions shape answers

  • Some examples

    • Behavioral frequency questions

    • Aided versus unaided questions

    • Framing of questions

    • Scales associated with questions

      • Odd versus even numbered scales

      • Semantic versus Stapel scales


How questions shape answers5

How questions shape answers

  • Some examples

    • Behavioral frequency questions

    • Aided versus unaided questions

    • Framing of questions

    • Scales associated with questions

    • Sequence of questions


Sequence of questions

Sequence of Questions

Q1: General Colin Powell, the hero of the Gulf War, recently decided to become a member of a political party. Do you happen to know which party that is?

Q2: Now rate this party.

Q1: General Colin Powell, the hero of the Gulf War, recently decided to become a member of a political party, but decided not to run for the presidential elections. Do you happen to know which party that is?

Q2: Now rate this party.

  • Specific – General Questions

    • Happiness and dating (How many dates have you had recently?)

      • Order 1 (Happiness then dates): Correlation = 0.03

      • Order 2 (Dates then happiness): Correlation = 0.67


Sequence of questions1

Sequence of Questions

  • Difficultly in answering one question can influence subsequent answers

  • List three (twelve) ways in which you are happy (unhappy) with your partner.

  • How happy are you with your partner?


Sensitive questions

Sensitive Questions

  • Respondents may not be willing to answer some questions, due to embarrassment, legal issues, etc.

  • This may result in

    • Item nonresponse

    • Termination

    • Distortion


Asking the embarrassing question

Asking the Embarrassing Question

  • The casual approach

    • “Do you happen to have killed your wife?”

  • The numbered card approach

    • “Would you please read off the number on this card which corresponds to what became of your wife?:

      • 1: Natural Death

      • 2: I killed her

      • 3: Other

  • The everybody approach (counter biasing method)

    • “As you know, many people have been killing their wives these days. Do you happen to have killed yours?”

  • The other people approach

    • “Do you know any people who have murdered their wives? How about yourself?”

Barton, POQ, 1958


How to ask a sensitive question

How to ask a sensitive question

  • Personal information – Build rapport, legitimize yourself

  • Embarrassing info – Downward bias

    • Counterbiasing statement

      • “Recent studies indicate that men often use their wives’ hair spray. Do you use your wife’s hair spray?”

    • Projective techniques


Projective techniques

Projective Techniques


Key points

Key Points

  • When wording questions…

    • Use simple words

    • Use clear words

    • Avoid leading questions

    • Avoid implicit alternatives

    • Avoid implicit assumptions

    • Avoid double-barreled questions

    • Consider frame of reference


Key points1

Key Points

  • Don’t forget about question sequence

    • It can influence the nature of the respondent’s answers and can be the cause of serious error in the results

      • Use a simple and interesting opening question

      • Ask general questions first

      • Place uninteresting and difficult questions late in the sequence

      • Arrange questions in logical order


The basic steps of questionnaire design

The Basic Steps of Questionnaire Design

  • Decide on question content

  • Decide how to ask each question

    • Response format

    • Question wording

  • Decide on question sequence

    • Funnel approach (general to specific)

  • Decide on physical characteristics

  • Carry out pretest, revise, and final draft


Survey and questionnaire design

So…

  • What is a good survey question?

  • How are we to interpret survey question output?


A managerial problem

A Managerial Problem

In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely is it that you will buy a Verizon iPhone?

  • Temptation:

    • Report those numbers to your manager as is

  • Problem:

    • Those numbers have no direct value

      • What does “somewhat likely” mean in real life?

Very

Unlikely

Somewhat

Unlikely

Somewhat

Likely

Very

Likely

15%

35%

35%

15%


A managerial problem1

A Managerial Problem

In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely is it that you will buy a Verizon iPhone?

  • Solutions:

    • 1. Benchmark

      • What do past surveys say?

Very

Unlikely

Somewhat

Unlikely

Somewhat

Likely

Very

Likely

15%

35%

35%

15%

Actual Purchase Rate


A managerial problem2

A Managerial Problem

In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely is it that you will buy a Verizon iPhone?

  • Solutions:

    • 1. Benchmark

      • What do past surveys say?

    • 2. Focus on questions you can answer

      • Relative responses on the same items

Very

Unlikely

Somewhat

Unlikely

Somewhat

Likely

Very

Likely

15%

35%

35%

15%


A managerial problem3

A Managerial Problem

In a few weeks, Verizon will start selling a version of the iPhone compatible with its network. How likely is it that you will buy a Verizon iPhone?

  • Solutions:

    • 1. Benchmark

      • What do past surveys say?

    • 2. Focus on questions you can answer

      • Relative responses on the same items

    • 3. Ask other questions/use other methods

Very

Unlikely

Somewhat

Unlikely

Somewhat

Likely

Very

Likely

15%

35%

35%

15%


Summary

Summary

  • Survey design is harder than it looks

    • Be sure to think about what the question suggests about the answer

      • Order and scale points matter, too

    • Be sure to keep respondents’ comfort in mind


  • Login