Survey research
1 / 63

Survey Research - PowerPoint PPT Presentation

  • Uploaded on

Survey Research. Prying into Your Personal Life in Ways Telemarketers Could Never Dream Of. Appropriate Research Questions for Surveys. Self-reported beliefs and behaviors Ask things at one time, measure many variables, and test several hypotheses. Appropriate Research Questions for Surveys.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Survey Research' - tuesday

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Survey research

Survey Research

Prying into Your Personal Life in Ways Telemarketers Could Never Dream Of

Appropriate research questions for surveys
Appropriate Research Questions for Surveys

  • Self-reported beliefs and behaviors

  • Ask things at one time, measure many variables, and test several hypotheses

Appropriate research questions for surveys1
Appropriate Research Questions for Surveys

  • Behavior

    • How often do you brush your teeth?

  • Attitudes/Beliefs/Opinions

    • What is the biggest problem facing the nation today?

  • Characteristics

    • Do you belong to a union?

  • Expectations

    • Do you plan to buy a new car in the next year?

  • Self-classification

    • Are you liberal or conservative?

  • Knowledge

    • Who was elected mayor in the last election?

Key limitation
Key Limitation

  • Surveys ONLY provide data on what a person or organization says

  • This can differ from what they actually do or think

  • Survey research is DEDUCTIVE

Steps in conducting a survey
Steps in Conducting a Survey

  • Step 1:

    • Develop hypotheses

    • Decide type of survey

    • Write survey questions

    • Decide on response categories

    • Design layout

Steps in conducting a survey1
Steps in Conducting a Survey

  • Step 2:

    • Plan how to record data

    • Pilot test survey instrument

  • Step 3:

    • Decide on target population

    • Get sampling frame

    • Decide on sample size

    • Select sample

Steps in conducting a survey2
Steps in Conducting a Survey

  • Step 4:

    • Locate respondents

    • Conduct interviews

    • Carefully record data

  • Step 5:

    • Enter data into computers

    • Recheck all data

    • Perform statistical analysis on data

Steps in conducting a survey3
Steps in Conducting a Survey

  • Step 6:

    • Describe methods and findings in research report

    • Present findings to others for critique and evaluation

Constructing the questionnaire
Constructing the Questionnaire

  • Introductory remarks and instructions

  • Keep it clear, keep it simple, keep respondent’s perspective in mind

  • Question writing is more of an art than a science

    • Skill, practice, patient, and creativity

Questionnaire hints
Questionnaire Hints

  • 1) Avoid jargon, slang, and abbreviations

    • No abbreviations

    • Only use this if you’re surveying a specialized population

    • “Did you drown in brew until you were totally blasted last night?”

Questionnaire hints1
Questionnaire Hints

  • 2) Avoid ambiguity, confusion, and vagueness

    • “What is your income?”

    • Confusion causes inconsistencies

  • 3) Avoid emotional language

    • Implicit connotative as well as explicit denotative

    • Use neutral language

    • “What do you think about a policy to pay murderous terrorists who threaten to steal the freedoms of peace-loving people?”

Questionnaire hints2
Questionnaire Hints

  • 4) Avoid prestige bias

    • Avoid associating statements with them

    • “Most doctors say that cigarette smoke causes lung disease for those near a smoker. Do you agree?”

  • 5) Avoid double-barreled questions

    • Each question = ONE QUESTION ONLY

    • “Does this company have pension and health insurance benefitted?”

Questionnaire hints3
Questionnaire Hints

  • 6) Do not confuse beliefs with reality

    • Don’t mix up respondents beliefs with what we want to measure

    • “Do you rate a teacher higher if the teacher tells many jokes?”

  • 7) Avoid leading questions

    • Don’t lead them to an answer

    • “You don’t smoke, do you?”

Questionnaire hints4
Questionnaire Hints

  • 8) Avoid asking questions that are beyond respondents’ capabilities

    • Frustrates respondents

    • Produces poor-quality responses

    • “Two years ago, how many hours did you watch TV every month?”

Questionnaire hints5
Questionnaire Hints

  • 9) Avoid false premises

    • Can’t put a premise that respondent could disagree with without risking alienating them

    • “When did you stop beating your girl/boyfriend?”

Questionnaire hints6
Questionnaire Hints

  • 10) Avoid asking about intentions in the distant future

    • Hypotheticals are bad

    • Not specific or concrete

    • “After you graduate college and have kids, will you invest in the stock market?”

Questionnaire hints7
Questionnaire Hints

  • 11) Avoid double negatives

    • Grammatically incorrect and confusing

    • “Do you disagree with those who do not want to build a new city swimming pool?”

Questionnaire hints8
Questionnaire Hints

  • 12) Avoid overlapping or unbalanced response categories

    • Mutually exclusive: categories do not overlap

    • Exhaustive: every respondent has a choice

    • Balanced: Offer polar opposites at each end of a continuum

    • “Did you find the service at our hotel to be, Outstanding, Excellent, Superior, or Good?”

Aiding recall
Aiding Recall

  • Ability to recall accurately declines over time

  • Memory is less trustworthy than once assumed

  • Give respondents special instructions and extra thinking time to guide them

Types of questions
Types of Questions

  • Threatening Questions

  • Socially Desirable Questions

  • Knowledge Questions

  • Skip or Contingency Questions


  • Sensitive issues…people don’t want to talk about them

  • Underreport socially undesirable answers

  • Context and wording are how to fix this

  • Issue examples:

    • Masturbation, sexual intercourse, illicit drug use, income

Socially desirable
Socially Desirable

  • Over report socially desirable answers (Social desirability bias)

    • Giving money to charity, voting etc.

  • Have to make the norm violation seem less objectionable then it really could be


  • We can’t answer elementary questions as Americans… Are You Smarter than a 5th Grader??

  • Pilot-test to make sure they’re at an appropriate level of difficulty

Skip or contingency
Skip or Contingency

  • Two or more part question

    • Answer to one part determines the next question you’ll receive

    • Sometimes called screening questions

Open v closed questions
Open v. Closed Questions

  • Open-ended: Respondent can give any answer they want

  • Close-ended: Gives respondents fixed answers from which to choose

  • Big surveys use close-ended…quicker and easier

  • Can mix both methods

  • Open-ended best in early or exploratory stages of research

  • Partially open questions leave final answer choice

  • as other

Advantages of closed
Advantages of Closed

  • Easy and quick to do

  • Easy to compare

  • Easier to code and analyze

  • Response choices can clarify meaning of question

  • Replication is easier

Disadvantages of closed
Disadvantages of Closed

  • Suggest ideas that respondent may not have thought of

  • No opinion people can answer anyways

  • Frustrating if desired answer is not a choice

  • Simplistic responses to complex ?s

  • Force us to make choices we wouldn’t make in the real world

Advantages of open
Advantages of Open

  • Unlimited responses

  • Answer in detail and clarify

  • Unanticipated feelings can be discovered

  • Creativity and self-expression

  • Reveal logic

Disadvantages of open
Disadvantages of Open

  • Different degrees of detail in answers

  • Can be irrelevant

  • Hard to compare/analyze

  • Coding is tough

  • Hard to write verbatim in interviews

  • More time necessary

  • Take up lots of space on questionnaire

Nonattitudes and middle positions
Nonattitudes and Middle Positions

  • Debate whether to include choices for middle, neutral, or nonattitudes

  • Two errors:

    • Accept nonattitude when respondent has one

    • Force choice when respondent doesn’t have one

  • Standard-format v. quasi-format v. full-format

Agree disagree rankings or ratings
Agree/Disagree, Rankings, or Ratings?

  • Agree/Disagree

  • Rankings

  • Ratings

  • Best to offer respondents explicit alternatives

  • Avoid the response set bias (just check agree to everything)

  • Respondents can rate several items equally high, but will put them in hierarchy if asked to

Word issues
Word Issues

  • Need to use simple vocabulary and grammar to minimize confusion

  • Need to watch for effects of specific words or phrases

    • Forbid v. not allow

    • Certain words trigger an emotional reaction

    • “To help the poor” v. “for welfare”

Length of survey
Length of Survey

  • Researchers prefer long because they’re cost effective

  • Phone: 5-20 minutes

  • Mail questionnaire: 3-4 pages

  • Face-to-face: Around an hour

Order effects
Order Effects

  • Strongest for people who lack strong views, less educated, elderly

  • May not perceive each element of a survey as isolated and separate

  • Influence through content and response

Context effects
Context Effects

  • Funnel sequence:

    • Start general and get more specific

  • Split sample

    • Half get questions in one order, other half different

  • More ambiguous the question, the more context effects matter

Survey layout
Survey Layout

  • Clear, neat, and easy to follow

  • Number questions

  • Identifying info

  • Cover sheet

  • Professional appearance

  • Provide instructions

Survey layout1
Survey Layout

  • Crucial in mail questionnaires

  • Use letterhead

  • Always thank them for participation

  • Circle response v. check box v. fill in dots v. put an X in a blank

  • List answer categories down, not across

Survey layout2
Survey Layout

  • Use arrows and instructions for contingency questions

  • Matrix questions

    • Compact way to present a series of questions with the same response categories

    • Box 7.5


  • If not enough people respond, we may not be able to generalize

  • Many people are burnt out with surveys these days

  • Improve eligibility rates through careful respondent selection


  • Decrease refusals through sending letters before calling, small incentives, adjusting behavior, using alternative interviewers, using alternative interview methods

  • Initial contact is HUGE in determining likelihood for success


  • Survey research is research based on direct or indirect interview methods.

  • Many types of surveys:

    • In person (including interviews)

    • Mail

    • Telephone

    • E-mail and Internet

    • Group surveys

Mail and self administered questionnaires
Mail and Self-Administered Questionnaires

  • Advantages:

    • Cheaper

    • Wide geographical area

    • Completed when convenient

    • Respondent can check personal records

    • Anonymity

Mail and self administered questionnaires1
Mail and Self-Administered Questionnaires

  • Disadvantages

  • Low response rate

  • Can’t control conditions under which its filled out

  • Someone other than intended can fill out

  • Can’t observe reactions

  • Ill suited for those who don’t have good grasp of English

Mail and self administered questionnaires2
Mail and Self-Administered Questionnaires

  • How to Increase Response

    • Address to specific person

    • Careful cover letter on letterhead

    • Postage-paid, addressed return envelope

    • Neat, attractive layout

    • Send two follow-up reminders

    • Nothing during major holidays

    • No questions on back page

    • Include a small monetary inducement

Web surveys
Web Surveys

  • Advantages

    • Very fast

    • Inexpensive

    • Flexible design

    • Can use visuals or audio

Web surveys1
Web Surveys

  • Disadvantages

    • Coverage (75% have internet)

    • Multiple e-mail addresses

    • Self-selection

    • Protecting privacy and confidentiality

      • Encrypt

    • Complexity of design depends on software

      • Screen-by-screen questions

      • Progress indicator

      • Needs to be easy to move back and forth across questions

Phone interviews
Phone Interviews

  • Advantages

    • 95% coverage rate

    • Can interview about 1500 in a few days

    • Half the cost of face-to-face interviews

    • CATI

Phone interviews1
Phone Interviews

  • Disadvantages:

    • Higher cost

    • Limited interview length

    • Use of interviewer reduces anonymity and introduces potential interviewer bias

Face to face

  • Advantages:

    • Highest response rates

    • Permit the longest questionnaires

    • Can observe surroundings

    • Ask all types of questions

Face to face1

  • Disadvantages:

    • High cost

      • Training, travel, supervision, personnel costs

    • Interviewer bias

Comparing the methods
Comparing the Methods

  • Cost:

    • I, M, P, F2F

  • Speed:

    • I, P, F2F, M

  • Length:

    • F2F, M, I, P

  • Response rate:

    • F2F, P, I, M


  • Obtain cooperation and build rapport, but remain neutral and objective

  • Interviewer must treat everything in a matter-of-fact manner

  • Pleasant, honest, accurate, mature, responsible, moderately intelligent, stable, and motivated

  • Poise and tact

Interviewer bias
Interviewer Bias

  • Interview expectations can create significant bias

  • Social setting can affect answers

    • Students answer differently at home and at school

  • Visible characteristics can play a role…such as race and gender


  • Computer-assisted telephone interviewing

  • Read questions from the screen and enter answers immediately

  • Automatically makes calls for you

  • Automatically stores answers in a database

The ethical survey
The Ethical Survey

  • Invasion of privacy

    • That’s what ANY survey is!!!!

  • Voluntary participation

    • Informed consent

  • Use of push polls—AKA “pseudosurveys”

  • Sloppy reporting of surveys by the media

    • No quality control standards


  • Five important factors influence the choice of survey type:

    • Cost

    • Completion rates

    • Sample-population congruence

    • Questionnaire length

    • Data-processing issues


  • Cost:

    • All surveys share basic expenses for materials.

    • Cost of a survey can be quite variable, based on

      • Professional questionnaire design

      • Length of the questionnaire

      • Geographic dispersion of the sample

      • Callback procedures

      • Respondent selection rules

      • Availability of trained staff

      • Method-specific costs like long distance rates or postage


  • Completion rates:

    • Low response rate on a survey limits the ability to make statistical inferences about the target population

    • Must make an effort to reach those selected for inclusion in the sample to maximize the representativeness of the sample…or risk bias


  • Sample-population congruence:

    • If those who fail to participate in the survey share common characteristics, the sample will be wrought with systematic sampling error and the conclusions made with the data could be incorrect.


  • Questionnaire length:

    • Keeping respondents’ attention during a survey helps achieve the best possible response quality.

    • A lengthy questionnaire can lead respondents to lose focus or speed through questions to finish sooner, resulting in low response quality.

    • Survey length ranges between three and four dozen items, depending on complexity and format.


  • Data-processing issues

    • A good deal of time and effort are required to process data.

    • Surveys using more elaborate or complex designs or a large number of questions will require more time and effort.


Each type of survey has advantages and disadvantages when it comes to the five factors.