survey research n.
Skip this Video
Download Presentation
Survey Research

Loading in 2 Seconds...

play fullscreen
1 / 63

Survey Research - PowerPoint PPT Presentation

  • Uploaded on

Survey Research. Prying into Your Personal Life in Ways Telemarketers Could Never Dream Of. Appropriate Research Questions for Surveys. Self-reported beliefs and behaviors Ask things at one time, measure many variables, and test several hypotheses. Appropriate Research Questions for Surveys.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Survey Research' - tuesday

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
survey research

Survey Research

Prying into Your Personal Life in Ways Telemarketers Could Never Dream Of

appropriate research questions for surveys
Appropriate Research Questions for Surveys
  • Self-reported beliefs and behaviors
  • Ask things at one time, measure many variables, and test several hypotheses
appropriate research questions for surveys1
Appropriate Research Questions for Surveys
  • Behavior
    • How often do you brush your teeth?
  • Attitudes/Beliefs/Opinions
    • What is the biggest problem facing the nation today?
  • Characteristics
    • Do you belong to a union?
  • Expectations
    • Do you plan to buy a new car in the next year?
  • Self-classification
    • Are you liberal or conservative?
  • Knowledge
    • Who was elected mayor in the last election?
key limitation
Key Limitation
  • Surveys ONLY provide data on what a person or organization says
  • This can differ from what they actually do or think
  • Survey research is DEDUCTIVE
steps in conducting a survey
Steps in Conducting a Survey
  • Step 1:
    • Develop hypotheses
    • Decide type of survey
    • Write survey questions
    • Decide on response categories
    • Design layout
steps in conducting a survey1
Steps in Conducting a Survey
  • Step 2:
    • Plan how to record data
    • Pilot test survey instrument
  • Step 3:
    • Decide on target population
    • Get sampling frame
    • Decide on sample size
    • Select sample
steps in conducting a survey2
Steps in Conducting a Survey
  • Step 4:
    • Locate respondents
    • Conduct interviews
    • Carefully record data
  • Step 5:
    • Enter data into computers
    • Recheck all data
    • Perform statistical analysis on data
steps in conducting a survey3
Steps in Conducting a Survey
  • Step 6:
    • Describe methods and findings in research report
    • Present findings to others for critique and evaluation
constructing the questionnaire
Constructing the Questionnaire
  • Introductory remarks and instructions
  • Keep it clear, keep it simple, keep respondent’s perspective in mind
  • Question writing is more of an art than a science
    • Skill, practice, patient, and creativity
questionnaire hints
Questionnaire Hints
  • 1) Avoid jargon, slang, and abbreviations
    • No abbreviations
    • Only use this if you’re surveying a specialized population
    • “Did you drown in brew until you were totally blasted last night?”
questionnaire hints1
Questionnaire Hints
  • 2) Avoid ambiguity, confusion, and vagueness
    • “What is your income?”
    • Confusion causes inconsistencies
  • 3) Avoid emotional language
    • Implicit connotative as well as explicit denotative
    • Use neutral language
    • “What do you think about a policy to pay murderous terrorists who threaten to steal the freedoms of peace-loving people?”
questionnaire hints2
Questionnaire Hints
  • 4) Avoid prestige bias
    • Avoid associating statements with them
    • “Most doctors say that cigarette smoke causes lung disease for those near a smoker. Do you agree?”
  • 5) Avoid double-barreled questions
    • Each question = ONE QUESTION ONLY
    • “Does this company have pension and health insurance benefitted?”
questionnaire hints3
Questionnaire Hints
  • 6) Do not confuse beliefs with reality
    • Don’t mix up respondents beliefs with what we want to measure
    • “Do you rate a teacher higher if the teacher tells many jokes?”
  • 7) Avoid leading questions
    • Don’t lead them to an answer
    • “You don’t smoke, do you?”
questionnaire hints4
Questionnaire Hints
  • 8) Avoid asking questions that are beyond respondents’ capabilities
    • Frustrates respondents
    • Produces poor-quality responses
    • “Two years ago, how many hours did you watch TV every month?”
questionnaire hints5
Questionnaire Hints
  • 9) Avoid false premises
    • Can’t put a premise that respondent could disagree with without risking alienating them
    • “When did you stop beating your girl/boyfriend?”
questionnaire hints6
Questionnaire Hints
  • 10) Avoid asking about intentions in the distant future
    • Hypotheticals are bad
    • Not specific or concrete
    • “After you graduate college and have kids, will you invest in the stock market?”
questionnaire hints7
Questionnaire Hints
  • 11) Avoid double negatives
    • Grammatically incorrect and confusing
    • “Do you disagree with those who do not want to build a new city swimming pool?”
questionnaire hints8
Questionnaire Hints
  • 12) Avoid overlapping or unbalanced response categories
    • Mutually exclusive: categories do not overlap
    • Exhaustive: every respondent has a choice
    • Balanced: Offer polar opposites at each end of a continuum
    • “Did you find the service at our hotel to be, Outstanding, Excellent, Superior, or Good?”
aiding recall
Aiding Recall
  • Ability to recall accurately declines over time
  • Memory is less trustworthy than once assumed
  • Give respondents special instructions and extra thinking time to guide them
types of questions
Types of Questions
  • Threatening Questions
  • Socially Desirable Questions
  • Knowledge Questions
  • Skip or Contingency Questions
  • Sensitive issues…people don’t want to talk about them
  • Underreport socially undesirable answers
  • Context and wording are how to fix this
  • Issue examples:
    • Masturbation, sexual intercourse, illicit drug use, income
socially desirable
Socially Desirable
  • Over report socially desirable answers (Social desirability bias)
    • Giving money to charity, voting etc.
  • Have to make the norm violation seem less objectionable then it really could be
  • We can’t answer elementary questions as Americans… Are You Smarter than a 5th Grader??
  • Pilot-test to make sure they’re at an appropriate level of difficulty
skip or contingency
Skip or Contingency
  • Two or more part question
    • Answer to one part determines the next question you’ll receive
    • Sometimes called screening questions
open v closed questions
Open v. Closed Questions
  • Open-ended: Respondent can give any answer they want
  • Close-ended: Gives respondents fixed answers from which to choose
  • Big surveys use close-ended…quicker and easier
  • Can mix both methods
  • Open-ended best in early or exploratory stages of research
  • Partially open questions leave final answer choice
  • as other
advantages of closed
Advantages of Closed
  • Easy and quick to do
  • Easy to compare
  • Easier to code and analyze
  • Response choices can clarify meaning of question
  • Replication is easier
disadvantages of closed
Disadvantages of Closed
  • Suggest ideas that respondent may not have thought of
  • No opinion people can answer anyways
  • Frustrating if desired answer is not a choice
  • Simplistic responses to complex ?s
  • Force us to make choices we wouldn’t make in the real world
advantages of open
Advantages of Open
  • Unlimited responses
  • Answer in detail and clarify
  • Unanticipated feelings can be discovered
  • Creativity and self-expression
  • Reveal logic
disadvantages of open
Disadvantages of Open
  • Different degrees of detail in answers
  • Can be irrelevant
  • Hard to compare/analyze
  • Coding is tough
  • Hard to write verbatim in interviews
  • More time necessary
  • Take up lots of space on questionnaire
nonattitudes and middle positions
Nonattitudes and Middle Positions
  • Debate whether to include choices for middle, neutral, or nonattitudes
  • Two errors:
    • Accept nonattitude when respondent has one
    • Force choice when respondent doesn’t have one
  • Standard-format v. quasi-format v. full-format
agree disagree rankings or ratings
Agree/Disagree, Rankings, or Ratings?
  • Agree/Disagree
  • Rankings
  • Ratings
  • Best to offer respondents explicit alternatives
  • Avoid the response set bias (just check agree to everything)
  • Respondents can rate several items equally high, but will put them in hierarchy if asked to
word issues
Word Issues
  • Need to use simple vocabulary and grammar to minimize confusion
  • Need to watch for effects of specific words or phrases
    • Forbid v. not allow
    • Certain words trigger an emotional reaction
    • “To help the poor” v. “for welfare”
length of survey
Length of Survey
  • Researchers prefer long because they’re cost effective
  • Phone: 5-20 minutes
  • Mail questionnaire: 3-4 pages
  • Face-to-face: Around an hour
order effects
Order Effects
  • Strongest for people who lack strong views, less educated, elderly
  • May not perceive each element of a survey as isolated and separate
  • Influence through content and response
context effects
Context Effects
  • Funnel sequence:
    • Start general and get more specific
  • Split sample
    • Half get questions in one order, other half different
  • More ambiguous the question, the more context effects matter
survey layout
Survey Layout
  • Clear, neat, and easy to follow
  • Number questions
  • Identifying info
  • Cover sheet
  • Professional appearance
  • Provide instructions
survey layout1
Survey Layout
  • Crucial in mail questionnaires
  • Use letterhead
  • Always thank them for participation
  • Circle response v. check box v. fill in dots v. put an X in a blank
  • List answer categories down, not across
survey layout2
Survey Layout
  • Use arrows and instructions for contingency questions
  • Matrix questions
    • Compact way to present a series of questions with the same response categories
    • Box 7.5
  • If not enough people respond, we may not be able to generalize
  • Many people are burnt out with surveys these days
  • Improve eligibility rates through careful respondent selection
  • Decrease refusals through sending letters before calling, small incentives, adjusting behavior, using alternative interviewers, using alternative interview methods
  • Initial contact is HUGE in determining likelihood for success
  • Survey research is research based on direct or indirect interview methods.
  • Many types of surveys:
    • In person (including interviews)
    • Mail
    • Telephone
    • E-mail and Internet
    • Group surveys
mail and self administered questionnaires
Mail and Self-Administered Questionnaires
  • Advantages:
    • Cheaper
    • Wide geographical area
    • Completed when convenient
    • Respondent can check personal records
    • Anonymity
mail and self administered questionnaires1
Mail and Self-Administered Questionnaires
  • Disadvantages
  • Low response rate
  • Can’t control conditions under which its filled out
  • Someone other than intended can fill out
  • Can’t observe reactions
  • Ill suited for those who don’t have good grasp of English
mail and self administered questionnaires2
Mail and Self-Administered Questionnaires
  • How to Increase Response
    • Address to specific person
    • Careful cover letter on letterhead
    • Postage-paid, addressed return envelope
    • Neat, attractive layout
    • Send two follow-up reminders
    • Nothing during major holidays
    • No questions on back page
    • Include a small monetary inducement
web surveys
Web Surveys
  • Advantages
    • Very fast
    • Inexpensive
    • Flexible design
    • Can use visuals or audio
web surveys1
Web Surveys
  • Disadvantages
    • Coverage (75% have internet)
    • Multiple e-mail addresses
    • Self-selection
    • Protecting privacy and confidentiality
      • Encrypt
    • Complexity of design depends on software
      • Screen-by-screen questions
      • Progress indicator
      • Needs to be easy to move back and forth across questions
phone interviews
Phone Interviews
  • Advantages
    • 95% coverage rate
    • Can interview about 1500 in a few days
    • Half the cost of face-to-face interviews
    • CATI
phone interviews1
Phone Interviews
  • Disadvantages:
    • Higher cost
    • Limited interview length
    • Use of interviewer reduces anonymity and introduces potential interviewer bias
face to face
  • Advantages:
    • Highest response rates
    • Permit the longest questionnaires
    • Can observe surroundings
    • Ask all types of questions
face to face1
  • Disadvantages:
    • High cost
      • Training, travel, supervision, personnel costs
    • Interviewer bias
comparing the methods
Comparing the Methods
  • Cost:
    • I, M, P, F2F
  • Speed:
    • I, P, F2F, M
  • Length:
    • F2F, M, I, P
  • Response rate:
    • F2F, P, I, M
  • Obtain cooperation and build rapport, but remain neutral and objective
  • Interviewer must treat everything in a matter-of-fact manner
  • Pleasant, honest, accurate, mature, responsible, moderately intelligent, stable, and motivated
  • Poise and tact
interviewer bias
Interviewer Bias
  • Interview expectations can create significant bias
  • Social setting can affect answers
    • Students answer differently at home and at school
  • Visible characteristics can play a role…such as race and gender
  • Computer-assisted telephone interviewing
  • Read questions from the screen and enter answers immediately
  • Automatically makes calls for you
  • Automatically stores answers in a database
the ethical survey
The Ethical Survey
  • Invasion of privacy
    • That’s what ANY survey is!!!!
  • Voluntary participation
    • Informed consent
  • Use of push polls—AKA “pseudosurveys”
  • Sloppy reporting of surveys by the media
    • No quality control standards
  • Five important factors influence the choice of survey type:
    • Cost
    • Completion rates
    • Sample-population congruence
    • Questionnaire length
    • Data-processing issues
  • Cost:
    • All surveys share basic expenses for materials.
    • Cost of a survey can be quite variable, based on
      • Professional questionnaire design
      • Length of the questionnaire
      • Geographic dispersion of the sample
      • Callback procedures
      • Respondent selection rules
      • Availability of trained staff
      • Method-specific costs like long distance rates or postage
  • Completion rates:
    • Low response rate on a survey limits the ability to make statistical inferences about the target population
    • Must make an effort to reach those selected for inclusion in the sample to maximize the representativeness of the sample…or risk bias
  • Sample-population congruence:
    • If those who fail to participate in the survey share common characteristics, the sample will be wrought with systematic sampling error and the conclusions made with the data could be incorrect.
  • Questionnaire length:
    • Keeping respondents’ attention during a survey helps achieve the best possible response quality.
    • A lengthy questionnaire can lead respondents to lose focus or speed through questions to finish sooner, resulting in low response quality.
    • Survey length ranges between three and four dozen items, depending on complexity and format.
  • Data-processing issues
    • A good deal of time and effort are required to process data.
    • Surveys using more elaborate or complex designs or a large number of questions will require more time and effort.

Each type of survey has advantages and disadvantages when it comes to the five factors.