Survey research
Download
1 / 63

Survey Research - PowerPoint PPT Presentation


  • 64 Views
  • Uploaded on

Survey Research. Prying into Your Personal Life in Ways Telemarketers Could Never Dream Of. Appropriate Research Questions for Surveys. Self-reported beliefs and behaviors Ask things at one time, measure many variables, and test several hypotheses. Appropriate Research Questions for Surveys.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Survey Research' - tuesday


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Survey research

Survey Research

Prying into Your Personal Life in Ways Telemarketers Could Never Dream Of


Appropriate research questions for surveys
Appropriate Research Questions for Surveys

  • Self-reported beliefs and behaviors

  • Ask things at one time, measure many variables, and test several hypotheses


Appropriate research questions for surveys1
Appropriate Research Questions for Surveys

  • Behavior

    • How often do you brush your teeth?

  • Attitudes/Beliefs/Opinions

    • What is the biggest problem facing the nation today?

  • Characteristics

    • Do you belong to a union?

  • Expectations

    • Do you plan to buy a new car in the next year?

  • Self-classification

    • Are you liberal or conservative?

  • Knowledge

    • Who was elected mayor in the last election?


Key limitation
Key Limitation

  • Surveys ONLY provide data on what a person or organization says

  • This can differ from what they actually do or think

  • Survey research is DEDUCTIVE


Steps in conducting a survey
Steps in Conducting a Survey

  • Step 1:

    • Develop hypotheses

    • Decide type of survey

    • Write survey questions

    • Decide on response categories

    • Design layout


Steps in conducting a survey1
Steps in Conducting a Survey

  • Step 2:

    • Plan how to record data

    • Pilot test survey instrument

  • Step 3:

    • Decide on target population

    • Get sampling frame

    • Decide on sample size

    • Select sample


Steps in conducting a survey2
Steps in Conducting a Survey

  • Step 4:

    • Locate respondents

    • Conduct interviews

    • Carefully record data

  • Step 5:

    • Enter data into computers

    • Recheck all data

    • Perform statistical analysis on data


Steps in conducting a survey3
Steps in Conducting a Survey

  • Step 6:

    • Describe methods and findings in research report

    • Present findings to others for critique and evaluation


Constructing the questionnaire
Constructing the Questionnaire

  • Introductory remarks and instructions

  • Keep it clear, keep it simple, keep respondent’s perspective in mind

  • Question writing is more of an art than a science

    • Skill, practice, patient, and creativity


Questionnaire hints
Questionnaire Hints

  • 1) Avoid jargon, slang, and abbreviations

    • No abbreviations

    • Only use this if you’re surveying a specialized population

    • “Did you drown in brew until you were totally blasted last night?”


Questionnaire hints1
Questionnaire Hints

  • 2) Avoid ambiguity, confusion, and vagueness

    • “What is your income?”

    • Confusion causes inconsistencies

  • 3) Avoid emotional language

    • Implicit connotative as well as explicit denotative

    • Use neutral language

    • “What do you think about a policy to pay murderous terrorists who threaten to steal the freedoms of peace-loving people?”


Questionnaire hints2
Questionnaire Hints

  • 4) Avoid prestige bias

    • Avoid associating statements with them

    • “Most doctors say that cigarette smoke causes lung disease for those near a smoker. Do you agree?”

  • 5) Avoid double-barreled questions

    • Each question = ONE QUESTION ONLY

    • “Does this company have pension and health insurance benefitted?”


Questionnaire hints3
Questionnaire Hints

  • 6) Do not confuse beliefs with reality

    • Don’t mix up respondents beliefs with what we want to measure

    • “Do you rate a teacher higher if the teacher tells many jokes?”

  • 7) Avoid leading questions

    • Don’t lead them to an answer

    • “You don’t smoke, do you?”


Questionnaire hints4
Questionnaire Hints

  • 8) Avoid asking questions that are beyond respondents’ capabilities

    • Frustrates respondents

    • Produces poor-quality responses

    • “Two years ago, how many hours did you watch TV every month?”


Questionnaire hints5
Questionnaire Hints

  • 9) Avoid false premises

    • Can’t put a premise that respondent could disagree with without risking alienating them

    • “When did you stop beating your girl/boyfriend?”


Questionnaire hints6
Questionnaire Hints

  • 10) Avoid asking about intentions in the distant future

    • Hypotheticals are bad

    • Not specific or concrete

    • “After you graduate college and have kids, will you invest in the stock market?”


Questionnaire hints7
Questionnaire Hints

  • 11) Avoid double negatives

    • Grammatically incorrect and confusing

    • “Do you disagree with those who do not want to build a new city swimming pool?”


Questionnaire hints8
Questionnaire Hints

  • 12) Avoid overlapping or unbalanced response categories

    • Mutually exclusive: categories do not overlap

    • Exhaustive: every respondent has a choice

    • Balanced: Offer polar opposites at each end of a continuum

    • “Did you find the service at our hotel to be, Outstanding, Excellent, Superior, or Good?”


Aiding recall
Aiding Recall

  • Ability to recall accurately declines over time

  • Memory is less trustworthy than once assumed

  • Give respondents special instructions and extra thinking time to guide them


Types of questions
Types of Questions

  • Threatening Questions

  • Socially Desirable Questions

  • Knowledge Questions

  • Skip or Contingency Questions


Threatening
Threatening

  • Sensitive issues…people don’t want to talk about them

  • Underreport socially undesirable answers

  • Context and wording are how to fix this

  • Issue examples:

    • Masturbation, sexual intercourse, illicit drug use, income


Socially desirable
Socially Desirable

  • Over report socially desirable answers (Social desirability bias)

    • Giving money to charity, voting etc.

  • Have to make the norm violation seem less objectionable then it really could be


Knowledge
Knowledge

  • We can’t answer elementary questions as Americans… Are You Smarter than a 5th Grader??

  • Pilot-test to make sure they’re at an appropriate level of difficulty


Skip or contingency
Skip or Contingency

  • Two or more part question

    • Answer to one part determines the next question you’ll receive

    • Sometimes called screening questions


Open v closed questions
Open v. Closed Questions

  • Open-ended: Respondent can give any answer they want

  • Close-ended: Gives respondents fixed answers from which to choose

  • Big surveys use close-ended…quicker and easier

  • Can mix both methods

  • Open-ended best in early or exploratory stages of research

  • Partially open questions leave final answer choice

  • as other


Advantages of closed
Advantages of Closed

  • Easy and quick to do

  • Easy to compare

  • Easier to code and analyze

  • Response choices can clarify meaning of question

  • Replication is easier


Disadvantages of closed
Disadvantages of Closed

  • Suggest ideas that respondent may not have thought of

  • No opinion people can answer anyways

  • Frustrating if desired answer is not a choice

  • Simplistic responses to complex ?s

  • Force us to make choices we wouldn’t make in the real world


Advantages of open
Advantages of Open

  • Unlimited responses

  • Answer in detail and clarify

  • Unanticipated feelings can be discovered

  • Creativity and self-expression

  • Reveal logic


Disadvantages of open
Disadvantages of Open

  • Different degrees of detail in answers

  • Can be irrelevant

  • Hard to compare/analyze

  • Coding is tough

  • Hard to write verbatim in interviews

  • More time necessary

  • Take up lots of space on questionnaire


Nonattitudes and middle positions
Nonattitudes and Middle Positions

  • Debate whether to include choices for middle, neutral, or nonattitudes

  • Two errors:

    • Accept nonattitude when respondent has one

    • Force choice when respondent doesn’t have one

  • Standard-format v. quasi-format v. full-format


Agree disagree rankings or ratings
Agree/Disagree, Rankings, or Ratings?

  • Agree/Disagree

  • Rankings

  • Ratings

  • Best to offer respondents explicit alternatives

  • Avoid the response set bias (just check agree to everything)

  • Respondents can rate several items equally high, but will put them in hierarchy if asked to


Word issues
Word Issues

  • Need to use simple vocabulary and grammar to minimize confusion

  • Need to watch for effects of specific words or phrases

    • Forbid v. not allow

    • Certain words trigger an emotional reaction

    • “To help the poor” v. “for welfare”


Length of survey
Length of Survey

  • Researchers prefer long because they’re cost effective

  • Phone: 5-20 minutes

  • Mail questionnaire: 3-4 pages

  • Face-to-face: Around an hour


Order effects
Order Effects

  • Strongest for people who lack strong views, less educated, elderly

  • May not perceive each element of a survey as isolated and separate

  • Influence through content and response


Context effects
Context Effects

  • Funnel sequence:

    • Start general and get more specific

  • Split sample

    • Half get questions in one order, other half different

  • More ambiguous the question, the more context effects matter


Survey layout
Survey Layout

  • Clear, neat, and easy to follow

  • Number questions

  • Identifying info

  • Cover sheet

  • Professional appearance

  • Provide instructions


Survey layout1
Survey Layout

  • Crucial in mail questionnaires

  • Use letterhead

  • Always thank them for participation

  • Circle response v. check box v. fill in dots v. put an X in a blank

  • List answer categories down, not across


Survey layout2
Survey Layout

  • Use arrows and instructions for contingency questions

  • Matrix questions

    • Compact way to present a series of questions with the same response categories

    • Box 7.5


Nonresponse
Nonresponse

  • If not enough people respond, we may not be able to generalize

  • Many people are burnt out with surveys these days

  • Improve eligibility rates through careful respondent selection


Nonresponse1
Nonresponse

  • Decrease refusals through sending letters before calling, small incentives, adjusting behavior, using alternative interviewers, using alternative interview methods

  • Initial contact is HUGE in determining likelihood for success


Surveys
Surveys

  • Survey research is research based on direct or indirect interview methods.

  • Many types of surveys:

    • In person (including interviews)

    • Mail

    • Telephone

    • E-mail and Internet

    • Group surveys


Mail and self administered questionnaires
Mail and Self-Administered Questionnaires

  • Advantages:

    • Cheaper

    • Wide geographical area

    • Completed when convenient

    • Respondent can check personal records

    • Anonymity


Mail and self administered questionnaires1
Mail and Self-Administered Questionnaires

  • Disadvantages

  • Low response rate

  • Can’t control conditions under which its filled out

  • Someone other than intended can fill out

  • Can’t observe reactions

  • Ill suited for those who don’t have good grasp of English


Mail and self administered questionnaires2
Mail and Self-Administered Questionnaires

  • How to Increase Response

    • Address to specific person

    • Careful cover letter on letterhead

    • Postage-paid, addressed return envelope

    • Neat, attractive layout

    • Send two follow-up reminders

    • Nothing during major holidays

    • No questions on back page

    • Include a small monetary inducement


Web surveys
Web Surveys

  • Advantages

    • Very fast

    • Inexpensive

    • Flexible design

    • Can use visuals or audio


Web surveys1
Web Surveys

  • Disadvantages

    • Coverage (75% have internet)

    • Multiple e-mail addresses

    • Self-selection

    • Protecting privacy and confidentiality

      • Encrypt

    • Complexity of design depends on software

      • Screen-by-screen questions

      • Progress indicator

      • Needs to be easy to move back and forth across questions


Phone interviews
Phone Interviews

  • Advantages

    • 95% coverage rate

    • Can interview about 1500 in a few days

    • Half the cost of face-to-face interviews

    • CATI


Phone interviews1
Phone Interviews

  • Disadvantages:

    • Higher cost

    • Limited interview length

    • Use of interviewer reduces anonymity and introduces potential interviewer bias


Face to face
Face-to-Face

  • Advantages:

    • Highest response rates

    • Permit the longest questionnaires

    • Can observe surroundings

    • Ask all types of questions


Face to face1
Face-to-Face

  • Disadvantages:

    • High cost

      • Training, travel, supervision, personnel costs

    • Interviewer bias


Comparing the methods
Comparing the Methods

  • Cost:

    • I, M, P, F2F

  • Speed:

    • I, P, F2F, M

  • Length:

    • F2F, M, I, P

  • Response rate:

    • F2F, P, I, M


Interviewers
Interviewers

  • Obtain cooperation and build rapport, but remain neutral and objective

  • Interviewer must treat everything in a matter-of-fact manner

  • Pleasant, honest, accurate, mature, responsible, moderately intelligent, stable, and motivated

  • Poise and tact


Interviewer bias
Interviewer Bias

  • Interview expectations can create significant bias

  • Social setting can affect answers

    • Students answer differently at home and at school

  • Visible characteristics can play a role…such as race and gender


CATI

  • Computer-assisted telephone interviewing

  • Read questions from the screen and enter answers immediately

  • Automatically makes calls for you

  • Automatically stores answers in a database


The ethical survey
The Ethical Survey

  • Invasion of privacy

    • That’s what ANY survey is!!!!

  • Voluntary participation

    • Informed consent

  • Use of push polls—AKA “pseudosurveys”

  • Sloppy reporting of surveys by the media

    • No quality control standards


Surveys1
Surveys

  • Five important factors influence the choice of survey type:

    • Cost

    • Completion rates

    • Sample-population congruence

    • Questionnaire length

    • Data-processing issues


Surveys2
Surveys

  • Cost:

    • All surveys share basic expenses for materials.

    • Cost of a survey can be quite variable, based on

      • Professional questionnaire design

      • Length of the questionnaire

      • Geographic dispersion of the sample

      • Callback procedures

      • Respondent selection rules

      • Availability of trained staff

      • Method-specific costs like long distance rates or postage


Surveys3
Surveys

  • Completion rates:

    • Low response rate on a survey limits the ability to make statistical inferences about the target population

    • Must make an effort to reach those selected for inclusion in the sample to maximize the representativeness of the sample…or risk bias


Surveys4
Surveys

  • Sample-population congruence:

    • If those who fail to participate in the survey share common characteristics, the sample will be wrought with systematic sampling error and the conclusions made with the data could be incorrect.


Surveys5
Surveys

  • Questionnaire length:

    • Keeping respondents’ attention during a survey helps achieve the best possible response quality.

    • A lengthy questionnaire can lead respondents to lose focus or speed through questions to finish sooner, resulting in low response quality.

    • Survey length ranges between three and four dozen items, depending on complexity and format.


Surveys6
Surveys

  • Data-processing issues

    • A good deal of time and effort are required to process data.

    • Surveys using more elaborate or complex designs or a large number of questions will require more time and effort.


Surveys7
Surveys

Each type of survey has advantages and disadvantages when it comes to the five factors.


ad