Survey research
This presentation is the property of its rightful owner.
Sponsored Links
1 / 63

Survey Research PowerPoint PPT Presentation


  • 40 Views
  • Uploaded on
  • Presentation posted in: General

Survey Research. Prying into Your Personal Life in Ways Telemarketers Could Never Dream Of. Appropriate Research Questions for Surveys. Self-reported beliefs and behaviors Ask things at one time, measure many variables, and test several hypotheses. Appropriate Research Questions for Surveys.

Download Presentation

Survey Research

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Survey research

Survey Research

Prying into Your Personal Life in Ways Telemarketers Could Never Dream Of


Appropriate research questions for surveys

Appropriate Research Questions for Surveys

  • Self-reported beliefs and behaviors

  • Ask things at one time, measure many variables, and test several hypotheses


Appropriate research questions for surveys1

Appropriate Research Questions for Surveys

  • Behavior

    • How often do you brush your teeth?

  • Attitudes/Beliefs/Opinions

    • What is the biggest problem facing the nation today?

  • Characteristics

    • Do you belong to a union?

  • Expectations

    • Do you plan to buy a new car in the next year?

  • Self-classification

    • Are you liberal or conservative?

  • Knowledge

    • Who was elected mayor in the last election?


Key limitation

Key Limitation

  • Surveys ONLY provide data on what a person or organization says

  • This can differ from what they actually do or think

  • Survey research is DEDUCTIVE


Steps in conducting a survey

Steps in Conducting a Survey

  • Step 1:

    • Develop hypotheses

    • Decide type of survey

    • Write survey questions

    • Decide on response categories

    • Design layout


Steps in conducting a survey1

Steps in Conducting a Survey

  • Step 2:

    • Plan how to record data

    • Pilot test survey instrument

  • Step 3:

    • Decide on target population

    • Get sampling frame

    • Decide on sample size

    • Select sample


Steps in conducting a survey2

Steps in Conducting a Survey

  • Step 4:

    • Locate respondents

    • Conduct interviews

    • Carefully record data

  • Step 5:

    • Enter data into computers

    • Recheck all data

    • Perform statistical analysis on data


Steps in conducting a survey3

Steps in Conducting a Survey

  • Step 6:

    • Describe methods and findings in research report

    • Present findings to others for critique and evaluation


Constructing the questionnaire

Constructing the Questionnaire

  • Introductory remarks and instructions

  • Keep it clear, keep it simple, keep respondent’s perspective in mind

  • Question writing is more of an art than a science

    • Skill, practice, patient, and creativity


Questionnaire hints

Questionnaire Hints

  • 1) Avoid jargon, slang, and abbreviations

    • No abbreviations

    • Only use this if you’re surveying a specialized population

    • “Did you drown in brew until you were totally blasted last night?”


Questionnaire hints1

Questionnaire Hints

  • 2) Avoid ambiguity, confusion, and vagueness

    • “What is your income?”

    • Confusion causes inconsistencies

  • 3) Avoid emotional language

    • Implicit connotative as well as explicit denotative

    • Use neutral language

    • “What do you think about a policy to pay murderous terrorists who threaten to steal the freedoms of peace-loving people?”


Questionnaire hints2

Questionnaire Hints

  • 4) Avoid prestige bias

    • Avoid associating statements with them

    • “Most doctors say that cigarette smoke causes lung disease for those near a smoker. Do you agree?”

  • 5) Avoid double-barreled questions

    • Each question = ONE QUESTION ONLY

    • “Does this company have pension and health insurance benefitted?”


Questionnaire hints3

Questionnaire Hints

  • 6) Do not confuse beliefs with reality

    • Don’t mix up respondents beliefs with what we want to measure

    • “Do you rate a teacher higher if the teacher tells many jokes?”

  • 7) Avoid leading questions

    • Don’t lead them to an answer

    • “You don’t smoke, do you?”


Questionnaire hints4

Questionnaire Hints

  • 8) Avoid asking questions that are beyond respondents’ capabilities

    • Frustrates respondents

    • Produces poor-quality responses

    • “Two years ago, how many hours did you watch TV every month?”


Questionnaire hints5

Questionnaire Hints

  • 9) Avoid false premises

    • Can’t put a premise that respondent could disagree with without risking alienating them

    • “When did you stop beating your girl/boyfriend?”


Questionnaire hints6

Questionnaire Hints

  • 10) Avoid asking about intentions in the distant future

    • Hypotheticals are bad

    • Not specific or concrete

    • “After you graduate college and have kids, will you invest in the stock market?”


Questionnaire hints7

Questionnaire Hints

  • 11) Avoid double negatives

    • Grammatically incorrect and confusing

    • “Do you disagree with those who do not want to build a new city swimming pool?”


Questionnaire hints8

Questionnaire Hints

  • 12) Avoid overlapping or unbalanced response categories

    • Mutually exclusive: categories do not overlap

    • Exhaustive: every respondent has a choice

    • Balanced: Offer polar opposites at each end of a continuum

    • “Did you find the service at our hotel to be, Outstanding, Excellent, Superior, or Good?”


Aiding recall

Aiding Recall

  • Ability to recall accurately declines over time

  • Memory is less trustworthy than once assumed

  • Give respondents special instructions and extra thinking time to guide them


Types of questions

Types of Questions

  • Threatening Questions

  • Socially Desirable Questions

  • Knowledge Questions

  • Skip or Contingency Questions


Threatening

Threatening

  • Sensitive issues…people don’t want to talk about them

  • Underreport socially undesirable answers

  • Context and wording are how to fix this

  • Issue examples:

    • Masturbation, sexual intercourse, illicit drug use, income


Socially desirable

Socially Desirable

  • Over report socially desirable answers (Social desirability bias)

    • Giving money to charity, voting etc.

  • Have to make the norm violation seem less objectionable then it really could be


Knowledge

Knowledge

  • We can’t answer elementary questions as Americans… Are You Smarter than a 5th Grader??

  • Pilot-test to make sure they’re at an appropriate level of difficulty


Skip or contingency

Skip or Contingency

  • Two or more part question

    • Answer to one part determines the next question you’ll receive

    • Sometimes called screening questions


Open v closed questions

Open v. Closed Questions

  • Open-ended: Respondent can give any answer they want

  • Close-ended: Gives respondents fixed answers from which to choose

  • Big surveys use close-ended…quicker and easier

  • Can mix both methods

  • Open-ended best in early or exploratory stages of research

  • Partially open questions leave final answer choice

  • as other


Advantages of closed

Advantages of Closed

  • Easy and quick to do

  • Easy to compare

  • Easier to code and analyze

  • Response choices can clarify meaning of question

  • Replication is easier


Disadvantages of closed

Disadvantages of Closed

  • Suggest ideas that respondent may not have thought of

  • No opinion people can answer anyways

  • Frustrating if desired answer is not a choice

  • Simplistic responses to complex ?s

  • Force us to make choices we wouldn’t make in the real world


Advantages of open

Advantages of Open

  • Unlimited responses

  • Answer in detail and clarify

  • Unanticipated feelings can be discovered

  • Creativity and self-expression

  • Reveal logic


Disadvantages of open

Disadvantages of Open

  • Different degrees of detail in answers

  • Can be irrelevant

  • Hard to compare/analyze

  • Coding is tough

  • Hard to write verbatim in interviews

  • More time necessary

  • Take up lots of space on questionnaire


Nonattitudes and middle positions

Nonattitudes and Middle Positions

  • Debate whether to include choices for middle, neutral, or nonattitudes

  • Two errors:

    • Accept nonattitude when respondent has one

    • Force choice when respondent doesn’t have one

  • Standard-format v. quasi-format v. full-format


Agree disagree rankings or ratings

Agree/Disagree, Rankings, or Ratings?

  • Agree/Disagree

  • Rankings

  • Ratings

  • Best to offer respondents explicit alternatives

  • Avoid the response set bias (just check agree to everything)

  • Respondents can rate several items equally high, but will put them in hierarchy if asked to


Word issues

Word Issues

  • Need to use simple vocabulary and grammar to minimize confusion

  • Need to watch for effects of specific words or phrases

    • Forbid v. not allow

    • Certain words trigger an emotional reaction

    • “To help the poor” v. “for welfare”


Length of survey

Length of Survey

  • Researchers prefer long because they’re cost effective

  • Phone: 5-20 minutes

  • Mail questionnaire: 3-4 pages

  • Face-to-face: Around an hour


Order effects

Order Effects

  • Strongest for people who lack strong views, less educated, elderly

  • May not perceive each element of a survey as isolated and separate

  • Influence through content and response


Context effects

Context Effects

  • Funnel sequence:

    • Start general and get more specific

  • Split sample

    • Half get questions in one order, other half different

  • More ambiguous the question, the more context effects matter


Survey layout

Survey Layout

  • Clear, neat, and easy to follow

  • Number questions

  • Identifying info

  • Cover sheet

  • Professional appearance

  • Provide instructions


Survey layout1

Survey Layout

  • Crucial in mail questionnaires

  • Use letterhead

  • Always thank them for participation

  • Circle response v. check box v. fill in dots v. put an X in a blank

  • List answer categories down, not across


Survey layout2

Survey Layout

  • Use arrows and instructions for contingency questions

  • Matrix questions

    • Compact way to present a series of questions with the same response categories

    • Box 7.5


Nonresponse

Nonresponse

  • If not enough people respond, we may not be able to generalize

  • Many people are burnt out with surveys these days

  • Improve eligibility rates through careful respondent selection


Nonresponse1

Nonresponse

  • Decrease refusals through sending letters before calling, small incentives, adjusting behavior, using alternative interviewers, using alternative interview methods

  • Initial contact is HUGE in determining likelihood for success


Surveys

Surveys

  • Survey research is research based on direct or indirect interview methods.

  • Many types of surveys:

    • In person (including interviews)

    • Mail

    • Telephone

    • E-mail and Internet

    • Group surveys


Mail and self administered questionnaires

Mail and Self-Administered Questionnaires

  • Advantages:

    • Cheaper

    • Wide geographical area

    • Completed when convenient

    • Respondent can check personal records

    • Anonymity


Mail and self administered questionnaires1

Mail and Self-Administered Questionnaires

  • Disadvantages

  • Low response rate

  • Can’t control conditions under which its filled out

  • Someone other than intended can fill out

  • Can’t observe reactions

  • Ill suited for those who don’t have good grasp of English


Mail and self administered questionnaires2

Mail and Self-Administered Questionnaires

  • How to Increase Response

    • Address to specific person

    • Careful cover letter on letterhead

    • Postage-paid, addressed return envelope

    • Neat, attractive layout

    • Send two follow-up reminders

    • Nothing during major holidays

    • No questions on back page

    • Include a small monetary inducement


Web surveys

Web Surveys

  • Advantages

    • Very fast

    • Inexpensive

    • Flexible design

    • Can use visuals or audio


Web surveys1

Web Surveys

  • Disadvantages

    • Coverage (75% have internet)

    • Multiple e-mail addresses

    • Self-selection

    • Protecting privacy and confidentiality

      • Encrypt

    • Complexity of design depends on software

      • Screen-by-screen questions

      • Progress indicator

      • Needs to be easy to move back and forth across questions


Phone interviews

Phone Interviews

  • Advantages

    • 95% coverage rate

    • Can interview about 1500 in a few days

    • Half the cost of face-to-face interviews

    • CATI


Phone interviews1

Phone Interviews

  • Disadvantages:

    • Higher cost

    • Limited interview length

    • Use of interviewer reduces anonymity and introduces potential interviewer bias


Face to face

Face-to-Face

  • Advantages:

    • Highest response rates

    • Permit the longest questionnaires

    • Can observe surroundings

    • Ask all types of questions


Face to face1

Face-to-Face

  • Disadvantages:

    • High cost

      • Training, travel, supervision, personnel costs

    • Interviewer bias


Comparing the methods

Comparing the Methods

  • Cost:

    • I, M, P, F2F

  • Speed:

    • I, P, F2F, M

  • Length:

    • F2F, M, I, P

  • Response rate:

    • F2F, P, I, M


Interviewers

Interviewers

  • Obtain cooperation and build rapport, but remain neutral and objective

  • Interviewer must treat everything in a matter-of-fact manner

  • Pleasant, honest, accurate, mature, responsible, moderately intelligent, stable, and motivated

  • Poise and tact


Interviewer bias

Interviewer Bias

  • Interview expectations can create significant bias

  • Social setting can affect answers

    • Students answer differently at home and at school

  • Visible characteristics can play a role…such as race and gender


Survey research

CATI

  • Computer-assisted telephone interviewing

  • Read questions from the screen and enter answers immediately

  • Automatically makes calls for you

  • Automatically stores answers in a database


The ethical survey

The Ethical Survey

  • Invasion of privacy

    • That’s what ANY survey is!!!!

  • Voluntary participation

    • Informed consent

  • Use of push polls—AKA “pseudosurveys”

  • Sloppy reporting of surveys by the media

    • No quality control standards


Surveys1

Surveys

  • Five important factors influence the choice of survey type:

    • Cost

    • Completion rates

    • Sample-population congruence

    • Questionnaire length

    • Data-processing issues


Surveys2

Surveys

  • Cost:

    • All surveys share basic expenses for materials.

    • Cost of a survey can be quite variable, based on

      • Professional questionnaire design

      • Length of the questionnaire

      • Geographic dispersion of the sample

      • Callback procedures

      • Respondent selection rules

      • Availability of trained staff

      • Method-specific costs like long distance rates or postage


Surveys3

Surveys

  • Completion rates:

    • Low response rate on a survey limits the ability to make statistical inferences about the target population

    • Must make an effort to reach those selected for inclusion in the sample to maximize the representativeness of the sample…or risk bias


Surveys4

Surveys

  • Sample-population congruence:

    • If those who fail to participate in the survey share common characteristics, the sample will be wrought with systematic sampling error and the conclusions made with the data could be incorrect.


Surveys5

Surveys

  • Questionnaire length:

    • Keeping respondents’ attention during a survey helps achieve the best possible response quality.

    • A lengthy questionnaire can lead respondents to lose focus or speed through questions to finish sooner, resulting in low response quality.

    • Survey length ranges between three and four dozen items, depending on complexity and format.


Surveys6

Surveys

  • Data-processing issues

    • A good deal of time and effort are required to process data.

    • Surveys using more elaborate or complex designs or a large number of questions will require more time and effort.


Surveys7

Surveys

Each type of survey has advantages and disadvantages when it comes to the five factors.


  • Login