surveys and questionnaires
Download
Skip this Video
Download Presentation
Surveys and Questionnaires

Loading in 2 Seconds...

play fullscreen
1 / 51

Surveys and Questionnaires - PowerPoint PPT Presentation


  • 163 Views
  • Uploaded on

Surveys and Questionnaires. Research Methods and Data College of Advancing Studies Brendan Rapple . Surveys Provide Important Knowledge. Economists, psychologists, health professionals, political scientists, and sociologists conduct surveys to study such topics as:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Surveys and Questionnaires' - tamyra


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
surveys and questionnaires
Surveys and Questionnaires

Research Methods and Data

College of Advancing Studies

Brendan Rapple

surveys provide important knowledge
Surveys Provide Important Knowledge
  • Economists, psychologists, health professionals, political scientists, and sociologists conduct surveys to study such topics as:
    • Income and expenditure patterns among households;
    • Voting behavior;
    • Effects on family life of women working outside the home, etc.
    • Auto manufacturers use surveys to find out how satisfied people are with their cars.
    • And a multitude of other topics

This presentation owes much to the American Statistical Association brochure series on survey research:

http://www.amstat.org/sections/srms/whatsurvey.html

specific purpose essential
Specific Purpose Essential
  • Objectives of a survey should be as
      • Specific
      • Clear-cut
      • Unambiguous as possible
slide4
"Men's Health Practices" is a very nebulous topic.
  • Better:

How often do African-American males aged 40-49 visit the dentist?

Or

A survey of 50-60 year old male professors at BC about their weekly exercise habits

steps in conducting a survey
Steps in Conducting A Survey
  • Define precise purpose
  • Specify population
  • Specify appropriate sample
  • How to administer survey?
  • Draft of survey instrument
  • Pretest it
  • Revise it
  • Administer survey
  • Analyze, write it up, and communicate the results
  • Use results meaningfully
decide on mode of data collection
Decide on Mode of Data Collection
  • Mailed Questionnaire
  • Telephone
  • In Person Interview
  • Computer Questionnaire
use mailed computer questionnaires or not
Use Mailed/Computer Questionnaires or Not?
  • Perhaps “better” responses by phone -- but phone interviewing is very time-consuming.
  • On other hand, people tend to be more truthful with anonymous questionnaires.
  • Generally cheaper than one-on-one interviews.
  • Perhaps a mix of questionnaire and interviewing?
for all surveys it is essential to
For All Surveys it is Essential to
  • explain purpose of survey very clearly and precisely.
  • explain any potential use the results will have for the respondents themselves.
  • stress voluntary nature - respondents are doing a favor to the researcher.
  • use letter-head – official – for a mailed/computer questionnaire.
  • explicitly promise confidentiality.
  • mention a clear expression of thanks.
whatever format used
Whatever Format Used . . . . . .
  • Important to specify a deadline for response.
  • Follow-up letter (e-mail) often advisable for a mailed/computer questionnaire
  • Always essential for PILOT STUDY or PRE-TEST

Main problems revolve about:

      • Question content, e.g. confusion with overall meaning of question as well as misinterpretation of individual terms or concepts
      • Formatting, e.g. problems with how to skip or navigate from question to question may result in missing data and frustration for both interviewers and respondents.
population sample
Population → Sample
  • Usually the population to be surveyed is too large.
  • Accordingly, one must select a smaller, representative sample.
  • “This sample is usually just a fraction of the population being studied.”
samples
Samples
  • The quality of the sample – whether it is up-to-date and complete – is probably the dominant feature for ensuring adequate coverage of the desired population to be surveyed.
    • Must be representative of population.
    • Are the distributions of attributes, opinions, and beliefs in the sample the same as in the population?
    • You want to be able to make inferences about the population as a whole based on what you find to be true of the sample.
variability
Variability
  • Variability is large, then sample should be large
  • Converse also true
2 barrels of apples
2 Barrels of Apples
  • Barrel A (low variability) -- all apples about 5 ins. in diameter (range 5.1 to 4.8 ins.)
  • Barrel B (high variability) -- apples range from 2 to 6 ins. in diameter
  • Picking 3 apples from Barrel B might give result well below (above) average.
still size of sample isn t everything
Still, Size of Sample Isn't Everything
  • Large numbers do not, in and of themselves, increase the representativeness of a sample.
  • Most professional survey conductors hold that a moderate sample size is enough statistically and operationally.
representative sample
Representative Sample
  • Survey:Success of unwed teenage mothers in a specific community in raising children?
      • To be representative, sample must contain sameproportion of unwed teenage mothers at --each age level --each educational level --each socio-economic statusin the community
population sample16
Population -- Sample
  • It is essential that you select sample in such a way that every name on the population list has an equal chance of being included in the sample.
random sample
Random Sample

Example: 500 part-time students in Advancing Studies

  • Sample of 20% is required
  • Assign each student a number from 1 to 500
  • Randomly select 100 numbers
systematic random sampling
Systematic Random Sampling

Example 1.

  • 2,000 in population and you want a sample of 200, then you might select every 10th name

Example 2.

  • 500 part-time students in Advancing Studies
  • Sample of 20% is required --Randomly Select a Number from 1 to 5 --Select Every 5th Person --002, 007, 012. 017, 022, and up to 497.
possible problem
Possible Problem
  • Staff in govt. agency may be listed unit by unit
  • Each unit has 9 line-level workers and 1 supervisor.
  • The supervisor is the 10th person on the list.
  • It’s a survey of 20% -- every 5th person is selected.
  • If first no. selected is 1, 2, 3, or 4 then no supervisor will be selected, though they comprise 10% of population.
  • If first number selected is 5, then supervisors will be greatly overrepresented.Thus, possibility of bias due to periodicity or patterns.
stratified sampling
Stratified Sampling

Population: 2,000 (800 females; 1,200 males)Sample required: 200If gender is an important variable in your survey, then both females and males should be included in appropriate numbers, that is, in proportions that correspond to their presence in the population.

strategy
Strategy:

Treat both sexes as separate populations and take 10% sample from each. ORMake sure that all females are listed first and then take every tenth name.Either way you will end up with 80 females and 120 males

convenience sampling
Convenience Sampling

Could ruin an otherwise well-conceived survey.

It’s simple and cheap to select a sample of names from a phone-directory to find out which candidate people intend to vote for.

However, this sampling procedure could give incorrect results since persons without telephones or with unlisted numbers would have no chance to be reflected in the sample,

Their voting preferences might be quite different from persons who have listed telephones.

confidentiality
Confidentiality
  • Confidentiality of data supplied by respondents is of prime concern to all reputable survey organizations.
  • Important that individual respondents are not identified in reporting survey findings.
  • All of the survey’s results should be presented in totally anonymous summaries, such as statistical tables and charts.
problems with volunteers
Problems with Volunteers

Example:

TV programs asking viewers to vote.

  • people call who are most committed to issue.
  • “stuffing of ballots” by multiple calls.
  • Time of day is important – who’s available?
margin of error
Margin of Error
  • Error margin of 1,000 randomly chosen individuals is said to be 3.1%.
  • Thus, if a random sample of 1,000 indicates that 59% will vote for Obama, the actual number could range from 55.9% to 62.1%.
questions in a questionnaire should be
Questions in a Questionnaire Should Be
  • Woven together
  • Flow smoothly
  • Avoid confusion

A good questionnaire forms an integrated whole.

different understandings
Different Understandings
  • Everyone should see/understand the exact same question – no ambiguity.
  • But people from different backgrounds, with diverse frames of reference, may have different perceptions of the same question.
kiss principle k eep i t s imple s tatistician
KISS PrincipleKeep It Simple, Statistician
  • Questions should be, as far as possible:
    • Simple
    • Clear,
    • Easy to answer
    • Personally relevant to them
  • Often recommended that questionnaires be written at the 5th grade reading level
avoid ambiguity
Avoid Ambiguity
  • Do you favor governmental involvement in health care?
  • “What is your income?” __________________

do you mean: weekly monthly annual pretax after tax from salary or from all sources

another example
Another Example:

(A blouse manufacturer wants to ascertain what type of sleeves teenage females prefer in their blouses)1. Do you like short sleeve blouses? YES__ NO__2. Do you like long sleeve blouses? YES__ NO__3. Do you like sleeveless blouses? YES__ NO__

another example31
Another Example
  • “Do you jog regularly?”Problem, of course, lies with REGULARLY
avoid jargon slang abbreviations
Avoid jargon, slang, abbreviations
  • Plumbers talk about “snakes”
  • Psychologists about “oedipus complex”
  • Lawyers about “mens rea”
nato usually means north atlantic treaty organization
“NATO” usually means North Atlantic Treaty Organization

But some respondents might take it to mean:--National Auto Tourist Organization

--Native Alaskan Trade Orbit

--North African Tea Office

types of questions
Types of Questions
  • Open-Ended Questions:
        • What is your age?
        • What is the total turnover in your company?
        • Which of the four seasons do you prefer?
        • How would you spend a a $1,000,000 lottery win?
disadvantages of open ended questions
Disadvantages of Open-ended Questions:
  • Variation in answers make coding/scoring difficult
advantages of open ended questions
Advantages of Open-ended Questions
  • Do not impose researcher's opinion on respondent
  • Can lead to a very precise answer
forced choice questions
Forced-Choice Questions
  • How many books do you read each year? Please check as appropriate below: ___ none ___ 1 to 5 ___ 6 to 10 ___ 11 to 20 ___ more than 20
slide38
What do you recall about the frequency of snowfall in Boston during the winter of 1994? ___ It snowed almost every day ___ It snowed about once a week ___ It snowed about once every two weeks ___ It snowed about once a month
importance of wording
Importance of Wording
  • Take a very simple question:

How many drinks do you have each day? (Check one of the following)

____ 5 or more ____ 4 ____ 3 ____ 2 ____ 1 ____ none

better to ask
Better to Ask
  • Are your daily drinking habits reasonably consistent --i.e. do you take about the same number of alcoholic drinks each day?

_____ YES _____ NO (if you mark "NO," skip the following question).

checking respondents consistency
Checking Respondents' Consistency

Q. 4. Check one of the following: _x_ I believe that manufacturing should be increasingly computerized, even if layoffs ensue.

___ Preserving the jobs of workers is more important than computerizing manufacturing.Later on, the Questionnaire might ask:

Q. 30. Check one of the following: ___ I support G.M.'s increasing use of robotics, despite the resulting massive layoffs.

_x_ The government should force G.M. to curtail computerization to ensure a reduction in layoffs.

avoid emotional language
Avoid Emotional Language
  • “What do you think about a policy to pay murderous terrorists who threaten to steal the freedoms of peace-loving people?”Problematic Words:murderous freedoms steal peace
avoid loaded questions
Avoid Loaded Questions:
  • "Should the mayor spend even more tax money trying to keep the streets in top shape?"
  • "Should the mayor fix the pot-holed and dangerous streets in our city?"
slide44
Arrangement Is Important

What is your present marital status?

1 never married 2 married 3 divorced 4 separated 5 widowed

a better arrangement would be
A Better Arrangement Would Be:
  • What is your present marital status? (circle number) 1. never married 2. married 3. divorced 4. separated 5. widowed
danger of overlapping
Danger of Overlapping
  • Check How Many Children are in Your Family:

a. 0-1 children

b. 1-2 children

c. 2 or more children

avoid prestige bias
Avoid Prestige Bias
  • “Most doctors say that cigarette smoke causes lung disease for those near a smoker”. Do you agree?”
  • “Do you support the president’s policy regarding Zanozui?”
avoid double barreled questions
Avoid Double-Barreled Questions:
  • "Does your company have pension and health insurance benefits?"
ranking answers likert scale format
Ranking Answers (Likert Scale Format):

Women should automatically receive three months maternity leave in your company

strongly agree partly agree disagree strongly

agree disagree

___ ___ ___ ___ ___

how often do you attend church
How often do you attend church?

Never Yearly Monthly Weekly Daily

____ ___ ___ ___ ___

visual analog scales bipolar scales
Visual Analog Scales BIPOLAR scales

I see myself as:Extremely Extremely interested in uninterested politics ______:______:______:______:______ in politics

ad