Questionnaire construction
1 / 19

Questionnaire Construction - PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Questionnaire Construction. January 28 & 31, 2011. Objectives. By the end of this meeting, participants should be able to: Distinguish open and closed-ended questions. Evaluate questions for their validity and reliability, as well as how well they follow basic rules of question wording.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Questionnaire Construction

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Questionnaire Construction

January 28 & 31, 2011


By the end of this meeting, participants should be able to:

  • Distinguish open and closed-ended questions.

  • Evaluate questions for their validity and reliability, as well as how well they follow basic rules of question wording.

Florida Datamar Democratic Polling

Likely Democratic Voters






The Vital Information- FL

  • Sample size 541 +/- 4.2 percent sampling error

  • January 5 – 7, 2008

  • Methodology: Findings are from a January 5 – 7, 2008 survey using a comprehensive predictive model of “likely” voters, based on election cycles and other factors of voters in Florida. The survey was conducted using an automated telephone dialer and the voice of a professional announcer.

  • The targeted-registration-based-sampling (TRBS)* selection criteria were based on election cycles and other voter factors in Florida. Datamar proprietary algorithms were used to generate random samples from the target group for calling.

  • Where necessary, responses were weighted according to age, gender, and political party.

The Vital Information- FL

Q18. If the primary election were held today, for whom would you vote? The candidates are: Joe Biden, U.S. Senator from Delaware; Chris Dodd, U.S. Senator from Connecticut; Bill Richardson, Governor of New Mexico; Mike Gravel, former U.S. Senator from Virginia; Dennis Kucinich, Congressman from Ohio; John Edwards, former U.S. Senator from North Carolina; Barack Obama, U.S. Senator from Illinois; Hillary Clinton, U.S. Senator from New York.

Research 2000 FL-D

Democratic Primary Voters

Clinton 50%





Vital Information- Florida

Research 2000

The Research 2000 Florida poll was conducted from January 14th through January 16th, 2008. There were a total of 500 likely Democratic and 500 Republican primary voters as well as 500 statewide primary voters interviewed by telephone. The margin of error is 4.5%.

If the Democratic Primary for President were held today, which of the following candidates would you vote for? (Rotated)

Types of Questions

  • Open-ended questions

    • What do you like about the Democratic Party?

    • What do you like about the Republican Party?

  • Closed-ended questions

    • Multiple choice questions

    • Rating scale/ Feeling thermometer

  • Disadvantages/ Advantages of Each Approach

Rating Scales

Common political types

  • Rating scales (generally seven or six points from very positive to very negative)

  • Thermometers (generally 1 to 100)

  • Semantic differential (very good-very bad)

  • Seven point scale (two concepts at each end)

Good Standard Practice

  • Use common questions

  • Use short questions with standard speech

  • Think of questions as a stimulus that the respondent will react to

  • Use a battery of questions for more difficult concepts

  • Tailor question wording to audience

Things to Consider

  • Branching format (particularly for scales)

  • Response choice order

  • Ambiguous wording

  • Biased questions

  • Social desirability bias

  • Double barreled questions

  • Double negatives

  • No opinion/ don’t know

Evaluating Questions

  • Reliability

  • Validity

    • Face validity

    • Convergent validity (similar answers)

    • Divergent validity (different answers)

    • Criterion validity (direct measure)

    • Content validity (all parts of a concept)

    • Construct validity (consistent with previous work)

Constructing a Questionnaire

  • Topic Order

  • Question order/ Consistency bias

  • Response choices/ Response set

  • Number of questions

    • half samples

    • rotating questions

  • Questionnaire layout

  • Pretesting the questionnaire

    • cognitive think-aloud

Example: Sensitive Questions

  • Suppose UGA wanted to survey student behavior on campus.

  • Specifically, the administration wants an accurate picture of what percent of students partake in unscrupulous activities.

  • How can the university ensure it gets accurate results?

Example: Sensitive Questions

Innocuous Question Technique

  • Flip a coin. If you get heads, answer question 1. If you get tails, answer question 2. Do not tell me what you get or what question you answer.

    1) I have had unprotected sex during the last month: Yes / No

    2) I got tails (on the coin flip): Yes / No

Example: Sensitive Questions

  • Proportion of “Yes” responses =(Yes to question 1) + (Yes to question2)/Total # of respondents =

  • Yes to question 1/Total # of respondents + Yes to question2/ Total # of respondents=

  • θP + (1 − θ)δ, where

    θ=probability of answering question 1

    P= proportion who had unprotected sex in the past month

    (1 − θ)= probability of responding to question 2

    δ= the probability of getting tails (yes: Q.2)

Random Half Sample

  • Divide survey sample in half (randomly).

  • Half of respondents get column 1 the other half get column 2.

For January 31

  • Read WKB chapter 4.

  • Answer the following questions:

    • Imagine we conducted a survey of 350 UGA students and found that 93% thought the Bulldogs would win the SEC championship. What is the margin of error (at 95%)? Based on that margin of error, we can be 95% sure that championship predictions lie in what range?

    • Imagine we conducted a survey of GA voters and found that Obama had approval of 55%. 750 people were surveyed, what is the margin of error (at 95%)? Based on that margin of error, we can be 95% sure that Obama’s support in GA lies in what range?

For February 2

  • Read WKB chapter 5

  • Answer questions 1, 2, & 3 on page 130.

  • Login