Questionnaire construction
1 / 19

Questionnaire Construction - PowerPoint PPT Presentation

  • Uploaded on

Questionnaire Construction. January 28 & 31, 2011. Objectives. By the end of this meeting, participants should be able to: Distinguish open and closed-ended questions. Evaluate questions for their validity and reliability, as well as how well they follow basic rules of question wording.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Questionnaire Construction' - rina-guthrie

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Questionnaire construction

Questionnaire Construction

January 28 & 31, 2011


By the end of this meeting, participants should be able to:

  • Distinguish open and closed-ended questions.

  • Evaluate questions for their validity and reliability, as well as how well they follow basic rules of question wording.

Florida datamar democratic polling
Florida Datamar Democratic Polling

Likely Democratic Voters

Clinton 40%

Obama 28%

Edwards 19%

Kucinich 2%

Richardson 2%

The vital information fl
The Vital Information- FL

  • Sample size 541 +/- 4.2 percent sampling error

  • January 5 – 7, 2008

  • Methodology: Findings are from a January 5 – 7, 2008 survey using a comprehensive predictive model of “likely” voters, based on election cycles and other factors of voters in Florida. The survey was conducted using an automated telephone dialer and the voice of a professional announcer.

  • The targeted-registration-based-sampling (TRBS)* selection criteria were based on election cycles and other voter factors in Florida. Datamar proprietary algorithms were used to generate random samples from the target group for calling.

  • Where necessary, responses were weighted according to age, gender, and political party.

The vital information fl1
The Vital Information- FL

Q18. If the primary election were held today, for whom would you vote? The candidates are: Joe Biden, U.S. Senator from Delaware; Chris Dodd, U.S. Senator from Connecticut; Bill Richardson, Governor of New Mexico; Mike Gravel, former U.S. Senator from Virginia; Dennis Kucinich, Congressman from Ohio; John Edwards, former U.S. Senator from North Carolina; Barack Obama, U.S. Senator from Illinois; Hillary Clinton, U.S. Senator from New York.

Research 2000 fl d
Research 2000 FL-D

Democratic Primary Voters

Clinton 50%

Obama 28%

Edwards 13%

Others 4%

Undecided 5%

Vital information florida
Vital Information- Florida

Research 2000

The Research 2000 Florida poll was conducted from January 14th through January 16th, 2008. There were a total of 500 likely Democratic and 500 Republican primary voters as well as 500 statewide primary voters interviewed by telephone. The margin of error is 4.5%.

If the Democratic Primary for President were held today, which of the following candidates would you vote for? (Rotated)

Types of questions
Types of Questions

  • Open-ended questions

    • What do you like about the Democratic Party?

    • What do you like about the Republican Party?

  • Closed-ended questions

    • Multiple choice questions

    • Rating scale/ Feeling thermometer

  • Disadvantages/ Advantages of Each Approach

Rating scales
Rating Scales

Common political types

  • Rating scales (generally seven or six points from very positive to very negative)

  • Thermometers (generally 1 to 100)

  • Semantic differential (very good-very bad)

  • Seven point scale (two concepts at each end)

Good standard practice
Good Standard Practice

  • Use common questions

  • Use short questions with standard speech

  • Think of questions as a stimulus that the respondent will react to

  • Use a battery of questions for more difficult concepts

  • Tailor question wording to audience

Things to consider
Things to Consider

  • Branching format (particularly for scales)

  • Response choice order

  • Ambiguous wording

  • Biased questions

  • Social desirability bias

  • Double barreled questions

  • Double negatives

  • No opinion/ don’t know

Evaluating questions
Evaluating Questions

  • Reliability

  • Validity

    • Face validity

    • Convergent validity (similar answers)

    • Divergent validity (different answers)

    • Criterion validity (direct measure)

    • Content validity (all parts of a concept)

    • Construct validity (consistent with previous work)

Constructing a questionnaire
Constructing a Questionnaire

  • Topic Order

  • Question order/ Consistency bias

  • Response choices/ Response set

  • Number of questions

    • half samples

    • rotating questions

  • Questionnaire layout

  • Pretesting the questionnaire

    • cognitive think-aloud

Example sensitive questions
Example: Sensitive Questions

  • Suppose UGA wanted to survey student behavior on campus.

  • Specifically, the administration wants an accurate picture of what percent of students partake in unscrupulous activities.

  • How can the university ensure it gets accurate results?

Example sensitive questions1
Example: Sensitive Questions

Innocuous Question Technique

  • Flip a coin. If you get heads, answer question 1. If you get tails, answer question 2. Do not tell me what you get or what question you answer.

    1) I have had unprotected sex during the last month: Yes / No

    2) I got tails (on the coin flip): Yes / No

Example sensitive questions2
Example: Sensitive Questions

  • Proportion of “Yes” responses =(Yes to question 1) + (Yes to question2)/Total # of respondents =

  • Yes to question 1/Total # of respondents + Yes to question2/ Total # of respondents=

  • θP + (1 − θ)δ, where

    θ=probability of answering question 1

    P= proportion who had unprotected sex in the past month

    (1 − θ)= probability of responding to question 2

    δ= the probability of getting tails (yes: Q.2)

Random half sample
Random Half Sample

  • Divide survey sample in half (randomly).

  • Half of respondents get column 1 the other half get column 2.

For january 31
For January 31

  • Read WKB chapter 4.

  • Answer the following questions:

    • Imagine we conducted a survey of 350 UGA students and found that 93% thought the Bulldogs would win the SEC championship. What is the margin of error (at 95%)? Based on that margin of error, we can be 95% sure that championship predictions lie in what range?

    • Imagine we conducted a survey of GA voters and found that Obama had approval of 55%. 750 people were surveyed, what is the margin of error (at 95%)? Based on that margin of error, we can be 95% sure that Obama’s support in GA lies in what range?

For february 2
For February 2

  • Read WKB chapter 5

  • Answer questions 1, 2, & 3 on page 130.