Q UESTIONNAIRE D ESIGN. O ften misused words generate misleading thoughts Herbert Spencer. The STOCK / LIVESTOCK Case The range of error due to use of imprecise words may be as high as 20 to 30 percent. Relevance and Accuracy. T he S teps in a S urvey P roject.
TheSTOCK / LIVESTOCKCase
The range of error due to
use of imprecise words
may be as high as 20 to 30
Relevance and Accuracy
Establish the goals of the project
Determine your sample - Whom to interview
Choose interviewing methodology - How to interview
Create your questionnaire
Choice of survey method will depend on several
factors : Speed, Cost, Internet Usage, Literacy Levels, Sensitive Questions
Researchers use three basic types of questions:multiple
choice, numeric open end and text open end. Examples of
each kind of question follow:
It’s all about the questions…
The Effect of One Word
How important do you think it is for the GOVERNMENT to set up a hotline telephone number people can call to get help when they have problems with their health plan?
How important do you think it is
to have a hotline telephone number people can call to get help when they have problems with their health plan?
To have a hotline
For government to set up a hotline
Not too important
Not at all important
Kaiser Family Foundation/Harvard University National Survey of Americans’ Views on Consumer Protections in managed Care, January 1998
Balance in Question Wording
Balance in Question Wording
Don’t Know Answers – offered or volunteered?
Do you favor or oppose the government offering parents money or "vouchers" to send their children to private or religious schools, or public schools outside their district?
Do you favor or oppose the government offering parents money or "vouchers" to send their children to private or religious schools, or public schools outside their district, or haven’t you heard enough about that to have an opinion?
NPR/Kaiser Family Foundation/Kennedy School of Government Education Survey, June 1999 (conducted June 25 – July 19, 1999)
Using scales – getting to “the most”
Now, I’m going to read you some different health care issues. As I read each one, please tell me how important you think it is for the President and Congress to deal with this issue – very important, somewhat important, not too important, or not important at all.
When forced to choose…
Which of the following issues you say are very important do you think is most important?*
Percent saying “most important”
Percent saying “very important”
Helping people age 65 and over pay for prescription medicines
Making Medicare more financially sound for future generations
Increasing the number of Americans covered by health insurance
Helping states with the cost of Medicaid
*Also includes those who named only one issue as “very important”
Lowering the cost of medical malpractice insurance for physicians
Source: Kaiser Family Foundation Health Poll Report
How questions can affect one another
The last step in questionnaire design is to test a questionnaire with a small number of interviews before conducting your main interviews. Ideally, you should test the survey on the same kinds of people you will include in the main study. This kind of test run can reveal unanticipated problems with question wording, instructions to skip questions, etc. It can help see if the interviewees understand your questions and giving useful answers.
If you change any questions after a pre-test, you should not combine the results from the pre-test with the results of post-test interviews.
Roper Center for Public Opinion Research: iPoll http://roperweb.ropercenter.uconn.edu/iPOLL/
American Association for Public Opinion Research http://aapor.org/
National Journal: Poll Track http://nationaljournal.com/
Kaiser Family Foundation: Kaiser Health Poll Report http://www.kff.org/healthpollreport
Strategos Institute at http://www.stategos.com/survey/
Zikmund G. William, Business Research Methods 7E, Chapter 15, Pg: 329 – 331, 344 – 347.