chapter 11 l.
Skip this Video
Loading SlideShow in 5 Seconds..
CHAPTER 11 PowerPoint Presentation
Download Presentation

Loading in 2 Seconds...

play fullscreen
1 / 30

CHAPTER 11 - PowerPoint PPT Presentation

  • Uploaded on

CHAPTER 11. DESIGNING DATA COLLECTION FORMS. The Questionnaire Development Process. Implementation. Determine Data Collection Methods. Determine Question Response Format. Prepare Final Copy. Determine Survey Objectives and Constraints. Decide Question Wording. Pretest and Revise.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
chapter 11



the questionnaire development process
The Questionnaire Development Process


Determine Data

Collection Methods

Determine Question

Response Format

Prepare Final Copy

Determine Survey

Objectives and


Decide Question


Pretest and Revise

Obtain Approval

from all Relevant



Questionnaire Flow

and Layout

Evaluate the

Questionnaire and


determine objectives resources and constraints
Determine Objectives, Resources, and Constraints
  • Survey (information) objectives should be spelled out as clearly and precisely as possible.
  • If this step is completed in a thorough fashion, the rest of the process will follow more smoothly and efficiently.
determine the data collection method
Determine the Data Collection Method
  • Chapter 9 discussed the variety of ways that survey data can be gathered, such as in-person, telephone, mail, internet or self-administration.
  • Each method will have an impact on survey design.
determine the question response format
Determine the Question Response Format
  • Open-Ended Questions
    • Open-ended questions are those in which the respondent can reply in his or her own words.
  • Closed-Ended Questions
    • A closed-ended question is one that requires the respondent to make a selection from a list of options.
    • The primary advantage of closed-ended questions is simply the avoidance of many of the problems (such as subjectivity) of open-ended questions.


determine the question response format cont
Determine the Question Response Format(cont.)
  • Dichotomous Questions
    • This simplest form of a closed-ended question is the dichotomous choice.
      • Example of a dichotomous question:
  • Would you rather live in a small town or a city?
  • A. Small Town 1
  • B. City 2
determine the question response format cont7
Determine the Question Response Format(cont.)
  • Multiple-Choice Questions
    • Multiple-Choice Questions permit the respondent to select from several alternatives.
      • Example of a multiple-choice question:
  • Who do you think will win the World Series?
  • A. New York Yankees
  • B. St. Louis Cardinals
  • C. Los Angeles Dodgers
  • D. Atlanta Braves
determine the question response format cont8
Determine the Question Response Format (cont.)
  • Scaled-Response Questions:
    • Question format that permits the measurement of the “intensity” of a respondents’ answers.
      • Example of a scaled-response question:
  • . Now that you have read a sample of the Wall Street
  • Journal, would you say that you would…(Check one):
  • Definitely buy it
  • Probably buy it
  • Might or might not buy it
  • Probably will not buy it
  • Definitely will not buy it
decide the question wording
Decide the Question Wording
  • Four Guidelines in Selecting Wording
    • The wording must be clear.
    • Select words so as to avoid biasing the respondent.
    • Consider the ability of the respondent to answer the question.
    • Consider the willingness of the respondent to answer the question.
establish questionnaire flow and layout
Establish Questionnaire Flow and Layout
  • After the questions have been properly formulated, the next step is to sequence them and develop a layout for the questionnaire.
  • The general guidelines concerning questionnaire flow are included in the next slide.
how a questionnaire should be organized
How a Questionnaire Should be Organized

Use the screener

questions to

identify qualified


After obtaining a

qualified respondent,

begin with a question

that obtains a

respondent’s interest-

Warm-up questions

Ask general

questions first.

Use skip questions

If it is necessary

Ask questions that

require “work” in

the middle of the


Transition questions

Insert “prompters”

at strategic points

Position sensitive,

threatening, and

Classification or


questions at the


evaluate the questionnaire
Evaluate the Questionnaire
  • Issues to Consider in Reviewing the Design of the Questionnaire:
    • Is the Question Necessary?
    • Is the Questionnaire Too Long?
    • Will the Questions Provide the Desired Information to Accomplish the Research Objectives?
    • Appearances of Mail and Self-Administered Questionnaires
    • Avoid a Cluttered Look
    • Allow Plenty of Space for Open-Ended Responses
    • Consider Color-Coding the Questionnaires
    • Instructions Printed Within The Questionnaire Should be in Capital Letters
obtain approval of all relevant parties
Obtain Approval of AllRelevant Parties
  • At this point in the questionnaire design process, the first draft of the questionnaire has been completed.
  • Approval for going forward with the questionnaire is the objective of this step.
pretest and revise
Pretest and Revise
  • When final managerial approval has been obtained, the questionnaire must be pre-tested.
  • The primary purpose of a pretest is to make certain that the questionnaire gives the respondent clear, understandable questions that will evoke clear, understandable responses.
prepare final copy
Prepare Final Copy
  • Even the final copy phase does not allow the researcher to relax.
    • Precise typing instructions, spacing, numbering, and pre-coding must be set up, monitored, and proofread.
implementing the survey
Implementing the Survey
  • Supervisor’s Instructions:
    • Inform them of the nature of the study, start and completion dates, quotas, reporting times, equipment and facility requirements, sampling instructions, number of interviewers required, and validation procedures.
    • Interviewer’s Instructions
    • Cover many of the same points as supervisor’s instructions but are geared to the actual interview.
implementing the survey cont
Implementing the Survey (cont.)
  • Call Record Sheets
    • Interviewers’ logs listing the number and results of a contact.
    • Call record sheets are used to measure the efficiency of the interviewers.
  • Visual Aids and Other Supplements
    • Many studies use visual aids to facilitate the interviewing process.
implementing the survey cont18
Implementing the Survey(cont.)
  • Field Management Companies
    • Field management companies such as Qfact, On-Line Communications, and Direct Resource generally provide questionnaire formatting, screener writing, development of instructional and peripheral materials, shipping departments, field auditing, and all coordination of data collection, coding, and tab services required for the project.
questionnaire challenges
Questionnaire Challenges

The Difficulty of Finding a Qualified Respondent in a Central Location Telephone Interview

1. Failed Attempts

- Busy

- No Answer

- Answering Machine

- Business Number

- Phone/Language Problem

- Discontinued Line

2. Cooperation Problems

- Respondent Not at Home

- Refused to be Interviewed

questionnaire challenges20
Questionnaire Challenges

The Difficulty of Finding a Qualified Respondent in a Central Location Telephone Interview

3. Screener Determines Respondent Not Eligible

- Failed security test (works for marketing research firm,

advertising agency, or the client)

- Doesn’t use the product

- Demographic disqualification (wrong gender, age, etc.)

- Quota filled (survey has quota of 500 users of Tide and 500

users of other clothing washing powder. Interviewer already

has 500 Tide users; the current respondent uses Tide.

4. Respondent Terminates During Interview

5. Completed interview


Questionnaire Design Checklist

Step 1. Specify The Information Needed

Step 2. Type of Interviewing Method

Step 3. Individual Question Content

Step 4. Overcome Inability and Unwillingness to Answer

Step 5. Choose Question Structure

Step 6. Choose Question Wording

Step 7. Determine the Order of Questions

Step 8. Form and Layout

Step 9. Reproduce the Questionnaire

Step 10. Pretest


Questionnaire Design Checklist

  • Step 1. Specify The Information Needed:
    • Ensure that the information obtained fully addresses all the components of the problem.
    • Review components of the problem and the approach, particularly the research questions, hypotheses, and characteristics that influence the research design.
  • Step 2. Individual Question Content:
    • Is the question necessary?
    • Are several questions needed instead of one to obtain the required information in an unambiguous manner?
    • Do not use double-barreled questions
    • Question should be focused on a single issue.
    • Question should be brief.
    • Question should be interpreted the same way by all responded.
    • Question should use the respondent’s core vocabulary.

Step 3. Overcoming Inability and Unwillingness to Answer:

    • Is the respondent informed?
    • If respondents are not likely to be informed, filter questions that measure familiarity, product use, and past experience should be asked before questions about the topics themselves.
    • Can the respondent remember?
    • Avoid errors of omission, telescoping and creation.
    • Questions which do not provide the respondent with cues can underestimate the actual occurrence of an event.
    • Can the respondent articulate?

Step 3 Overcoming Inability and Unwillingness to Answer (cont.):

    • Minimize the effort required of the respondents.
    • Is the context in which the questions are asked appropriate?
    • Make the request for information seem legitimate.
    • If the information is sensitive:
      • Place sensitive topics at the end of the questionnaire.
      • Preface the question with a statement that the behavior of interest is common.
      • Ask the question using the third-person technique.
      • Hide the question in a group of other questions which respondents are willing to answer.
      • Provide response categories rather than asking for specific figures.

Step 4 What Should Not be in Question Wording:

    • Question should not assume criteria that are not obvious.
    • Question should not be beyond the respondent’s ability or experiences.
    • Question should not use a specific example to represent a general case.
    • Question should not ask the respondent to recall specific when only generalist will be remembered.
    • Question should not require the respondent to guess a generalization.
    • Question should not ask for details that cannot be related.
    • Question should not use words that overstate the condition.
    • Question should not ambiguous wording.
    • Question should not be double barreled.
    • Question should not lead the respondent to particular answer.
    • Question should not have “loaded” wording or phrasing.

Step 5. Choosing Question Structure:

    • Open-ended questions are useful in exploratory research and as opening questions.
    • Use structured questions whenever possible.
    • In multiple-choice questions, the response alternatives should include the set of all possible choices and should be mutually exclusive.
    • In a dichotomous question, if a substantial proportion of the respondents can be expected to be neutral, include a neutral alternative.
    • Consider the use of the split ballot technique to reduce order bias in dichotomous and multiple-choice questions.
    • If the response alternatives are numerous, consider using more than one question to reduce the information processing demands on the respondents.

Step 6. Choosing Question Wording:

    • Define the issue in terms of who, what, when, where, why, and way (the six Ws).
    • Use ordinary words. Words should match the vocabulary level of the respondents.
    • Avoid ambiguous words: usually, normally, frequently, often, regularly, occasionally, sometimes, etc.
    • Avoid leading questions that clue the respondent to what the answer should be.
    • Avoid implicit alternatives that are not explicitly expressed in the options.
    • Avoid implicit assumptions.
    • Respondent should not have to make generalizations or compute estimates.
    • Use positive and negative statements.

Step 7. Determine the Order of Questions:

    • The opening questions should be interesting, simple, and non-threatening.
    • Qualifying questions should serve as the opening questions.
    • Basic information should be obtained first, followed by classification, and, finally, identification information.
    • Difficult, sensitive, or complex questions should be placed late in the sequence.
    • General questions should precede the specific questions.
    • Questions should be asked in a logical order.
    • Branching questions should be designed carefully to cover all possible contingencies.
    • The question being branched should be placed as close as possible to the question causing the branching, and (2) the branching questions should be ordered so that the respondents cannot anticipate what additional information will be required.

Step 8. Form and Layout:

    • Divide a questionnaire into several parts.
    • Questions in each part should be numbered.
    • The questionnaire should be pre-coded.
    • The questionnaires themselves should be numbered serially.
  • Step 9. Reproduction of the Questionnaire:
    • The questionnaire should have a professional appearance.
    • Booklet format should be used for long questionnaires.
    • Each question should be reproduced on a single page (or double-page spread).
    • Vertical response columns should be used.
    • Grids are useful when there are a number of related questions which use the same set of response categories.
    • The tendency to crowd questions to make the questionnaire look shorter should be avoided.
    • Directions or instructions for individual questions should be placed as close to the questions as possible.

Step 10. Pre-testing::

    • Pre-testing should be done always.
    • All aspects of the questionnaire should be tested, including question content, wording, sequence, form and layout, question difficulty, and instructions.
    • The respondents in the pretest should be similar to those who will be included in the actual survey.
    • Begin the pretest by using personal interviews.
    • Pretest should also be conducted by mail or telephone if those methods are to be used in the actual survey.
    • A variety of interviewers should be used for pretests.
    • The pretest sample size is small, varying from 15 to 30 respondents for the initial testing.
    • Use protocol analysis and debriefing to identify problems.
    • After each significant revision of the questionnaire, another pretest should be conducted, using a different sample of respondents.
    • The responses obtained from the pretest should be coded and analyzed.