Survey Design and Implementation. Elizabeth C Wells Doctoral Candidate Ag & Extension Education http://www.msu.edu/~wellselz/. Why do people fill out surveys? What do you want to ask/measure? Accuracy/Survey Error Random Sampling Writing Questions Questionnaire Design
Elizabeth C Wells
Ag & Extension Education
What do you want to ask/measure?
Telephone, Interview and Internet Surveys
Theory of human behavior in which actions of individuals are motivated by the return these actions are expected to bring.
Elements of reward, cost and trust
show positive regard
say thank you
ask for advice
support group values
give tangible rewards
make the questionnaire interesting
avoid subordinating language
make it short and easy
minimize requests for personal information
keep requests consistent
provide token of appreciation in advance
sponsorship by legitimate authority
make task appear important
invoke other exchange
behavior, attitudes, feelings, demographics
Instrument (whole questionnaire not returned)
Item (question not answered)
compare early to late respondents
Pop Size 50/50 split 80/20 split
1000 278 198
Sample size for a 95% confidence level
And a + 5% sampling error
A www site that will generate random numbers which you can use for random selection of participants on a mailing list.
If you fixed dinner last night, did you eat meat as part of that meal?
Are tall people more likely to be elected President?
How much television did you watch last month?
most important………………..top priority
your answers………………….your responses to this
How often did you attend religious services last year?
How often did you attend religious services during the past year?
Not at all
A few times
About once a month
Two or three times a month
About once a week
More than once a week
“To what extent do you agree….”
“To what extent do you agree or disagree with this statement…. “
16. Should the new school have a swimming pool that includes lanes for swimming laps and is not enclosed for winter use?
5. Do you favor or oppose not allowing the state to approve a casino without approval from 60% of the voters?
5. Do you favor or oppose requiring 60% approval by voters in order for a casino to be built.
Favor requiring 60% approval
Oppose requiring 60% approval
To what extent do you agree or disagree with this statement: “Listening to classical music is good for a persons emotional health.”
Neither agree nor disagree
Have you ever shoplifted something from a store?
Have you ever taken something from a store without paying for it?
22. What was your total income from all sources in 2003? “Listening to classical music is good for a persons emotional health.”
______________ Total income for 2003
22. Which category best describes your total income from all sources in 2003?
$10,000. or less
$10,001. to $20,000.
$20,001. to $35,000.
$35,001. to $50,000.
$50,001. or above
Doing so may impact respondents perception of the scale.
1= strongly agree, 2 = agree,
3 = disagree, 4 = strongly disagree
People tend to check the first ones more than the last ones and sometimes check a few and feel that is enough to satisfy the researcher.
Extension Agent “Listening to classical music is good for a persons emotional health.”
Fertilizer and other salesmen
Fertilizer and other salesmen
Extension AgentQuestion Principles
booklet format preferred
one column for legal size
two columns for full size
avoid upper left staple design
avoid unusual size, shape or fold
1. Please think about the things that make MSU Extension a pleasant place to work and write down the five most important aspects and rate them one to five in importance to you.
1. Please describe in your own words what you consider good about working for MSU Extension.
1. What year were you born?
1. Thinking about MSU Extension, how would you rate it as a place to work?
graphic symbols and arrangements
Start here: “Listening to classical music is good for a persons emotional health.”
Other (Please Specify) _________________
START HERE: “Listening to classical music is good for a persons emotional health.”
Which of the following is your main work activity?
Other (Please specify) ____________________
5. “Listening to classical music is good for a persons emotional health.”How many months have you worked at your current job?
_________Number of months
Please be as specific as possible in answering the next question, including any area of specialization. Example: Extension Agent – Agriculture. If you had more than one job, answer for the job for which you worked the most hours.
6. What kind of work do you do in your current job?
_________________________________ Kind of work
5. “Listening to classical music is good for a persons emotional health.”How many months have you worked in your current job?
_____________________ Number of months
6. What kind of work do you do in your current job? Please be as specific as possible in answering. Include any are of specialization, for example: “Extension Agent – Agriculture”. If you had more than one job, answer for the job which you worked the most hours.
______________________________Kind of work
1 “Listening to classical music is good for a persons emotional health.”. Which one of the following best describes the 4-H project with the largest enrollment within your club?
1. Which one of the following best describes the 4-H project with the largest enrollment within your club?
__________ Times a week
12. Normally do you eat breakfast every day? with the largest enrollment within your club?
No (Skip to 18)
13. (If Yes) How often do you drink orange juice with your breakfast?
____________ Times a week
Place at end of questionnaire but not on
Only ask what is necessary
Consider simple art work
Include name of survey
Include name and address for return
Try not to ask specific questions
May place open ended question here
Give opportunity to add comments
Conclude with a thank you
Brief pre notice letter
Questionnaire mailing with SASE
Thank you postcard
Final contact with “special” mailing
Date at top
What will happen
What it is about
Usefulness of survey
P.S. Token incentive
Mail out a few days to one week before mailing instrument
4-6% higher response rate
Include personalized cover letter
Self addressed stamped envelope with real stamp
Use first class postage – plain envelope
Serves to jog memory
Gives opportunity to call for replacement if neededImplementation – Contact 3
Mail one week after questionnaire
Increase up to 13%
Sent only to non-respondents
Two to four weeks after 3rd contact
Final contact is a week or more after 4th contact
Responses differ by mode:
“Excellent Health” = 44% by personal interview
= 37% by telephone
= 30% by mail
primacy vs recency effect
Anonymous – No way to know who returned the survey
Confidential – Only researcher can connect the respondent with their answers. All answers remain confidential.
University Committee on Research Involving Human Subject
Mail and Internet Surveys – The Tailored
Design Method by Don Dillman 2000