What is a Survey?

1. What is a Survey? • A scientific social research method that involves • Selecting a random sample of people to answer some questions • Designing a standardized questionnaire to get information about the research question(s) • Administering the questionnaire • Coding the responses in a standardized form • Analyzing the results to provide descriptions about the people in the sample and find relationships different responses • Generalizing the results to the population from which the sample was drawn

2. Basic Formats • Self-administered mail survey • Telephone interview • Face-to-face interview • Focus group • Self-administered Web survey • Self-administered Email survey

3. Why do a Survey? • To collect information that is not available from other sources • Attitudes and opinions • Using a standardized measurement (questionnaire) provides comparable information from everyone taking the survey, which allows for meaningfulanalysis • With a good sample, you can have confidence that the results can be generalized to the population in which you are interested

4. Before you do a Survey.. • Think About Available Resources • What information is already available from non-survey sources? • Have any surveys already been done on this question? • Think About Who is going to: • Design the survey • Administer the survey • Enter the data • Analyze and interpret the data • Write up the results • Present the findings • Use the findings • Pay for it all...

5. Questionnaire Construction • To have confidence in your survey results your questions must be • Reliable = provide consistent measures in comparable situations • Valid = measure what they are intended to measure • Careful wording will decrease bias

6. Questions/Items • Closed-end or Forced-choice Questions • Respondent selects question answer from among a list of possible responses • It’s a good idea to ask multiple indicators of a single concept, and then create a scale by combining those indicators • Open-end Questions • Respondents write question response in own words • might need to enter data by hand • develop coding scheme for responses • It’s a good idea to always include an open-ended question giving the respondent the opportunityto add any additional comments they might have

7. Good Questions • Are clear and use simple language • Are concise • Are specific • Are possible to answer • Are relevant to the respondent • Do not use negatives • Avoid biased terms • Have only 1 part (not “double-barrel”)

8. Examples of Good and Bad Questions… • Use simple language... • How often do you discuss course readings outside of class? • Is better than : • How often do you confabulate about course readings with others outside of class?

9. Examples of Good and Bad Questions… • Are concise... • How often do you work with students on class projects outsideof class? • Is better than: • Sometimes instructors assign class projects and have students work on those projects with other students who are in the class, but the students sometimes need to work on the project outside of class time. How often do you have to dothis?

10. Examples of Good and Bad Questions… • Are specific... • In which year and semester did you take your first class at NC State? • Is better than: • When did you start taking classes here?

11. Examples of Good and Bad Questions… • Do not use negatives... • I usually see students raise their hand in class. • Is better than: • I have rarely seen students not raise their hand in class.

12. Examples of Good and Bad Questions… • Do not use biased terms or suggestions... • Do you agree or disagree with the following statement: Inquiry guided learning enhances critical thinking. • Is better than: • Don’t you agree with Chancellor Fox when she says that inquiry guided leaning enhances critical thinking?

13. Examples of Good and Bad Questions… • Do not ask two questions in one (“doublebarrel”)... • How often did you work with other students on class projects during class time? How often did you work with other students on class projects outside of class time? • How often did you work with other students on class projects during and outside of class time?

14. Response Options • Response options should reflect the concepts you are trying to measure, and fit with the question wording. • Suggestion: Avoid simple “yes” or “no” type responses and try to measure intensity of feeling when possible.

15. Response Options continued… • Must be mutually exclusive • can select only one appropriate answer • On most days, what is the total amount of time you spend working on your course assignments outside of class? (IS mutually exclusive) (is NOT mutually exclusive) Less than 1 hour 1 hour or less 1-2 hours 1-2 hours more than 2 hours 2 hours or more

16. Response Options continued… • Must be exhaustive • all possible answers are listed (including e.g., “other, “ “don’tknow,” etc.) • On most days, what is the total amount of time you spend working on your course assignments outside of class? (IS exhaustive) (is NOT exhaustive) Less than 1 hour Less than 1 hour 1 to 2 hours 1 to 2 hours more than 2 hours 3 to 4 hours

17. Question Order • Be attentive: Initial questions affect answers to subsequent ones. • Start with easy, salient, non-threatening but necessary questions. • Put more difficult or threatening questions near the end (e.g., demographic questions). • Keep questions dealing with the same topic together. • Do not include questions that are redundant or are not likely to be analyzed.

18. Questionnaire Format • Should be: • as short as possible • visually attractive and nicely reproduced • spread out and uncluttered • broken into logical sections when possible • have clear skip patterns for contingency questions

19. Questionnaire Format continued… • Should have clear spaces for respondents to mark answers • use boxes • parentheses • numbers to circle • avoid lines to put check on • The amount of space provided for responses to open-end questions will affect how much respondents write.

20. Instructions for Questionnaire • Provide general instructions in beginning of self-administered survey • Brief explanation could include: • why doing survey • why answers are important • how to answer questions • stress confidentiality (if appropriate) • Provide specific instructions to questions as necessary

21. Example of Instructions • Instructions: The following questions ask about your classroom experience. Please tell us how satisfied or dissatisfied you feel with these experiences by circling one of the response options listed below each statement. • There are no right or wrong answers. All replies are confidential so give your honest opinion on each of the statements. Please do not discuss your answers with other people. It is your opinion which matters. • Even though it may be hard to decide, please be sure to answer all questions. Thank you for your help.