Effective Survey Design and Data Coding Workshop Insights

1. MSU RStats Institute Workshop Survey/Questionnaire Design and Data Coding Presenters: Dr. Chantal Levesque-Bristol Dr. Jeanne Phelps

2. Some up-front considerations: Two Cardinal Rules #1 cardinal rule of survey research: “First, do no harm” #2 cardinal rule of survey research: “Never forget that you are sending as well as collecting information – be sure you understand the message your survey sends”

3. Some up-front considerations:What are some pitfalls? • Sampling error – obtaining survey responses from too few people who represent the population of interest; too small a sample will limit your ability to precisely estimate the characteristics of that population. • Coverage error – failure to randomly sample from the population of interest; not allowing all members of that population an equal or known chance of being sampled. Lead to lack of generalizability.

4. Some up-front considerations:Pitfalls, continued… • Measurement error – can be the result of poor question wording, or questions that produce inaccurate or impossible-to-interpret answers. • Nonresponse error – can occur when people who respond are systematically different from those who don’t respond. • Missing data error - Aim to have less than 5% of missing data.

5. Nuts-and-bolts considerations: • Good questions are devilishly difficult to write – consider professionally developed and validated scales, if available • Avoid questions that ask respondents to “check all answers that apply” • Avoid questions that reduce or limit the richness of the information that you could collect • Surveys will give you about 20% to 30% response rate • Always pilot-test surveys

6. Levels of Measurements • Likert Scales • Continuous variables • Provides the most information • Example: 1 (not at all) to 7 (completely) scale • Use odd number of values so that there is only one mid-point • Dichotomous Scales • Yes/No, True/False types of questions • Provides the least amount of information

7. Levels of Measurements Likert Scales Advantages • You obtain the richest kind of information • You can always dichotomize your scale afterwards Disadvantages • Harder to construct

8. Levels of Measurements Dichotomous Scales Advantages • Easy to construct • Easy to answer Disadvantages • You get poor information • You can’t obtain information on the extent to which participants agree with a survey item

9. Data Coding • Nominal Variable (e.g. gender) • Select a numerical code (1 = female/ 2 = male) • Dichotomous Variable • Select a numerical code (yes =1/ no =0) • Check all that apply questions • Select a numerical code (checked =1/ not checked =0) • Continuous or Likert scale • No coding necessary. Use the number participants provided or circled

10. Data Coding • Each item on your questionnaire/survey becomes a variable • “Check all that applies” question generate lots of variables • Do not give complex code that combine two variables together • Do not code a girl in 4th grade as “14” • Create 2 variables and code separately: • Gender = 1 and grade = 4

11. Recent developments… • Increasing use of internet to administer surveys, but problems inherent in internet surveys will probably continue to keep this delivery mode from taking over completely • Increasing use of more than one delivery mode (i.e., internet, mail, phone, face-to-face interviews)

12. References • Dillman, D.A. (2007). Mail and internet surveys: The tailored design method. Hoboken, NJ: Wiley • Salkind, N.A. (2006). Exploring research (6th Ed.). Upper Saddle River, NJ: Prentice Hall

13. Questions?