Addressing Survey Fatigue Annemieke Rice Assessment Consultant StudentVoice

1. Addressing Survey FatigueAnnemieke RiceAssessment ConsultantStudentVoice

2. 2001-03: Idea The StudentVoice Story

5. Survey response rates have been falling Difficult to contact people Refusals to participate increasing National survey response rates fell from about 60% in the 1960s to just above 20% in the 1980s Nonresponse may not be random 2 strategies for correcting low response rates: Weight the data for nonresponse Implement strategies to increase response rates

6. Research on Survey Fatigue Multiple surveys are seen as a burden; would expect response rates to decrease as requests increase Time is a major issue Example - 1998 study at Air Force Academy: Anecdotal evidence � survey about surveys 97 percent: somewhat over surveyed Should be surveyed only 3-4 times a year Over surveyed: �combination of frequent surveys perceived as irrelevant�

7. Implications from Research Potential of multiple surveys can reduce response rates Nonrespondents cite time concerns as reason Effects of survey fatigue may be moderated by salience of survey content Number of previous surveys may have an impact on current survey response Survey fatigue may have biggest impact on survey administered back-to-back Feeling of �I have done enough� (reciprocity) pg. 66

8. Reasoned action approach Calculation of costs and benefits Social exchange: rewards, cost, and trust Increase rewards Reduce costs Establish trust Psychological approach Informal decision rules Norm of reciprocity Norm of social responsibility

9. What this means in action: More likely to comply when request appears to be from a legitimate authority Requests for help should be clear and straightforward in survey invitation Emphasize that recipients are part of a select group

10. Technique 1: Multiple Contacts Pre-notifications: Provides information about the project including data collection times, incentives, purpose, contact information Invitation with survey information: Repeat of pre-notification information Follow-up reminders: Ideally only to non-respondents Range of 2-5 day intervals

11. Technique 2: Length The longer the survey the higher the perceived cost Moderate correlation between length and nonresponse Studies: 22 questions 13 minutes Also consider type of questions: text fields vs. answer choices

12. Technique 3: Incentives Research on the effect of incentives on college students is lacking Prepaid incentives consistently raise levels of response Effect of postpaid incentives is minimal Payment contingent upon completion = compensation 2 effects: Getting paid removes reciprocity aspect Expectations for future participation

13. Technique 4: Salience Salience: How important or relevant a survey topic is to the survey recipient 12-14% increase in response rates for salient surveys Difficult to control Highlight salience in survey invitations

14. Technique 5: Statement of Confidentiality Anonymous vs. confidential Anonymous: responses are unidentifiable Confidential: can identify but will not share Voluntary statement assuring participants of confidence of information ascertained through survey (may be required) Reduces perceived cost Establishes trust Heightens awareness of what may be asked Misuse can lead to decrease in response rates

15. Other Techniques Technique 6: Request for Help Follows social responsibility theory Be careful of wording in contact, especially with subject lines Technique 7: Sponsorship Who is coordinating the survey No differences within an institution Collaborations with external entity may need elaboration Technique 8: Deadlines Scarce/limited opportunity: more valuable Results are mixed: won�t hurt, may not help

16. Contact/Invitation Checklist Should convey the importance of the study Should include a request for help Should guarantee confidentiality (as appropriate) Should provide information on how long it will take to complete the survey Should indicate the time period for which the survey will be accepting responses Include information on incentives

17. Timing of Contact/Administration Avoid busy times or holidays Send email/ preannouncement 2-3 days prior to survey mailing According to Karen Bauer research, first mailing should arrive on Thursday/Friday First half of semester/term may be better if you are surveying in an academic environment

18. Web Survey Practices Possible ways of motivating respondents to continue: Keep design simple Make directions clear Include questions on the first page Break the survey up into multiple pages Utilize a progress tool Utilize skip patterns so that respondents do not have to see questions that do not apply to them

19. Sampling Sampling: A way to obtain information about a large group by examining a smaller selection (the sample) of group members. If the sampling is conducted correctly, the results will be representative of the group as a whole. Margin of error: Maximum amount by which the sample results are expected to differ from the actual population.

20. Sampling Strategies Simple random sample gives everyone in sampling population an equal chance of selection; a probability sample Stratified random sample breaks total sample into subpopulations and then selects randomly from each stratum Systematic sample involves randomly selecting the first member of the sample from a list and then selects others thereafter using a sampling fraction (e.g., picking every 9th person) Convenience sample selects respondents based on easy accessibility or availability (e.g., everyone at point X in a theme park over three time periods) Purposeful sample selects potential respondents based on needs of research project and because the sample is deemed to be information-rich per the research problem(s) Any many more�. (snowball, cluster random, etc.)

21. Sample Size What is the appropriate sample size? Factors to consider: Population size Sampling error Time and money Type of analysis (descriptive, correlational, experimental) Online tool for calculating sample size: http://www.touchpoll.com/calculator.htm

23. Coordination of Survey Activity Overseeing body: person or committee that monitors activity for department, division, or institution Assessment calendar for division and institution Think about what other areas on campus are doing and collaborate with them Partnerships between units with related missions and objectives Find opportunities to share assessment activities and results (e.g., �Assessments of the Month�) Share results widely (internally and externally): Publish an Assessment Update

25. How is the issue of survey fatigue or over-surveying addressed in your department, in your division, and on campus in general?

35. Questions/DiscussionAnnemieke RiceAssessment ConsultantStudentVoice716-652-9400arice@studentvoice.comPresentation developed by Marissa Cope, mcope@studentvoice.com