A survey of surveys PETE&C February 21, 2006

1. A survey of surveysPETE&C February 21, 2006 • Helmut Doll • Bloomsburg University • hdoll@bloomu.edu

2. Surveys in education • 124,000,000 results for Google search “surveys in education” • Why surveys? • To find out what is going on • To discuss topics with your stakeholders • To prioritize your actions • to satisfy data requirements • What does the community think about our school? • How are the students responding to my lesson?

3. Example survey • “The Utilization of Technology by Teachers: A Follow-up Study of Special Education Graduates from Bloomsburg University” • Dr. Sheila Jones • Survey posted as online web page • 290 Initial and follow-up letters • 58 Online surveys completed (20% completion rate)

4. Survey process addresses from alumni office follow-up letter

5. Examples • Lakeview Middle School - Online Surveys • School Climate Survey (Parent, Teacher and/or Student Satisfaction Surveys) Principals give an annual school climate survey to parents, teachers and/or students to assess their satisfaction with school goals, programs and activities. The survey results are used for on-going school planning and program improvement. • Webfeedback Sample Education Survey

6. Education surveys • Faculty and course evaluations • Student affairs surveys • Alumni surveys • School climate surveys • School safety surveys • Community surveys • Employee surveys • Professional development needs assessment • Community and adult education surveys • Student drug awareness survey • Lunch room count

7. Survey participants in education Community Students Teachers DoE Administrators Parents

8. What do you want to accomplish with your survey?

9. What about your audience? • Do you know their e-mail address? • Do they have Internet access? • Are they already visiting your web site? • Are they computer literate? • Do they have Internet access at the location where you are trying to collect data?

10. Distribution options • send the survey as an email (text-based) • post the survey online, as a web form • hosted on your site • hosted on third-party site • send a paper-based survey • send a paper form that can be scanned • send a disk with the survey • use a pdf form • do phone or in-person interviews

11. Electronic survey special features • Prevention of retaking the survey • Limiting access to authorized people • Dynamically branching surveys • Saving partially finished survey • Randomized survey questions • Use of multimedia in survey • Time limits on responses • Individualized surveys

12. Advantages of online surveys • Avoiding data entry errors • Eliminating some incorrect answers • Elimination of geographic proximity as a limiting factor • Efficient data collection • Time savings in data analysis • Anonymity can result in more honest answers

13. Disadvantages of online surveys • Does the use of the Internet/computer already limit the sample • What happens, if the computer/Internet crashes during the survey? • Fear of loss of anonymity can result in less honest answers • Respondents may be random if open web survey

14. Printed survey • Time delay in responses • Widely available • Low response rate • Data entry effort

15. Adobe PDF form • Good printing support • Allows ‘intelligent’ forms • Automatic data collection possible • Requires Adobe Acrobat Reader • Can be sent by e-mail

16. E-mailed PDF form Live sample form

17. Form data collection • as spreadsheet in Acrobat • in database using server middleware (CGI, PHP,...)

18. Survey software integrated into SIS • Access can be limited to students • Access is often limited to students • Data can be linked • Anonymity not guaranteed

19. Web form for surveys

20. The survey

21. One more survey screen

22. Web forms info • Only requires web browser • Requires live Internet connection • Allows special features • Concerns about Internet Security • Accessibility issues (Section 508)

23. Dynamic branching

24. Dynamic branching

25. Control of audience • Code number in notification letter • Code number in link • Please click to start your survey • Track IP address of respondent • Anonymity will be questioned

26. Data validation

27. Contacting the respondents • Do I have the addresses of all participants? • Mail? • Email? • Unreliability of email • Changing addresses • Spam filters • Are the participants technology-savvy? • How will I ‘enter the data’?

28. Notification by mail or email • E-mail: • …Please click on this survey link to start the survey. • Mail: • … Go to the following address using your web browsers: http://iit.bloomu.edu/techsurvey/

29. Creating a survey

30. Creating a question

31. Reports • Charts • Statistical summaries • Exporting options • Filtering data

32. Reports

33. Report Filtering

34. Subscription software/Hosted surveys • Company runs the survey on their web site • Survey is created by • sending questions to host company • or using an administrator account and web pages • Examples • WebSurveyor • SurveyMonkey • Zoomerang • Perseus SurveySolutions® Express

35. Subscription software $$$ • Professional Subscription • A professional subscription is only $19.95 USD/month (or only $200.00 USD/year), and includes up to 1000 responses per month.  If you exceed 1000 survey responses in any given month, there is an additional charge of $0.05 USD per survey response.  There are no long-term contracts, and you can cancel at any time.  As a professional subscriber, you have access to all of the advanced features of SurveyMonkey.  You can create an unlimited number of surveys, with an unlimited number of pages and questions.  In addition, all of your surveys are completely unbranded. • Basic Subscription • A basic subscription is totally free and includes all of the basic features of SurveyMonkey.  It's a great option for individuals, students, and anyone who doesn't need the advanced features of SurveyMonkey.  Unlike other services, there are no annoying banner ads on your surveys.  In addition, all of your survey responses remain absolutely private.  Please note that basic subscribers are limited to a total of 10 questions and 100 responses per survey. • (from SurveyMonkey)

36. Questions for survey hosts • How does the cost change when we have a large number of respondents? • Who owns the survey data? • Will you have access to the address list of respondents? • Can we customize the survey look and feel?

37. Hosting your own survey • Build survey application from scratch or modify prebuilt system • Customize a purchased or free survey application • Requires programming skills • Requires server and server admin • Does not cost more for more uses.

38. Open source survey software • Usually requires additional software on the server • php, jsp, coldfusion • PHPSurveyor.org (PHP) • Projects/phpESP - Butterfat • php Easy Survey Package • Soundings ColdFusion Survey Application

39. Purchased survey software • Often self-contained • E.g. confirmit

40. Example: Limitations • RSVME is a free application that integrates with Outlook and other email packages, and makes obtaining feedback from people a snap! You can quickly and easily put together a questionnaire on any subject, from the best time for soccer practice to who's bringing what food to a party. Then select names out of your address book you want to send the questionnaire to, and send. It's that simple! • System requirements • Microsoft Windows 98/ME/2000/XP

41. Resources • ITS Online Surveys - Introduction • Survey Tutorials • Online Survey Articles