Software/Tools For Running Web Experiments

1. Software/Tools For Running Web Experiments By Robert Slater Texas A&M -Corpus Christi

2. Considerations • Skills • Technology • Data Privacy • Features

3. Skills • What is your proficiency? • HTML, ASP, VB, JAVA, NONE • Hire external party • Do they understand what you are doing? • How important is their friendship? • Learn new skills? • learning curve.

4. Technology • What type of computer systems will access the software? • Windows XP, ME, 2000, 98, 95, 3.1, DOS, MAC • Will the code you have work on the different operating systems or web browsers? • Do you have a web-server you can use to host your study? • Would a third party notify you in advance of server maintenance?

5. Data Privacy • Where will the data be stored and who will have access to it? • If you use a third party hosting site can you keep your data confidential? • Can you control backup copies of your data?

6. Features • What kind of features do you need in your study? • I needed: • Randomization into treatment groups • Piping with mathematical manipulation • Multiple Question formats • Data Export • Customization • User sign in options (specific/anonymous) • Java script capability within survey pages

7. My Solution • Adapted an open source survey software to meet experimental needs (www.ClassApps.com) • ASP based open source software • Multiple database support – • Supports data export to excel • Professional looking finished product • No programming experience needed • Quick learning curve

8. SelectSurveyASP

9. SelectSurveyASP

11. Finished Product

12. Survey Options

15. Export Options

16. Limitations and Solutions • Software not designed for multiple treatments: • Create multiple questionnaires based on treatment groups • Created one master survey and copied it to create separate treatments • Software would not randomize subjects: • Use separate web page and basic java script to randomize subjects into treatment groups • Software allowed hyperlinks to surveys

17. Multiple Treatments

18. Limitations and Solutions (cont) • Software allowed for piping but not mathematical manipulation as needed in my study. • I needed to be able to: • Take users estimate of Earnings Per Share • Give them management’s estimate that was calculated as a percentage of user’s response.

19. Piping • Take data input by participants and use in the study. • Personalize questions • To develop rankings using data input by users.

20. Taking User Input

21. Using the Input

22. Capture Input for Ranking

23. Ranking The Captured Input

24. Piping Solution • Program already allowed for piped responses. • Open source software allowed me to manipulate the ASP source code to perform mathematical function on piped response.

25. Users Input EPS

26. Design Screen

27. Pipe Instructions

28. Actual Question Text

29. User View Piped Response

30. Conclusions • Quick learning curve. Little expertise necessary. • Customizable software allowed for unique functionality. • Locally housed database on web server allowed for data privacy and further customized data gathering. (Timed responses) • Customized programming problems led to incompatibility with some operating systems. • Participants were dropped from the study before completion.

31. Other Commercial Software *(list of survey companies found on www.surveymonkey.com/Pricing.asp)

32. Recent Academic Studies on Web Experiments • Bryant, B., Hunton, J., and Stone, D., 2004, “Web Experiments: Prospects and Possibilities for Behavioral Accounting Research,” Behavioral Research In Accounting, 16: 1-24 • Birnbaum, M., 2004, “Human Research and Data Collection via The Internet,” Annual Review of Psychology, 55: 803-32