web based research issues problems and an example technique l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Web-Based Research: Issues, Problems and an Example Technique PowerPoint Presentation
Download Presentation
Web-Based Research: Issues, Problems and an Example Technique

Loading in 2 Seconds...

play fullscreen
1 / 36

Web-Based Research: Issues, Problems and an Example Technique - PowerPoint PPT Presentation


  • 306 Views
  • Uploaded on

Web-Based Research: Issues, Problems and an Example Technique John H. Krantz Hanover College Outline Brief History Issues in Internet Research An Example Method Problems and Solutions History Email for 2-3 Decades 1995 Krantz, Ballard, & Scher (1997) Within Subjects

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Web-Based Research: Issues, Problems and an Example Technique' - paul


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
outline
Outline
  • Brief History
  • Issues in Internet Research
  • An Example Method
  • Problems and Solutions
history
History
  • Email for 2-3 Decades
  • 1995
    • Krantz, Ballard, & Scher (1997) Within Subjects
    • Reips (1997) Between Subjects
    • First True Web Experiments
  • July 1995 List of Online Psychological Research – sponsored by APS for awhile
    • http://psych.hanover.edu/research/exponnet.html
    • Almost 200 studies and study sites listed
why do internet research
Why Do Internet Research
  • Musch & Reips (2000)
    • Sample Size
    • Statistical Power
    • High Speed
    • Ability to reach participants in other countries
    • High external or ecological validity
    • Low cost
    • Ability to replicate a lab experiment with more power
    • Special populations
issues
Issues
  • Is the Sample Representative
  • Yes and No
    • Important to remember base of comparison
      • Ideal vs. Lab
    • Krantz & Dalal (2000)
    • Much more diverse that most lab samples
    • Especially in age and education range
    • Race is generally limited as is nationality
sample characteristics gender
Sample Characteristics: Gender
  • Are Internet Samples Male Dominated?
    • Krantz & Dalal (2000) % Female
      • GVU 1st (1994) 5%
      • Reips (1996) English: 43% German: 18%
      • Krantz, et al. (1997) 44%
      • Pasveer & Ellard (1998) 3rd Study 71%
    • More Recent
      • Caddell & Utt (2004) 77%
      • Meyerson & Tryon (2003) 45%
sample characteristics age
Sample Characteristics: Age
  • Are we still testing college sophomores?
  • Krantz, et al. (1997) 43% > 30
  • Smith & Leigh (1997) 35% > 30
  • Pasveer & Ellard (1998) 45% > 25
  • GVU 1st 36% > 30
  • Caddell & Utt (2004) 60% > 30
  • Pattison & Rouse(2003) 16% > 30
is the college sophomore be making a comeback
Is the College Sophomore be Making a Comeback
  • Plot of activity of Psychological Research on the Net
  • Pattison & Rouse (2004) 76% 18-22
sample characteristics race
Sample Characteristics: Race
  • How diverse are the samples?
  • Unfortunately in general samples, diversity is still limited
  • Krantz et al. (1997) 89% White
  • Smith & Leigh (1997) 86% White
  • GVU 10th (1998b) 87% White
  • O’Neil, Penrod, & Bornstein (2003) 82% White
  • Meyerson & Tryon (2003) 93% White
  • However, as will be discussed later, web can make it possible to access special populations
sample characteristics nationality
Sample Characteristics: Nationality
  • Where to do the subjects come from?
  • Largely North American, even US, even in some European studies
  • Krantz, et al. (1997) 86% N. Am.
  • Senior, et al (1999) >80% N. AmThis study was conducted in England.
is the data any good
Is the Data Any Good
  • Emphatically Yes
data quality direct comparisons
Data Quality: Direct Comparisons
  • A number of studies, still, run both laboratory and internet samples.
  • Krantz, et al. (1997)
  • Regression of web means on laboratory samples (even though different types of samples): lab mean = 1*(Web mean) + 0 r2 = .99
compare to established data
Compare to Established Data
  • Compare web results to previously published data sets
  • Myerson & Tryon (2003)
  • Studied Sexual Boredom Scale of Watt & Ewing (1996)
  • Matched sample characteristics
  • Found same internal consistency
  • Form of administration was not a significant factor
direct validity comparison
Direct Validity Comparison
  • Use same techniques to validate results
  • Pasveer & Ellard (1998)
  • Developed new scale
  • Internal consistency
  • Psychomectric properties
data validity arguments
Data Validity: Arguments
  • Reips (2000)
    • Statistical Power
    • Limited Sample Population
    • Limited External Validity
    • Less than Optimal Voluntariness
    • Motivational Confounding
    • Experimenter Bias
    • Nontransparency
    • Limitations of what is feasible to research
interesting deviations
Interesting Deviations
  • Stern & Faber (1997)
    • Milgram’s lost letter technique
    • Milgram, sent on
    • Stern & Faber, returned to sender
    • From doing favor to doing easy task
  • Often effects are smaller
getting subjects general
Getting Subjects: General
  • http://psych.hanover.edu/research/exponnet.html
  • Other Pages:
    • Social Psychology: http://www.socialpsychology.org/expts.htm
    • The Web Experiment List: http://genpsylab-wexlist.unizh.ch/
getting subjects special populations
Getting Subjects: Special Populations
  • Advertising
  • Email Groups
    • Careful, get permission
    • Can be thought spam
    • Netiquette
  • Discussion Groups
    • Same, permission
sample method for doing survey
Sample Method for Doing Survey
  • Birnbaum, M. H. (2001). Introduction to Behavioral Research on the Internet. Upper Saddle River, NJ: Prentice Hall.
  • Lots of resources at http://ati.fullerton.edu/
    • NSF sponsored classes at Fullerton on Social Psychology
    • One more next January, look for the announcement at this site
first step where to put it
First Step: Where to put it
  • Setting up a server can be easy
  • Apache web server
  • http://www.apache.org
  • Most popular server
  • Freeware
  • Any computer connected to web is now a server
  • May have issues of firewalls & dynamic IP addresses
second step surveywiz
Second Step: SurveyWiz
  • In Birnbaum (2001)
  • Also linked to from http://ati.fullerton.edu/
    • Under Birnbaum’s page
  • Freely available
  • We will build a very short survey to illustrate
forms
Forms
  • SurveyWiz uses web forms to collect data
  • What are forms:
    • Elements to allow web page viewers to input
    • This data is then sent back to the server to be processed
  • SurveyWiz
  • What the page looks likeMy Take
  • Can edit it with any web page editor
examples i have done this year
Examples I have done this year
  • Caddell & Utt
  • Pattison & Reese
third step collecting the data
Third Step: Collecting the data
  • First way, let Birnbaum do it for you
    • SurveyWiz is set up this way
    • <FORM action=http://psych.fullerton.edu/cgi-win/polyform.exe/generic method=post>
    • FTP://guest:guest99@psych.fullerton.edu
    • You can type in the address box of your browser. Note that this is FTP site. At the present time, password is guest99, as shown above, and permits download only.
keeping the data local
Keeping the Data Local
  • Second, get this perl script (found at http://psych.fullerton.edu/mbirnbaum/programs/PERL_script2.htm)
  • The script and instructions are at this site
  • Written by Billy Schmidt
    • Also change code in form command
    • At my school the line reads:
    • <FORM action=http://psych.hanover.edu/cgi-bin/survey_wiz.pl method=post>
what is cgi
What is CGI
  • Common Gateway Interface
  • Method for other programs to interact with web servers
  • In this case, this perl program takes data from web forms and stores them in a file
  • Stores data in Excel or SPSS readable files
fourth step example data file
Fourth Step: Example Data File
  • Data File
  • Data coding scheme: all variable names begin two numbers, see example
  • The perl program sorts the variables by numerical order
another method of survey generation
Another Method of Survey Generation
  • Schmit’s Survey Assistant
    • He manages data
    • More flexibility
    • http://or.psychology.dal.ca/~wcs/hidden/home.html
what do i need
What Do I Need?
  • At NSF ATI site
  • List of software resources
what problems are important
What Problems are important
  • Musch & Reips (2000) – 5 point scale
    • No control over participant’s behavior 3.6
    • No control over motivation 3.4
    • Inability of participants to ask questions 3.3
    • Nonrepresentative sample 2.9
    • Manipulation and fraud 2.4
    • Ethical problems 1.5
problems
Problems
  • Eliminating Multiple Entries
  • Dropout
  • Security
  • Data Integrity
multiple entries
Multiple Entries
  • People submit too fast, while waiting for feedback
  • People will fake being two people
  • Usually have same IP address
  • SurveyWiz and most other methods, sends the IP address of machine where survey is being taken
  • Can eliminate more than one from same IP, e.g. Schmidt (1997)
security data integrity
Security & Data Integrity
  • If on public server, others can access data and download
  • Others might fake pages to send data (unlikely)
  • Keep data in non-public directories so only researchers have access
  • Have CGI check for origin of survey to make sure it is yours (Schmidt, 1997)
dropout
Dropout
  • People will come but not finish
  • Or data is incomplete
  • Survey will sends a complete signal to help you track
  • Can use: (O’Neil, Penrod, & Bornstein, 2003)
    • Short sweet
    • Financial incentives.
    • If multiple pages to survey, several pages warm-up so not drop-out during
ethical issues
Ethical Issues
  • Should not think of Ethical Issues of Web Research Alone
  • Compare to Ethics of doing Traditional Research
  • Not let our comfort with what we have done blind us to those problems
conclusion
Conclusion
  • Many Benefits
  • Easy Methods
  • Easy Resources
  • Not alternative to traditional methods
  • A new tool still needs to be used thoughtfully