1 / 33

Overview: Online Surveys

Handbook of Online Research Methods Colloquium 28-29 March 2007. Overview: Online Surveys. Vasja Vehovar University of Ljubljana, Slovenia http://WebSM.org. Introduction. Structure. Computer-assisted survey information collection Key methodological issues Related issues Applications.

carlotta
Download Presentation

Overview: Online Surveys

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Handbook of Online Research Methods Colloquium 28-29 March 2007 Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia http://WebSM.org

  2. Introduction Structure • Computer-assisted survey information collection • Key methodological issues • Related issues • Applications

  3. Computer-assisted survey information collection Technology and surveys 1930s: applications of probability sampling 1960s: expansion of telephone surveys 1970s: computer technology appears in surveys 1980s: computer-assisted surveying 1990s: Internet mediated surveys

  4. Computer-assisted survey information collection Early CASIC interviewer assisted modes • CATI – Computer-assisted telephone interviewing • CAPI – Computer-assisted personal interviewing • CASI – Computer-assisted self-interviewing

  5. Computer-assisted survey information collection CASIC benefits • Reduced time and costs for data input • Elimination of errors during data transcription • Implementation of advanced features: • automatic skips and branching • randomization of questions and response options • control of answers • inclusion of multimedia elements…

  6. Computer-assisted survey information collection Computerized self-administered questionnaires (CSAQ) Different modes of CSAQ: • disk-by-mail • touch-tone dataentry (TDE) • interactive voice response (IVR) • e-mail surveys • web surveys

  7. Computer-assisted survey information collection Online, Internet, Web, CASIC, CSAQ, CADAC Internet surveys Web surveys

  8. Computer-assisted survey information collection Online, Internet, Web, CASIC, CSAQ, CADAC CSAQ Internet surveys Web surveys

  9. Computer-assisted survey information collection Online, Internet, Web, CASIC, CSAQ, CADAC CSAQ Internet surveys Web surveys CADAC

  10. Computer-assisted survey information collection Interviewer-less and paper-less surveys

  11. Computer-assisted survey information collection Interviewer-less and paper-less data collection

  12. Computer-assisted survey information collection CSAQ input-output technology

  13. Computer-assisted survey information collection CSAQ input-output technology

  14. Computer-assisted survey information collection Technological aspects • Developments of the Internet and related technologies • Importance of broadband Internet access • Possibilities of distribution across various platforms and devices • Fast and easy implementation using specialized software tools

  15. Key methodological issues 1. Probability vs. non-probability sampling • Two major problems of probability samples in Internet surveys: • non-coverage • sampling frame problems • Image of Internet surveys as inherently non-probability ones

  16. Key methodological issues The spurious link Issue of a statistical inference: probability vs. non-probability samples Information-communication technologies Internet survey Management of the survey process Mode of survey data collection

  17. Key methodological issues 2. Sampling and invitation

  18. Key methodological issues 3. Non-response • Generally low and variable response rates to web surveys • Complexity of the response process • Measures to increase response rates: • incentives (e.g. using PayPal) • multiple follow-up contacts • other motivations

  19. Key methodological issues Involvement process Target population Complete respondents Operational population Partial respondents Frame population Starting units Sampled units Solicitation Clickers Absorbed units Aware units Informed units

  20. Key methodological issues Participation in web surveys Respondent’s characteristics Partially under researcher’s control No researcher’s control Researcher-respondent interaction Social environment Technological environment Survey design

  21. Key methodological issues 4. Questionnaire design • Question types and visual elements • Advanced features of computerized questionnaires • Inclusion of multimedia • Problems and considerations: • standardization • impact on respondent’s answers • technical problems measurement error

  22. Key methodological issues 5. Post-survey adjustments • Commonly utilized for correcting: • deviations from probability selections • non-coverage and non-response problems • Importance for non-probability Internet surveys • Promising approaches: • calibration methods • propensity score weighting • modelling and causal analysis

  23. Related issues 1. Costs, errors and management • The managerial process of survey implementation: • costs optimization • data quality • relations between data quality and costs • Cost effectiveness of web surveys: • costs for increasing response rates • evaluations of costs in relation to errors

  24. Related issues 2. Survey mode • Web surveys vs. other modes: • lower response rates (Lozar Manfreda et. al., 2007) • comparable or higher level of validity and reliability (e.g. Fricker et al., 2005; Chang & Krosnick, 2002)

  25. Related issues Mixing survey modes Solicitation Interview PAPI, CAPI, CATI, CASI,… Contact with the respondent personal, telephone, e-mail, mail,… Centralized data management

  26. Related issues Selecting the right mixture Structure of the data (MAR) Willingness to participate Modelling mixed modes Cost models and impact of environment Optimal combination Response rates and behaviour Data quality requirements

  27. Related issues Programing of the questionaire User-friendly interface Standardized questionnaire description in QML CAPI CATI CASI Paper questionnaire Internet survey

  28. Related issues 3. Mixing research methods • Quantitative and qualitative research methods • Flexible combinations of approaches (Morgan, 1998): • qualitative preliminary • quantitative preliminary • qualitative follow-up • quantitative follow-up • New possibilities with the Internet

  29. Related issues Flexible on-line mixed methods (FOMM) Analysis Qualitative method Quantitative method Conclusion Beginning

  30. Related issues 4. Ethics, guides and standards • New ethical dilemmas in survey research: • solicitation process, spam and privacy • data protection • children and minors • combining data… • Development of new standards (e.g. ESOMAR, AoIR, MRA…)

  31. Current and future applications Applications • Different applications according to: • type (internet access panels, on-line voting, intercept…) • organizations (official, academic, commercial) • topics (marketing research, polling opinions, health…) • complexity (simple one-shot surveys vs. complex integrateddata collection)

  32. Current and future applications Related areas • psychological research • online experiments • paradata and online measurement • specific research fields (usability research, HCI...) • emerging areas (e-learning, e-government…)

  33. Current and future applications Trends • Expansion of CASIC modes, options and mixtures • Integration of devices (TV, mobile,..) • Speech recognition and touch-screens • Multimedia and interactivity • Integration with other CADAC (GPS, paradata, observations,...) • Specialisation and professionalisation

More Related