research design and tools for internet research
Skip this Video
Download Presentation
Research Design and Tools for Internet Research

Loading in 2 Seconds...

play fullscreen
1 / 19

Research Design and Tools for Internet Research - PowerPoint PPT Presentation

  • Uploaded on

Research Design and Tools for Internet Research. Claire Hewson University of Bolton ESRC e-society Programme and Sage Handbook of Online Research Methods Colloquium, 28-29 March, 2007. Aims.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Research Design and Tools for Internet Research' - cece

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
research design and tools for internet research

Research Design and Tools for Internet Research

Claire Hewson

University of Bolton

ESRC e-society Programme and Sage Handbook of Online Research Methods Colloquium, 28-29 March, 2007

  • Summarise some key issues and principles involved in designing Internet-mediated research (IMR) studies
  • Consider some of the approaches possible (tools, procedures), and reasons for choosing between these alternatives
  • Highlight areas which need further exploration, evaluation and development in relation to IMR design principles
research design in imr
Research Design in IMR
  • Research design - the process of making choices at various levels, or stages, within the research (including developing a research question, specifying a theoretical orientation, selecting data collection and analysis procedures)
  • Research design in IMR - focus primarily on the data-gathering stage
why use imr
Why Use IMR?
  • Advantages
    • cost / time efficiency (no data input costs, or multiple copies of materials, etc.)
    • easy access to potentially vast, geographically diverse participant pool (IUP) [enhanced reach, statistical power, access to small specialist populations]
    • unique features of online environment can lead to various potential benefits (e.g. reduced social desirability effects, increased candour, balanced power relationships, etc.)
why use imr5
Why Use IMR?
  • Disadvantages
    • biased nature of IUP(?) [inherent bias in Internet-accessed samples; generalisability issues]
    • reduced levels of control (over procedures / stimuli, participation context, participant behaviour)
    • features of online interaction could lead to various disadvantages (e.g. ambiguities in communication due to lack of extra-linguistic cues)
tools procedures design considerations in quantitative imr
Tools, Procedures, Design Considerations in Quantitative IMR
  • Design choices essentially linked to the criteria of validity, reliability and generalisability
  • Surveys, experiments, observation, and document analysis, have been implemented to date
internet surveys
Internet Surveys
  • Tools:
    • www forms, email, ftp; lower or higher tech solutions (e.g. javascript, server-side scripting); audio / video capabilities
  • Issues influencing design choices:
    • levels of expertise / equipment available
    • levels of control required (over presentation / response format)
    • sophistication of procedures / functionalities required (ID-checking, etc.)
    • data security (validity / ethical concerns)
internet surveys8
Internet Surveys
  • Overall, web-based approaches offer far greater scope and are likely to be preferable for most types of survey-based research, issues relating to levels of technical support / ease of implementation aside
internet experiments
Internet Experiments
  • Tools:
    • essentially same range of tools and approaches as for survey-based research, though web-based approaches likely to be required in many cases (precise timings, control over presentation, etc.)
  • Issues influencing design choices:
    • essentially, the same issues as with survey-based approaches emerge (ease of implementation, levels of control, functionalities required, data security)
internet experiments10
Internet Experiments
  • Overall, web-based approaches are almost always to be preferred (and will often be necessary) in experimental IMR, given the levels of control and sophistication of functionalities that will often be required. Though in some research contexts alternative approaches may be possible (e.g. emailing experimental text-based materials to participants, e.g. Hewson, 1994)
summary of imr design in quantitative research
Summary of IMR Design in Quantitative Research
  • A trade-off emerges between simple, low tech implementations which may suffer reliability and validity issues, and higher tech solutions which allow greater levels of control (etc.), but place greater demands on the researcher (and possibly participant, perhaps restricting access and thus affecting generalisability). Clearly specific research aims and goals, as well as practical considerations, will direct the design choices made.
tools procedures design considerations in qualitative imr
Tools, Procedures, Design Considerations in Qualitative IMR
  • Design choices essentially linked to criteria of trustworthiness (dependability, etc.)
  • Interviews, observation, and document analysis, have been implemented to date
internet interviews
Internet Interviews
  • Tools:
    • asynchronous (email, mailing lists, discussion groups) and synchronous (IM, online chat) approaches; audio / video capabilities
  • Issues influencing design choices:
    • nature of interaction / data desired (rich, detailed, reflective; factual, quick to obtain; candid, sensitive, personal, etc. [relates to: timescale; levels of anonymity; availability of extra-linguistic information])
    • demands placed on participant and researcher (skill / experience levels, etc.)
    • reach, access and diversity (e.g. issue of different time zones, accessibilty by e.g. disabled participants)
internet observation document analysis
Internet Observation / Document Analysis
  • Tools:
    • essentially same range as for interviews, though often archives of online interactions will be most useful, which tend to involve asynchronous communications (real-time observation is also feasible, and observation beyond text-based interactions, e.g. in MU*s)
  • Issues influencing design choices:
    • specificity of material required (e.g. rumour transmission, Bordia, 1996)
    • resources / expertise available (e.g. chat software)
    • disclosed / undisclosed (relates to research aims, and ethics [e.g. public / private domain])
    • naturalistic / solicited (e.g. Hessler et al, 2003)
summary of imr design in qualitative research
Summary of IMR Design in Qualitative Research
  • Trade-offs will tend to emerge, e.g. asynchronous [depth, reflexivity, empowering participants, accessibility / reach] versus synchronous [enhanced flow, extra-linguistic information, speedier]. Greater anonymity may enhance candour, but inhibit rapport. Audio / video approaches currently suffer reliability issues. Cross-cultural approaches would seem to demand asynchronous methods. Naturalistic (archives) versus solicited approaches. Ethical considerations (disclosure, consent, anonymity / confidentiality).
sampling issues
Sampling Issues
  • Tools:
    • email, discussion group / mailing list postings, www adverts, dedicated IMR study pages, sampling offline
  • Issues:
    • inherent biases in IUP
    • probability sampling problematic (of IUP, and of sub-populations, e.g. dormant email addresses, etc.)
    • sampling procedures may lead to biases, but difficulty in measuring sampling frame / nature of biases
    • how to maximise response (follow-ups; incentives; issue salience; group readership / volume of postings)
    • ethics (netiquette, e.g. permission of moderators, topic relevance; email requests?)
imr some proposed good design principles
IMR - Some Proposed Good Design Principles
  • Use procedures which can restrict variations on parameters which must crucially remain constant to ensure validity / reliability, but which maximise accessibility
  • Gather as much information as feasible to help detect unwanted variations / validity threats
  • Provide clear, explicit instructions, beyond what may normally be necessary for offline approaches
  • Pilot extensively (e.g. in different browsers)
imr some proposed good design principles18
IMR - Some Proposed Good Design Principles
  • Aim for procedures where sampling frame can be estimated, and sample bias assessed, when generalisability is important
  • Carefully consider ethical issues, especially when sensitive / personal information is being obtained
  • Consider features of synchronous and asynchronous interactions, and their effects, and effects of greater or lower levels of anonymity, and assess the relevant trade-offs in order to select procedures that best meet research aims / goals
issues for further research in imr design
Issues for Further Research in IMR Design
  • Validation studies which support IMR procedures (e.g. adaption of psychometric tests, experimental procedures)
  • Nature of IUP and relationship between sampling procedures and sample characteristics
  • Effects of online interactional medium
  • Issues surrounding public / private domain online
  • Reliability of technologies (e.g. audio / video)
  • Further piloting and exploration of advantages / disadvantages of novel approaches, e.g. mixed methods studies