experimental design l.
Download
Skip this Video
Download Presentation
Experimental Design

Loading in 2 Seconds...

play fullscreen
1 / 57

Experimental Design - PowerPoint PPT Presentation


  • 296 Views
  • Uploaded on

Experimental Design. Playing with variables. The nature of experiments. allow the investigator to control the research situation so that causal relationships among variables may be evaluated

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Experimental Design' - olympe


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
experimental design

Experimental Design

Playing with variables

slide2

The nature of experiments

  • allow the investigator to control the research situation so that causal relationships among variables may be evaluated
  • One variable is manipulated and its effect upon another variable is measured, while other variables are held constant
slide3

So… you’ve decided to do an experiment

  • Decisions… decisions… decisions
slide4

Decision 1: Independent Variable?

  • value is changed or altered independently of other variables
  • hypothesized to be the causal influence
  • categorical or continuous (?)

Experimental Treatments:

  • alternative manipulations of the Independent Variable
slide5

Experimental and Control Groups

  • Control Group
  • Experimental Groups
  • there can be more than one treatment level of the Independent Variable (basic or factorial)
  • there can be more than one IV

Experimental Groups

slide6

Decision 2: Dependent Variable

  • The criterion or standard by which the results are judged
  • It is presumed that changes in the Dependent Variable are the result of changes in one or more Independent Variable
  • the choice of Dependent Variable determines the type of answer that is given to the research question
slide7

Decision 3: Test units/unit of analysis

  • The subjects or entities whose responses to the experimental treatment are being measured
  • People are the most common test unit in business research
slide8

Decision 4: Extraneous variables

  • A number of extraneous or “other” variables may affect the dependent variable and distort the results

Conditions of constancy:

  • When extraneous variables cannot be eliminated we strive to hold Extraneous Variables constant for all subjects
slide9

But, what about ___________?

  • Problems… problems…
slide10

IMPACT OF THE RESEARCH SITUATION

Demand Characteristics: experimental design procedures that unintentionally hint to subjects about the experimenter’s hypothesis

  • rumour
  • instructions
  • status and personality of researcher
  • unintentional cues from experimenter
  • experimental procedure itself
  • Setting: Field versus Laboratory
field versus laboratory
Field versus Laboratory
  • Field experiments: usually used to fine-tune strategy and determine sales volume
  • Laboratory: used when control over the experimental setting is more important
slide13

The Hawthorne effect

  • Subjects perform differently just because they know they are are experimental subjects
  • Western Electric’s Hawthorne Plant 1939 study of light intensity

The Guinea Pig effect

  • exhibit the behaviour that they think is expected
  • Potential Solutions:
    • run experiment for a longer period
    • use a control group
    • Deception (?)
slide14

Experimental Treatment Diffusion

  • if treatment condition perceived as very desirable relative to the control condition, members of the control group may seek access to the treatment condition
  • Potential Solutions:

-have control group in another site

-of course, this introduces new variables!

slide15

John Henry Effect

  • legend of black railway worker
  • control group overcompensates
  • Potential Solutions:
    • don’t do threatening experiments
    • don’t set up obviously competitive situations
    • don’t tell control group that they are control group
      • conduct in another location somewhere else
      • unfortunately, produces new variable of different location, neighbourhood, etc.!
slide16

Resentful Demoralization of Control Group

  • Control group artificially demoralized if perceives experimental group receiving desirable treatment being withheld from it
  • Potential Solutions?
    • what about giving control group some perk to compensate?
    • don’t tell them they are control group! (but what about informed consent?)… Use of Placebo… use of blinding…
slide17

Getting control….

  • Design decisions
slide18

Physical Control

    • Holding the value or level of extraneous variables constant throughout the course of an experiment.
  • Statistical Control
    • Adjusting for the effects of confounding variables by statistically adjusting the value of the dependent variable for each treatment conditions.
  • Design Control
    • Use of the experimental design to control extraneous causal factors.
slide19

Blinding

  • Blinding is utilized to control subjects knowledge of whether or not they have been given a particular experimental treatment
  • double-blind experiment
  • secrecy
    • but then violate principle of informed consent
  • screen out or balance number of placebo reactors in treatment & control groups
sampling

Sampling

Who and How

And How to Screw It up

terms
Terms
  • Sample
  • Population (universe)
  • Population element
  • census
why use a sample
Why use a sample?
  • Cost
  • Speed
  • Sufficiently accurate (decreasing precision but maintaining accuracy)
  • More accurate than a census (?)
  • Destruction of test units
slide23

Stages in the Selection of a Sample

Step 7: Conduct

Fieldwork

Step 2: Select

The Sampling

Frame

Step 3: Probability

OR Non-probability?

Step 1: Define the

the target population

Step 6: Select

Sampling units

Step 5: Determine

Sample Size

Step 4: Plan

Selection of

sampling

units

step 1 target population
Step 1: Target Population
  • The specific, complete group relevant to the research project
  • Who really has the information/data you need
  • How do you define your target population
operational definition
Operational Definition
  • A definition that gives meaning to a concept by specifying the activities necessary to measure it.
    • “The population of interest is defined as all women in the City of Lethbridge who hold the most senior position in their organization.”
    • What variables need further definition?
step 2 sampling frame
Step 2: Sampling Frame
  • The list of elements from which a sample may be drawn.
    • Also known as: working population.
    • Examples?
slide28
Sampling Frame Error:
  • error that occurs when certain sample elements are not listed or available and are not represented in the sampling frame.
slide29
Sampling Units:
  • A single element or group of elements subject to selection in the sample.
    • Primary sampling unit
    • Secondary sampling unit
slide30
Error: Less than perfectly.

representative samples.

  • Random sampling error.
    • Difference between the result of a sample and the result of a census conducted using identical procedures; a statistical fluctuation that occurs because of chance variation in the selection of the sample.
slide31
…Error
  • Systematic or non-sampling error.
    • Results from some imperfect aspect of the research design that causes response error or from a mistake in the execution of the research
    • Examples: Sample bias, mistakes in recording responses, non-responses, mortality etc,.
slide32
…Error
  • Non-response error.
    • The statistical difference between a survey that includes only those who responded and a survey that also includes those that failed to respond.
step 3 choice
Step 3: Choice!
  • Probability Sample:
    • A sampling technique in which every member of the population will have a known, nonzero probability of being selected
step 3 choice34
Step 3: Choice!
  • Non-Probability Sample:
    • Units of the sample are chosen on the basis of personal judgment or convenience
    • There are no statistical techniques for measuring random sampling error in a non-probability sample. Therefore, generalizability is never statistically appropriate.
slide35

Classification of Sampling Methods

Sampling

Methods

Probability

Samples

Non-

probability

Systematic

Stratified

Convenience

Snowball

Cluster

Simple

Random

Judgment

Quota

slide36

Probability Sampling Methods

  • Simple Random Sampling
    • the purest form of probability sampling.
    • Assures each element in the population has an equal chance of being included in the sample
    • Random number generators

Sample Size

Probability of Selection =

Population Size

slide37

Advantages

    • minimal knowledge of population needed
    • External validity high; internal validity high; statistical estimation of error
    • Easy to analyze data
  • Disadvantages
    • High cost; low frequency of use
    • Requires sampling frame
    • Does not use researchers’ expertise
    • Larger risk of random error than stratified
slide38

Systematic Sampling

    • An initial starting point is selected by a random process, and then every nth number on the list is selected
    • n=sampling interval
      • The number of population elements between the units selected for the sample
      • Error: periodicity- the original list has a systematic pattern
      • ?? Is the list of elements randomized??
slide39

Advantages

    • Moderate cost; moderate usage
    • External validity high; internal validity high; statistical estimation of error
    • Simple to draw sample; easy to verify
  • Disadvantages
    • Periodic ordering
    • Requires sampling frame
slide40

Stratified Sampling

    • Sub-samples are randomly drawn from samples within different strata that are more or less equal on some characteristic
    • Why?
  • Can reduce random error
  • More accurately reflect the population by more proportional representation
slide41

How?

1.Identify variable(s) as an efficient basis for stratification. Must be known to be related to dependent variable. Usually a categorical variable

2.Complete list of population elements must be obtained

3.Use randomization to take a simple random sample from each stratum

slide42

Types of Stratified Samples

    • Proportional Stratified Sample:
      • The number of sampling units drawn from each stratum is in proportion to the relative population size of that stratum
    • Disproportional Stratified Sample:
      • The number of sampling units drawn from each stratum is allocated according to analytical considerations e.g. as variability increases sample size of stratum should increase
slide43

Types of Stratified Samples…

    • Optimal allocation stratified sample:
      • The number of sampling units drawn from each stratum is determined on the basis of both size and variation.
      • Calculated statistically
slide44

Advantages

    • Assures representation of all groups in sample population needed
    • Characteristics of each stratum can be estimated and comparisons made
    • Reduces variability from systematic
  • Disadvantages
    • Requires accurate information on proportions of each stratum
    • Stratified lists costly to prepare
slide45

Cluster Sampling

    • The primary sampling unit is not the individual element, but a large cluster of elements. Either the cluster is randomly selected or the elements within are randomly selected
    • Why?
  • Frequently used when no list of population available or because of cost
  • Ask: is the cluster as heterogeneous as the population? Can we assume it is representative?
slide46

Cluster Sampling example

    • You are asked to create a sample of all Management students who are working in Lethbridge during the summer term
    • There is no such list available
    • Using stratified sampling, compile a list of businesses in Lethbridge to identify clusters
    • Individual workers within these clusters are selected to take part in study
slide47

Types of Cluster Samples

    • Area sample:
      • Primary sampling unit is a geographical area
    • Multistage area sample:
      • Involves a combination of two or more types of probability sampling techniques. Typically, progressively smaller geographical areas are randomly selected in a series of steps
slide48

Advantages

    • Low cost/high frequency of use
    • Requires list of all clusters, but only of individuals within chosen clusters
    • Can estimate characteristics of both cluster and population
    • For multistage, has strengths of used methods
  • Disadvantages
    • Larger error for comparable size than other probability methods
    • Multistage very expensive and validity depends on other methods used
slide49

Classification of Sampling Methods

Sampling

Methods

Probability

Samples

Non-

probability

Systematic

Stratified

Convenience

Snowball

Cluster

Simple

Random

Judgment

Quota

slide50

Non-Probability Sampling Methods

  • Convenience Sample
    • The sampling procedure used to obtain those units or people most conveniently available
    • Why: speed and cost
    • External validity?
    • Internal validity
    • Is it ever justified?
slide51

Advantages

    • Very low cost
    • Extensively used/understood
    • No need for list of population elements
  • Disadvantages
    • Variability and bias cannot be measured or controlled
    • Projecting data beyond sample not justified.
slide52

Judgment or Purposive Sample

    • The sampling procedure in which an experienced research selects the sample based on some appropriate characteristic of sample members… to serve a purpose
slide53

Advantages

    • Moderate cost
    • Commonly used/understood
    • Sample will meet a specific objective
  • Disadvantages
    • Bias!
    • Projecting data beyond sample not justified.
slide54

Quota Sample

    • The sampling procedure that ensure that a certain characteristic of a population sample will be represented to the exact extent that the investigator desires
slide55

Advantages

    • moderate cost
    • Very extensively used/understood
    • No need for list of population elements
    • Introduces some elements of stratification
  • Disadvantages
    • Variability and bias cannot be measured or controlled (classification of subjects0
    • Projecting data beyond sample not justified.
slide56

Snowball sampling

    • The sampling procedure in which the initial respondents are chosen by probability methods, and then additional respondents are obtained by information provided by the initial respondents
slide57

Advantages

    • low cost
    • Useful in specific circumstances
    • Useful for locating rare populations
  • Disadvantages
    • Bias because sampling units not independent
    • Projecting data beyond sample not justified.