- 143 Views
- Uploaded on
- Presentation posted in: General

Experimental Design

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Experimental Design

Playing with variables

The nature of experiments

- allow the investigator to control the research situation so that causal relationships among variables may be evaluated
- One variable is manipulated and its effect upon another variable is measured, while other variables are held constant

So… you’ve decided to do an experiment

- Decisions… decisions… decisions

Decision 1: Independent Variable?

- value is changed or altered independently of other variables
- hypothesized to be the causal influence
- categorical or continuous (?)
Experimental Treatments:

- alternative manipulations of the Independent Variable

Experimental and Control Groups

- Control Group
- Experimental Groups
- there can be more than one treatment level of the Independent Variable (basic or factorial)
- there can be more than one IV

Experimental Groups

Decision 2: Dependent Variable

- The criterion or standard by which the results are judged
- It is presumed that changes in the Dependent Variable are the result of changes in one or more Independent Variable
- the choice of Dependent Variable determines the type of answer that is given to the research question

Decision 3: Test units/unit of analysis

- The subjects or entities whose responses to the experimental treatment are being measured
- People are the most common test unit in business research

Decision 4: Extraneous variables

- A number of extraneous or “other” variables may affect the dependent variable and distort the results
Conditions of constancy:

- When extraneous variables cannot be eliminated we strive to hold Extraneous Variables constant for all subjects

But, what about ___________?

- Problems… problems…

IMPACT OF THE RESEARCH SITUATION

Demand Characteristics: experimental design procedures that unintentionally hint to subjects about the experimenter’s hypothesis

- rumour
- instructions
- status and personality of researcher
- unintentional cues from experimenter
- experimental procedure itself
- Setting: Field versus Laboratory

- Field experiments: usually used to fine-tune strategy and determine sales volume
- Laboratory: used when control over the experimental setting is more important

The Hawthorne effect

- Subjects perform differently just because they know they are are experimental subjects
- Western Electric’s Hawthorne Plant 1939 study of light intensity
The Guinea Pig effect

- exhibit the behaviour that they think is expected
- Potential Solutions:
- run experiment for a longer period
- use a control group
- Deception (?)

Experimental Treatment Diffusion

- if treatment condition perceived as very desirable relative to the control condition, members of the control group may seek access to the treatment condition
- Potential Solutions:
-have control group in another site

-of course, this introduces new variables!

John Henry Effect

- legend of black railway worker
- control group overcompensates
- Potential Solutions:
- don’t do threatening experiments
- don’t set up obviously competitive situations
- don’t tell control group that they are control group
- conduct in another location somewhere else
- unfortunately, produces new variable of different location, neighbourhood, etc.!

Resentful Demoralization of Control Group

- Control group artificially demoralized if perceives experimental group receiving desirable treatment being withheld from it
- Potential Solutions?
- what about giving control group some perk to compensate?
- don’t tell them they are control group! (but what about informed consent?)… Use of Placebo… use of blinding…

Getting control….

- Design decisions

- Physical Control
- Holding the value or level of extraneous variables constant throughout the course of an experiment.

- Statistical Control
- Adjusting for the effects of confounding variables by statistically adjusting the value of the dependent variable for each treatment conditions.

- Design Control
- Use of the experimental design to control extraneous causal factors.

Blinding

- Blinding is utilized to control subjects knowledge of whether or not they have been given a particular experimental treatment
- double-blind experiment
- secrecy
- but then violate principle of informed consent

- screen out or balance number of placebo reactors in treatment & control groups

Sampling

Who and How

And How to Screw It up

- Sample
- Population (universe)
- Population element
- census

- Cost
- Speed
- Sufficiently accurate (decreasing precision but maintaining accuracy)
- More accurate than a census (?)
- Destruction of test units

Stages in the Selection of a Sample

Step 7: Conduct

Fieldwork

Step 2: Select

The Sampling

Frame

Step 3: Probability

OR Non-probability?

Step 1: Define the

the target population

Step 6: Select

Sampling units

Step 5: Determine

Sample Size

Step 4: Plan

Selection of

sampling

units

- The specific, complete group relevant to the research project
- Who really has the information/data you need
- How do you define your target population

- Bases for defining the population of interest include:
- Geography
- Demographics
- Use
- Awareness

- A definition that gives meaning to a concept by specifying the activities necessary to measure it.
- “The population of interest is defined as all women in the City of Lethbridge who hold the most senior position in their organization.”
- What variables need further definition?

- The list of elements from which a sample may be drawn.
- Also known as: working population.
- Examples?

Sampling Frame Error:

- error that occurs when certain sample elements are not listed or available and are not represented in the sampling frame.

Sampling Units:

- A single element or group of elements subject to selection in the sample.
- Primary sampling unit
- Secondary sampling unit

Error: Less than perfectly.

representative samples.

- Random sampling error.
- Difference between the result of a sample and the result of a census conducted using identical procedures; a statistical fluctuation that occurs because of chance variation in the selection of the sample.

…Error

- Systematic or non-sampling error.
- Results from some imperfect aspect of the research design that causes response error or from a mistake in the execution of the research
- Examples: Sample bias, mistakes in recording responses, non-responses, mortality etc,.

…Error

- Non-response error.
- The statistical difference between a survey that includes only those who responded and a survey that also includes those that failed to respond.

- Probability Sample:
- A sampling technique in which every member of the population will have a known, nonzero probability of being selected

- Non-Probability Sample:
- Units of the sample are chosen on the basis of personal judgment or convenience
- There are no statistical techniques for measuring random sampling error in a non-probability sample. Therefore, generalizability is never statistically appropriate.

Classification of Sampling Methods

Sampling

Methods

Probability

Samples

Non-

probability

Systematic

Stratified

Convenience

Snowball

Cluster

Simple

Random

Judgment

Quota

Probability Sampling Methods

- Simple Random Sampling
- the purest form of probability sampling.
- Assures each element in the population has an equal chance of being included in the sample
- Random number generators

Sample Size

Probability of Selection =

Population Size

- Advantages
- minimal knowledge of population needed
- External validity high; internal validity high; statistical estimation of error
- Easy to analyze data

- Disadvantages
- High cost; low frequency of use
- Requires sampling frame
- Does not use researchers’ expertise
- Larger risk of random error than stratified

- Systematic Sampling
- An initial starting point is selected by a random process, and then every nth number on the list is selected
- n=sampling interval
- The number of population elements between the units selected for the sample
- Error: periodicity- the original list has a systematic pattern
- ?? Is the list of elements randomized??

- Advantages
- Moderate cost; moderate usage
- External validity high; internal validity high; statistical estimation of error
- Simple to draw sample; easy to verify

- Disadvantages
- Periodic ordering
- Requires sampling frame

- Stratified Sampling
- Sub-samples are randomly drawn from samples within different strata that are more or less equal on some characteristic
- Why?

- Can reduce random error
- More accurately reflect the population by more proportional representation

- How?
1.Identify variable(s) as an efficient basis for stratification. Must be known to be related to dependent variable. Usually a categorical variable

2.Complete list of population elements must be obtained

3.Use randomization to take a simple random sample from each stratum

- Types of Stratified Samples
- Proportional Stratified Sample:
- The number of sampling units drawn from each stratum is in proportion to the relative population size of that stratum

- Disproportional Stratified Sample:
- The number of sampling units drawn from each stratum is allocated according to analytical considerations e.g. as variability increases sample size of stratum should increase

- Proportional Stratified Sample:

- Types of Stratified Samples…
- Optimal allocation stratified sample:
- The number of sampling units drawn from each stratum is determined on the basis of both size and variation.
- Calculated statistically

- Optimal allocation stratified sample:

- Advantages
- Assures representation of all groups in sample population needed
- Characteristics of each stratum can be estimated and comparisons made
- Reduces variability from systematic

- Disadvantages
- Requires accurate information on proportions of each stratum
- Stratified lists costly to prepare

- Cluster Sampling
- The primary sampling unit is not the individual element, but a large cluster of elements. Either the cluster is randomly selected or the elements within are randomly selected
- Why?

- Frequently used when no list of population available or because of cost
- Ask: is the cluster as heterogeneous as the population? Can we assume it is representative?

- Cluster Sampling example
- You are asked to create a sample of all Management students who are working in Lethbridge during the summer term
- There is no such list available
- Using stratified sampling, compile a list of businesses in Lethbridge to identify clusters
- Individual workers within these clusters are selected to take part in study

- Types of Cluster Samples
- Area sample:
- Primary sampling unit is a geographical area

- Multistage area sample:
- Involves a combination of two or more types of probability sampling techniques. Typically, progressively smaller geographical areas are randomly selected in a series of steps

- Area sample:

- Advantages
- Low cost/high frequency of use
- Requires list of all clusters, but only of individuals within chosen clusters
- Can estimate characteristics of both cluster and population
- For multistage, has strengths of used methods

- Disadvantages
- Larger error for comparable size than other probability methods
- Multistage very expensive and validity depends on other methods used

Classification of Sampling Methods

Sampling

Methods

Probability

Samples

Non-

probability

Systematic

Stratified

Convenience

Snowball

Cluster

Simple

Random

Judgment

Quota

Non-Probability Sampling Methods

- Convenience Sample
- The sampling procedure used to obtain those units or people most conveniently available
- Why: speed and cost
- External validity?
- Internal validity
- Is it ever justified?

- Advantages
- Very low cost
- Extensively used/understood
- No need for list of population elements

- Disadvantages
- Variability and bias cannot be measured or controlled
- Projecting data beyond sample not justified.

- Judgment or Purposive Sample
- The sampling procedure in which an experienced research selects the sample based on some appropriate characteristic of sample members… to serve a purpose

- Advantages
- Moderate cost
- Commonly used/understood
- Sample will meet a specific objective

- Disadvantages
- Bias!
- Projecting data beyond sample not justified.

- Quota Sample
- The sampling procedure that ensure that a certain characteristic of a population sample will be represented to the exact extent that the investigator desires

- Advantages
- moderate cost
- Very extensively used/understood
- No need for list of population elements
- Introduces some elements of stratification

- Disadvantages
- Variability and bias cannot be measured or controlled (classification of subjects0
- Projecting data beyond sample not justified.

- Snowball sampling
- The sampling procedure in which the initial respondents are chosen by probability methods, and then additional respondents are obtained by information provided by the initial respondents

- Advantages
- low cost
- Useful in specific circumstances
- Useful for locating rare populations

- Disadvantages
- Bias because sampling units not independent
- Projecting data beyond sample not justified.