Experimental design
1 / 57

Experimental Design - PowerPoint PPT Presentation

  • Updated On :

Experimental Design. Playing with variables. The nature of experiments. allow the investigator to control the research situation so that causal relationships among variables may be evaluated

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Experimental Design' - olympe

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Experimental design l.jpg

Experimental Design

Playing with variables

Slide2 l.jpg

The nature of experiments

  • allow the investigator to control the research situation so that causal relationships among variables may be evaluated

  • One variable is manipulated and its effect upon another variable is measured, while other variables are held constant

Slide3 l.jpg

So… you’ve decided to do an experiment

  • Decisions… decisions… decisions

Slide4 l.jpg

Decision 1: Independent Variable?

  • value is changed or altered independently of other variables

  • hypothesized to be the causal influence

  • categorical or continuous (?)

    Experimental Treatments:

  • alternative manipulations of the Independent Variable

Slide5 l.jpg

Experimental and Control Groups

  • Control Group

  • Experimental Groups

  • there can be more than one treatment level of the Independent Variable (basic or factorial)

  • there can be more than one IV

Experimental Groups

Slide6 l.jpg

Decision 2: Dependent Variable

  • The criterion or standard by which the results are judged

  • It is presumed that changes in the Dependent Variable are the result of changes in one or more Independent Variable

  • the choice of Dependent Variable determines the type of answer that is given to the research question

Slide7 l.jpg

Decision 3: Test units/unit of analysis

  • The subjects or entities whose responses to the experimental treatment are being measured

  • People are the most common test unit in business research

Slide8 l.jpg

Decision 4: Extraneous variables

  • A number of extraneous or “other” variables may affect the dependent variable and distort the results

    Conditions of constancy:

  • When extraneous variables cannot be eliminated we strive to hold Extraneous Variables constant for all subjects

Slide9 l.jpg

But, what about ___________?

  • Problems… problems…

Slide10 l.jpg


Demand Characteristics: experimental design procedures that unintentionally hint to subjects about the experimenter’s hypothesis

  • rumour

  • instructions

  • status and personality of researcher

  • unintentional cues from experimenter

  • experimental procedure itself

  • Setting: Field versus Laboratory

Field versus laboratory l.jpg
Field versus Laboratory

  • Field experiments: usually used to fine-tune strategy and determine sales volume

  • Laboratory: used when control over the experimental setting is more important

Slide13 l.jpg

The Hawthorne effect

  • Subjects perform differently just because they know they are are experimental subjects

  • Western Electric’s Hawthorne Plant 1939 study of light intensity

    The Guinea Pig effect

  • exhibit the behaviour that they think is expected

  • Potential Solutions:

    • run experiment for a longer period

    • use a control group

    • Deception (?)

Slide14 l.jpg

Experimental Treatment Diffusion

  • if treatment condition perceived as very desirable relative to the control condition, members of the control group may seek access to the treatment condition

  • Potential Solutions:

    -have control group in another site

    -of course, this introduces new variables!

Slide15 l.jpg

John Henry Effect

  • legend of black railway worker

  • control group overcompensates

  • Potential Solutions:

    • don’t do threatening experiments

    • don’t set up obviously competitive situations

    • don’t tell control group that they are control group

      • conduct in another location somewhere else

      • unfortunately, produces new variable of different location, neighbourhood, etc.!

Slide16 l.jpg

Resentful Demoralization of Control Group

  • Control group artificially demoralized if perceives experimental group receiving desirable treatment being withheld from it

  • Potential Solutions?

    • what about giving control group some perk to compensate?

    • don’t tell them they are control group! (but what about informed consent?)… Use of Placebo… use of blinding…

Slide17 l.jpg

Getting control….

  • Design decisions

Slide18 l.jpg

  • Physical Control

    • Holding the value or level of extraneous variables constant throughout the course of an experiment.

  • Statistical Control

    • Adjusting for the effects of confounding variables by statistically adjusting the value of the dependent variable for each treatment conditions.

  • Design Control

    • Use of the experimental design to control extraneous causal factors.

Slide19 l.jpg


  • Blinding is utilized to control subjects knowledge of whether or not they have been given a particular experimental treatment

  • double-blind experiment

  • secrecy

    • but then violate principle of informed consent

  • screen out or balance number of placebo reactors in treatment & control groups

Sampling l.jpg


Who and How

And How to Screw It up

Terms l.jpg

  • Sample

  • Population (universe)

  • Population element

  • census

Why use a sample l.jpg
Why use a sample?

  • Cost

  • Speed

  • Sufficiently accurate (decreasing precision but maintaining accuracy)

  • More accurate than a census (?)

  • Destruction of test units

Slide23 l.jpg

Stages in the Selection of a Sample

Step 7: Conduct


Step 2: Select

The Sampling


Step 3: Probability

OR Non-probability?

Step 1: Define the

the target population

Step 6: Select

Sampling units

Step 5: Determine

Sample Size

Step 4: Plan

Selection of



Step 1 target population l.jpg
Step 1: Target Population

  • The specific, complete group relevant to the research project

  • Who really has the information/data you need

  • How do you define your target population

Slide25 l.jpg

Operational definition l.jpg
Operational Definition

  • A definition that gives meaning to a concept by specifying the activities necessary to measure it.

    • “The population of interest is defined as all women in the City of Lethbridge who hold the most senior position in their organization.”

    • What variables need further definition?

Step 2 sampling frame l.jpg
Step 2: Sampling Frame

  • The list of elements from which a sample may be drawn.

    • Also known as: working population.

    • Examples?

Slide28 l.jpg

Sampling Frame Error:

  • error that occurs when certain sample elements are not listed or available and are not represented in the sampling frame.

Slide29 l.jpg

Sampling Units:

  • A single element or group of elements subject to selection in the sample.

    • Primary sampling unit

    • Secondary sampling unit

Slide30 l.jpg

Error: Less than perfectly.

representative samples.

  • Random sampling error.

    • Difference between the result of a sample and the result of a census conducted using identical procedures; a statistical fluctuation that occurs because of chance variation in the selection of the sample.

Slide31 l.jpg


  • Systematic or non-sampling error.

    • Results from some imperfect aspect of the research design that causes response error or from a mistake in the execution of the research

    • Examples: Sample bias, mistakes in recording responses, non-responses, mortality etc,.

Slide32 l.jpg


  • Non-response error.

    • The statistical difference between a survey that includes only those who responded and a survey that also includes those that failed to respond.

Step 3 choice l.jpg
Step 3: Choice!

  • Probability Sample:

    • A sampling technique in which every member of the population will have a known, nonzero probability of being selected

Step 3 choice34 l.jpg
Step 3: Choice!

  • Non-Probability Sample:

    • Units of the sample are chosen on the basis of personal judgment or convenience

    • There are no statistical techniques for measuring random sampling error in a non-probability sample. Therefore, generalizability is never statistically appropriate.

Slide35 l.jpg

Classification of Sampling Methods
















Slide36 l.jpg

Probability Sampling Methods

  • Simple Random Sampling

    • the purest form of probability sampling.

    • Assures each element in the population has an equal chance of being included in the sample

    • Random number generators

Sample Size

Probability of Selection =

Population Size

Slide37 l.jpg

  • Advantages

    • minimal knowledge of population needed

    • External validity high; internal validity high; statistical estimation of error

    • Easy to analyze data

  • Disadvantages

    • High cost; low frequency of use

    • Requires sampling frame

    • Does not use researchers’ expertise

    • Larger risk of random error than stratified

Slide38 l.jpg

  • Systematic Sampling

    • An initial starting point is selected by a random process, and then every nth number on the list is selected

    • n=sampling interval

      • The number of population elements between the units selected for the sample

      • Error: periodicity- the original list has a systematic pattern

      • ?? Is the list of elements randomized??

Slide39 l.jpg

  • Advantages

    • Moderate cost; moderate usage

    • External validity high; internal validity high; statistical estimation of error

    • Simple to draw sample; easy to verify

  • Disadvantages

    • Periodic ordering

    • Requires sampling frame

Slide40 l.jpg

  • Stratified Sampling

    • Sub-samples are randomly drawn from samples within different strata that are more or less equal on some characteristic

    • Why?

  • Can reduce random error

  • More accurately reflect the population by more proportional representation

Slide41 l.jpg

  • How?

    1.Identify variable(s) as an efficient basis for stratification. Must be known to be related to dependent variable. Usually a categorical variable

    2.Complete list of population elements must be obtained

    3.Use randomization to take a simple random sample from each stratum

Slide42 l.jpg

  • Types of Stratified Samples

    • Proportional Stratified Sample:

      • The number of sampling units drawn from each stratum is in proportion to the relative population size of that stratum

    • Disproportional Stratified Sample:

      • The number of sampling units drawn from each stratum is allocated according to analytical considerations e.g. as variability increases sample size of stratum should increase

Slide43 l.jpg

  • Types of Stratified Samples…

    • Optimal allocation stratified sample:

      • The number of sampling units drawn from each stratum is determined on the basis of both size and variation.

      • Calculated statistically

Slide44 l.jpg

  • Advantages

    • Assures representation of all groups in sample population needed

    • Characteristics of each stratum can be estimated and comparisons made

    • Reduces variability from systematic

  • Disadvantages

    • Requires accurate information on proportions of each stratum

    • Stratified lists costly to prepare

Slide45 l.jpg

  • Cluster Sampling

    • The primary sampling unit is not the individual element, but a large cluster of elements. Either the cluster is randomly selected or the elements within are randomly selected

    • Why?

  • Frequently used when no list of population available or because of cost

  • Ask: is the cluster as heterogeneous as the population? Can we assume it is representative?

Slide46 l.jpg

  • Cluster Sampling example

    • You are asked to create a sample of all Management students who are working in Lethbridge during the summer term

    • There is no such list available

    • Using stratified sampling, compile a list of businesses in Lethbridge to identify clusters

    • Individual workers within these clusters are selected to take part in study

Slide47 l.jpg

  • Types of Cluster Samples

    • Area sample:

      • Primary sampling unit is a geographical area

    • Multistage area sample:

      • Involves a combination of two or more types of probability sampling techniques. Typically, progressively smaller geographical areas are randomly selected in a series of steps

Slide48 l.jpg

  • Advantages

    • Low cost/high frequency of use

    • Requires list of all clusters, but only of individuals within chosen clusters

    • Can estimate characteristics of both cluster and population

    • For multistage, has strengths of used methods

  • Disadvantages

    • Larger error for comparable size than other probability methods

    • Multistage very expensive and validity depends on other methods used

Slide49 l.jpg

Classification of Sampling Methods
















Slide50 l.jpg

Non-Probability Sampling Methods

  • Convenience Sample

    • The sampling procedure used to obtain those units or people most conveniently available

    • Why: speed and cost

    • External validity?

    • Internal validity

    • Is it ever justified?

Slide51 l.jpg

  • Advantages

    • Very low cost

    • Extensively used/understood

    • No need for list of population elements

  • Disadvantages

    • Variability and bias cannot be measured or controlled

    • Projecting data beyond sample not justified.

Slide52 l.jpg

  • Judgment or Purposive Sample

    • The sampling procedure in which an experienced research selects the sample based on some appropriate characteristic of sample members… to serve a purpose

Slide53 l.jpg

  • Advantages

    • Moderate cost

    • Commonly used/understood

    • Sample will meet a specific objective

  • Disadvantages

    • Bias!

    • Projecting data beyond sample not justified.

Slide54 l.jpg

  • Quota Sample

    • The sampling procedure that ensure that a certain characteristic of a population sample will be represented to the exact extent that the investigator desires

Slide55 l.jpg

  • Advantages

    • moderate cost

    • Very extensively used/understood

    • No need for list of population elements

    • Introduces some elements of stratification

  • Disadvantages

    • Variability and bias cannot be measured or controlled (classification of subjects0

    • Projecting data beyond sample not justified.

Slide56 l.jpg

  • Snowball sampling

    • The sampling procedure in which the initial respondents are chosen by probability methods, and then additional respondents are obtained by information provided by the initial respondents

Slide57 l.jpg

  • Advantages

    • low cost

    • Useful in specific circumstances

    • Useful for locating rare populations

  • Disadvantages

    • Bias because sampling units not independent

    • Projecting data beyond sample not justified.