Experimental design l.jpg
Sponsored Links
This presentation is the property of its rightful owner.
1 / 57

Experimental Design PowerPoint PPT Presentation

  • Updated On :
  • Presentation posted in: General

Experimental Design. Playing with variables. The nature of experiments. allow the investigator to control the research situation so that causal relationships among variables may be evaluated

Download Presentation

Experimental Design

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Experimental design l.jpg

Experimental Design

Playing with variables

Slide2 l.jpg

The nature of experiments

  • allow the investigator to control the research situation so that causal relationships among variables may be evaluated

  • One variable is manipulated and its effect upon another variable is measured, while other variables are held constant

Slide3 l.jpg

So… you’ve decided to do an experiment

  • Decisions… decisions… decisions

Slide4 l.jpg

Decision 1: Independent Variable?

  • value is changed or altered independently of other variables

  • hypothesized to be the causal influence

  • categorical or continuous (?)

    Experimental Treatments:

  • alternative manipulations of the Independent Variable

Slide5 l.jpg

Experimental and Control Groups

  • Control Group

  • Experimental Groups

  • there can be more than one treatment level of the Independent Variable (basic or factorial)

  • there can be more than one IV

Experimental Groups

Slide6 l.jpg

Decision 2: Dependent Variable

  • The criterion or standard by which the results are judged

  • It is presumed that changes in the Dependent Variable are the result of changes in one or more Independent Variable

  • the choice of Dependent Variable determines the type of answer that is given to the research question

Slide7 l.jpg

Decision 3: Test units/unit of analysis

  • The subjects or entities whose responses to the experimental treatment are being measured

  • People are the most common test unit in business research

Slide8 l.jpg

Decision 4: Extraneous variables

  • A number of extraneous or “other” variables may affect the dependent variable and distort the results

    Conditions of constancy:

  • When extraneous variables cannot be eliminated we strive to hold Extraneous Variables constant for all subjects

Slide9 l.jpg

But, what about ___________?

  • Problems… problems…

Slide10 l.jpg


Demand Characteristics: experimental design procedures that unintentionally hint to subjects about the experimenter’s hypothesis

  • rumour

  • instructions

  • status and personality of researcher

  • unintentional cues from experimenter

  • experimental procedure itself

  • Setting: Field versus Laboratory

Field versus laboratory l.jpg

Field versus Laboratory

  • Field experiments: usually used to fine-tune strategy and determine sales volume

  • Laboratory: used when control over the experimental setting is more important

Experimental design effects l.jpg

Experimental Design effects….

Slide13 l.jpg

The Hawthorne effect

  • Subjects perform differently just because they know they are are experimental subjects

  • Western Electric’s Hawthorne Plant 1939 study of light intensity

    The Guinea Pig effect

  • exhibit the behaviour that they think is expected

  • Potential Solutions:

    • run experiment for a longer period

    • use a control group

    • Deception (?)

Slide14 l.jpg

Experimental Treatment Diffusion

  • if treatment condition perceived as very desirable relative to the control condition, members of the control group may seek access to the treatment condition

  • Potential Solutions:

    -have control group in another site

    -of course, this introduces new variables!

Slide15 l.jpg

John Henry Effect

  • legend of black railway worker

  • control group overcompensates

  • Potential Solutions:

    • don’t do threatening experiments

    • don’t set up obviously competitive situations

    • don’t tell control group that they are control group

      • conduct in another location somewhere else

      • unfortunately, produces new variable of different location, neighbourhood, etc.!

Slide16 l.jpg

Resentful Demoralization of Control Group

  • Control group artificially demoralized if perceives experimental group receiving desirable treatment being withheld from it

  • Potential Solutions?

    • what about giving control group some perk to compensate?

    • don’t tell them they are control group! (but what about informed consent?)… Use of Placebo… use of blinding…

Slide17 l.jpg

Getting control….

  • Design decisions

Slide18 l.jpg

  • Physical Control

    • Holding the value or level of extraneous variables constant throughout the course of an experiment.

  • Statistical Control

    • Adjusting for the effects of confounding variables by statistically adjusting the value of the dependent variable for each treatment conditions.

  • Design Control

    • Use of the experimental design to control extraneous causal factors.

Slide19 l.jpg


  • Blinding is utilized to control subjects knowledge of whether or not they have been given a particular experimental treatment

  • double-blind experiment

  • secrecy

    • but then violate principle of informed consent

  • screen out or balance number of placebo reactors in treatment & control groups

Sampling l.jpg


Who and How

And How to Screw It up

Terms l.jpg


  • Sample

  • Population (universe)

  • Population element

  • census

Why use a sample l.jpg

Why use a sample?

  • Cost

  • Speed

  • Sufficiently accurate (decreasing precision but maintaining accuracy)

  • More accurate than a census (?)

  • Destruction of test units

Slide23 l.jpg

Stages in the Selection of a Sample

Step 7: Conduct


Step 2: Select

The Sampling


Step 3: Probability

OR Non-probability?

Step 1: Define the

the target population

Step 6: Select

Sampling units

Step 5: Determine

Sample Size

Step 4: Plan

Selection of



Step 1 target population l.jpg

Step 1: Target Population

  • The specific, complete group relevant to the research project

  • Who really has the information/data you need

  • How do you define your target population

Slide25 l.jpg

  • Bases for defining the population of interest include:

    • Geography

    • Demographics

    • Use

    • Awareness

Operational definition l.jpg

Operational Definition

  • A definition that gives meaning to a concept by specifying the activities necessary to measure it.

    • “The population of interest is defined as all women in the City of Lethbridge who hold the most senior position in their organization.”

    • What variables need further definition?

Step 2 sampling frame l.jpg

Step 2: Sampling Frame

  • The list of elements from which a sample may be drawn.

    • Also known as: working population.

    • Examples?

Slide28 l.jpg

Sampling Frame Error:

  • error that occurs when certain sample elements are not listed or available and are not represented in the sampling frame.

Slide29 l.jpg

Sampling Units:

  • A single element or group of elements subject to selection in the sample.

    • Primary sampling unit

    • Secondary sampling unit

Slide30 l.jpg

Error: Less than perfectly.

representative samples.

  • Random sampling error.

    • Difference between the result of a sample and the result of a census conducted using identical procedures; a statistical fluctuation that occurs because of chance variation in the selection of the sample.

Slide31 l.jpg


  • Systematic or non-sampling error.

    • Results from some imperfect aspect of the research design that causes response error or from a mistake in the execution of the research

    • Examples: Sample bias, mistakes in recording responses, non-responses, mortality etc,.

Slide32 l.jpg


  • Non-response error.

    • The statistical difference between a survey that includes only those who responded and a survey that also includes those that failed to respond.

Step 3 choice l.jpg

Step 3: Choice!

  • Probability Sample:

    • A sampling technique in which every member of the population will have a known, nonzero probability of being selected

Step 3 choice34 l.jpg

Step 3: Choice!

  • Non-Probability Sample:

    • Units of the sample are chosen on the basis of personal judgment or convenience

    • There are no statistical techniques for measuring random sampling error in a non-probability sample. Therefore, generalizability is never statistically appropriate.

Slide35 l.jpg

Classification of Sampling Methods
















Slide36 l.jpg

Probability Sampling Methods

  • Simple Random Sampling

    • the purest form of probability sampling.

    • Assures each element in the population has an equal chance of being included in the sample

    • Random number generators

Sample Size

Probability of Selection =

Population Size

Slide37 l.jpg

  • Advantages

    • minimal knowledge of population needed

    • External validity high; internal validity high; statistical estimation of error

    • Easy to analyze data

  • Disadvantages

    • High cost; low frequency of use

    • Requires sampling frame

    • Does not use researchers’ expertise

    • Larger risk of random error than stratified

Slide38 l.jpg

  • Systematic Sampling

    • An initial starting point is selected by a random process, and then every nth number on the list is selected

    • n=sampling interval

      • The number of population elements between the units selected for the sample

      • Error: periodicity- the original list has a systematic pattern

      • ?? Is the list of elements randomized??

Slide39 l.jpg

  • Advantages

    • Moderate cost; moderate usage

    • External validity high; internal validity high; statistical estimation of error

    • Simple to draw sample; easy to verify

  • Disadvantages

    • Periodic ordering

    • Requires sampling frame

Slide40 l.jpg

  • Stratified Sampling

    • Sub-samples are randomly drawn from samples within different strata that are more or less equal on some characteristic

    • Why?

  • Can reduce random error

  • More accurately reflect the population by more proportional representation

Slide41 l.jpg

  • How?

    1.Identify variable(s) as an efficient basis for stratification. Must be known to be related to dependent variable. Usually a categorical variable

    2.Complete list of population elements must be obtained

    3.Use randomization to take a simple random sample from each stratum

Slide42 l.jpg

  • Types of Stratified Samples

    • Proportional Stratified Sample:

      • The number of sampling units drawn from each stratum is in proportion to the relative population size of that stratum

    • Disproportional Stratified Sample:

      • The number of sampling units drawn from each stratum is allocated according to analytical considerations e.g. as variability increases sample size of stratum should increase

Slide43 l.jpg

  • Types of Stratified Samples…

    • Optimal allocation stratified sample:

      • The number of sampling units drawn from each stratum is determined on the basis of both size and variation.

      • Calculated statistically

Slide44 l.jpg

  • Advantages

    • Assures representation of all groups in sample population needed

    • Characteristics of each stratum can be estimated and comparisons made

    • Reduces variability from systematic

  • Disadvantages

    • Requires accurate information on proportions of each stratum

    • Stratified lists costly to prepare

Slide45 l.jpg

  • Cluster Sampling

    • The primary sampling unit is not the individual element, but a large cluster of elements. Either the cluster is randomly selected or the elements within are randomly selected

    • Why?

  • Frequently used when no list of population available or because of cost

  • Ask: is the cluster as heterogeneous as the population? Can we assume it is representative?

Slide46 l.jpg

  • Cluster Sampling example

    • You are asked to create a sample of all Management students who are working in Lethbridge during the summer term

    • There is no such list available

    • Using stratified sampling, compile a list of businesses in Lethbridge to identify clusters

    • Individual workers within these clusters are selected to take part in study

Slide47 l.jpg

  • Types of Cluster Samples

    • Area sample:

      • Primary sampling unit is a geographical area

    • Multistage area sample:

      • Involves a combination of two or more types of probability sampling techniques. Typically, progressively smaller geographical areas are randomly selected in a series of steps

Slide48 l.jpg

  • Advantages

    • Low cost/high frequency of use

    • Requires list of all clusters, but only of individuals within chosen clusters

    • Can estimate characteristics of both cluster and population

    • For multistage, has strengths of used methods

  • Disadvantages

    • Larger error for comparable size than other probability methods

    • Multistage very expensive and validity depends on other methods used

Slide49 l.jpg

Classification of Sampling Methods
















Slide50 l.jpg

Non-Probability Sampling Methods

  • Convenience Sample

    • The sampling procedure used to obtain those units or people most conveniently available

    • Why: speed and cost

    • External validity?

    • Internal validity

    • Is it ever justified?

Slide51 l.jpg

  • Advantages

    • Very low cost

    • Extensively used/understood

    • No need for list of population elements

  • Disadvantages

    • Variability and bias cannot be measured or controlled

    • Projecting data beyond sample not justified.

Slide52 l.jpg

  • Judgment or Purposive Sample

    • The sampling procedure in which an experienced research selects the sample based on some appropriate characteristic of sample members… to serve a purpose

Slide53 l.jpg

  • Advantages

    • Moderate cost

    • Commonly used/understood

    • Sample will meet a specific objective

  • Disadvantages

    • Bias!

    • Projecting data beyond sample not justified.

Slide54 l.jpg

  • Quota Sample

    • The sampling procedure that ensure that a certain characteristic of a population sample will be represented to the exact extent that the investigator desires

Slide55 l.jpg

  • Advantages

    • moderate cost

    • Very extensively used/understood

    • No need for list of population elements

    • Introduces some elements of stratification

  • Disadvantages

    • Variability and bias cannot be measured or controlled (classification of subjects0

    • Projecting data beyond sample not justified.

Slide56 l.jpg

  • Snowball sampling

    • The sampling procedure in which the initial respondents are chosen by probability methods, and then additional respondents are obtained by information provided by the initial respondents

Slide57 l.jpg

  • Advantages

    • low cost

    • Useful in specific circumstances

    • Useful for locating rare populations

  • Disadvantages

    • Bias because sampling units not independent

    • Projecting data beyond sample not justified.

  • Login