Experimental design
Download
1 / 57

Experimental Design - PowerPoint PPT Presentation


  • 295 Views
  • Updated On :

Experimental Design. Playing with variables. The nature of experiments. allow the investigator to control the research situation so that causal relationships among variables may be evaluated

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Experimental Design' - olympe


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Experimental design l.jpg

Experimental Design

Playing with variables


Slide2 l.jpg

The nature of experiments

  • allow the investigator to control the research situation so that causal relationships among variables may be evaluated

  • One variable is manipulated and its effect upon another variable is measured, while other variables are held constant


Slide3 l.jpg

So… you’ve decided to do an experiment

  • Decisions… decisions… decisions


Slide4 l.jpg

Decision 1: Independent Variable?

  • value is changed or altered independently of other variables

  • hypothesized to be the causal influence

  • categorical or continuous (?)

    Experimental Treatments:

  • alternative manipulations of the Independent Variable


Slide5 l.jpg

Experimental and Control Groups

  • Control Group

  • Experimental Groups

  • there can be more than one treatment level of the Independent Variable (basic or factorial)

  • there can be more than one IV

Experimental Groups


Slide6 l.jpg

Decision 2: Dependent Variable

  • The criterion or standard by which the results are judged

  • It is presumed that changes in the Dependent Variable are the result of changes in one or more Independent Variable

  • the choice of Dependent Variable determines the type of answer that is given to the research question


Slide7 l.jpg

Decision 3: Test units/unit of analysis

  • The subjects or entities whose responses to the experimental treatment are being measured

  • People are the most common test unit in business research


Slide8 l.jpg

Decision 4: Extraneous variables

  • A number of extraneous or “other” variables may affect the dependent variable and distort the results

    Conditions of constancy:

  • When extraneous variables cannot be eliminated we strive to hold Extraneous Variables constant for all subjects


Slide9 l.jpg

But, what about ___________?

  • Problems… problems…


Slide10 l.jpg

IMPACT OF THE RESEARCH SITUATION

Demand Characteristics: experimental design procedures that unintentionally hint to subjects about the experimenter’s hypothesis

  • rumour

  • instructions

  • status and personality of researcher

  • unintentional cues from experimenter

  • experimental procedure itself

  • Setting: Field versus Laboratory


Field versus laboratory l.jpg
Field versus Laboratory

  • Field experiments: usually used to fine-tune strategy and determine sales volume

  • Laboratory: used when control over the experimental setting is more important



Slide13 l.jpg

The Hawthorne effect

  • Subjects perform differently just because they know they are are experimental subjects

  • Western Electric’s Hawthorne Plant 1939 study of light intensity

    The Guinea Pig effect

  • exhibit the behaviour that they think is expected

  • Potential Solutions:

    • run experiment for a longer period

    • use a control group

    • Deception (?)


Slide14 l.jpg

Experimental Treatment Diffusion

  • if treatment condition perceived as very desirable relative to the control condition, members of the control group may seek access to the treatment condition

  • Potential Solutions:

    -have control group in another site

    -of course, this introduces new variables!


Slide15 l.jpg

John Henry Effect

  • legend of black railway worker

  • control group overcompensates

  • Potential Solutions:

    • don’t do threatening experiments

    • don’t set up obviously competitive situations

    • don’t tell control group that they are control group

      • conduct in another location somewhere else

      • unfortunately, produces new variable of different location, neighbourhood, etc.!


Slide16 l.jpg

Resentful Demoralization of Control Group

  • Control group artificially demoralized if perceives experimental group receiving desirable treatment being withheld from it

  • Potential Solutions?

    • what about giving control group some perk to compensate?

    • don’t tell them they are control group! (but what about informed consent?)… Use of Placebo… use of blinding…


Slide17 l.jpg

Getting control….

  • Design decisions


Slide18 l.jpg

  • Physical Control

    • Holding the value or level of extraneous variables constant throughout the course of an experiment.

  • Statistical Control

    • Adjusting for the effects of confounding variables by statistically adjusting the value of the dependent variable for each treatment conditions.

  • Design Control

    • Use of the experimental design to control extraneous causal factors.


Slide19 l.jpg

Blinding

  • Blinding is utilized to control subjects knowledge of whether or not they have been given a particular experimental treatment

  • double-blind experiment

  • secrecy

    • but then violate principle of informed consent

  • screen out or balance number of placebo reactors in treatment & control groups


Sampling l.jpg

Sampling

Who and How

And How to Screw It up


Terms l.jpg
Terms

  • Sample

  • Population (universe)

  • Population element

  • census


Why use a sample l.jpg
Why use a sample?

  • Cost

  • Speed

  • Sufficiently accurate (decreasing precision but maintaining accuracy)

  • More accurate than a census (?)

  • Destruction of test units


Slide23 l.jpg

Stages in the Selection of a Sample

Step 7: Conduct

Fieldwork

Step 2: Select

The Sampling

Frame

Step 3: Probability

OR Non-probability?

Step 1: Define the

the target population

Step 6: Select

Sampling units

Step 5: Determine

Sample Size

Step 4: Plan

Selection of

sampling

units


Step 1 target population l.jpg
Step 1: Target Population

  • The specific, complete group relevant to the research project

  • Who really has the information/data you need

  • How do you define your target population


Slide25 l.jpg


Operational definition l.jpg
Operational Definition

  • A definition that gives meaning to a concept by specifying the activities necessary to measure it.

    • “The population of interest is defined as all women in the City of Lethbridge who hold the most senior position in their organization.”

    • What variables need further definition?


Step 2 sampling frame l.jpg
Step 2: Sampling Frame

  • The list of elements from which a sample may be drawn.

    • Also known as: working population.

    • Examples?


Slide28 l.jpg

Sampling Frame Error:

  • error that occurs when certain sample elements are not listed or available and are not represented in the sampling frame.


Slide29 l.jpg

Sampling Units:

  • A single element or group of elements subject to selection in the sample.

    • Primary sampling unit

    • Secondary sampling unit


Slide30 l.jpg

Error: Less than perfectly.

representative samples.

  • Random sampling error.

    • Difference between the result of a sample and the result of a census conducted using identical procedures; a statistical fluctuation that occurs because of chance variation in the selection of the sample.


Slide31 l.jpg

…Error

  • Systematic or non-sampling error.

    • Results from some imperfect aspect of the research design that causes response error or from a mistake in the execution of the research

    • Examples: Sample bias, mistakes in recording responses, non-responses, mortality etc,.


Slide32 l.jpg

…Error

  • Non-response error.

    • The statistical difference between a survey that includes only those who responded and a survey that also includes those that failed to respond.


Step 3 choice l.jpg
Step 3: Choice!

  • Probability Sample:

    • A sampling technique in which every member of the population will have a known, nonzero probability of being selected


Step 3 choice34 l.jpg
Step 3: Choice!

  • Non-Probability Sample:

    • Units of the sample are chosen on the basis of personal judgment or convenience

    • There are no statistical techniques for measuring random sampling error in a non-probability sample. Therefore, generalizability is never statistically appropriate.


Slide35 l.jpg

Classification of Sampling Methods

Sampling

Methods

Probability

Samples

Non-

probability

Systematic

Stratified

Convenience

Snowball

Cluster

Simple

Random

Judgment

Quota


Slide36 l.jpg

Probability Sampling Methods

  • Simple Random Sampling

    • the purest form of probability sampling.

    • Assures each element in the population has an equal chance of being included in the sample

    • Random number generators

Sample Size

Probability of Selection =

Population Size


Slide37 l.jpg

  • Advantages

    • minimal knowledge of population needed

    • External validity high; internal validity high; statistical estimation of error

    • Easy to analyze data

  • Disadvantages

    • High cost; low frequency of use

    • Requires sampling frame

    • Does not use researchers’ expertise

    • Larger risk of random error than stratified


Slide38 l.jpg

  • Systematic Sampling

    • An initial starting point is selected by a random process, and then every nth number on the list is selected

    • n=sampling interval

      • The number of population elements between the units selected for the sample

      • Error: periodicity- the original list has a systematic pattern

      • ?? Is the list of elements randomized??


Slide39 l.jpg

  • Advantages

    • Moderate cost; moderate usage

    • External validity high; internal validity high; statistical estimation of error

    • Simple to draw sample; easy to verify

  • Disadvantages

    • Periodic ordering

    • Requires sampling frame


Slide40 l.jpg

  • Stratified Sampling

    • Sub-samples are randomly drawn from samples within different strata that are more or less equal on some characteristic

    • Why?

  • Can reduce random error

  • More accurately reflect the population by more proportional representation


Slide41 l.jpg

  • How?

    1.Identify variable(s) as an efficient basis for stratification. Must be known to be related to dependent variable. Usually a categorical variable

    2.Complete list of population elements must be obtained

    3.Use randomization to take a simple random sample from each stratum


Slide42 l.jpg

  • Types of Stratified Samples

    • Proportional Stratified Sample:

      • The number of sampling units drawn from each stratum is in proportion to the relative population size of that stratum

    • Disproportional Stratified Sample:

      • The number of sampling units drawn from each stratum is allocated according to analytical considerations e.g. as variability increases sample size of stratum should increase


Slide43 l.jpg

  • Types of Stratified Samples…

    • Optimal allocation stratified sample:

      • The number of sampling units drawn from each stratum is determined on the basis of both size and variation.

      • Calculated statistically


Slide44 l.jpg

  • Advantages

    • Assures representation of all groups in sample population needed

    • Characteristics of each stratum can be estimated and comparisons made

    • Reduces variability from systematic

  • Disadvantages

    • Requires accurate information on proportions of each stratum

    • Stratified lists costly to prepare


Slide45 l.jpg

  • Cluster Sampling

    • The primary sampling unit is not the individual element, but a large cluster of elements. Either the cluster is randomly selected or the elements within are randomly selected

    • Why?

  • Frequently used when no list of population available or because of cost

  • Ask: is the cluster as heterogeneous as the population? Can we assume it is representative?


Slide46 l.jpg

  • Cluster Sampling example

    • You are asked to create a sample of all Management students who are working in Lethbridge during the summer term

    • There is no such list available

    • Using stratified sampling, compile a list of businesses in Lethbridge to identify clusters

    • Individual workers within these clusters are selected to take part in study


Slide47 l.jpg

  • Types of Cluster Samples

    • Area sample:

      • Primary sampling unit is a geographical area

    • Multistage area sample:

      • Involves a combination of two or more types of probability sampling techniques. Typically, progressively smaller geographical areas are randomly selected in a series of steps


Slide48 l.jpg

  • Advantages

    • Low cost/high frequency of use

    • Requires list of all clusters, but only of individuals within chosen clusters

    • Can estimate characteristics of both cluster and population

    • For multistage, has strengths of used methods

  • Disadvantages

    • Larger error for comparable size than other probability methods

    • Multistage very expensive and validity depends on other methods used


Slide49 l.jpg

Classification of Sampling Methods

Sampling

Methods

Probability

Samples

Non-

probability

Systematic

Stratified

Convenience

Snowball

Cluster

Simple

Random

Judgment

Quota


Slide50 l.jpg

Non-Probability Sampling Methods

  • Convenience Sample

    • The sampling procedure used to obtain those units or people most conveniently available

    • Why: speed and cost

    • External validity?

    • Internal validity

    • Is it ever justified?


Slide51 l.jpg

  • Advantages

    • Very low cost

    • Extensively used/understood

    • No need for list of population elements

  • Disadvantages

    • Variability and bias cannot be measured or controlled

    • Projecting data beyond sample not justified.


Slide52 l.jpg

  • Judgment or Purposive Sample

    • The sampling procedure in which an experienced research selects the sample based on some appropriate characteristic of sample members… to serve a purpose


Slide53 l.jpg

  • Advantages

    • Moderate cost

    • Commonly used/understood

    • Sample will meet a specific objective

  • Disadvantages

    • Bias!

    • Projecting data beyond sample not justified.


Slide54 l.jpg

  • Quota Sample

    • The sampling procedure that ensure that a certain characteristic of a population sample will be represented to the exact extent that the investigator desires


Slide55 l.jpg

  • Advantages

    • moderate cost

    • Very extensively used/understood

    • No need for list of population elements

    • Introduces some elements of stratification

  • Disadvantages

    • Variability and bias cannot be measured or controlled (classification of subjects0

    • Projecting data beyond sample not justified.


Slide56 l.jpg

  • Snowball sampling

    • The sampling procedure in which the initial respondents are chosen by probability methods, and then additional respondents are obtained by information provided by the initial respondents


Slide57 l.jpg

  • Advantages

    • low cost

    • Useful in specific circumstances

    • Useful for locating rare populations

  • Disadvantages

    • Bias because sampling units not independent

    • Projecting data beyond sample not justified.


ad