experiments l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Lecture 5 PowerPoint Presentation
Download Presentation
Lecture 5

Loading in 2 Seconds...

play fullscreen
1 / 30

Lecture 5 - PowerPoint PPT Presentation


  • 175 Views
  • Uploaded on

Experiments . Lecture 5. Agenda. Labs (update and questions) STATA Introduction Intro to Experiments and Experimental Design. Experiments vs Non-Experiments. Experiment is a study in which a treatment is introduced

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Lecture 5' - adlai


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
agenda
Agenda
  • Labs (update and questions)
  • STATA Introduction
  • Intro to Experiments and Experimental Design
experiments vs non experiments
Experiments vs Non-Experiments
  • Experiment is a study in which a treatment is introduced
  • The type of experiment (such as a true, randomized experiment) depends on several factors
  • Non-Experiments include responses from natural groups (e.g., most surveys)

http://imgs.xkcd.com/comics/the_data_so_far.png

famous experiments
Famous Experiments

Tuskegee Study

Milgrim Obedience Study

Stanford Prison Experiment

two essential criteria in true randomized experimental design
Two Essential Criteria in True Randomized Experimental Design
  • (1) Independent Variables must be manipulated (usually by experimenter, sometimes by context)
  • (2) Participants must be assigned randomly to various conditions or groups

Courtesy http://psychology.ucdavis.edu/SommerB/

active versus attribute independent variables
Active versus Attribute Independent Variables
  • Active independent variable(s):
    • The I.V. is “given” to the participants, usually for some specified time period. It is often manipulated and controlled by the investigator.
  • Attribute independent variable(s):
    • A predictive, defining characteristic of individuals. Cannot be manipulated.
randomization in sample and assignment
Randomization in Sample and Assignment
  • Random Sample
    • System for choosing participants from a population
    • Generally, the larger the sampling population the better your generalizability becomes.
  • Random Assignment
    • Method for assigning participants randomly to experiment conditions
true experiments
True Experiments
  • True experiments protect against both time and group threats to internal validity by randomly assigning subjects to treatment and control groups. The treatment (independent variable) is active.
  • If we cannot randomly assign subjects to different groups, then it is a quasi-experiment. The independent variable is active.
  • If we cannot randomly assign subjects to groups because the groups contain the attribute of interest, and if we give all groups the same treatment, then it is an associational non-experiment. The independent variable is not active.
pre test experimental manipulation and post testing
Pre-test, experimental manipulation and post-testing
  • Pre-test: allows us to check group equivalence before the intervention X is introduced.
  • Experimental manipulation: An independent variable (X) that the experimenter manipulates.
  • Post-test: allows us to check group equivalence after intervention X has been introduced.

Hypothesis  Random Assignment  Measure D.V.  Treatment  Measure D.V.

(pre-test)

(post-test)

common types of true experiments
Common types of true experiments

O

X

O

Pretest-Posttest Control

R

O

O

X

O

Post-only Control

R

O

O

X

O

(1)

O

O

(2)

R

Solomon 4-group

X

O

(3)

O

(4)

example pen study
Example: Pen Study
  • Question: Do individuals in Japan and the US make differential choices about ‘unique’ versus ‘less unique’ items when given a choice?
pen study
Pen Study
  • Independent Variable
    • Cultural difference: Japanese students compared to US students
  • Assignment
    • Subjects were not randomly assigned because they already fell into one of the two groups.
  • Dependent Variable:
    • Pen layout (3 of one type, 1 of another)
    • Would they choose the ‘common’ pen or the ‘unique’ one?
example trust building study
Example: Trust-Building Study
  • Question: Do increased risk-taking behaviors over time increase interpersonal trust?
trust building study
Trust-Building Study
  • Independent Variable
    • Experiment Condition (2 conditions):
      • Fixed partner on every trial, cannot control amount to entrust to partner
      • Fixed partner on every trial, can control amount to entrust to partner
  • Assignment
    • Random assignment of participants to one of the 2 conditions.
    • Same experiment conducted in Japan and US, and comparisons made between the two studies.
  • Dependent Variable
    • Cooperation rate (i.e., whether they returned the coins to the partner or not)
measurement and design validity
Measurement and Design Validity
  • Measurement Concerns
    • Construct Validity
  • Design Concerns
    • Internal Validity
    • External Validity
    • Ecological Validity
construct validity
Construct Validity

How do we know that our independent variable is reflecting the intended causal construct and nothing else?

  • “Face” validity deals with subjective judgement of appropriate operationalization
  • “Content” validity is a more direct check against relevant content domain for the given construct.
internal validity
Internal Validity

Internal Validity deals with questions about whether changes in the dependent variable were caused by the treatment.

slide18

?

?

Cause

Effect

?

?

threats to internal validity
Threats to Internal Validity
  • History
    • additional I.V. that occurs between pre-test and post-test
  • Maturation
    • Subjects simply get older and change during experiment
  • Testing
    • Subjects “get used” to being tested
  • Regression to the Mean
    • Issue with studies of extremes on some variable
contamination and internal validity
Contamination and Internal Validity
  • Demand Characteristics
    • Anything in the experiment that could guide subjects to expected outcome
  • Experimenter Expectancy
    • Researcher behavior that guides subjects to expected outcome (self-fulfilling prophecy)
general demand characteristics
General Demand Characteristics
  • Evaluation Apprehension
  • Solutions
    • Double-blind experiments
    • Experiments in natural setting (i.e., subjects do not know they are in an experiment)
    • Cover stories
    • Hidden measurements
reducing the role of the experimenter solving expectancy effects
Reducing the role of the experimenter: solving expectancy effects
  • Naïve experimenter
    • Those conducting study are not aware of theory or hypotheses in the experiment
  • Blind
    • Researcher is unaware of the experiment condition that he/she is administering
  • Standardization
    • Experimenter follows a script, and only the script
  • “Canned” Experimenter
    • Audio/Video/Print material gives instructions
and more
And More!
  • Selection Bias
    • Issue with non-random selection of subjects
  • Mortality
    • Departure of subjects in the experiment
  • Diffusion, Sharing of Treatments
    • Control group unexpectedly obtains treatment
  • Other ‘social’ threats?
    • Compensatory rivalry, resentful demoralization, etc.
three threats to external validity generalizability in experiments
Three threats to external validity (generalizability) in experiments
  • Setting
  • Population
  • History

External Validity– How far does the given experiment generalize to similar groups, individuals, etc?

the validity tradeoff truth and myth
The Validity Tradeoff: Truth and Myth

Internal

Validity

External & Ecological

Validity

Balance is important between the types of validity, but internal validity is usually (if not always) the more important factor.

true experiments in the field
True Experiments in the Field
  • Some experiments can be conducted in a real-world setting while maintaining random assignment and manipulation of treatments

Milliman (1986) Study of music tempo and restaurant customer behavior

Cheshire and Antin (2008) Study of Incentives and Contributions of Information in an Online Setting

natural experiments
Natural Experiments
  • 1998 Total Solar Eclipse: testing temperature of sun’s corona
pro s and con s of experiments
Pro’s and Con’s of Experiments
  • Pro’s
    • Gives researcher tight control over independent factors
    • Allows researcher to test key relationships with as few confounding factors as possible
    • Allows for direct causal testing
  • Con’s
    • Often very small N; enough for statistical purposes but not ideal for generalizability
    • Sometimes give up large amounts of external validity in favor of construct validity and direct causal analysis
    • Require a large amount of planning, training, and time– sometimes to test relationship between only 2 factors!
additional considerations before using experiments
Additional considerations before using experiments
  • Cost and Effort
    • Is the effort worth it to test the concepts you are interested in?
  • Manipulation and Control
    • Will you actually be able to manipulate the key concept(s)?
  • Importance of Generalizability
    • Are you testing theory, or trying to establish a real-world test?