Establishing a cause effect relationship
Download
1 / 61

Establishing a Cause-Effect Relationship - PowerPoint PPT Presentation


  • 130 Views
  • Uploaded on

Establishing a Cause-Effect Relationship. Internal Validity. Is the relationship causal between. The “treatment” and the “outcomes” The independent and dependent variables. Alternative cause. Alternative cause. Treatment. Outcomes. What you do. What you see. Alternative cause.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Establishing a Cause-Effect Relationship' - Leo


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Internal validity
Internal Validity

Is the relationship causal between...

  • The “treatment” and the “outcomes”

  • The independent and dependent variables.

Alternative

cause

Alternative

cause

Treatment

Outcomes

What you do

What you see

Alternative

cause

Alternative

cause

Observation

In this study


Establishing cause and effect
Establishing Cause and Effect

Temporal precedence


Establishing cause and effect1
Establishing Cause and Effect

Temporal precedence

then

Cause

Effect

Time

It can get complicated through:

-sloppiness (campaign contributions

- Chicken and egg cyclical functions (democracy and GDP)


Establishing cause and effect2

Cause

Effect

then

Time

Establishing Cause and Effect

Temporal precedence

Covariation of cause and effect


Establishing cause and effect3

Cause

Effect

then

Time

Establishing Cause and Effect

Temporal precedence

Covariation of cause and effect

if X, then Y

if not X, then not Y

if treatment given, then outcome observed (usually)

if program not given, then outcome not observed


Establishing cause and effect4

Cause

Effect

then

Time

Establishing Cause and Effect

Temporal precedence

Covariation of cause and effect

if X, then Y

if not X, then not Y

if program given, then outcome observed

if program not given, then outcome not observed

Dosage effects or comparative statics:

If more of treatment, then more of outcome observed

if less of treatment given, then less of outcome observed


Establishing cause and effect5

Cause

Effect

then

Time

Establishing Cause and Effect

Temporal precedence

if X, then Y

if not X, then not Y

Covariation of cause and effect

No alternative explanations

Treatment

Outcome

Micromediation


Establishing cause and effect6

Cause

Effect

then

Time

Establishing Cause and Effect

Temporal precedence

if X, then Y

if not X, then not Y

Covariation of cause and effect

Alternative cause

(substantive)

No alternative explanations

Alternative

cause

Treatment

Outcome

Micromediation

Alternative

cause

Alternative cause

(nuisance)


In lab or field experiments
In Lab or Field Experiments…

Temporal precedence

  • Is taken care of because you intervene before you measure outcome

  • Is measured by comparing treated and untreated groups

  • Is the central issue of internal validity -- usually taken care of through random assignment

Covariation of cause and effect

No alternative explanations



The single group case
The Single Group Case

Two designs:


The single group case1

Administer

program

Measure

outcomes

X

O

The Single Group Case

Two designs:

  • “Post-test only single-group design”

    • X is the treatment

    • O is the observation


The single group case2

Administer

program

Measure

outcomes

X

O

The Single Group Case

Two designs:

“pre-test, post-test single-group design”

or

“interrupted time-series”

Measure

baseline

O


The single group case3

Administer

program

Measure

outcomes

X

O

Administer

program

Measure

outcomes

X

O

The Single Group Case

Alternative

explanations

Two designs:

Alternative

explanations

Measure

baseline

O

Alternative

explanations


Example
Example

  • After the 2003 recall election, did Democrats in the California Assembly move to the center?

  • California ran a full legislative “season” before the October, 2003 election, then ran another “season” afterward.

  • We can look at roll call vote behavior



History threat

Pretest

Program

Posttest

O

X

O

History Threat

  • Any other event that occurs between pretest and posttest

  • Perhaps the nation was just shifting to the center at this time.

  • How might we rule it out?


Maturation threat

Pretest

Program

Posttest

O

X

O

Maturation Threat

  • Normal growth between pretest and posttest.

  • Coming into an election year, state legislators always shift to the center.



Testing threat

Pretest

Program

Posttest

O

X

O

Testing Threat

  • The effect on the posttest of taking the pretest

  • Legislators may have learned that the state was watching them. When real tests are given, this is a big problem.


Instrumentation threat

Pretest

Program

Posttest

O

X

O

Instrumentation Threat

  • Any change in the test from pretest and posttest

  • A different test may have been used if a different roll call estimation technology used.


Mortality threat

Pretest

Program

Posttest

O

X

O

Mortality Threat

  • Nonrandom dropout between pretest and posttest

  • If some legislators had been recalled along with Gray Davis, this would be a problem.


Regression threat

Pretest

Program

Posttest

O

X

O

Regression Threat

  • Group is a nonrandom subgroup of population.

  • The 2003 session was particularly extreme, any other session would look more centrist.



The central issue
The Central Issue

  • When you move from single to multiple group research the big concern is whether the groups are comparable.

  • Usually this has to do with how you assign units (for example, persons) to the groups (or select them into groups).

  • If you are not careful, may mistake a selection effect for a treatment effect.


The multiple group case

O

X

O

O

O

The Multiple Group Case

Alternative

explanations

Measure

baseline

Administer

treatment

Measure

outcomes

Do not administer

treatment

Measure

baseline

Measure

outcomes

Alternative

explanations


Example1
Example

  • Suppose USAID looked before and after at countries where it did and didn’t run governance programs in the last decade

  • Pre-post program-comparison group design

  • Measures (O) are all of the things Clark hates, but let’s set that aside for now.


Selection threats

O

X

O

O

O

Selection Threats

  • Any factor other than the program that leads to posttest differences between groups.

  • USAID did not randomly select the countries in which it ran programs, and sent aid to those with the lowest-rated governments


Selection history threat

O

X

O

O

O

Selection-History Threat

  • Any other event that occurs between pretest and posttest that the groups experience differently.

  • For example, countries that begin with more stable democracies faced fewer challenges in the past decade.


Selection maturation threat

O

X

O

O

O

Selection-Maturation Threat

  • Differential rates of normal growth between pretest and posttest for the groups.

  • It is easier to move from a semi-democracy to a full democracy than it is to move from a non-democracy to a semi-democracy


Selection testing threat

O

X

O

O

O

Selection-Testing Threat

  • Differential effect on the posttest of taking the pretest.

  • At least these measures are “unobtrusive,” so this probably is not a grave threat


Selection instrumentation threat

O

X

O

O

O

Selection-Instrumentation Threat

  • Any differential change in the test used for each group from pretest and posttest

  • For example, the Polity measures may give some countries credit for having a USAID program


Selection mortality threat

O

X

O

O

O

Selection-Mortality Threat

  • Differential nonrandom dropout between pretest and posttest.

  • Perhaps the countries with weak governments are more likely to cease being a country over the past decade.


Selection regression threat

O

X

O

O

O

Selection-Regression Threat

  • Different rates of regression to the mean because groups differ in extremity.

  • For example, the countries that USAID chooses may have nowhere to go but up.



What are social threats
What Are “Social” Threats?

  • All are related to social pressures in the research context, which can lead to posttest differences that are not directly caused by the treatment itself.

  • Most of these can be minimized by isolating the two groups from each other, but this leads to other problems (for example, hard to randomly assign and then isolate, or may reduce generalizability).



Types of designs1
Types of Designs

Random assignment?


Types of designs2
Types of Designs

Random assignment?

Yes


Types of designs3
Types of Designs

Random assignment?

Yes

Randomized or

true experiment?


Types of designs4
Types of Designs

Random assignment?

Yes

No

Randomized or

true experiment?


Types of designs5
Types of Designs

Random assignment?

Yes

No

Control group or

multiple measures?

Randomized or

true experiment?


Types of designs6
Types of Designs

Random assignment?

Yes

No

Control group or

multiple measures?

Randomized or

true experiment?

Yes


Types of designs7
Types of Designs

Random assignment?

Yes

No

Control group or

multiple measures?

Randomized or

true experiment?

Yes

Quasi-experiment


Types of designs8
Types of Designs

Random assignment?

Yes

No

Control group or

multiple measures?

Randomized or

true experiment?

Yes

No

Quasi-experiment


Types of designs9
Types of Designs

Random assignment?

Yes

No

Control group or

multiple measures?

Randomized or

true experiment?

Yes

No

Quasi-experiment

Nonexperiment


Design notation example
Design Notation Example

R O X O

R O O

Os indicate different

waves of

measurement.


Elements of a design
Elements of a Design

  • Observations and measures

  • Treatments

  • Groups

  • Assignment to group

  • Time


Design notation example1
Design Notation Example

Vertical alignment

of Os shows that

pretest and posttest

are measured at same time.

R O X O

R O O


Design notation example2
Design Notation Example

X is the treatment.

R O X O

R O O


Design notation example3
Design Notation Example

There are two

lines, one for

each group.

R O X O

R O O


Design notation example4
Design Notation Example

R O X O

R O O

R indicates

the groups

are

randomly

assigned.


Design notation example5
Design Notation Example

R O1 X O1, 2

R O1 O1, 2

Subscripts

indicate

subsets of

measures.


Design notation example6
Design Notation Example

R O X O

R O O

Pretest-posttest (before-after)

Treatment versus comparison group

Randomized experimental design


Design example
Design Example

Posttest Only Randomized Experiment


Design example1
Design Example

Posttest Only Randomized Experiment

R X O

R O


Design example2
Design Example

Pretest-Posttest Nonequivalent Groups Quasi-Experiment


Design example3
Design Example

Pretest-Posttest Nonequivalent Groups Quasi-Experiment (note multiple groups or multiple observations are REQUIRED to have a quasi-experiment)

N O X O

N O O


Design example4
Design Example

Posttest Only Nonexperiment


Design example5
Design Example

Posttest Only Nonexperiment

X O


ad