Loading in 5 sec....

Randomized Evaluations: ApplicationsPowerPoint Presentation

Randomized Evaluations: Applications

- By
**shiro** - Follow User

- 61 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Randomized Evaluations: Applications' - shiro

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Randomized Evaluations:Applications

Kenny Ajayi

September 22, 2008

Global Poverty and Impact Evaluation

Review

- Causation and Correlation (5 factors)
- X → Y
- Y → X
- X → Y and Y → X
- Z → X and Z → Y
- X ‽ Y

Review

- Causation and Correlation (5 factors)
- X → Y causality
- Y → X reverse causality
- X → Y and Y → X simultaneity
- Z → X and Z → Y omitted variables / confounding
- X ‽ Y spurious correlation

- Reduced-form relationship between X and Y

Review

Exogenous

Practical Applications

- Noncompliance
- Cost Benefit Analysis
- Theory and Mechanisms

Kling, Liebman and Katz (2007)

- Do neighborhoods affect their residents?

Do neighborhoods affect their residents?

- Positively

Do neighborhoods affect their residents?

- Positively
- Social connections, role models, security, community resources

Do neighborhoods affect their residents? Negatively

- Positively
- Social connections, role models, security, community resources

Do neighborhoods affect their residents? Negatively

- Positively
- Social connections, role models, security, community resources

- Discrimination, competition with advantaged peers

Do neighborhoods affect their residents? Negatively Not at all

- Positively
- Social connections, role models, security, community resources

- Discrimination, competition with advantaged peers

Do neighborhoods affect their residents? Negatively Not at all

- Positively
- Social connections, role models, security, community resources

- Discrimination, competition with advantaged peers

- Only family influences, genetic factors, individual human capital investments or broader non-neighborhood environment matter

Do neighborhoods affect their residents?

- What is Y?

Do neighborhoods affect their residents?

- What is Y?
- Adults
- Self sufficiency
- Physical & mental health

- Youth
- Education (reading/math test scores)
- Physical & mental health
- Risky behavior

- Adults

Kling, Liebman and Katz (2007)

- Do neighborhoods affect their residents?
- X → Y causality
- Y → X reverse causality
- X → Y and Y → X simultaneity
- Z → X and Z → Y omitted variables / confounding
- X ‽ Y spurious correlation

Neighborhood Effects

- Natural Experiment

Neighborhood Effects

- Natural Experiment
- Hurricane Katrina

- Can we randomly assign people to live in different neighborhoods?

Neighborhood Effects

- Natural Experiment
- Hurricane Katrina

- Can we randomly assign people to live in different neighborhoods?
- Sort of…

Moving to Opportunity

- Solicit volunteers to participate
- Randomly assign each volunteer an option
- Control
- Section 8
- Section 8 with restriction (low poverty neighborhood)

- Volunteers decide whether to move
- Compliers: use voucher
- Non-compliers: don’t move

Non-random Participation

- Why is this OK?
- Volunteers presumably care about their neighborhood environment, so should be the target when thinking about uptake for the use of housing vouchers
BUT

- Results might not generalize to other populations with different characteristics

- Volunteers presumably care about their neighborhood environment, so should be the target when thinking about uptake for the use of housing vouchers

Treatment and Compliance

- With perfect compliance:
- Pr(X=1│Z=1)=1
- Pr(X=1│Z=0)=0

- With imperfect compliance:
- 1>Pr(X=1│Z=1)>Pr(X=1│Z=0)>0
Where X - actual treatment

Z - assigned treatment

- 1>Pr(X=1│Z=1)>Pr(X=1│Z=0)>0

Some Econometrics

- Intention-to-treat (ITT) Effects
- Differences between treatment and control group means

Some Econometrics

- Intention-to-treat (ITT) Effects
- Differences between treatment and control group means

ITT = E(Y|Z = 1) – E(Y|Z = 0)

Y - outcome

Z - assigned treatment

Some Econometrics

- Intention-to-treat (ITT) Effects
- Writing this relationship in mathematical notation

Y = Zπ1 + Wβ1 + ε1

Individual wellbeing (Y) depends on whether or not the individual is assigned to treatment (Z), baseline characteristics (W), and whether or not the individual got lucky (ε)

π1- effect of being assigned to treatment group

= (effect of actually moving) x (compliance rate)

Some Econometrics

- Effect of Treatment on the Treated (TOT)
- Differences between treatment and control group means
Differences in compliance for treatment and control groups

- Differences between treatment and control group means

Some Econometrics

- Effect of Treatment on the Treated (TOT)
- Differences between treatment and control group means
Differences in compliance for treatment and control groups

- TOT = “Wald Estimator” = E(Y|Z = 1) – E(Y|Z = 0)
E(X|Z = 1) – E(X|Z = 0)

Y - outcome

Z - assigned treatment

X - actual treatment (i.e. compliance)

- Differences between treatment and control group means

Some More Econometrics

- Estimate TOT using instrumental variables
- Use offer of a MTO voucher as an instrument for MTO voucher use

Some More Econometrics

- Estimate TOT using instrumental variables
- Use offer of a MTO voucher as an instrument for MTO voucher use. In mathematical notation:
(1) X = Zρd + Wβd + εd

- Predict whether people use voucher (X), given their assigned treatment (Z) and observable characteristics (W)

- Use offer of a MTO voucher as an instrument for MTO voucher use. In mathematical notation:

Some More Econometrics

- Estimate TOT using instrumental variables
- Use offer of a MTO voucher as an instrument for MTO voucher use. In mathematical notation:
(1) X = Zρd + Wβd + εd

(2)Y = Xγ2 + Wβ2 + ε2

- Predict whether people use voucher (X), given their assigned treatment (Z) and observable characteristics (W)
- Estimate how much individual wellbeing (Y) depends on predicted compliance (X) and observable characteristics (W)

- Use offer of a MTO voucher as an instrument for MTO voucher use. In mathematical notation:

Some Econometrics

- Effect of Treatment on the Treated (TOT)
- Writing the reduced-form relationship in mathematical notation

Y = Xγ2 + Wβ2 + ε2

Individual wellbeing (Y) depends on whether or not the individual complies with treatment (X), baseline characteristics (W), and whether or not the individual got lucky (ε)

Note:γ2-effect of actuallymoving (i.e. compliance)

= effect of being assigned to treatment group compliance rate

Moving to Opportunity

INTENTION TO TREAT

Moving to Opportunity

EFFECT OF TREATMENT ON THE TREATED

Results

- Positive effects
- Teenage girls
- Adult mental health

- Negative effects
- Teenage boys

- No effects
- Adult economic self-sufficiency & physical health
- Younger children

- Increasing effects for lower poverty rates

Limitations

- Can’t fully separate relocation effect from neighborhood effect
- Can’t observe spillovers into receiving neighborhoods

Some Thoughts

- We can still estimate impacts in cases where we don’t have full compliance
- Policies sometimes target people who are different from the general population (i.e. those more likely to take up a program)
BUT

- We have to be careful about generalizing these results for other populations (external validity)

Duflo, Kremer & Robinson (2008)

- Why don’t Kenyan farmers use fertilizer?

Duflo, Kremer & Robinson (2008)

- Why don’t Kenyan farmers use fertilizer?
- Low returns
- High cost
- Preference for traditional farming methods
- Lack of information

Is Fertilizer Use Profitable?

- What is Y (outcome)?
- Crop yield
- Rate of return over the season

- What is X (observed)?
- Fertilizer
- Hybrid seed

Is Fertilizer Use Profitable?

- What is Z (unobserved)?

Is Fertilizer Use Profitable?

- What is Z (unobserved)?
- Farmer experience/ability
- Farmer resources
- Weather
- Soil quality
- Market conditions

Correlation or Causation?

Fertilizer Use (X) and Crop Yield (Y)

Correlation or Causation?

Fertilizer Use (X) and Crop Yield (Y)

- X → Y causality
- Y → X reverse causality
- X → Y and Y → X simultaneity
- Z → X and Z → Y omitted variables / confounding
- X ‽ Y spurious correlation

Fertilizer Experiment

- How could we test the effect of fertilizer use on crop yield using randomization?

Fertilizer Experiment

- Randomly select participating farmers
- Measure 3 adjacent equal-sized plots on each farm
- Randomly assign farming package for each plot
- Top dressing fertilizer (T1)
- Fertilizer and hybrid seed (T2)
- Traditional seed (C)

- Weigh each plot’s yield at harvest

Fertilizer Experiment

- Calculate rate of return over the season:
value of value of

treatment - comparison - value of

plot output plot output input

RR = ----------------------------------------------------------- - 100

value of input

Some Thoughts

- More comprehensive package increased yield
BUT

- Lower cost package had highest rate of return
- Lab results don’t always apply in the field
- Think about the COSTS and BENEFITS

Karlan and Zinman (2007)

- What is the optimal loan contract?

Objectives

- Profitability
- Gross Revenue
- Repayment

- Targeted Access
- Poor households
- Female microentrepreneurs

- Sustainability

Theory of Change

Loan Profits

contract

- Mechanisms
- How do we expect a change to affect the outcome?
- What assumptions are we making?
- Can we test if they are true?

Loan Offer Experiment What is X? How sensitive are borrowers to interest rates and maturities?

- What is Y?
- Take-up
- Loan size
- Default

- Interest rate
- Maturity

Results

- Downward sloping demand curves
- Loan size more responsive to loan maturity than to interest rate

Some Limitations

- Again, results apply only to people who had previously borrowed from the lender.
- Elasticities apply to direct mail solicitations and might not hold for other loan offer methods

Final Thoughts

- Randomized evaluation can have many applications beyond merely looking at reduced-form relationships between
X and Y

Download Presentation

Connecting to Server..