Research methods Festival 2012: Bringing the lab to the field

1. Research methods Festival 2012: Bringing the lab to the field ANANDI MANI, UNIVERSITY OF WARWICK & CAGE

2. Types of Experiments (Harrison-List JEL) ______AFE_________FFE_________NFE______________________________ Lab [field experiments] NE, PSM, IV, STR, etc. • Conventional lab experiment (Lab) • employs a standard subject pool of students, an abstract framing, and an imposed set of rules • Artefactual field experiment (AFE) • same as a conventional lab experiment but with a non-standard subject pool • Framed field experiment (FFE) • same as an artefactual field experiment but with field context in the commodity, task, information, stakes, time frame, etc. • Natural field experiment (NFE) • same as a framed field experiment but where the environment is the one that the subjects naturally undertake these tasks, such that the subjects do not know that they are in an experiment

3. Motivation • Development Economists have been doing Field Experiments using Randomized Control Trials (RCTs) for over a decade now, addressing a wide range of questions, e.g. • Do Cameras in Schools improve Teacher Attendance & Student Outcomes? • Does Microfinance spur Business Investment among the poor? • Some of these RCTs come under criticism for a lack of light on the Mechanisms underlying the observed findings (Deaton(2009)) • Lab Experiments may help identify • Potential Reasons for Certain Outcomes Observed in Survey Data • Mechanisms Underlying Some Field Experiment Findings, which would help increase the External Validity

4. Advantages of Lab Experiments • Better Control: Lab Experiment Design makes it feasible to generate results ceteris paribus • Testing alternative theoretical mechanisms • Test Institutions (e.g. Auction formats) • Scope for Replication & Comparison across Cultural Settings • Cheaper Market Design Pilots

5. Outline of Talk Applications: (A) GENDER DIFFERENCES IN PRODUCTIVITY & PAY • Fact 1: Capital Returns (de Mel et al -- 2009): Lower returns of Women owned firms associated with less supportive spouses • Q: Could Intra-Household Decision-Making Play a role in this? • Fact 2: Women in the US earn 75% of what men do on the labor market – and education, experience, hours worked don’t explain more than 50% of it • Q: Gender Differences in Competitive Behavior explain this gap? (B) POVERTY & DECISION-MAKING • Fact 3: Poor seem to make irrational decisions on Savings, Human Capital Investment • Q: Could Poverty Affect Stress Levels & Cognitive Ability? (C) PITFALLS OF LAB EXPERIMENTS & POSSIBLE SOLUTIONS

6. Intra-HH Decisions How does decision- making work within the family? Does it amplify the inequities of market outcomes for its members, or does it mitigate them? Laws about Property Rights and Inheritance Schemes to encourage Human capital investment Entrepreneurship/ Income Generation Programs

7. HH Decision-Making: Experiment Questions • Assuming that HH members do not share a common set of preferences… • Q1: Is HH decision-making efficient – i.e. do members maximize HH (Investment) returns ? OR … • Q2: If not, why do they sacrifice HH income? Is it • …for economic reasons – for instance, greater bargaining power/ control over HH resources (how much?) • …or for other socially influenced reasons? • …and do they do this only when their spouse won’t know?

8. Arguments for an Experimental Approach • One-phrase Summary of Survey Based Empirical Studies of HH resource allocation decisions: Can’t be sure! • Allocation decisions directly observed real-time • No need to infer decisions from data reported ex-post • Survey responses may “adjusted to fit” local social and cultural norms(Bertrand-Mullainathan(2001)), whereas... • Actions speak better than words • Focus on Investment rather than Consumption Decisions • No scope for effects arising from possible substitutions outside the experiment

9. Experiment Location & Sample • Anantapur district, Andhra Pradesh (2nd most drought prone) • 300 couples, from 32 villages -- all wives members of Self-Help Groups (SHGs) run by a single NGO • Promised Participation Fee (Rs.50 – about 62p), roughly equal to daily wages, with scope to make more based on their performance

10. Experiment Protocol • 3-4 villages participating daily (10 day experiment) • Participating couples from each village brought in to NGO location • Separate waiting area for men and women • Three couples taken to six separate rooms, where.. • Experiment explained and options presented by a coordinator • Data recorded by two independent data entry staff • Separate waiting areas for male and female participants who completed the experiment • Individual payment to participants upon completion • Participants taken back to village when all payments completed

11. No Tradeoff Investor Tradeoff between higher HH income and own control over it Experiment Tasks • 4 Investment Decisions for each Spouse, individually presented in random order • Task: Allocate Rs.50 (seed money) across two Investment options Blue and Red • Efficient Investment Allocation = Rs. 50 in Blue

12. Investment Means • Efficient Investment: Rs.50 in Blue option • Both Men and Women are Inefficient. • They’re willing to sacrifice HH income, to gain more control over it for themselves. • But Men don’t maximize HH returns even if their share of control is Fixed! WHY??

13. Fraction of Efficient HHs • A third of men are inefficient even when their share of control is Fixed – i.e. they undercut their own income (and their wife’s) rather than maximize HH returns • “Irrationality” not explained by low education/confusion, lack of experience with financial decisions, longer term effects on bargaining power within HH

14. Why are Men inefficient under Fixed Shares • Men don’t like it when their wife’s share exceeds theirs • When her share>50% • They are willing to undercut their own income.. • to ensure she does not earn too much more than themselves. • Wives of “Spiteful” Husbands are more inefficient in other three decisions, where Control over HH income depends upon investment allocation. • CONCLUSION: Consistent with de Mel et al(2009) finding, Less cooperative spouses => Lower Productivity on Women’s Businesses

15. Gender Earnings Gap: Motivation • POTENTIAL SOURCES OF GENDER GAP IN LABOR MARKET OUTCOMES • Occupation choice • Experience & Continuity in labor market participation • Discrimination • Psychological factors • Stereotype threat • Claude Steele (1997): Additional anxiety causes choking under pressure when performing a task • Ambady et al (1999), Psychological Science • Self-Confidence • Competitive Behavior

16. Performance under Competition Gender Differences: Gneezy-Niederle-Rustichini(2003, QJE)-Summary • Lab Experiment conducted in Israel with students from Technion • Participants Task: Solving Mazes on a computer • Studied Participants’ performance under three payment schemes • (a) Non-competitive (Piece rate compensation) • (b) Competitive (Winner-take-all tournament) • (c) Random pay setting (One person in Group of 6 is paid, rest are not) • Main findings: • Men’s performance improves considerably going from (a) to (b), whereas women’s performance does not change • Women’s performance is much worse when their tournament group includes men than when it has only women

17. Do Women prefer to Compete less? Niederle-Vesterlund( Aug 2007, QJE) • Women may choose lower powered jobs for multiple reasons: • Responsibility & Time demands of such jobs, given family considerations • Discrimination may discourage attempts to obtain these jobs • Competitive Pressure of such jobs? • Experiments allow choice of tasks with similar time demands, where innate abilities do not differ among men and women, and discrimination is ruled out • Theories (about why women shy away from high-profile jobs): They may • Dislike Competition • Lack Confidence, relative to men • Be Risk Averse • Have Feedback Aversion (They’re more discouraged by negative feedback). • Experimental Design makes it possible to distinguish among various channels

18. Experiment Details Lab experiment with students at University of Pittsburgh, groups Task: Addition of sets of five 2 digit-numbers, for five minutes Information to Participants: Only on own absolute performance, no information on others’ performance. Information provided real time, as task is performed. Studied Payment Scheme Choice of Men vs. Women: Piece-rate vs. Winner-take all (Competitive) scheme, given information above. 4 participants per group, two male and two female (20 groups)

19. Experiment Design • Task 1: Piece rate (PR) of $0.5 per correctly solved addition • Task 2: Tournament (T; winner take all) rate of $2 per correctly solved addition • At a 25% chance of tournament win, both payment schemes generate the same expected payoff. • Tournament payoff is in per task terms to avoid guesswork about what would be a high enough fixed payment to induce tournament entry among high performers • Task 3: First choose payment scheme (PR or T) and then do addition task • Participants evaluated against others’ performance in Task2 –why? • Eliminates effects of beliefs about others’ choice on decision • 10,000 (feasible) groups made with replacement from the data, avg. across 100 trials to determine individual success probability in tournament. • Task 4: Choose payment scheme (PR or T) for (previous) Task 1; No new task • To separate the preferences for competition from other factors such as risk aversion & feedback aversion , on tournament entry decision • Ask participants to guess their rank in task 1 and task 2 in their group of four • To measure effects of self-confidence on tournament entry and performance

20. Main Findings • Men and Women are equally good at Addition Task under Piece Rate and Tournament. Despite this, • being a woman reduces probability of selecting Tournament payment scheme in Task 3 by 38% • Not explained by individual performance in previous rounds (T1,T2) or current round (T3) itself. • For women, total expected cost of under-entry is much larger than cost of over-entry; for men it’s the reverse • Despite accounting for differences in Self-confidence, being female still reduces Tournament entry probability by 27.8%

21. Taking the Lab Design to the Field • Potential Concerns with the above 2 Experiments: • Experiment 1: Performance could be influenced by Task specific differences in ability (men have advantage in spatial ability and arm-throwing capacity) -- so mazes may not be to women’s advantage. • Experiment 2: Women’s observed Preferences for Competition may be due to being socialized to believe they are worse competitors than men – or that their behavior should be “ladylike” (less aggressive) ? • Gneezy-Leonard-List (2007) address both these concerns – How? • Socialization: Repeat similar experiment design in one Matrilineal & Matrilocal tribe and one Patriarchal tribe • Task: Task unfamiliar to people in both tribes

22. Lab-in-the-Field: Design • Maasai (Tanzania): Patriarchal • “Men treat us like donkeys” Maasai woman (Hodgson (2001)) • Khasi (NW India): Matrilineal • “We are sick of playing the roles of breeding bulls and baby-sitters” Khasi man (Ahmed (1994) • Subjects in 2 groups, randomly paired with 1person from other group (paired subject identity/sex not known) • Task: Throw tennis ball into bucket 3 metres away (10 chances per subject) • Payment Scheme: X per “success” irrespective of paired subject performance OR 3X per “success” if own performance better than paired person • X=Rs.20 in India; X = 500 shillings in Tanzania

23. Maasai • Maasai men choose to compete at twice the rate that women do • Similar to findings in Western settings

24. Maasai vs. Khasi • Khasi women choose to compete at twice the rate that men do • And even at a rate slightly higher than Maasai men Authors’ Conclusion: Any number of subtle influences on children or adults can cause differences in attitudes to competition -- even if the behavior is broadly framed by genetic endowment

25. Poverty, Stress & Cognitive Capacity USING SUGARCANE HARVESTS TO UNDERSTAND THE PSYCHOLOGY OF POVERTY

26. Poverty, Cognitive Capacity & Decisions-1 • A fundamental assumption of Economics is the Scarcity of Resources… • Yet the Rational Model assumes that Mental Capacity is Infinite ! • But Decision-Making Takes Mental Effort, and its Tiring! • Question: Does the State of Being Poor affect Cognitive Capacity? (Mani-Mullainathan-Shafir)

27. Sugarcane Harvests • Long Cycle Crop – about 11 months • Farmers are down to the wire a few weeks before Harvest • Receive Lump sum Returns a few weeks after Harvest • Sugar Mills assign Cutting Dates to individual farmers, hence farmers don’t have control over when their Income arrives • Methodology: Compare Individual Farmers’ before vs. after Harvest on Measures of: • Stress: Blood Pressure, Heart Rate (Round 1-- 2009) • Cognitive Capacity & Attention: IQ(Raven’s) tests, Stroop tests (Round 2 – 2011)

28. Raven’s IQ Test

29. Stroop Tests Coffee

30. House

31. Train

32. Window

33. Monkey

34. Brick

35. Stroop Tests Green

36. Stroop Tests Red

37. Stress Measures: Blood Pressure

38. IQ Measure: Raven’s Test

39. Cognitive Depletion: Stroop

40. Summary of Findings • Main Findings: Poverty in the Pre-Harvest Period • Raises Stress Levels • Lowers IQ & Cognitive Capacity • Comments: These findings are not driven by • Adverse Nutritional changes pre-harvest • Learning Effects post-harvest (for IQ tests)

41. Potential Pitfalls – and some Solutions • Lack of Anonymity • May elicit more pro-social behavior when observed • Solution: Double blind experiments, Outcome measure unclear • Context and Framing • Label “Wall Street” game vs. “Community” game affects play • Solution: Neutral wording; Collect Background data on subjects • Self-Selection in Participants • Biased sample -- Could be a problem in all Field Experiments • Solution: Conduct experiment in different settings • Low Stakes may elicit non-serious behavior • Solutions: Vary stakes, Treat results as lower/upper bound, Use suitable subjects • Relevance of Lab decisions to “real” behavior? • Track correlation b/w the two (e.g. Karlan(2005) AER) – Trust game outcome and Repayment of Microfinance Loan a year later

42. THANK YOU!