slide1
Download
Skip this Video
Download Presentation
For Peace and Goodwill? Using an Experimental Game and a Social Dilemma to Analyse the Effect of the Desarollo y Paz Programmes in Colombia.

Loading in 2 Seconds...

play fullscreen
1 / 56

Introduction - PowerPoint PPT Presentation


  • 467 Views
  • Uploaded on

For Peace and Goodwill? Using an Experimental Game and a Social Dilemma to Analyse the Effect of the Desarollo y Paz Programmes in Colombia. Luca PELLERANO Oxford Policy Management, Oxford Institute for Fiscal Studies, London World Bank Washington, 5 th May 2009 Introduction

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Introduction' - Solomon


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

For Peace and Goodwill?

Using an Experimental Game and a Social Dilemma to Analyse the Effect of the Desarollo y Paz Programmes in Colombia.

Luca PELLERANO

Oxford Policy Management, Oxford

Institute for Fiscal Studies, London

World Bank

Washington, 5th May 2009

introduction
Introduction
  • There is an increasing need to evaluate the effects of social interventions on a broad set of “socio-cultural” dimensions that exceed the traditional socioeconomic variables:
    • values, aptitudes, perceptions, motivations, capabilities, empowerment
    • relationships, networks, social capital
  • Rigorous impact evaluation in this domain is extremely challenging as:
    • Expected socio-cultural changes are often not clearly specified (Theory of Change)
    • They are difficult to measure
    • As the analysis moves in the domain of the “intangibles”, the evaluation must be driven my a multi method and multidisciplinary approach.
  • In this work (Guarin, Navarro and Pellerano, 2009; Attanasio, Pellerano and Phillips, 2009):
    • We evaluate the effect of a complex an very peculiar CDD-Social Fund type intervention in Colombia on social capital and conflict management
    • We explore innovative methodological options for the measurement and evaluation of “socio cultural” dimensions.
slide3
The Setting and the Programme
  • Theory of Change
  • The Instruments
  • The Data
  • Descriptive Evidence
  • The Evaluation Strategy
  • Results
  • Conclusions
slide4
The Setting and the Programme
  • Theory of Change
  • The Instruments
  • The Data
  • Descriptive Evidence
  • The Evaluation Strategy
  • Results
  • Conclusions
slide5

The Setting. Conflict in Colombia.

  • Colombia has been affected by a complex civil conflict along the whole course of its recent history.
  • The drivers of the conflict have evolved with time (political violence, control over natural resources, coca).
  • Kidnapping, selective assassinations and internal displacement are amongst the most evident consequences.
  • Two main illegal factions: Guerrilla(s) and Paramilitary groups (+ new emerging groups after process of desmovilización).
  • The regional dimension of the conflict:
    • Despite progress, violence still affects many rural areas of the country in a very clustered way.
    • An atypical civil conflict, as the illegal groups are small in number and normally “outsiders” to the communities
    • Strong association between the presence of illegal groups and violence levels, local political instability, inequality (Ibañez and Moya, 2009)
slide6

The Peace and Development Regional Programmes (PRDPs)

  • A major Community Driven Development / Community Driven Peace Building initiative
  • Networks of grassroots organizations formed under the auspices of the civil society (labour unions, the church, private firms) in some of the most poor and violent regions of Colombia
  • Aim at building conditions of peace and development in these regions
  • Claim that peace is the result of equitable and inclusive models of development
  • Support a wide range of development processes informed by the principles of protecting life (“Primero la vida”), equity, solidarity and participatory democracy
  • The PRDPs work as an “umbrella fund”, financing a series of projects along three main axes:
      • Productive Development
      • Institutional Development and Governance
      • Human Rights and “Culture of Peace”
slide7

The Peace and Development Regional Programmes

  • A truly bottom up approach
    • Projects are promoted, formulated and executed by grassroots organizations in a participatory way
    • They express the own views and priorities (identity) of the local communities (“La vida que queremos”)
    • Beneficiaries are normally members of the grassroots organizations
  • PRPDs operate through a complex structure of networks:
      • network of grassroots organizations
      • community networks (nucleos de pobladores, red de jovenes, red de mujeres)
      • liaising with local public institutions
slide8

The Peace and Development Regional Programmes

  • Started in Magdalena Medio in the early 1990s (Jesuits)
  • Have become such a credible actor to engage in continued debate with international donors, national and local governments, and even in even in direct negotiations with the illegal groups at the local level
  • 6 PRDPs have been supported by the UNDP, the World Bank (LIL1 and LIL2 grants, 1998-2003; Paz y Desarrollo Loan, 2004-2008) and the European Commission (Laboratorios de Paz I, II and III, 2002-2010)
  • Total investment of more than 180 million USD
  • Resources are managed and executed at the regional level by a Organización Socia
  • Although they are not governmental programmes, they are endorsed in the National Development Plan
  • The Presidential Secretary for Social Affairs of Government (Acción Social) coordinates the funding
slide9

The Peace and Development Regional Programmes

6 Regions

9 Departments

125 Municipalities

1363 projects

800 grassroots organizations

180,000 estimated beneficiaries

On average:

300 beneficiaries

per project

2 years project

duration

120,000 USD project

value

slide10
The Setting and the Programme
  • Theory of Change
  • The Instruments
  • The Data
  • Descriptive Evidence
  • The Evaluation Strategy
  • Results
  • Conclusions
slide11

PRDPs. Theory of Change

  • How can the PRDPs contribute to development and peace building?
    • A community-wide process
  • Through projects (using the projects as an excuse) the PRDPs aim at:
    • regenerating the social fabric and (re)establishing bonds of trust amongst people
    • favouring the flourishing of new community level leaderships and “institutions”
    • (re)activating mechanisms of participatory democracy
  • … in order to:
    • make collective action possible
    • increase local government’s transparency and accountability
    • promote new means of conflict management
  • … and achieve in the long run:
    • peace and development
prdps theory of change

The PRDPs identify

Initial Conditions

in the regions

…and identify Modes of

Relationship

that correspond to them

PRDP

PRDPs. Theory of Change
slide13

New Conditions

Peace and Development

PRDPs. Theory of Change

Traditional Modes of Relationship

Initial Conditions

Poverty and Violence

New Modes of

Relationship

  • With the illegal groups
  • With public institutions
  • Within the community
slide14

Approach

  • ¿How to address the challenge of evaluating PRDPs’ Theory of Change from
  • a rigorous perspective?
  • In this work:
    • A quasi experimental setting
    • We evaluate the effect of PRPDs on two crucial dimensions of the “modes of relationship”: social capital and conflict management.
    • We use two innovative instruments of data collection to explore impacts in the domain of the “intangibles”:
  • Experimental Games
  • Social Dilemma

Modes of Relationship with the Community

(Social Capital)

Modes of Relationship with the Conflict

(Conflict Management)

slide15
The Setting and the Programme
  • Theory of Change
  • The Instruments
  • The Data
  • Descriptive Evidence
  • The Evaluation Strategy
  • Results
  • Conclusions
slide16

Instruments. Social Dilemma

  • Beneficiaries are invited to express their views on a tale that is read to them in third person
  • “Juana’s tale”
      • “una señora de su casa, una campesina común y corriente, hasta el día en que los violentos le mataron a Jacinto, su esposo...”
      • “... la vecina y el esposo, que eran cultivadores de fique, convencieron a Juana de que también se metiera en se negocio...”
      • “... su prestigio fue creciendo cada vez más, hasta el punto que Pedraza, un concejal y político muy conocido, le propuso que se lanzara como candidata al Concejo...”
      • “... sin embargo, antes de la posesión comenzaron a llegar mensajes anónimos con amenazas de muerte, dirigidas a Juana, provenientes del mismo grupo armado que había asesinado a su marido...”
  • Juana faces a dilemma. How should she manage a threat from an illegal group?
  • Beneficiaries are expected to identify with Juana. The instrument explores the processes of moral judgement (what is right? what should be done?) on decisions (like conflict management) that can be hardly assessed through direct survey questions.
slide17

Instruments. Social Dilemma

Immediate Displacement

Individual conflict management

Community conflict management

Institutional conflict management

Life danger associated to community leadership

  • Note that options are not mutually exclusive
slide18

Instruments. Experimental Games

Social capital

  • Competing definitions and competing measuring tools
  • Standard survey techniques (WB Soccat)
    • Structural approach (membership) versus motivational/value based approach (trust, reciprocity)
    • People tend to respond to value based survey questions according to social expectations and social norms (Glaeser et al., 2000).
  • Our definition: social cooperation (act collectively for mutual gain)
  • Our measure: a behavioural measure of contribution to a local public good in a field experimental setting
slide19

Instruments. Experimental Games

  • Simulated decision scenarios where the beneficiaries face alternative on the use of a monetary endowment
  • Based on the theoretical framework of the experimental economics: players’ decisions should reveal their preferences/motivations
  • The rules of the game (institutions) define a potential conflict (a dilemma) between competing motivations
    • Normally extrinsic self-interested preferences against non self-interested or intrinsic preferences (Deci and Ryan, 2000).
  • The “controlled” experiments are designed in such a way that decisions only have monetary consequences (no reputation effects)
  • As the decisions imply a real monetary payoff (on average the equivalent of 3 USD) participants are expected to reveal their preferences more similarly to how they would behave in a real life situation
slide20

Instruments. Experimental Games

Voluntary Contribution Mechanism – VCM (Marwell and Ames, 1979)

  • Played in group (40 participants on average)
  • The experiment is a simple public good game in which subjects decide to invest in a ‘public’ or a ‘private’ project (public or private account)
  • Every token invested in the public account yields benefits for the whole group, while the token inverted in the private account only yields benefits to the owner of the account
  • The dilemma consists in that there is a conflict between the self interest and the optimum social outcome
  • If everyone invests in the public project the total earnings for the group are bigger than if everyone invests in the private account
  • However an individual is always better off by investing in the private account (Nash Equilibrium)
the basic structure of the dilemma
The basic structure of the dilemma

Not cooperating (~C) is a dominant strategy for the (j-1) player

If everyone invests in P

If everyone invests in G

slide22

contribution to local public good

communication

The game is played in Two Rounds: First Round. Anonymous decision Second Round. 10 minutes communicationInformation feedback after First Round. We also collect data on social networks in the group.

slide23
The Setting and the Programme
  • Theory of Change
  • The Instruments
  • The Data
  • Descriptive Evidence
  • The Evaluation Strategy
  • Results
  • Conclusions
slide24

The Data

  • Two main data sources
  • (Late) Baseline for the evaluation of PyD and LP (Nov 2006 – Feb 2007).
    • Treated Locations
    • 18 months of exposure to the activities of the PRDPs on average
    • High variability in exposure because of the roll out of the PRDPs both within and across municipalities
  • Second follow up for the evaluation of Familias en Acción (Nov 2005 – Apr 2006)
    • Control Locations
    • Sample for the evaluation of the famous CCT programme.
    • Targeted to the poor (SISBEN1), mainly women
slide25

The Data

FA Data

PRDP Data

PRDP Municipalities

FA Municipalities

Sample of Treated Locations

Sample of Control Locations

Members of Other Civic Organizations

PRPD Project Beneficiaries

Sample of non Beneficiaries

(Control Group 2)

Sample of Beneficiaries

(Treatment Group)

Sample of Beneficiaries

(Control Group 1)

Beneficiaries with lower exposure

Beneficiaries with higher exposure

slide26

The Data

  • The challenge of defining project beneficiaries for projects with universal vocations
slide28

Credits

  • The impact evaluation of the PRDPs is designed and promoted by the Departamento Nacional de Planeación (DNP) of the Colombian Government with funding of the WB and the EC
  • The evaluation plan contemplates 2 rounds of data collection and a mixed qualitative-quantitative methods approach
  • The first phase of the evaluation was conducted by the consortium SEI-Economertía-IFS
  • Full results of the evaluation have been published in an official Policy Report by the DNP (DNP (2009), ‘Hacia la consolidación de una propuesta para evaluar el impacto de los Programas Regionales de Desarrollo y Paz.Evaluación de Impacto de los programas Paz y Desarrollo y Laboratorios de Paz: Línea de Base e Impactos Preliminares‘, Evaluación de Políticas Publicas, Bogotá)
  • This new work draws extensively on the conceptual framework, analysis and results of the Policy Report
  • A second phase of the study is currently undergoing, coordinated by the DNP
slide29
The Setting and the Programme
  • Theory of Change
  • The Instruments
  • The Data
  • Descriptive Evidence
  • The Evaluation Strategy
  • Results
  • Conclusions
slide31

Descriptive Evidence

  • High degree of heterogeneity in beneficiary characteristics according to project types
slide32

Descriptive Evidence

Social Capital (Contribution to the Public Good)

slide33

Descriptive Evidence

Conflict Management

slide34
The Setting and the Programme
  • Theory of Change
  • The Instruments
  • The Data
  • Descriptive Evidence
  • The Evaluation Strategy
  • Results
  • Conclusions
slide35

Evaluation Strategy

  • Identification strategy

a) across locations (treated vs. control matched municipalities)

b) within locations (treated vs. control households in treated locations)

c) within treated households (based on the intensity of exposure)

  • Comparison of a) and b) may provide insights on the presence and magnitude of spill-over effects
  • No random assignment -> Potential endogenous programme placement (selection and self selection)
    • Across locations
    • Within locations
    • Within treated households
  • No pre-programme information (apart from municipality characteristics)
  • Multivariate regression models. All models are based on the “selection on observables” assumption
  • Serious treatment heterogeneity issue
  • Different time frames
slide36

Evaluation Strategy. Across Municipalities

  • We match treatment and control locations on a complete set of pre-programme municipality characteristics
  • 2 control locations dropped because of missing information
  • 20 control locations and 16 treated locations are dropped because they fall outside the common support
  • 2,427 households (58% of the original sample) are on the common support)
slide38

Evaluation Strategy. Within Treated Municipalities

  • We use all treated locations
  • We compare outcomes between treated and control households controlling for individual, households and municipality characteristics
  • We assess the effect of the differential exposure to the PRDPs across treated households:
    • By comparing households across the median treatment duration (0-12 months versus 13+)
    • By comparing three groups of households with different treatment duration (0-6, 7-14, 15+)
    • In a continuous way (allowing for non linear functional forms – cubic and quadratic terms)
  • We also analyze the effect of average municipality exposure
slide40
The Setting and the Programme
  • Theory of Change
  • The Instruments
  • The Data
  • Descriptive Evidence
  • The Evaluation Strategy
  • Results
  • Conclusions
slide42

Results. Experimental Game

Effects across municipalities

  • The level of contribution to the public good is much higher in municipalities where the PRDPs operate, than in other municipalities in the country with comparable characteristics.
  • Results hold if CS is imposed based on matching on individual characteristics
slide43

Results. Experimental Game

Effects within treated municipalities

  • There is no significant difference in the contribution rates between treated and control households within treated locations.
slide44

Results. Experimental Game

Exposure effects

  • We find some light evidence that, ceteris paribus, increased exposure to the PRDPs lead to higher contribution to the public good, particularly in the long run.
slide45

Results. Experimental Game

  • The previous findings suggest that there might be some contamination effect in social capital formation at the community level between treated and control households within the same municipality.

Examining the case for Spill Over effects.

  • Exploiting knowledge networks, social capital effects seem to propagate within treated municipality through a spill-over mechanism.
  • The fact of knowing the PRPD could however be endogenous to social capital.
slide46

Results. Experimental Game

The importance of session composition

  • Heterogeneity and social capital formation (Alesina and La Ferrara, 1999; Cardenas, 2003)
slide47

Results. Experimental Game

Critical Mass or Conditional Cooperation?

  • Because of the sampling procedures, the proportion of beneficiaries in the session doesn’t reflect coverage rates in the municipality
  • These findings confirms the hypothesis that cooperation is conditional to the fact of being interacting with a groups of like-minded socially oriented people (Fischbacher et al., 2000; Gächter and Thöni, 2004; Burlando and Guala 2003)
slide48

Results. Experimental Game

Critical Mass or Conditional Cooperation?

  • In the second round, the group effect seems to be operative for control households only
  • This suggests that it is mainly driven by reputation factors (Brosig, 2002)
  • Complementary evidence suggests that PRPD beneficiaries are actually considered more “trustworthy” than the rest of the population
slide49

Results. Experimental Game

Exploring the “critical mass” hypothesis

slide50

Results. Social Dilemma

  • Because of the structure of the data we can only test our hypothesis within treated locations
  • Even after controlling for observables, there is no evidence of significant differences between treated and control households within treated locations
slide51

Results. Social Dilemma

  • The exposure models indicate a convincing positive effect on community conflict management, as well as a reduction of the propensity to displace and a lower perception of risk.
slide52

Results. Social Dilemma

  • Taken all together, these results may suggest the existence of spill over effect in the modification of conflict management behaviour at the community level

Municipality level effects

slide53
The Setting and the Programme
  • Theory of Change
  • The Instruments
  • The Data
  • Descriptive Evidence
  • The Evaluation Strategy
  • Results
  • Conclusions
slide54

Some methodological points for discussion on the instruments

  • Experimental games
    • What’s the meaning of (the motivations behind) the experimental behaviour?
    • Does the experimental behaviours disclose elements of real life behaviour?
      • Do people understand the rules of the game?
      • Real decisions don’t have only monetary consequences
      • Are 3 USD a big enough incentive to inform decisions?
      • Need to further differentiate heterogeneous types of motivations
  • Social Dilemma
    • What is the degree of coincidence between moral deliberations (Juana should) and concrete actions?
      • To what extent does the hypothetical filter put a bias?
      • To what extent does the third person filter put a bias?
    • Would a single option approach reflect reality?
  • Can these instruments be used in the context of a panel?
    • Potential biases arising from repetition (learning effect)
  • Is a core quantitative approach enough?
slide55

Conclusions

  • The work of the PRDPs is purposefully directed to strengthening the social fabric in the community. In facts, the PRDPs centre their efforts on the promotion of civic participation, leadership and active citizenship.
  • Indeed, the PRDPs contribute to building social capital and creating new aptitudes/abilities for conflict management in communities affected by violence.
  • Does this lead to increased socioeconomic status and a reduction in the violation of human rights? Yes in some important development indicators. No, or not yet, in peace?
  • These results shall not be generalized because of the peculiarity of the PRDPs influence strategy.
  • More research should be undertaken on strength and limitations of Experimental Games and Social Dilemmas as instruments for the evaluation of the effect of social interventions on “socio cultural” dimensions.
  • A follow-up study, for the impact evaluation of the PRDPs is currently undergoing, with new expanded data collection and improved instruments.
slide56

For Peace and Goodwill?

Using an Experimental Game and a Social Dilemma to Analyse the Effect of the Desarollo y Paz Programmes in Colombia.

more information on the PRDPs can be found at:

http://www.redprodepaz.org/

http://www.accionsocial.gov.co/

THANKS

World Bank

Washington, 5th May 2009

ad