Gabriela p rez yarahu n universidad iberoamericana
This presentation is the property of its rightful owner.
Sponsored Links
1 / 17

Gabriela Pérez Yarahuán Universidad Iberoamericana PowerPoint PPT Presentation


  • 61 Views
  • Uploaded on
  • Presentation posted in: General

Exploring Politics, Accountability and Evaluation Use in the Mexican Federal Government Education Programs. Gabriela Pérez Yarahuán Universidad Iberoamericana. Motivation. Late 1990’s – 2000’s political changes leading to democratization in Mexico.

Download Presentation

Gabriela Pérez Yarahuán Universidad Iberoamericana

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Gabriela p rez yarahu n universidad iberoamericana

Exploring Politics, Accountability and Evaluation Use in the Mexican Federal Government Education Programs

Gabriela Pérez Yarahuán Universidad Iberoamericana


Motivation

Motivation

  • Late 1990’s – 2000’s political changes leading to democratization in Mexico.

  • Late 1990’s, changes in social policy. National large scale poverty program (PROGRESA) was implemented and evaluated.

  • Since the late 90’s several institutions for accountability, transparency, evaluation are created.

  • Since the early 2000’s social programs evaluated once a year.

  • ….so?

  • How can we study the effects of these institutions on policy/programs?


Accountability performance evaluation

Accountability Performance Evaluation


Evaluation use

Evaluation use

Types of use

Determinants of use

Evaluationprocess

Quality and credibility of evaluation studies

Communication and publication practice

Applicability and relevance of findings

Timeliness

Unit responsible for evaluation

Externalfactors

Legal framework

Political climate

Availability of other sources of information

Personal characteristics of policy decision makers

Commitment or receptiveness to evaluation.

  • Symbolic

  • Conceptual

  • Instrumental

(Weiss 2008; Chelimsky 2006; Rossi and Freeman 2004; Pitman, Feinstein, and Ingram 2005; Cousins and Leithwood 1986; Patton 1997; Weiss 1999; Marra 2000; Balthasar's 2008)


Program evaluation system in mexico two periods

ProgramEvaluationSystem in MexicoTwoperiods

2001-2006

2007-2009

Aspects such as the communication and publication practice of evaluation results are present, as well as the intention to make evaluations have more applicability and relevance of findings.

LGDS and LFPRH

First Annual Evaluation Program was issued in 2007.

“Consistency and Results Evaluation” .

Mechanism to follow up on recommendations.

Specific Performance Evaluation implemented.

  • Official norms, guidelines and laws were vague, not explicit and at most ambiguous regarding evaluations objective, content and use.

  • Except for the references of their use in the Budget bill discussions, there is an absence, during this period, of specific mechanisms to incorporate evaluation in the policy process regarding the improvement, change, modification of social programs and policies.


Methods

Methods

  • In order to investigate on the instrumental use of evaluations:

  • 9 programs were selected with variability of budget, agency, year of creation.

  • Four categories of changes were established.

    • General Objective

    • Target Population

    • Selection criteria

    • Type of benefits provided

  • Operation Rules were analyzed to count changes in each of the categories for each year of program operation.

  • Evaluation were analyzed to count reccomendations on each of the categories established.


Methods ii

Methods II

Survey

  • In order to understand how the evaluation studies are used by public officials in charge of program operation or in evaluation areas in federal agencies, this research also included a survey of evaluation use applied to public officials.

  • The survey was designed to measure perception on types of evaluation use and the variables that may affect evaluation use.

  • The survey was voluntarily and anonymous and was sent to 293 public officials in 19 agencies of the federal government.


Results

Results


Gabriela p rez yarahu n universidad iberoamericana

  • The total count of changes registered were 111, with the highest percentage (37%) of those being in the Selection Criteria of beneficiaries. The total count for recommendations was 80. The highest percentage of recommendations was in the Target Population category (36%). The spearman correlation coefficient between changes and recommendations is high for the Target Population category (0.812), and for Benefits (0.667); moderate for General Objective (0.432) and rather low for Selection Criteria (0.098).

  • The three programs with the greatest amount of recommendations and change, as can be seen in the figure are FONART, OPORTUNIDADES and LICONSA, all three from the Department of Social Development.

  • Changes in ORs seem to have a moderate to low negative correlation with size of budget (-.319), while the opposite happens with recommendations with a moderate positive correlation (0.439).


Survey

Survey

  • The effective rate of response was 19% (56) with respondents from 16 of the 19 agencies.

  • 54% of respondents belonged to operation units of programs, and 46% to evaluation units.

  • Their demographics showed that 54% of respondents were men, and the average age was between 46 and 55 years.

  • The agency with the greatest group of respondents was Social Development with 21% of the sample.

  • The sample had a wide spectrum of administrative level officials, being the greatest group a middle level manager (Director de Area).

  • The average level of years of experience in their current position was between 3 and 5 years.


Gabriela p rez yarahu n universidad iberoamericana

  • The great majority of respondents (88%) thought that evaluations had some type of use for programs or agencies.

  • A majority of respondents (79%) mentioned impact evaluation as the most useful type of evaluation. When asked to rank (on a scale from 1 to10) most common uses, the results were as follows:

    • To provide information about a program´s design, process and impact: 8.2

    • To modify Operating Rules: 8.2

    • For accountability: 7.2

    • To position the program to public opinion: 6.4

    • To negotiate budget: 5.2

  • When asked to rank users of evaluations, the three best ranked actors were: CONEVAL (8.0), Evaluation Units within agencies (7.8), Directors of Programs (7.2); the three worst ranked users were Congress (5.2), Beneficiaries (4.6) and Citizens (4.0).


Gabriela p rez yarahu n universidad iberoamericana

  • In a scale from 1 to 10, respondents ranked the following elements for their importance in the use of evaluations.

    • Qualifications of evaluator: 8.9

    • Quality and adequate communication of findings: 8.7

    • Receptiveness to evaluation: 8.6

    • Clarity in evaluation design: 8.5

    • Involvement in evaluation design by program directors: 8.1

    • Legal framework for evaluation use 7.8

    • Organizational climate 6.6

    • Political external factors: 6.2


Final comments

Final Comments

  • The relationship between accountability institutions and government performance must be explored in detail. Evaluation and its use play an important role.

  • Mexico shows a clear development towards a more structured and coherent set of rules regulating evaluation making and its use.

  • This research makes use of small sample of programs and a survey of perception on evaluation use. With this data we observe a rinsing pattern on instrumental evaluation use.

  • The results of the survey confirm the idea that evaluation appears to be highly ranked as an instrument for government change.

  • More research is necessary to asses the effectiveness of the system to produce the necessary changes to increase government performance.


  • Login