Assessing Public Investment Management

1. Assessing Public Investment Management Lessons from the 2008 EU Infrastructure Study and Beyond Bernard Myers, July 24, 2008

2. Administrative Context Affects PIM in our Region • Coalition governments, with weak MOFs • Political incentives to maximize absorption of funds • Reliance on formal laws and regulations • High level of political involvement in administrative decision-making • Lack of an evaluation culture • Incentives for performance are weak and ad hoc

3. Fragmented Institutional Responsibilities: MoF can’t be the only client • Potential TA targeted at multiple levels • Financing decisions at MOF – and possibly split across directorates • Medium Term Planning by MoPlan or MoEcon • Appraisal Standards and Screening at MOF or MOP • Preparation and appraisal by sponsoring ministry • Other bodies: State Expertise; Procurement Office; SAI • Private sector firms conducting feasibility studies

4. EU Report: Questions posed at the beginning of the process • How do countries determine the overall envelope for investment • How do they determine sectoral allocations • How are projects selected within the sector • How do they monitor implementation of the projects to assure effective outcomes • How do they evaluate project performance ex-post

5. Characteristics of the EU Report Preparation • Local consultants/former civil servants prepared cases • Common questionnaire used, but strong reliance on consultant judgment to guide process • Balance criticism of local practice with comparisons to better practice in old member states • Drilled down into experience with individual projects – comparing “formal” rules with the informal practice (e.g., political decision-making) • Two dimensions: Institutional/Process issues versus Technical Tools/Capacity

6. Some Questions Are Difficult to Assess Accurately • Strategic Planning: How relevant is it for decision-making • Cost-Benefit Analysis: What is the quality, and is it really used • Alternative Options: Are the alternatives proposed credible • Project Selection: What really drives decision-making; the role of politics • Project Outcomes: How well are projects being implemented (i.e., cost-effectiveness)

7. Limitations of the Report: Not all issues can be addressed • Difficult to benchmark across countries • Recommendations are general and not country specific • Possible risk that good practice examples may appear too challenging • Limited attention to other relevant issues: • How overall investment level is decided • Asset registries and under-funding of maintenance • Public Procurement processes and anti-corruption • Ear-marked revenues and off-budget financing • Screening for small, standard projects – e.g., school construction • Improving absorption capacity of EU funds • Reasons for cost over-runs

8. Main Findings from EU Study(1) • Strategic Plans – usually disconnected from medium term budget projections and lacking genuine prioritization of projects • Budgeting – Funding is fragmented due to annual budget process; not well supported by medium term budgeting tools. Within-year funding rigidities • Project Appraisal and Selection • Use of CBA on EU-funded projects, less so for national projects • Impact of CBA on decision-making is questionable. • Risk of project over-design and limited consideration of project alternatives.

9. Main Findings from EU Study(2) • Risk mitigation and project planning – Pro forma part of feasibility study, but little risk mitigation. Sometimes hurried project design leads to poor implementation • Role of MOF and external bodies: Weak MOF oversight role in setting quality standards. Inability to stop bad projects early. Little external checks on quality • PPPs and off-budget entities: Private financing used primarily to get around fiscal constraints. Project selection is ad hoc and internal expertise to manage public risk is limited

10. Main Findings from EU Study(3) • Procurement strategies: Do not necessarily balance risk between contractor and purchaser. • Project Monitoring: Little comparison of actual cost versus planned cost over course of project life. Limited non-financial indicators of performance • Audits and Ex-post Review: Internal and external audit limited to financial compliance • Capacity development: Training efforts on CBA but limited in scope. Unlikely to include project management, procurement, or policy analysis

11. Experience from Kazakhstan(1) • Concerns about the quality of project selection and cost-effectiveness of projects • Requirement for CBA is generally enforced, but quality of analysis is inconsistent and capacity for MOF to assess is weak. Limited private sector capacity. • Strategic plans are too general to provide adequate prioritization of projects or enable poor projects to go forward (e.g., soviet-style norms) • No effective mechanism to prioritize projects or rank them even within a sector

12. Experience from Kazakhstan(2) • High turnover of public investment staff in MOF, insufficient clout to challenge • Information on project implementation not well organized at MOF (no database) • Procurement procedures favoring lowest bid, questionable bidders • Budget calendar forces rushed design phase, leading to costly changes during implementation

13. Experience from Kazakhstan(3) • Discrepancies between cost estimate in CBA and actual contract award and final completion cost. • PPPs not well integrated into overall project prioritization processes • Ministry staff supervising contracts lack expertise to challenge contractors • No ex-post evaluation of projects

14. Survey results from 2008 Istanbul conference on PIM • Which results are intuitive and which are unexpected?

15. What have been the most serious weaknesses in the way public investment is managed in your country?

16. Where a cost benefit analysis is prepared, are the results of that analysis a major factor in deciding which projects to proceed with?

17. What other factors affect the decision to proceed with a particular project?