Managing for transition impact at the EBRD

1. Managing for transition impact at the EBRD World Bank Donor Meeting Paris, May 21, 2008

2. Introduction • EBRD mandate: foster transition to open market economies and private and entrepreneurial activity • “Systemic change”, not “development outcomes” • No existing template for project appraisal • Approach evolved in steps over ten years • Further refinements under consideration (environmental impact)

3. Aspects of transition impact management • Translate institutional mandate into project-level framework • Link operational objectives to economic priorities • Appraise, monitor and evaluate project impact • Governance and decision-making

4. Overview: Managing for transition impact in EBRD operations

5. Aspects of transition impact management • Translate institutional mandate into project-level framework • Link operational objectives to economic priorities • Appraise, monitor and evaluate project impact • Governance and decision-making

6. Operationalising the transition mandate • Define market economy and project contributions • Three dimensions of a market economy • Extent and structure of markets • Institutions and policies that support markets • Skills and behaviours of a modern market economy • Project transition impact checklist • Not quantifiable: ordinal approach, typologies

7. Transition impact checklist

8. Aspects of transition impact management • Translate institutional mandate into project-level framework • Link operational objectives to economic priorities • Appraise, monitor and evaluate project impact • Governance and decision-making

9. Linking project choice and design to economic priorities • Transition checklist does not per se express priorities • Four main instruments to inform project choice • Analysis: Assess country and sector status, gaps in economic development (triennial) • Strategy: Prepare country (triennial) and sector strategies (periodically) • Monitoring: Annual monitoring of transition progress across countries • Evaluating broader impact: Transition Impact Retrospective (every 5 years), experimenting with Randomized Impact Assessments

10. Assessing gaps in transition • Attempt to rate challenges systematically • 13 sectors x 27 countries x 3 dimensions = 1053 ratings • Define benchmarks of good practice • Compare actual achievements against benchmarks • Value judgments, supported by data • Rate magnitude of remaining challenges • Scale: large; medium; small; negligible

11. Framework for transition gap assessments: Illustration

12. Annual monitoring: Transition Report • Country-by-country review of progress on several reform dimensions • Progress is rated on a scale of 1-4+ (transition indicators) • Published in November and presented in roadshows • Brief update prior to Annual Meetings (Spring)

13. Transition indicators 2007

14. Transition indicators 2007 CEB SEE CIS/M

15. Aspects of transition impact management • Translate institutional mandate into project-level framework • Link operational objectives to economic priorities • Appraise, monitor and evaluate project impact • Governance and decision-making

16. Project appraisal: Ex ante impact assessment • At appraisal, draw together gap analysis and project checklist • Does project selection reflect transition priorities? • Does project design (e.g., covenants, technical assistance) maximize impact? • How likely is the project to achieve its transition goals (e.g., policy risks, commercial strength)? • Rating for transition potential: • Excellent, good, satisfactory, marginal, unsatisfactory • Only projects rated “satisfactory” or better can proceed • Rating for risks to transition impact: • Negligible, low, medium, high, excessive • Projects with “excessive” rating do not proceed Project monitoring

17. Stylised ratings guide • No market distortions; transition impact via demonstration effects or impact on markets to be achieved through project selection • Sector requires policy changes but these are being implemented by client’s own accord or externally driven (e.g. EU accession; • Sector requires policy change by government but client is a private company – Bank’s role limited to policy dialogue; • Company requires substantial restructuring but Bank has no leverage. Satisfactory • Project addresses a clear transition challenge in company or sector specifically identified in Bank country or sector strategy; • Sector requires policy changes that client is implementing and project facilitates with covenants, TC component or policy undertakings; • Repeat project with company, or in sector, where market or demonstration effects or other transition impact sources are far from exhausted. Good • Project is a “first” in sector or country with large market or demonstration effects; • Project addresses a key transition challenge in company or sector with strong and comprehensive conditionality; • Sector requires policy change and Bank plays a key policy dialogue role; • Flagship company restructuring/privatisation in sector or country. Excellent

18. Risks to Transition Impact • Likelihood of not achieving transition potential • e.g., due to business environment, government policies, legal framework, project or client performance • Likelihood of “negative” transition impact from perverse attributes of the project • e.g., reputation risks, subsidies, trade barriers, other market distortions • Many transition risks are correlated with commercial risk • if a project does not perform, transition impact potential will not be realised

19. Distribution of transition impact potential and risk ratings 2002- 2006 Satisfactory Good Excellent

20. Project monitoring • TIMS – information system to monitor and assess the transition impact of the Bank’s portfolio • Project documents set detailed impact targets (transition benchmarks) and timeline • At least annual review of performance in conjunction with credit reviews • Project teams prepare brief update of benchmark achievement • Action taken as necessary to mitigate concerns • Ratings adjusted: • “Risk” rating improves as benchmarks are met • “Potential” ratings adjusted if major change in circumstances

21. Aspects of transition impact management • Translate institutional mandate into project-level framework • Link operational objectives to economic priorities • Appraise, monitor and evaluate project impact • Governance and decision-making

22. Governance considerations • Institutional Scorecard • Board sets annual targets for transition impact in context of budget • For 2008: • Transition potential ratings of signed projects >80% good or excellent • Expected transition impact of portfolio 50-60% equivalent to good/medium or better • Quarterly Board review based on Institutional Performance Report + annual Strategic Portfolio Review • Incentive management: Allocation of salary and bonus pool to banking teams • transition scores, financial and operational performance

23. Decision-making process • Project approval cycle • Operations Committee (OCE key role) • Concept Review: transition priorities, sound banking principles, additionality; guidance on design • Structure Review: have major issues identified at CR been addressed? (often off-line) • Final Review: detailed design, draft loan / shareholder agreement, document issues • Board • Project document sets out transition rationale / ratings, credit rationale / ratings, etc. • Board can request Chief Economist to justify transition rating / rationale

24. Issues to consider • Centrality of assessor (impact) • Independence vs. involvement • Design, implementation, learning • Assessment vs. evaluation • Internal checks and balances • Quality of assessment • Quality of information • Experimental design (learning) • Coverage of assessment • Range of products: Frameworks (+ BAS), TC... • Range of effecs: Environment, social issues...

25. Maintain transition focus • Selectivity: systemic change, not global public goods per se • Process, not specific development outcomes • Institutional change - where transition and development meet