Driving better Project Evaluation in Queensland - the role of the Revised Project Evaluation Guidelines Presentation

1. Driving better Project Evaluation in Queensland - the role of the Revised Project Evaluation Guidelines  Presentation to the Australasian Evaluation Societyby Scott Hook and Mark RiceMay 2006 

2. What is today about? Understand the reasons for having the Project Evaluation Guidelines (PEG). The PEG framework and key concepts. Importance of ex-ante and ex-post evaluation. Assess the changes in the guidelines since the previous 1997 version. Using the PEG in your analysis. Key lessons.

3. Objective � section 1 Key concepts. Rationale for evaluation at the beginning of a project. Why it is important to use the Project Evaluation Guidelines. When to use the PEG?

4. Key Concepts � what to understand Cost-benefit analysis Cost-effectiveness analysis Discount rates Externality Multiplier Opportunity cost Shadow pricing

5. Mistakes happen!

6. Things can go wrong! Projects can become more expensive, take longer to complete and do not achieve the initial objective. Consider Victoria�s Federation square Commonwealth�s Seasprite acquisition UK government experience with planning for infrastructure projects suggests that: Timing for non-standard engineering projects be revised in the range of 3-25 per cent; Costs be revised in the range of 6-66 per cent.

7. Fail to plan, plan to fail Consistent, transparent and accurate information is needed to Align agencies� policies, projects, programs and activities to the Government�s priorities; Prioritise individual projects within programs; Ensure that their project procurement and resource allocation decisions achieve maximum value and benefit for the community.

8. What are the Project Evaluation Guidelines? The development of the PEG reflected a concern that proposals for new projects, services or policy require guidelines. The PEG provides the rationale, process and requirements for project evaluation in Queensland Government Agencies. There is also an increasing need to use cost-benefit analysis in the evaluation process.

9. PEG Background Previous Project Evaluation Guidelines � February 1997. Requirement for more detailed guidelines, with inclusion of templates. Inclusion of additional material on risk management, governance arrangements and project management. The revised Project Evaluation Guidelines have been circulated to all agencies for comment and use.

10. PEG background (cont) Charter of Social and Fiscal Responsibility (April 2004) requires The direction of resources and activity towards the Government�s highest priority areas; and The provision of strategies to achieve value for money in delivering infrastructure and services to the community. PEG seeks to assist rigorous and robust analysis across Queensland Government Agencies.

11. Why do we need the PEG? To provide a standard methodology and whole-of-Government approach to the preparation of project evaluations and the analyses they contain. Increased recognition of the need to improve rigour and clarity for decision making purposes by increasing the use of cost-benefit analysis. Establishment of a whole-of-Government framework to clarify uses and linkages between other guidelines, such as PPP.

12. Part of Good Governance Thinking about options and then ranking them can improve the quality of Government decision making. Cabinet seeks the maximum value benefit for the whole community.

13. What PEG is not A one-stop project management tool. A method to micro-manage all projects; slow agency new policy development; and only evaluate physical infrastructure. Instead, PEG is Part of the policy development toolkit for agencies. PEG can used for infrastructure, service delivery and new policy initiatives.

14. Ex-what? Traditional evaluation has been conducted at the end of a project. Increasingly, risks of a project in terms of time, cost and quality require evaluation upfront. PEG and business cases are an example of ex-ante evaluation. Ex-post evaluation still has an important role.

15. Changes in 2005 PEG 2005 PEG applies to all projects (involving capital or recurrent expenditure or regulation), rather than capital projects. Increased detail on how to undertake each of the analyses. Social and environmental analyses integrated into CBA. Budget analysis on accrual basis. New section on regulatory analysis. Stronger statement discouraging use of multipliers.

16. Objectives � Part 2 PEG contents. Project Evaluation framework. Understanding the cost-benefit analysis methodology. Role of agencies.

17. PEG Contents Introduction. Framework for Evaluating Projects. Project Evaluation Pro forma. Common Project Evaluation Techniques.

18. What is Cost-benefit analysis? Identifies values and compares the costs and benefits derived from investing scarce resources, such as land, labour and capital, in project options to achieve a particular objective. Costs and benefits are valued in dollar terms and adjusted for market distortions or imputed where the market does not exist.

19. Project Life Cycle

20. Framework for Evaluating Projects Five Steps Objective, priority and justification of project. Project options identified. Preliminary assessment. Evaluation of project options (explained further shortly). Selection of preferred option.

21. Evaluation of Project Options For each of the project options, agencies prepare a: Risk Analysis Financial Analysis Cost Benefit or Cost Effectiveness Analysis Budget Analysis Regulatory Analysis (where appropriate)

22. Risk Analysis Risk Identification Determining what, why, where, when and how events could prevent, degrade, delay or enhance the project. Risk Assessment Sources of project risk, their positive and negative consequences and probabilities that they will occur. Allocation and Mitigation Can it be prevented and/or what to do with it once it has materialised.

23. Financial Analysis Measures costs and revenues for the sponsoring agency or agencies. Involves cash expenditure and revenue, including capital. Measures the ability of the agency to cover expenses of the project option through a revenue stream.

24. Cost Benefit Analysis Adopts economy wide perspective (usually for the State as a whole). A financial analysis is the starting point. Includes a wider range of social and environmental costs and benefits than the financial and budget analyses, eg pollution, land degradation, traffic congestion, health outcomes, etc. Measures true economic worth of resources. Project option with the greatest net present value � most economic use of resources.

25. Identify and estimate the expected economic benefits and costs We are measuring �social� costs and benefits Not calculating the distribution of those costs and benefits. Valued in dollar terms at their �true� economic worth. Not confined to market transactions Non-market costs and benefits can be valued by imputation or by appropriate adjustment of the market price.

26. Calculate the net present economic value Discounting allows benefits and costs in different options occurring at different times to be compared on the same basis, and recognises that a dollar now is valued more highly than a dollar in the future. The difference between the discounted streams of benefits and costs of each project option is the Net Present Economic Value of the project option. A positive or negative value helps determine the economic viability of a project. Choice of the discount rate will be an important determinant.

27. Expected economic benefits (continued) Consideration should be given to estimating: Avoided costs; Cost savings; Revenues; Benefits to consumers and to the community as a whole; and Residual value of assets.

28. Key Points � the NPV analysis Capital expenditure estimates are reasonable and accurate All legitimate costs and benefits have been included. No invalid costs and benefits. All costs and benefits have been valued at their market value or economic value. A reasonable timeframe for the project and costs and benefits have been forecast reasonably and with transparency.

29. Budget Analysis Covers the budget impact for the relevant agency or agencies. Involves accrual expenses and revenue and includes non cash items, such as depreciation. Measures impact on Operating Statement and Statement of Financial Position. Measures impact on the State Government�s fiscal position.

30. Regulatory Analysis Projects are assessed to ensure they: Do not restrict competition under the TPA; Produce a net benefit for State where regulations are changed; and A Regulatory Impact Statement and /or Public Benefit Test may be required for changes to legislation and/or subordinate legislation.

31. Don�t forget we need to� Discount the value of money to account for different flows occurring over different time periods. Use sensitivity analysis for the Financial Analysis and the Cost Benefit Analysis (eg, changes in prices, construction costs, usage of services or discount rates). Include project implementation and post-implementation review.

32. Garbage in � Garbage out The analysis should contain Textually and numerically explicit explanations of the assumptions underlying all capital and recurrent estimates regarding labour costs, energy costs, demand growth, charges etc. Clear and referenced data sources for validation purposes.

33. When to use the PEG The PEG should guide the assessment of any project under consideration by an agency. Can be used for infrastructure, service delivery and new policy initiatives However, judgement is needed. A detailed financial and economic analysis may be impractical for small projects or expensive and complex. Assessment should be made according to scope, cost, complexity, riskiness and sensitivity of the project.

34. What are some alternatives Economic impact analysis, which estimates flow-on economic impacts of a project, can supplement cost-benefit analysis. Input-output modelling measures a gross flow-on from project expenditure (a multiplier). Applying input-output multipliers usually overstates project impacts. General equilibrium modelling uses a mathematical model of the economy to estimate the whole of economy impact of changes in policies or new projects. Why are they different to CBA? Focuses on economic impacts (not valuing social impacts) Includes flow-on impacts to all sectors (possibility of overestimating benefits).

35. What is the role of Agencies? The agency must make a decision about the level of analysis and the allocation of suitable resources. Identify and quantify expenses, costs, revenues and benefits and project affordability. Undertake a rigorous cost-benefit analysis. Liaise with the relevant business branch and OESR for guidance on specific technical issues.

36. What is Treasury�s role? Provide training, advice and guidance to an agency to support the preparation of a cost-benefit analysis. Advise the Treasurer and CBRC on the information quality of project evaluations and the merits of output/investment proposals put forward by Government departments and agencies. Establishment of a data base that agencies can access when preparing project evaluations

37. The process Several stages Consultation on the PEG. Refinement and finalisation of the PEG. Integrate PEG within the Project Assurance Framework.

38. Relationship with other frameworks Other guidelines that are related include the Public-Private Partnership Policy Guidelines, Public Benefits Test Guidelines and Regulatory Impact Statement Guidelines. A Project Assurance Framework is being developed by FMB The PAF will include guidance on a gateway process to ensure success management of project processes. The PEG is being integrated into the PAF.

40. Key Learnings That any project evaluation should be using the PEG as a template and all data and assumptions are documented, and costs and benefits are explained. Cost-benefit analysis allows choices to be recommended to Government. Include the status quo as an option but exclude estimates of costs and benefits do not follow logically from assumptions. Never include invalid costs and benefits and uses a multiplier.

41. Questions? Views? Opinions? Comments?