Current issues in the Dutch CBA practice

1. Current issues in the Dutch CBA practice Niek Mouter (TU Delft) Bert van Wee (TU Delft) Jan Anne Annema (TU Delft)

2. Context • Since the year 2000 it is obligatory to construct a CBA in the planning process of large / special national infrastructure projects in the Netherlands. • Since the year 2008 it is obligatory to construct a CBA in the planning process of spatial-infrastructure projects that apply for monetary resources by the central government. • Research question PhD: • Substantive improvement regarding the use of CBA in the decision making process of spatial-infrastructure projects.

3. Problem analysis • Start with a thorough problem analysis before starting research concerning substantive improvements. • Interviewed 72 people who construct, use or reflect on CBAs • 19 consultants, 28 scientists, 25 policymakers • Structure interviews: • What are the five mostimportant substantive problems u experience with CBA? (plus 40 minute discussion about problems) • What is the most important advantage and most important disadvantage regarding the use of CBA in the decision making process of spatial-infrastructure projects? • How do u weigh the advantages and disadvantages?

4. I think that advantages of using CBA in decision making process are larger than disadvantages 84,5% I think that advantages of using CBA in decision making process are larger than disadvantages 8,5% 7,0% Neutral

5. 0: Rejecting the use of welfare economics as the fundamental theory of an ‘ex ante’ evaluation method (19) 1: Problem analysis (7) 2A: Reference case (28) 2B: Project alternatives (5) Non-specific CBA problems 3A: Estimating primary physical effects (269) 3B: Deduce the different impacts of primary physical effects (30) 7: Problems with uncertainty in CBA (26) 4: Monetizing effects / impacts (88) Specific CBA problems 5: Discounting effects (27) 6B: Presenting distributional effects (14) 6A: Presenting conclusions (72) Non-specific CBA problems

6. ‘I think it is a problem to determine the effects of an infrastructure project on the quality of the biodiversity and nature in the long run’ 3A: Estimating primary physical effects (269) Impacts on non-users ‘I think it is possible to determine the primary effects of an infrastructure project on the quality of the biodiversity and nature in the long run’. However, a change in biodiversity could have different impacts. Some impacts are easier to determine than others. For instance, it is possible to determine the impact on users and not possible to determine the impact on non-users.’ 3B: Deduce the different impacts of primary physical effects (30) Impacts on Users ‘I think it is impossible to transfer primary effects and or impacts into money.’ ‘ I have got problems with the methods that are used to transfer values into money’ 4: Monetizing effects / impacts (88)

7. 0: Rejecting the use of welfare economics as the fundamental theory of an ‘ex ante’ evaluation method (19) 1: Problem analysis (7) 2A: Reference case (28) 2B: Project alternatives (5) Non-specific CBA problems 3A: Estimating primary physical effects (269) 3B: Deduce the different impacts of primary physical effects (30) 7: Problems with uncertainty in CBA (26) 4: Monetizing effects / impacts (88) Specific CBA problems 5: Discounting effects (27) 6B: Presenting distributional effects (14) 6A: Presenting conclusions (72) Non-specific CBA problems

8. 2A: Problems with constructing reference case (27) 2B: Problems with constructing project alternatives (5) 1: Problems with problem analysis (7) Problems with constructing a reference case for a spatial - infrastructure project (15) Problems with constructing a reference case for an infrastructure project (8)

9. 3A: Problems with estimating primary physical effects (256) General problems with estimating primary effects (9) Problems with estimating intangible effects (80) Problems with estimating tangible effects (180) tta tta Synergy-effects (29) External effects (17) Improvement business climate (16) tta Direct effects (106) tta Agglomeration / cluster benefits (17) Indirect effects (28) Problems with other intangible effects (47) Transport effects (95) Estimating costs (11)

10. Intangible effect • Absence of consensus whether or not effect will arise • Difficult to determine causality between project and effect • Difficult to determine whether effect is an additional welfare effect for the country or a distributional effect • In general effect is non-physical

11. 3A: Problems with estimating primary physical effects (256) General problems with estimating primary effects (9) Problems with estimating intangible effects (80) Problems with estimating tangible effects (180) tta tta Synergy-effects (29) External effects (17) Improvement business climate (16) tta Direct effects (106) tta Agglomeration / cluster benefits (17) Indirect effects (28) Problems with other intangible effects (47) Transport effects (95) Estimating costs (11)

12. 4: Problems with monetizing (88) Problems with monetizing value of a human life (5) Decision whether or not to monetize the effect (27) Problems with monetizing effects on air quality (2) Problems with valuation methods (29) n Problems with monetizing of specific effects (32) Problems with Hedonic Pricing (3) Standardized indicators method (15) Problems with monetizing effects on recreation (2) Problems with Stated Preference (7) Problems with Revealed preference (1) Problems with monetizing effects on biodiversity (4) Problems with monetizing effects on reliability (3) Problems with monetizing travel time savings (13) Problems with monetizing effects on landscape / open space (3)

13. 6A: Presenting conclusions (72) Problems with presenting effects and computations (19) Problems with presenting uncertainty (22) Problems with presenting difficult to monetize effects (12) Problems with the absence of sufficient nuance in the CBA report (10) Problems with presenting which aspects of political decision-making the CBA take into account and which aspects the CBA does not take into account. (5) Problem that the report must put effort into convincing its users of the method CBA (4)

14. Chi-Square test: significant relation between group of respondents and whether or not this problems is mentioned

15. Conclusions • Use of CBA in the decision making process is evaluated positive by majority of respondents. • Although CBA is used actively for decade in Dutch practice, respondents experience problems with every aspect of the CBA. • The majority of the problems mentioned, are general problems for all effect studies.

16. 0: Rejecting the use of welfare economics as the fundamental theory of an ‘ex ante’ evaluation method (19) 1: Problem analysis (7) 2A: Reference case (28) 2B: Project alternatives (5) Non-specific CBA problems 3A: Estimating primary physical effects (269) 3B: Deduce the different impacts of primary physical effects (30) 7: Problems with uncertainty in CBA (26) 4: Monetizing effects / impacts (88) Specific CBA problems 5: Discounting effects (27) 6B: Presenting distributional effects (14) 6A: Presenting conclusions (72) Non-specific CBA problems

17. Many thanks for your attention More information / questions: Niek Mouter n.mouter@tudelft.nl 0031623581315

18. Most important advantages CBA • Information is presented in a systematic and objective way (26) • CBA brings rationality in the discussion (29) • As a result of the use of CBA the chances that a project with a negative effect on the welfare of the country is constructed (13) • With the help of a CBA it is possible to improve the project (18) • A CBA is helpful in the process of selecting the best project alternative (16) Most important advantages CBA Most important disadvantages CBA • Use of CBA in ‘ex ante’ evaluation implicates that there is less attention for values and effects that are difficult to monetize in ‘ex ante’ evaluation (21) • Actors that participate in the decision making process, overestimate the importance of the CBA score in the decision making process (19) • The CBA is used in a strategic way by actors that participate in the decision making process (11)

19. Next steps in PhD research • Finishing problem analysis • Repeat the analysis of 72 interviews in order to sharpen the categorization I presented. • Constructs questionnaire with propositions for the 72 respondents in order to test importance of different problems. • Analyze the answers of the respondents. • Aim is to write three papers: • Problem analysis: Do scientists and policy makers experience different substantive problems? • What are different perceptions concerning most important problems + confrontation perceptions with literature. • Agenda for further research regarding substantive problems.

20. Remarks • The problem analysis focuses on the use of CBA by the ‘ex ante’ appraisal of spatial-infrastructural projects. • Problems regarding the use of CBA by the ‘ex ante’ appraisal of land use redevelopment projects are included in the problem analysis.

21. Remarks • During the analysis it turned out that some respondents mention more then 5 perceptions concerning substantive problems • I decided that all perceptions that are mentioned during the interviews are taken into account. • Due to that decisions it is possible that for respondent A 11 perceptions are taken into account, whilst for respondent B 5 perceptions are taken into account

22. Demarcation Problems with the way calculations are presented in CBA report Problems with the way effects are calculated in the CBA Problems with the way CBA is used in decision making process YES YES NO

23. O: Criticism on status quo: welfare economics is used as a fundamental theory in CBA and thus in ‘ex ante’ evaluation (19) tta tta Problems when ‘ex ante’ evaluation tool that is based on welfare economics is used in practice. Fundamental problems with an ‘ex ante’ evaluation tool that is based on the theory welfare economics. CBA does not take into account: -Ethical aspects of the policy -Impact on social (in)equality -Difference in capacity to pay (3) 1 Welfare economics / CBA pretends to take into account all the welfare effects. 2 Respondents claim that other actors use the CBA outcome as the only argument to make a decision ‘Other actors use it as the holy grail’. 3 Respondents claim welfare economics is an imperfect theory and CBA an imperfect tool, reject welfare economics and CBA if used as the holy grail. (6) -Effects that are easy to monetize fit better in theory of welfare economics / CBA than effects that are impossible / difficult to monetize. -As a result; lessattention for effects that are impossibleto monetize in ‘ex ante’ evaluation -less attention for those effects in the decision making process. (10)

24. 3B: Problems with Deducing utility effects out of physical effects (29) Problems deducing utility effects out of physical effects in landscape and open space (5) Problems deducing utility effects out of physical effects on noise pollution (1) Problems deducing utility effects out of physical effects air pollution (2) Problems deducing utility effects out of physical effects in biodiversity (6) Problems deducing utility effects out of physical effects in recreation (1) Problems deducing utility effects out of physical effects in cultural chronicle (3) Problems deducing utility effects out of physical effects in spatial quality (4) Problems deducing utility effects out of physical effects in transport effects (7)

25. 4 Problems with Discounting Problems with effects that arise at a longer term then evaluation period (2) Problems with discount rate and discounting long term effects (16) Problems with risk component discount rate (9)