NETLIPSE Infrastructure Project Assessment Tool

1. NETLIPSEInfrastructure Project Assessment Tool Professor Hans-Rudolf Schalcher, ETH Zürich Amsterdam, 21 April 2009

2. Motivation (1) The European Union and (new) member states: • address the need for a Trans European Transport Network (TEN-T) • have already invested billions in the construction of several Large Infrastructure Projects (LIPs) to create the TEN-T. But, these organisations: • have limited possibilities for forecasting and monitoring the effectiveness of these projects; • face large delays and cost overruns on the supported projects and experience local opposition; • notice that knowledge exchange between Large Infrastructure Projects (LIPs) is scarce.

3. Motivation (2) These organisations have a need for: • Improvement of the current management and organisation of LIPs. • Insight in ‘the vitality’ of projects on certain moments, e.g. financing (gate review): • to have a reliable insight in risks and opportunities before deciding; and if decision is ‘go’: as a basis to manage risks and opportunities • to allocate budgets to the most vital projects. • Better insight in the progress of LIPs (risks, opportunities). • Benchmark projects. For this NETLIPSE will develop: • Knowledge exchange programme • Infrastructure Project Assessment Tool • Training programmes

4. IPAT:Infrastructure Project Assessment Tool

5. Measurement • The IPAT is an assessment method of ‘the vitality’ of the project for the client/sponsor and project delivery organisation. When can the assessment take place? • On moments of ‘go/no go-decisions’ by sponsors: ex ante ‘gate review’. Gate reviews can indicate weaknesses and strengths in the organisation and management in advance. • During execution: monitoring. The ‘in between’ assessments can indicate the ability of the management to execute the planning and their adaptive abilities towards changes taking place during implementation. • Evaluation afterwards: ex post, in order to expand our knowledge about vital forms of organisation and management approaches of project planning and implementation. Ex post evaluation generates a comparative perspective on weaknesses and strengths on different implementation strategies. • Benchmarks. The assessments generates the ability to benchmark projects in different stages.

6. Improvement • The IPAT can also be used as a method by the organisation itself to improve their management. • The organisation can improve themselves: • As a stimulus from the IPAT assessment, to improve themselves to have a better review next time • To use the IPAT as a basis for improvement of strategy, organisation and processes. • In that way the organisation will facilitate their learning process. • The organisation can use the IPAT as a ‘self assessment’. A self assessment can be an important first step in an external peer review.

7. Assessment process Effective assessments will be based on professional use of the IPAT: • In order to get comparable information, the IPAT will be based on a questionnaire. • Self assessment and facts & figures as a starting point. • Essential is the team of assessors, • with assessors that have a track record: • in implementation to give professional judgement • in project evaluation and analysis to give a scientific judgement. • Assessors will be facilitated by an IPAT-training. The training will focus on the IPAT itself and the use of it.

8. Focus of the IPAT • The external reviewers will not decide on which approach is most appropriate in a certain situation. • Based on past information we know that each new project will ask for unique answers, that suit the objectives and requirements of that specific project and environment. • What we will evaluate is the quality of thoughts and reflections of sponsors/clients and programme managers on how they deal with the four elements of their challenge and they are going to realise their project within scope, budget and time.

9. Control Interaction Control & Interaction • Mature management of LIPs asks for a hybrid approach combining control and interaction.

10. Objectives and Scope Hardware and software Project and Context Risks and Opportunities IPAT Research Framework (1)3 Levels

11. Control Commitment Anticipation Support IPAT Research Framework (2) Hardware of projects: Planning & Control finance, objectives, risk, contracts, technology Software of projects: Shareholder management, Team culture, HRM Hardware of contexts: Legal consents, Policy dynamics, Market dynamics Software of contexts: Stakeholder management, Quality of external relations, Societal relation mgt. (SRM)

12. 11 Themes (draft) T1 Objectives, Purpose and Business Case T2 Functional Specification and Scope T3 Interfaces T4 Stakeholders T5 Finances T6 Legal T7 Technology T8 Knowledge T9 Organisation and Management T10 Contracting (PPP) T11 Risks (Threats and Opportunities)

13. Milestones M 1 Initiation of the project M 2 Funding assembly M 3 Official approval official planning authority M 4 Start of execution M 5 Completion M 6 Start operation M 7 5 years after start of operation

14. Prioritisation: Level 1: Minimal importance Level 2: Little importance Level 3: Medium importance Level 4: Important Level 5: Crucial Levels of Importance

15. Target Groups • European Commission • Member States (Parliaments and Ministries) • Rail, road, waterway and airport authorities • Programme and project organisations • Financial institutes (e.g. EIB) • Insurance companies

16. Criteria IPAT 4 scientific criteria: • Practical Can we get the data? • Reliable Is the measurement consistent? • Validity Do we actually measure what we are supposed to measure? • Utility Is it useful, worth doing? 2 practical criteria: • Understandable: Is the outcome understandable for clients and project managers? • Presentable: Are we able to present the outcome unambiguously in an easy way?

17. Development of the IPAT • The IPAT needs acceptation of a sufficient number of (member) states. • We need to find a balance between effective development of the IPAT and involvement and commitment of (member) states.

18. Development of the IPAT Further steps: • Until September 2009: • Headlines of the IPAT • Concept manual, including questionnaires. • October 2009 – January 2010: • Test the IPAT on 3 to 4 projects. • February – April 2010: • Evaluation and sharpening the IPAT. • Complete and deliver the IPAT Assessors manual. • IPAT-assessors Course.