Realising Benefits from e-government – a fool’s errand? ECEG, 29 th June 2009 Stephen Jenner

1. Realising Benefits from e-government – a fool’s errand? ECEG, 29th June 2009 Stephen Jenner

2. Change programmes - the rationale “The fundamental reason for beginning a programme is torealise benefits through change. The change may be to do things differently, to do different things, or to do things that will influence others to change.” Office of Government Commerce (OGC) “It is only possible to be sure that change has worked if we can measure the delivery of benefits it is supposed to bring.” Cabinet Office

3. But the track record is not good in the UK • “Implementation of IT systems has resulted in delay, confusion and inconvenience to the citizen and, in many cases poor value for money to the taxpayer.” • Public Accounts Committee • “Deficiencies in benefits capture bedevils nearly 50% of government projects” and,“30-40% of systems to support business change deliver no benefits whatsoever.” • Office of Government Commerce

4. Or in Europe… • “After at least a decade of large investments (running into billions of Euro) at digitalizing the public sector, governments in Europe are still mostly unable to objectively quantify and show the benefits and returns of such investments.” • e GEP, 2006 • “The Implementation of eGovernment solutions is expected to bring a number of advantages including efficiency, increased user satisfaction, and reduction of administrative burden. However, this cannot be taken for granted any longer.” • eGovMoNet, 2008

5. Or in Australia… • “Data from Australia on achieved benefit/cost ratios [in] central government indicates that rates of return on ICT investments often are low or negative.” • OECD • “benefits realisation and the measurement of benefits arising from investments in ICT are areas where there is substantial scope for improvement” • Sir Peter Gershon

6. Or indeed in the US… • “The United States federal government will spend over USD 60 billion on ICT technology and systems in 2006. This huge expenditure has led many to ask if ICT investment is providing more than USD 60 billion in value to government agencies and, ultimately, tax payers. The answer is increasingly that nobody knows.” • OECD, 2006 • “the committee has no confidence that the amounts being assessed have any relationship to the benefits anticipated.” • US Senate Appropriations Committee, 2006

7. And empirical research concludes… “There is a demonstrated, systemic tendency for project appraisers to be overly optimistic. This is a worldwide phenomenon that affects both the private and public sectors…appraisers tend to overstate benefits, and underestimate timings and costs.” HM Treasury Forecasts are “highly, systematically and significantly misleading (inflated). The result is large benefit shortfalls”. Flyvbjerg “Delusional optimism: we overemphasise projects’ potential benefits and underestimate likely costs, spinning success scenarios while ignoring the possibility of mistakes.”Lovallo and Kahneman

8. Four explanations… • The technical explanation • The psychological explanation – cognitive biases • 3. The economic explanation • 4. The political explanation – “Strategic misrepresentation”?

9. Delusion or Deception? “the planned, systematic, deliberate misstatement of costs and benefits to get projects approved.” In short, “that is lying”.Flyvbjerg et al “Figures don’t lie, but liars can figure” Sharpe and Keelin 38% of respondents in one survey openly admitted to overstating benefits to get funding. Ward

10. Ensuring benefits claimed are robust and realisable. Planning for all potential forms of value created. Realising benefits and creating value – exploiting the capability created. The 3 Benefit Challenges

11. Capturing all forms of value Moving beyond hurdle rates of return to ask – have all potential forms of value been identified?

12. Managing benefits from a portfolio perspective. Continuous Participative Engagement. Learning from experience. Going beyond forecast to creating value From optimism in planning and pessimism in implementation to realism in planning and enthusiasm in implementation

13. Spend more time doing your homework and be clear about the benefits you are buying. Triangulate and Validate benefit estimates. Appraise ‘Attractiveness’ in the context of ‘Achievability’. Use stage/phase ‘Gates with teeth’. Independent review. Maintaining a ‘Clear line of Sight’. But – the problem starts with the business case so…

14. “It has often been observed when analysing IS/IT project failures that all the ingredients for failure existed on Day One of the project, or even before it started!” Ward and Taylor The Solutions: 1. Understand the benefits you are buying

15. The Solutions: 1. Understand the benefits you are buying • Beware ‘Strategic alignment’ as an investment justification, • “Our CEO defines ‘strategic projects’ • as expensive projects without a • Business case.” • Corporate Executive Board paper

16. Strategic Contribution Analysis

17. Strategic Contribution Analysis

18. Beware staff time savings – they are vouchers The Solutions: Understand the benefits you are buying

19. The Solutions: 2. Triangulate and Validate • Use Reference class forecasting. • Validate the benefits with the recipients – before investment. • Establish effective accountability for benefits realisation & ‘book’ them.

20. The Solutions: 3. Appraise ‘Attractiveness’ in the context of ‘Achievability’

21. The Solutions: 4. ‘Gates with teeth’ • Challenge and Support. • Based on joint planning for success. • Concludes with a formal re-commitment to benefits realisation.

22. The Solutions: 5. Independent review “humans not only are prone to make biased predictions, we’re also damnably overconfident about our predictions and slow to change them in the face of new evidence. In fact, these problems of bias and overconfidence become more severe the more complicated the prediction.” Ayers

23. The Solutions: 5. Independent review • Ayerssuggests an, “‘Advocatus Diaboli’… whose job it is to poke holes in pet projects. These professional “No” men could be an antidote to overconfidence bias.” • Davidson Frame proposes the use of “murder boards” to pull a proposal apart to, “make sure that arguments in support of project ideas do not have built into them the seeds of their own destruction.” • Steve Jenner - I suggest a ‘fool’ to ask the questions others don’t dare to ask and identify those, ‘assumptions that masquerade as facts’.

24. The Solutions: 6. A Clear line of sight

25. And remember… • Spend more time doing your homework - be clear about what benefits you are buying. • Look beyond the hurdle rate of return. • Use stage/phase gates with formal re-commitment to benefits realisation. • Joint accountability - planning for success, with a forward looking perspective. • Track performance and use it to inform investment appraisals. • Use summary documentation – size is the enemy of understanding. • Ensure there is a clear line of sight from strategic intent through to benefits realisation. • Manage benefits from a portfolio perspective. • Shift the focus to - realism in planning and enthusiasm in implementation and….

26. 10. Be ‘wise enough to play the fool’ Available from www.academic-publishing.org