Session 3: Benefits Definition and Measurement

1. eHealth Benefits Management Toolkit Session 3:Benefits Definition and Measurement v4.0 May 2009

2. Benefits measurement categorisation Benefits profile template Benefits Realisation Plan Transition to business as usual Session 3 Outline

3. Session 3 purpose • At the end of the session you will have an understanding of: • what methods can be used to define quantifiable benefits and estimate their value; • how to finalise detailed benefit definitions using SMARTT and ensure the benefits are measurable; • how to pull all the outputs of the previous sessions together to create the Benefits Realisation Plan; and • what further activities need to take place at the end of the project.

4. Benefits measurement categorisation Benefits profile template Benefits Realisation Plan Transition to business as usual Benefits Measurement Categorisation

5. Specific Definitions of Benefits Benefits should be: • clearly and succinctly defined • aligned with the projects objectives and the overall NHS strategy • focused on improving service and patient care, not focused on IT Effective measurement and monitoring will: • indicate the extent to which benefits are being realised; • give early warning of potential problems; • create the opportunity to adapt the benefits or changes to enable the overall objectives; and • ensure that achieved benefits are measured, reported and communicated.

6. Measurable Outcomes • The next step is to: • devise measures that will show when the benefits have been delivered and to what degree; • determine which benefits can be quantified; • determine the financial value of benefits where possible; • investigate additional data capture activities required; and • establish baseline, targets and benchmarking figures.

7. Defining Measures • Use existing measures where possible • Agree all new measures with those concerned • Ask yourself: • Is it achievable? • Is it an appropriate and valid indication that the benefit has been delivered? • Is it directly attributable to a particular change in the project? • Consider the effort and cost of capturing and collating the data • Decide who will be responsible for ensuring that the data is available • Ensure the measures encourage the right type of behaviour • Define when and how often the measurements should be taken

8. Benefits Categorisation(JW)

9. Non Quantifiable Benefits • Observable benefits • lowest category of benefits in term of robust measuring • often the softer / people benefits • can help get buy-in essential to the implementation of changes and the realisation of their resulting benefits • agree realisation criteria and who will say whether they are met • Measurable benefits • performance is easily measured; but • potential improvement won’t be known until after the event • take baseline measurements before any changes are made for comparison

10. Quantifiable Benefits • Quantifiable benefits • aspect of performance easily and consistently measured • improvement can be predicted and targets defined • usually applicable to embedded processes where performance is well documented and predictable • Financial benefits • highest category of benefits in term of robust measuring • focus on benefits that can easily be translated in terms of financial gain or reduced cost: if it is too complicated or not robust don’t assign a monetary value to it • don’t confuse value with savings

11. Progressing Non Quantifiable Measures to Quantifiable Measures • The following methods can be used to get from measurable to quantifiable measures: • Modelling or simulation: extrapolate expected benefits from existing data. It may be necessary to establish trends/times series. • Benchmarking: evaluate changes in relation to “best practices” in the comparable organisations. • References sites: can also be used to establish a comparative base but assess relevance and feasibility in your own organisation. • Pilots: test the new way of working and system on a small scale.

12. Examples for Benefits Template

13. Benefits measurement categorisation Benefits profile template Benefits Realisation Plan Transition to business as usual Benefits Profile Template

14. SMARTT Benefits • Detailed benefit descriptions contain the following categories:

15. Filling in the Benefit Profiles • ACTION: • Expand on the benefits information from the previous workshops in the template for each priority benefit identified: • Utilise measurements that are already being used by the Health Board where possible • Take into account pre-existing government targets of efficiency estimates when setting benchmarks and targets for benefits • Ensure that dependencies are taken into account when setting realisation timelines

16. Preparing to Measure • Are there pre-set requirements for benefits realisation timelines? • Who is going to be collecting the measurements? • How will the measurement be collected? (data sources) • Who will oversee the benefits collection? • Where will the results be reported to? • Who will conduct benefits reviews? • Who will be responsible for communicating about the benefits side of the project?

17. Preparing for Benefits Monitoring • In order that the benefits monitoring is as productive as possible: • define a robust mechanism to monitor progress and ensure that reports are distributed to the appropriate people; • ensure that the Benefits Management templates are up to date; • hold regular benefits reviews to track the progress of benefits realisation against the project and programme objectives; • use benefits reviews as an opportunity to highlight areas where benefits are not being achieved, and identify the barriers to achieving them; • for those benefits that are under-realised, refer to the dependencies shown in the BDN for potential root causes; • assign responsibilities for taking and coordinating corrective actions where benefits are not being met; and • identify and communicate successes too!

18. Benefits measurement categorisation Benefits profile template Benefits Realisation Plan Transition to business as usual Benefits Realisation Plan

19. Formalising it • The Benefit Realisation Plan provides evidence to: • Justify investment decisions • Prioritise competing work in context of finite resources • Establish clear ownership / governance arrangements • Realise benefits • Demonstrate value and effective use of resources • Store the Benefit Realisation Plan with the other key project management work.

20. Benefits Realisation Plan (BRP) • When drafting the Benefit Realisation Plan remember to: • Consider the dependencies identified between benefits, changes and IS&T capabilities; • Identify timeline and milestones that take those dependencies into account; • Work with stakeholders towards minimising the negative impact of identified disbenefits; and • Extend the plan beyond the system implementation date (often viewed as the end of the project) to schedule regular reviews of benefits realised and consider potential further benefits.

21. BRP Content • The Benefit Realisation Plan should include: • the completed Benefits Dependency Network (BDN) • the completed benefit profiles, including dates for capture of baseline figures at an early stage • the completed change templates • the completed stakeholders analysis • Prioritise the benefits so that the most important always has the most focus. This ensures that the project makes the greatest impact. • Identify dates for expected delivery of the benefits.

22. Benefits Strategy • A benefits strategy should also be prepared for the business case, including: • the rationale linking the drivers, objectives and benefits • defined roles and responsibilities • a mechanism for regular measurements and reporting • a mechanism for regular reviews and corrective action • arrangements for the transfer of responsibilities at close of project

23. Linking Business Case & Benefits Management • Providing all this, the business case will demonstrate that: • benefits outweigh disbenefits; • a shared vision is held which is strategically aligned; • the BDN clearly shows how the shared vision will be achieved; • the change required, when considered in relation to the overall schedule of change planned, is not overwhelming for any stakeholders; and, • the benefits realised will be worth the effort required to achieve them. • It will also show that: • there is a clear process for identifying, monitoring and realising the benefits; and • the baseline benefits position has been recorded to enable comparison with projected targets for monitoring the achievement of benefits.

24. Benefits measurement categorisation Benefits profile template Benefits Realisation Plan Transition to business as usual Transition to Business as Usual

25. Close of project activities • Following full implementation of information systems, the achievement of the business case should be reviewed. • The review will produce 3 reports: • formal end-of-project review • lessons learned • potential further benefits

26. End of Project Review • The end of project review will: • assess to what extend the project was successful in delivering the benefits; • decide what further actions are required, by whom and when they should be reviewed; and • detail handover arrangements for any tasks outstanding (including benefits monitoring and reporting). • The Benefits Plan should be updated as relevant to ensure a reference document is available to handover to the “business as usual” team.

27. Lessons Learned • The purpose of the “lessons learned report” is to: • reflect on the lessons learned for future investments; • disseminate useful lessons learned during the project for the benefit of other projects, internal and external; • provide a useful control as part of the functions of an independent quality assurance group; • consider how much effort was required to complete the benefits management documentation and monitor the benefits; and • make recommendations for future enhancement or modification of the benefits management method.

28. Further Benefits • The Review Team should also consider and communicate: • what opportunities exist for the identification and realisation of additional benefits; and • who will take the business case for them forward. • Additional benefits should be defined, planned and implemented following the same methodology as the initial project benefits.

29. Transition to Business As Usual • Leave no loose ends: each outstanding activity needs to be allocated to a named individual • Have end dates for decommissioning of legacy systems to ensure the new IM&T are used – this will ensure new processes are used too and related benefits are delivered • Monitor the implementation of change over a period: • people may revert to the “old ways” once the initial enthusiasm has worn off • people may find work-arounds where the technology or new processes are not working instead of reporting the problems • address teething problems

30. Summary of Benefits Management Lifecycle

31. References and Further Reading • The Clinical Indicators Team has produced a presentation on statistical process control (SPC): • http://www.indicators.scot.nhs.uk/Workshops/SPC.html • HM Treasury’s “Green Book” provides examples of methods used to estimate the financial value of activities or resources: • http://www.hm-treasury.gov.uk/d/green_book_complete.pdf • No Delays Scotland have devised a methodology for measuring benefits: • http://www.nodelaysscotland.scot.nhs.uk/ServiceImprovement/Tools/Pages/IT211_Methodology_for_measuring_benefits.aspx • Statistics for NHS England and Wales for benchmarking on: • http://www.ic.nhs.uk/statistics-and-data-collections (cont’d)

32. References and Further Reading (cont’d) • Benchmark statistics (Scotland) are available from ISD: • http://www.isdscotland.org/isd/782.html • Quarterly and annual figures on patient safety are published by NPSA: • http://www.npsa.nhs.uk/nrls/patient-safety-incident-data/quarterly-data-reports/ • The Department of Health publishes annual and quarterly healthcare performance statistics: • http://www.dh.gov.uk/en/Publicationsandstatistics/Statistics/Performancedataandstatistics/index.htm • Population numbers by NHS Boards – births, deaths, totals, by age, by gender – are available from GRO: • http://www.gro-scotland.gov.uk/statistics/publications-and-data/population-estimates/index.html

33. Congratulations on completing the benefits management programme and on all your efforts to ensure that maximum benefit is gained from eHealth Systems.