Chapter 9: Evaluating the Outcomes of Information Systems Plans Managing information technology evaluation – techniques

1. Chapter 9: Evaluating the Outcomes of Information Systems PlansManaging information technology evaluation – techniques and processes* By: Satchanawit Phongkriangkrai * An earlier version of this chapter appeared in the European Management Journal, Vol. 10, No.2. June, pp. 220–229.

2. Overview: Information systems planning

3. Agenda • Evaluation: emerging problems • Strategy and information systems • Evaluating feasibility: findings • Linking strategy and feasibility techniques • CODA: From development to routine operations

4. Introduction • The size and continuing growth in IT investments from the early 1990 served to place IT issue above the ranks in most organization . • Many organizations have encountered IT investment problems, e.g., high risks, and hidden cost processes. • At least 20% of IT investment is wasted and between 30% and 40% of IT projects realize no net benefits. • Major problems occur in how the IT investment is evaluated and controlled.

5. Evaluation: emerging problems • Evaluation brings into question the notion of cost, benefit, risk, value, and process. • There are many problems that impact on IT evaluation: • inappropriate measures • understating human and organizational costs • overstating cost • neglecting intangible benefits, etc. • The fundamental and common failure relates to strategy alignment.

6. Strategy and information systems • Organizational investment climate is affected by: • The financial health and market position of the organization • Industry sector pressures • The organizational business strategy and direction • The management and decision-making culture • Decision-making culture • Conservative • Innovative • Decision-making style • Directive • Consensus-driven

7. Alignment • Alignment creates strategic climate in which IT investment can be related to business/organizational direction. • Lack of alignment is a common problem in public sector information. • When the organization lacks alignment of IT evaluation practice, they tend to become separated from business needs and plans, and from organizational realities. • Another critical alignment is that between: • what is done with IT • how it fits with the information needs of the organization

8. IT Strategic grid • Developed by McFarlan and McKenney in 1983 • Classifies systems where IT investment has been made and where it should be applied. • Demonstrates whether the IT vestments are being made: • into core system, • business growth, • competitiveness. • Identifies four classes of firms.

9. Figure Strategic grid analysis

10. Value chain • Established by Porter and Millar in 1991 • Consists of the primary and support activities. • Primary activities of a typical manufacturing company: • inbound and outbound logistics • operations • marketing and sales • support service • Support activities: • firm infrastructure • human resource management • technology development • procurement

11. IT investment mapping • Developed by Peters in 1993 • The basic dimensions of map were arrived at after reviewing the main investment concerns. • Two dimension over 50 IT projects • investment orientation • Infrastructure –software/hardware environment • Business Process – finance and accounts • Market Influence – increasing repeat sales • and benefit: • Business Expansion • Risk Minimization • Enhance Productivity

12. Figure IT investment mapping

13. Figure Investment map comparing business and IT plans

14. Multiple methodology • Developed by Earl in 1989 • A multiple methodology inquires IT investment more closely with the strategic aims and direction of the organization, and its key needs. • Three strategy formulas are: • A top-down approach – use critical factor success (CFC) to establish objectives; • A bottom-up evaluation – evaluate current systems; • Inside-out innovation – create new strategic options. • The purpose is to identify through internal and external analysis of business needs and opportunities to relate the development of IS applications to business strategy.

15. Evaluating feasibility: findings • Evaluating IT projects at its feasibility stage are based on the right strategic climate. • Feasibility evaluation provides results of interest and to sponsor of research. • Value of IT/IS are justified by understating cost and using notational figure for benefit realization. • Willcocks and Lester looked at 50 organizations from both private and public section of manufacturing and service. • There is little evidence of a concern for assessing risk in any formal manner.

16. Figure IT evaluation: feasibility findings

17. Linking strategy and feasibility techniques • A method uses evaluation techniques to • measure the type of IT project, • develop techniques relating the IT investment to business/organization value. • The method of evaluation needs to be • reliable, • consistent in its measurement over time. • The three evaluation techniques are: • Return on management (ROM), • Matching objectives, projects and techniques, • From cost-benefit to value.

18. Linking strategy and feasibility techniques (Return on management (ROM)) • Developed by Strassman in 1990 • ROM is modern IT investment evaluation. • ROM measures the performance based on the added value – the difference between net revenues and payment external supplier – to an organization provided by management. • However, ROM is not widely used in UK. • The problem is whether it really represents what IT has contributed to business performance.

19. Linking strategy and feasibility techniques (Matching objectives, projects and techniques) • Butler Cox (1990) suggests five main purposes: • surviving and functioning as a business • improving business performance by cost reduction/increasing sales • achieving a competitive leap • enabling the benefits of other IT investments to be realized • being prepared to compete effectively in the future • The matching of IT investment can now be categorized as follows: • Mandatory investments • Investments to improve performance • Competitive edge investments • Infrastructure investments • Research investments

20. Figure Classifying IT investments

21. Figure Matching projects to techniques

22. Linking strategy and feasibility techniques(From cost-benefit to value) • Known as the information economics approach • Developed by Parker et al. 1988 • This technique copes with many problems of IT evaluation, both at the level of methodology and of process. • Information economics looks beyond benefit to value. • Value is the sum of: • Enhanced return on investment, • Business domain assessment, • Technology domain assessment.

23. Figure The information economic approach

24. Linking strategy and feasibility techniques(From cost-benefit to value (Cont’d)) • Enhanced ROI • Value Linking – assesses indirect benefits added to other departments. • Value Acceleration – reduced time-scales for operations. • Value Restructuring – measures the benefits of restructuring a department, or job. • Innovation valuation – considers the value of competitive advantage and calculate risks, pioneer cost, or failing cost. • Business Domain Assessment • Strategic Match: matches the business goals to the proposed project/IT investment. • Competitive Advantage – assesses the degree of an advantage in the market. • Management Information – assesses the contribution toward the management need for information. • Competitive Response – assesses the degree of corporate risk associated with not undertaking the project. • Organization Risk – implements the project in terms of personnel, skills, and experience.

25. Linking strategy and feasibility techniques(From cost-benefit to value (Cont’d)) • Technology Domain Assessment • Strategic IS Architecture – measures the degree to which the project fits into the overall information system plan. • Definitional Uncertainty – assesses the complexity of the area and probability of non-routine change. • IS Infrastructure Risk – evaluates a project’s dependence on new or untried technologies.

26. CODA: From development to routine operations

27. CODA: From development to routine operations (Cont’d) • The evaluation cycle is useful for • controlling a specific project, • building organizational know-how on IT and management. • However, some of limitations in evaluation techniques and processes are: • Weak linkage between evaluations carried out different stages, • Projects shelved, • Stakeholders not included in feasibility stage, • Continual problems from feasibility stage.

28. CODA: From development to routine operations (Cont’d) • Suggestion to improve evaluation: • Linking evaluation across stages and time • Determining stakeholders who are participants in evaluation at all stages • Avoiding the fall-of interest in evaluation at later stage • Providing adequate evaluation techniques for the long term process.

29. Conclusions • The profile of IT evaluation has increased • high expenditure, • disappointed expectations, • underdeveloped and undermanaged area. • Need more effective evaluation practice: • traditional techniques, • tailor modern techniques. • In the past of the evaluation practice, value is price. • The future challenge is to continue the measurement of value and build techniques and processes.

30. Thank youQ&A