CISA REVIEW

1. CISA REVIEW The material provided in this slide show came directly from Certified Information Systems Auditor (CISA) Review Material 2010 by ISACA.

2. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Learning Objectives: • Evaluate the business case for the proposed system development/acquisition to ensure that it meets the organization's business goals. • Evaluate the project management framework and project governance practices to ensure that business objectives are achieved in a cost-effective manner while managing risks to the organization. • Evaluate proposed control mechanisms for systems and/or infrastructure during specification, development/acquisition and testing to ensure that they will provide safeguards and comply with the organization's policies and other requirements. • Evaluate the processes by which systems and/or infrastructure are developed/acquired and tested to ensure that the deliverables meet the organization's objectives. • Evaluate the readiness of the system and/or infrastructure for implementation and migration into production.

3. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Learning Objectives, cont. • Perform post-implementation review of systems and/or infrastructure to ensure that they meet the organization's objectives and are subject to effective internal control. • Perform periodic reviews of systems and/or infrastructure to ensure that they continue to meet the organization's objectives and are subject to effective internal control. • Evaluate the process by which systems and/or infrastructure are maintained to ensure the continued support of the organization's objectives and are subject to effective internal control. • Evaluate the process by which systems and/or infrastructure are disposed to ensure that they comply with the organization's policies and procedures.

4. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Organizations need to develop and change through a systematic process that maximizes benefits. With the growth of information technology, most business benefits are obtained through technology-enabled changes. Every proposed IT system for an organization should have identifiable benefits for both the organization and its customers. Identifying these benefits will require an understanding of the work processes of the organization and its customers.

5. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Organizational benefits of new IT systems may include: • enabling some personnel to perform two different jobs with little or no extra training, • allowing organizational changes that reduce the number of managers, • permitting some jobs to be eliminated entirely. • These benefits are usually measured in terms of productivity gains, staffing reductions and improved organizational effectiveness.

6. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • IT governance – as part of enterprise governance – should be driven by business goals and objectives. • The IS auditor should evaluate whether there is a business strategic planning process in place by considering questions such as: • Is there a clear definition of business vision and mission? • Is there a business strategic planning methodology used? • Is the level of the individuals involved in this process appropriate? • Is this planning periodically updated?

7. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • The benefits realization process involves: • Assigning a measure and target, • Establishing a tracking/measuring system, • Documenting any assumptions, • Establishing key roles and responsibilities, • Planning for the benefit to be realized, • Validating the benefits predicted in the business. • This is a continuous process, and enterprise wide benefits realization studies should be collected and synthesized to fine-tune the benefit realization process.

8. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Examples of benefits for IT systems include: Accuracy – Will the proposed system provide better accuracy by reducing the number of data entry errors? Availability – How long will it take to develop and implement the system? Will one alternative be available sooner than another? Compatibility – How compatible is the proposed alternative with existing facilities and procedures? Will one alternative require less training of personnel or less new equipment or software? Efficiency – Will one alternative provide faster or more accurate processing of inputs? Will one alternative require fewer resources for the processing? Maintainability – Will the maintenance costs for one alternative be lower than the costs for the others? Are the maintenance resources easier to acquire for one alternative? An example of this would be availability and cost of programmers to maintain the software.

9. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Examples of benefits for IT systems include: Reliability – Does one alternative provide greater hardware or software reliability? Greater reliability translates to higher productivity in using and/or operating the system and less time for operations and user support. Security – Does one alternative provide better security to prevent fraud, waste or abuse? Are privacy, confidentiality and data integrity enhanced?

10. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • The business case needs to provide enough detail to justify the decision to develop and continue a project. • The business case for a new system should, for example: • Determine the strategic benefits of implementing the system, either in productivity gains or in future cost avoidance, • Identify and quantify the cost savings of the new system, • Estimate a payback schedule for the cost incurred in implementing the system or show the projected return on investment (ROI), • Identify intangible benefits, such as improved morale, and quantify these wherever possible.

11. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • The objective of the feasibility study is to give an assessment of the business requirement or opportunity and determine whether there are feasible solutions before the commitment of full life cycle resources. • Several key questions need to be addressed in advance of conducting the study. • What is the specific requirement or opportunity and what group within the organization is responsible? • What are the new information needs that are associated with this requirement or opportunity? • What should be the scope of the solution?

12. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management An impact assessment looks at the potential effects of a proposed development project on current projects and resources. The report of the impact assessment should outline the benefits and drawbacks of pursuing a specific course of action. Value analysis uses a team approach to analyze the functions of facilities, processes, systems, equipment, services and supplies. The objective is to achieve the essential functions at the lowest life cycle cost (LCC) consistent with required performance, reliability, quality, safety and achievement of the organization's stated mission priorities.

13. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Exercise: Complete each statement by choosing one of the four terms below. A business caseA feasibility studyAn impact assessmentA value analysis [Blank] looks at how to achieve the organization's essential functions at the lowest life cycle cost consistent with requirements. [Blank] includes decision criteria, comparisons of potential solutions and a proposed solution. [Blank] looks at the potential effects of a proposed development project on current projects and resources. [Blank] for a new system should identify both intangible benefits and return on investment.

14. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Answer: A value analysis looks at how to achieve the organization's essential functions at the lowest life cycle cost consistent with requirements. A feasibility study includes decision criteria, comparisons of potential solutions and a proposed solution. An impact assessment looks at the potential effects of a proposed development project on current projects and resources. A business case for a new system should identify both intangible benefits and return on investment.

15. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Systems development life cycle (SDLC) • The phases deployed in the development or acquisition of a software system. Typical phases of SDLC include: • feasibility study, • requirements study, • requirements definition, • design, • development, • testing, • installation and post-implementation review.

16. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • The requirements definition identifies and specifies the business requirements of the system chosen for development during the feasibility study. • Requirements include descriptions of: • What the system should do, • How users will interact with the system, • Conditions under which the system will operate, • Information criteria the system should meet.

17. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management The requirements definition includes these tasks: Identify stakeholders – Consult with them to determine their expectations. Analyze requirements – Determine priorities. Look for conflicts and correct them. Identify system boundaries – Define what is part of the system and what the system will be interacting with. Convert user requirements into system requirements – For example, create a prototype user interface that demonstrates the screen look and feel. Record requirements – Requirements decisions need to be presented and retained in a structured format. Verify requirements – Ensure they are complete, consistent, unambiguous, verifiable, modifiable, testable and traceable. Resolve conflicts – Identify where the requirements do not match the available resources and where the requirements of various stakeholders differ and determine a course of action.

18. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management There is a large payoff to an effective review of requirements because of the high cost of rectifying requirements problems in the downstream phases of development.

19. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • To ensure control over the IT process of identifying automated solutions to satisfy the business requirements, the IS auditor needs to verify that technically feasible and cost-effective solutions are achieved. The IS auditor must ensure: • Business and technical requirements have been defined, • Feasibility studies have been completed, • Approval (or rejection) of the requirements and the feasibility study results is measured by: • Number of projects where stated beliefs were not achieved due to incorrect feasibility assumptions, • Percent of feasibility studies signed off by the business process owner, • Percent of users satisfied with functionality delivered. • Alternate solutions satisfying the business requirements should be identified to help ensure the optimal solution is selected.

20. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Key Personnel in the Systems Development Process • The project manager is appointed by the IS Steering committee and is responsible for: • Providing leadership and project management, • Ensuring the project follows the overall direction outlined in its charter, • Involving the affected departments, • Complying with local standards, • Ensuring deliverables meet the quality expectations of key stakeholders, • Resolving interdepartmental conflicts, • Monitoring and controlling costs and schedules.

21. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Key Personnel in the Systems Development Process • Senior Management is responsible for: • Approving the resources to undertake and complete the project, • Ensuring, through its commitment, the involvement of others.

22. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Key Personnel in the Systems Development Process • User Management is responsible for: • Assuming ownership of the project and resulting system, • Providing qualified representatives to the team, • Actively participating in business process redesign, system requirements definition, test case development, acceptance testing and user training, • Reviewing and approving system deliverables as they are defined and implemented.

23. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Key Personnel in the Systems Development Process • Project Steering Committee is responsible for: • Providing overall project direction, • Ensuring appropriate representation of major stakeholders, • Retains ultimate responsibility for all deliverables, project costs and schedules.

24. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Key Personnel in the Systems Development Process • Project Sponsor is responsible for: • Providing funding for the project, • Working closely with the project manager to define critical success factors (CSFs) and metrics for the project.

25. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Key Personnel in the Systems Development Process • Quality Assurance is responsible for: • Reviewing results and deliverables within each phase and at the end of each phase, • Confirming compliance with requirements.

26. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Exercise: Match each role to its corresponding responsibility. Key Roles Project sponsorProject steering committeeQuality assurance Senior managementUser management Responsibilities Works with project manager to define CSFs,Retains ultimate responsibility for all deliverables, project costs and schedules,Confirms compliance with requirements,Approves the resources to undertake and complete the project, Assumes ownership of the project and resulting system.

27. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Answer: Project sponsorWorks with project manager to define CSFs Project steering committeeRetains ultimate responsibility for all deliverables, project costs and schedules Quality assuranceConfirms compliance with requirements Senior managementApproves the resources to undertake and complete the project User managementAssumes ownership of the project and resulting system

28. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Exercise: Think About It At the minimum, what are the basic elements necessary for ensuring the success of a life cycle project?

29. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Answer: Elements critical to ensuring the overall success of a project include but are not limited to: • Ensuring the project has strong executive sponsorship, • Developing project management skills and awareness at staff and executive levels, • Staying true to the strategy and vision, • Maintaining the enterprise perspective, • Delivering on promises, • Ensuring trading partner agreements are clear and concise, • Providing representation on the steering committee for all major stakeholders.

30. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management System Design: The project requirements definition provides a general preliminary design and user requirements. Based on this, the detailed design is to be developed. Typically this is done by a team of programmers and business and system analysts who define the software architecture, depicting a general blueprint of the system and then detailing or decomposing the system into its constituent parts such as modules and components. This permits resources to be allocated to design and to defining how the system will address all its information requirements.

31. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • The design phase of the project includes these activities: • Developing system flowcharts and entity relationship models to illustrate, how information will flow through the system, • Describing inputs and outputs, such as screen designs and reports, • Determining processing steps and computation rules when addressing functional requirement needs, • Determining data file or database system file design, • Preparing program specifications for the types of requirements or information criteria defined, • Developing test plans for various levels of testing, • Developing data conversion plans to convert data and manual procedures from the old system into the new system.

32. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • The focus for the IS auditor in the design phase of a project is: • Determining if an adequate system of controls is incorporated into system specifications and test plans • Determining if continuous online auditing functions are built into the system (particularly for e-commerce applications and other types of paperless environments) • Evaluating the effectiveness of the design process including the use of: Structured design techniques • Prototyping and test plans • Software baselining • The IS auditor should verify the implementation of a formal software change process that effectively freezes the inclusion of any changes to system requirements without a formal review and approval process.

33. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Continuous online auditing allows IS auditors to monitor the operation of a system and gather selective audit evidence while normal processing takes place. The audit evidence is stored in a separate audit file for later verification by the IS auditor.

34. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • System Development: The detailed design is used in the development phase to start coding. This phase is largely the responsibility of the programmers and systems analysts. The principal activities performed during the development and testing phase include: • Coding and developing the program and system-level documents, • Debugging and testing the programs developed, • Developing programs to convert data from the old system for use on the new system, • Creating procedures to enable users to transition to the new system, • Providing training on the new system for selected users, • Ensuring modifications are documented and applied accurately and completely to vendor-acquired software to ensure future updated versions of the vendor's code can be applied.

35. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • In auditing systems development, acquisition and maintenance, IS auditors should meet with key members of the systems development and user project team to: • Determine the system's principal components, objectives and user requirements in order to identify the areas that require controls. • Determine major risks to and exposures of the system and rank these items. • Identify controls to mitigate the risks to and exposures of the system using references to authoritative sources and meetings with the project team. • Advise the project team regarding the design of the system and implementation of controls through evaluation of available controls and discussions with the team. • Monitor the systems development process to ensure controls are implemented, user and business requirements are met, and the systems development/acquisition methodology is being followed through review of the documentation and deliverables as well as in team meetings.

36. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • IS auditors should also ensure documented controls are in place to address all security, edit and processing issues by reviewing and evaluating application system audit trails. Any program changes should be handled by a change management system that includes: • History of all work order activity (work order date, programmer assignment, changes made and date closed), • History of logons and logoffs by programmers, • History of program deletions, • Evaluation of system maintenance standards and procedures, • Testing of system maintenance procedure to ensure procedures are being applied as described in the standards, • Evaluation of the system maintenance process to determine whether control objectives were achieved by analyzing test results and other audit evidence • Identification and testing of existing controls to assess the adequacy of production library security in order to ensure the integrity of the production resources

37. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • The activities the IS auditor should perform in reviewing the detailed design and development of a system include: • Reviewing system flowcharts to see that they comply with the general design, • Verification that all changes were discussed and approved by appropriate user management, and that all approvals are recorded, • Reviewing the appropriateness of the input, processing and output controls that are designed into the system, • Interviewing system users to assess their understanding of the system and their degree of input into screen format and output report design, • Determining if the audit trails can adequately provide traceability and accountability of system transactions, • Verifying the integrity of key calculations and processes, • Verifying that the system can correctly identify and process erroneous data, • Verifying all corrections to programming errors were made and the audit trails or embedded audit modules were coded as recommended into the appropriate programs.

38. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management In general, systems are easier to maintain and enhance to the degree that individual software items perform a single, dedicated function (cohesion) and retain independence from other comparable items (coupling), because this facilitates determining where and how to apply a change and reduces the chance of unintended consequences.

39. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Exercise: Determine if each statement pertains to the Requirements, Design, or Development stage of the project.

40. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Answer:

41. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Following are examples of possible sources of risk for a system development project: • System requirements factors: complexity, difficulty, feasibility, novelty, verifiability and volatility • SDLC deliverables: correctness, integrity, maintainability, performance, reliability, security, testability and usability • Developmental model: manageability, measurability, quality and traceability of the processes used to satisfy customer requirements • Communication, cooperation, domain knowledge, experience, technical knowledge and training of the personnel associated with technical and support work on the project • Budget, external constraints, politics, resources and schedule of the external system environment • Methods, tools and supporting equipment for system development: capacity, documentation, familiarity, tool support, and usability

42. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Question: Think About It What factors and/or elements should you as an IS auditor consider when assessing systems development and related risks?

43. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Answer: You should determine whether audit procedures for systems development and acquisition and related risk management adequately consider: • Level of support for systems development by senior management and the board of directors • Adequacy of the management structures to establish accountability and responsibility for IT systems and technology initiatives • Development of IT solutions that meet the needs of end users • Extent of end-user involvement in the systems development process • Adequacy of the institution's systems development methodology and programming standards • Quality of practices followed by developers, operators, executive management, vendors and end users • The independence of the QA function and the adequacy of controls over program changes • Quality and thoroughness of system documentation

44. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management System Maintenance Once a system has been developed and moved into production, it will be subject to changes in response to alterations in the business, the IT structure, adverse incidents such as intrusions and viruses, changes in the classification of an item, and audits. System maintenance practices must be used to manage changes to system processes and configurations and to ensure the continued integrity of both the production source and executable code.

45. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • System Maintenance: A standard process for performing and recording changes is needed to control the ongoing maintenance of the system. This process should mirror the organization's SDLC process to make sure that all system changes are: • Appropriate to the needs of the organization, • Authorized, • Documented, • Tested thoroughly, • Approved by management. • This process needs to be applied not only to software changes but to the overall IT and business structure.

46. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Change requests may be initiated by end users, operational staff or system development/maintenance staff. The request should be submitted in a formal document format such as a change request form, a memo or e-mail. • The request should include: • Requestor's name, • Date of the request, • Date change is needed, • Priority of the request, • Description of the change request, • Description of any anticipated effects on other systems or programs, • Reason for change, such as a cost-justification analysis, and expected benefits of the change, • Changes submitted by end users should also include evidence that the change request has been reviewed and approved by user management.

47. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management • Testing Changes: To ensure the changes perform the functions intended, changed programs should be tested and certified with the same rigor as is applied to newly developed systems. • Depending on the outcome of a risk analysis of the change, additional testing may be required to ensure: • Existing functionality is not damaged by the change. • System performance is not degraded because of the change. • No security exposures have been created because of the change.

48. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Deploying Changes Back Into Production After a change has been approved by user management, the modified programs can be moved into the production environment. A group that is independent of computer programming – such as computer operations, QA or a designated change control group – should perform the migration of programs from test to production. Appropriate access restrictions must be in place to ensure only authorized individuals can migrate programs into production. This may be accomplished through operating system security or an external security package. Migrating changes into an application in production requires a detailed plan that includes checkpoints at which "go" or "no-go" decisions are to be made. This allows the change to be rolled back if problems are encountered.

49. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Question: Think About It What is process change management and what is it designed to accomplish?

50. CISA REVIEWChapter 3 – Systems Infrastructure and Lifecycle Management Answer: Think About It What is process change management and what is it designed to accomplish? Process change management involves defining process improvement goals and, with senior management sponsorship, proactively and systematically identifying, evaluating and implementing continuous improvements to the standard software process of the organization and defined software processes of the project. The purpose of process change management is to continually improve the software processes used in the organization. The objective is to improve software quality, increase productivity and decrease the cycle time for product development.