Presenter: Brian E. Kruk, CIA, CCSA, CGAP, CCA, CISA Director Contract and Construction Audit

1. Quality Assurance & Improvement Program: Audit Process Versus Program: The Difference…and Why It Matters Presenter: Brian E. Kruk, CIA, CCSA, CGAP, CCA, CISA Director Contract and Construction Audit Union Pacific Railroad Topeka Chapter April 5, 2016

2. Today’s Agenda • A brief history of QA • Discuss the available QA&IP guidance • Examine common misconceptions in QA&IP development • Explore the differences between basic internal audit processes and effective components of a QA&IP • Utilization of the Old IIA PA 1311-2 to create an appropriate, right-sized QA&IP • Understand how a CMM can be used to facilitate the path to quality

3. Today’s Focus • Has anyone recently completed a QA? • Has anyone performed as a Validator? • Is anyone working on their Internal Assessment or Self Assessment? • What do you want out of today’s session? • Are there any questions before we begin?

4. “Quality is not an act – it is a habit.” ~Aristotle “Quality means doing it right when no one is looking.” ~ Henry Ford

5. Quality Assessment Defined The process of evaluating the efficiency and effectiveness of an internal auditing organization through a comprehensive, qualitative review of audit procedures, leading to recommendations for improving controls, reducing risk and the introductions of successful innovative best practices. It should also provide assurance conformity with the International Standards for the Professional Practice of Internal Auditing and other relevant organizational and departmental policies and procedures.

6. QAR 1984

7. Synopsis of QA History - Other professions have required peer reviews • IIA first publication on QA in 1984 • IIA recommended peer reviews in previous Standards • IIA began conducting QAs in 1986 • Some QAs also conducted by other providers • GTF Brings Focus to Quality Initiative • QA Manual, 4th Edition, released in 2002 • QA Manual, 5th Edition, released in 2006 • QA Manual, 6th Edition, released in 2009 • QA Manual, 7th Edition, released in 2013

8. A Vision for the Future: Professional Practices for Internal Auditing Report of GTF to IIA Board of Directors • Adopt New Framework • Revise Definition of IA • Update Code of Ethics and Standards • Establish Oversight Committee • Develop Guidance to Support the Standards

9. Professional Practices Framework - 2002 The “Path to Quality” gets its formal start with the creation of: 7 New Quality Standards & 5 Practice Advisories OH 2-3

10. Continuous Improvement Highlights Onward and Upward

11. Continuous Improvement HighlightsExamples of Shortfalls • Addressing the applicability of the Standards for specialty groups • Further clarification of Assurance & Consulting services • Need for some level of basic fraud (Red Flags) • Knowledge of key IT risk, controls and technology-based audit techniques • Periodic Internal and External QA and ongoing monitoring as part of QA&IP • Inclusion of overall opinion and/or conclusion where appropriate, in final communications

12. Continuous Improvement Highlights • By January 2004 -24 changes to the PPF • 11 New Standards • 13 Additions to Glossary • 11 New Practice Advisories • 5 Revisions to PA’s

13. Continuous Improvement Highlights July 2007 - Arrival of the New International Professional Practice Framework

14. Continuous Improvement Highlights • By the end of 2009 - changes to the IPPF • 6 New Standards • 19 New Interpretations • 13 Additions to Glossary • Practice Advisories reduction to 58 • 3 New Practice Guides, • New 13 GTAG’s • New 3 GAIT’s

15. Continuous Improvement Highlights • 2010 to 2011 - changes to the IPPF • 3 New 1 Deleted • 15 Revised Standards • 9 New and Revised Interpretations • 5 Revisions to Glossary • 13 New Practice Advisories • 8 New Practice Guides, • 3 New GTAG’s

16. Continuous Improvement Highlights

17. Continuous Improvement Highlights The New IPPF • Mandatory Guidance • Core Principles • Standards • DIA • COE • Recommended Guidance • Implementation Guidance • Supplemental Guidance (PGs, GTAGs, & GAITs)

18. IIA - Core Principles • Demonstrates integrity. • Demonstrates competence and due professional care. • Is objective and free from undue influence (independent). • Aligns with the strategies, objectives, and risks of the organization. • Is appropriately positioned and adequately resourced. • Demonstrates quality and continuous improvement. • Communicates effectively. • Provides risk-based assurance. • Is insightful, proactive, and future-focused. • Promotes organizational improvement.

19. Attribute Standards Attribute Standards address the attributes of organizations and individuals performing internal auditing. • 1000: Purpose, Authority and Responsibility • 1100: Independence and Objectivity • 1200: Proficiency and Due Professional Care • 1300: Quality Assurance and Improvement Program

20. Performance Standards Performance Standards describe the nature of internal auditing and provide quality criteria against which the performance of these services can be measured. • 2000: Managing the Internal Audit Activity • 2100: Nature of Work • 2200: Engagement Planning • 2300: Performing the Engagement • 2400: Communicating Results • 2500: Monitoring Progress • 2600: Management’s Acceptance of Risks

21. QA Related Standards 1300: Quality Assurance and Improvement Programs The chief audit executive should develop and maintain a quality assurance and improvement program that covers all aspects of the IAA and continuously monitors its effectiveness. The program should be designed to help the internal auditing activity add value and improve the organization’s operations and to provide assurance that the internal audit activity is in conformity with the Standards and the Code of Ethics. Note: 2nd half drop in new Standard: See Interpretation next slide

22. QA Related Standards Standard 1300 – Interpretation A quality assurance and improvement program is designed to enable an evaluation of the IAA’s conformance with the Standards and an evaluation of whether internal auditors apply the COE. The program also assesses the efficiency and effectiveness of the IAA and identifies opportunities of improvement.

23. QA Related Standards Original 1310: Quality Program Assessments The internal audit activity should adopt a process to monitor and assess the overall effectiveness of the quality program. The process should include both internal and external assessments. Revised 1310 – Requirements of the QA&IP The QA&IP must include both internal and external assessments.

24. QA Related Standards Original 1311 - Internal Assessments Should include: • Ongoing reviews of the performance of the IAA; and • Periodic reviews performed through self-assessment or by other persons within the organization, with knowledge of internal auditing practices and the Standards. Revised1311 - Internal Assessments Internal Assessments mustinclude: • Ongoing monitoring of the performance of the IAA; and • Periodic self-assessment or assessments by other persons within the organization with sufficientknowledge of internal audit practices.

25. QA Related Standards 1311- Internal Assessments Interpretation: Ongoing monitoring is an integral part of the day-to-day supervision, review and measurement of the IAA. Ongoing monitoring incorporated intothe routine policies and practices used to manage the IAA and uses processes, tools and information considered necessary to evaluateconformance with the DIA, COE and Standards. Periodic reviewsare assessments conducted to evaluate conformancewith the DIA, COE and Standards. Sufficient knowledge of IA practices requires at leastan understanding of all elements of the IPPF.

26. QA Related Standards Original 1312: - External Assessments External assessments such as quality assurance reviews, should be conducted at least once every five yearsby a qualified, independent reviewer or review team from outsidethe organization.

27. QA Related Standards Revised 1312: External Assessments External assessments should be conducted at least once every five years by a qualified, independent reviewer or review team from outside the organization. The potential need for more frequent external assessments as well as the qualifications and independence of the external reviewer or review team, including any potential conflict of interest, should be discussed by the CAE with the Board. Such discussions should also consider the size, complexity and industry of the organization in relation to the experience of the reviewer or review team.

28. QA Related Standards Current 1312 : External Assessments External assessments must be conducted at least once every five years by a qualified independent assessor or assessment team from outside the organization. The CAE must discuss with the board: • The form and frequency of external assessment; and • The qualifications and independence of the external assessor or assessment team, including any potential conflict of interest.

29. QA Related Standards 1312 - External Assessments Original Interpretation: A qualified reviewer or review team consists of individuals who are competentin the professional practice of internal auditing and the external assessment process. The evaluation of the competency of the reviewer and review team is a judgment that considers the professional internal audit experience and professional credentials of the individuals selected to perform the review. The evaluation of qualifications also considers the size and complexity of the organizations that the reviewers have been associated with in relation to the organization for which the IAA is being assessed, as well as the need for particular sector, industry, or technical knowledge. An independent reviewer or review team means not having either a real or an apparent conflict of interest and not being a part of, or under the control of, the organization to which the IAA belongs.

30. QA Related Standards 1312 - External Assessments Revised Interpretation: A qualified reviewer or review team demonstrates competence in two areas: the professional practice of internal auditing and the external assessment process.Competencecan be demonstrated through a mixture of experience and theoretical learning.Experiencegained in organizations of similar size, complexity, sector or industry, and technical issues is more valuable than less relevant experience. In the case of a review team, not all members of the team need to have all the competencies; it is the team as a whole that is qualified. The CAE uses professional judgment when assessingwhether a reviewer or review team demonstrates sufficient competence to be qualified. An independent reviewer or review team means not having either a real or an apparent conflict of interest and not being a part of, or under the control of, the organization to which the IAA belongs.

31. QA Related Standards 1312 - External Assessments Proposed Interpretation: External assessments enhance a complete QA&IP and may be accomplished through a full external assessment, or a self-assessment with independent validation. The external assessor must conclude as to conformance with the Standards; the external assessment may also include operational and strategic comments. 3rd paragraph adjustments “ real or an apparent ” changed to read “ actual or a perceived” conflict of interest ____* Added 3rd sentence: The CAE should encourage board participation in the QA&IP to reduce perceived or potential conflicts of interest.

32. QA Related Standards Original 1320 – Reporting on Quality Program The chief audit executive should communicate the results of external assessments to the board. Revised 1320 – Reporting on Quality Program The CAE must communicate the results of the QA&IP to senior management and the board. Review interpretation narrative

33. QA Related Standards 1320 - Reporting on the QA&IP Interpretation: The form, content and frequency of communicating the results of the QA&IP is established through discussions with the senior management and the board and considers the responsibilities of the IAA and CAE as contained in the IA Charter. To demonstrate conformance with the DIA, the COE, and the Standards, the results of external and periodic internal assessments are communicated upon completion of such assessments and the results of ongoing monitoring are communicated at least annually.The results include the assessor’s or assessment team’s evaluation with respect to the degree of conformance.

34. QA Related Standards Original -1330: Use of “Conducted in Accordance with the Standards” Internal auditors are encouraged to reportthat their activities are “conducted in accordance with the International Standards for the Professional Practice of Internal Auditing.” However, internal auditors may use the statement only if assessmentsof the quality improvement program demonstrate that the internal audit activity is in compliance with the Standards. Current -1321: Use of “Conforms with the International Standards for the Professional Practice of Internal Auditing” Indicating that the IAA conforms with the ISPPIA is appropriate only if the results of the QA&IP supports such a statement.

35. QA Related Standards Original 1340: Disclosure of Noncompliance Although the IAA should achieve full compliance with the Standards and internal auditors with the Code of Ethics, there may be instances in which full compliance is not achieved. When noncompliance impacts the overall scope or operation of the IAA, disclosure should be madeto senior management and the board. Current –1322: Disclosure of Nonconformance When nonconformance with DIA, the COE, or the Standards impacts the overall scope or operation of the IAA, the CAE must disclose the nonconformance and the impact to senior management and the board.

36. QA Related Practice Advisories • 1300 - 1 Quality Assurance & Improvement Program • 1310 – 1 Requirement of the QA&IP (Deleted from IPPF) • 1311 - 1 Internal Assessments • 1311 - 2 Internal Assessment: Establishing Measures to Support Reviews of IAA (Deleted from IPPF) • 1312 - 1 External Assessments • 1312 - 2 External Assessment- SAWIV • 1312 - 3 Independence of External Assessment Team – Private • 1312 - 4 Independence of External Assessment Team – Public • 1320 -1 Reporting Results of QA&IP • 1321 - 1 Use of “Conforms with the International Standards for the Professional Practice of Internal Auditing” • 1322 - 1 Disclosure of Nonconformance w/ the ISPPIA • 2120- 2 Managing the Risk of the IAA

37. QA Related Practice Advisories PA 2120- 2 Managing the Risk of the Internal Audit Activity • Managing the risk of not achieving IA Objectives • IAA must manage its own risk • 3 categories: audit failure, false assurance, and reputation risks • Where were the internal auditors? • IAA can implement the practices to mitigate its risk: • QA&IP • Periodic reviews of audit plan • Effective planning • Effective audit design • Effective management review and escalation • Proper Resource Allocation • 6 through 14 - additional topics of further guidance

38. External Assessments Areas of focus: • Review IA Activity’s charter, audit plans, policies and procedures • Review a sample of audit reports, special projects and supporting work papers • Review staff composition, supervision, professional development and response to client needs

39. External Assessments Areas of focus: • Assess staff and client satisfaction through interviews and surveys • Specifically interview audit committee chairperson, a representative sample of officers, senior executives and management clients and the external auditing partner • Risk assessment methodology • Approach and adequacy of IT audit coverage

40. External Assessment Activities Tools Review • Self Study/Benchmarking • Customer/Staff Survey • On-site Activities • Interviews (Board, Management, External Auditor, Staff) • QA Program • Work Paper Reviews • Issue Report

41. QA - Assessment Objectives • Assess the efficiency and effectiveness of the internal audit activity in light of: - Its charter and mission - Expectations of the board, senior management, audit clients, and the CAE - Identify opportunities and offer ideas and counsel to the CAE and staff for: - Improving their performance - Increasing the value they add to the enterprise - Provide an opinion on the internal audit activity’s conformance to the spirit and intent of the Standards

42. QA - Assessment Approach • - Self Study& Audit Management Questionnaire • - Survey of Clients and Staff • - Interviews with Senior Managers & Staff • - Review Tools (Programs) • Organization of the Internal Audit Activity • Risk Assessment and Engagement Planning • Staff Professional Proficiency • Information Technology • Production and Value Added • Sample of Workpapers and Reports • - Rating of Conformity with IIA Standards

43. QA – Conforming Evaluation Definitions • GC – “Generally Conforms” means the assessor has concluded that the Activity’s charter, structure, policies, and procedures, as well as the processes by which they are applied, are judged to be in conformity with a majority of the Standardswith some opportunities for improvement being possible. • PC – “Partially Conforms” means the assessor has concluded that a good faith effort exist but deviations from conformity for a majority of the Standards exists and corrective action is needed. These deviations are not, however, significant enough to preclude the Activity from carrying out its responsibilities in an acceptable manner. • DNC – “Does Not Conform” means the evaluator has concluded that the Activity is not aware of, is not making good-faith efforts to comply with, or is failing to achieve conformity with the majority of the Standards, thus impacting its ability to carry out its mission.

44. QA Overall Evaluation • OVERALL EVALUATION- Generally Conforms (GC) • Attribute StandardsGC • 1000 Purpose, Authority & Responsibility GC • 1100 Independence & Objectivity GC • 1200 Proficiency and Due Professional Care GC 1300 Quality Assurance and Improvement PC • Performance Standards GC2000 Managing the IA Activity GC • 2100 Nature of Work GC • 2200 Engagement Planning GC • 2300 Performing the Engagement GC • 2400 Communicating Results GC • 2500 Monitoring Progress GC • 2600 Communicating the Acceptance of Risk GC • IIA Code of Ethics GC

45. QA - Potential Issues Reporting Categories • Opportunities to Improve Conformity with Standards • Opportunities for IA Consideration • Suggestions for Senior Management • Verbal Comments

46. QA – Validation Reporting Process Two Options: • Validator signs internally prepared report • Validator prepares separate report referencing internally prepared report

47. Quality Assessment Process Map (IIA Manual 7th Edition)

48. Quality Assessment Process Map (IIA Manual 7th Edition) • IA Governance (1000,1100,1300, COE, & DIA) • IA Staff (1200) • IA Management (2000, 2100, & 2600) • IA Process (2200, 2300, 2400, & 2500)

49. 5 Minute Break

50. QA Related Standards - Revisit Original 1311 - Internal Assessments Should include: • Ongoing reviews of the performance of the IAA; and • Periodic reviews performed through self-assessment or by other persons within the organization, with knowledge of internal auditing practices and the Standards. Revised1311 - Internal Assessments Internal Assessments mustinclude: • Ongoing monitoring of the performance of the IAA; and • Periodic self-assessment or assessments by other persons within the organization with sufficientknowledge of internal audit practices.