Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation

19th World Continuous Auditing and Reporting Symposium Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Philip ElsasComputationalAuditing.com Newark, New Jersey November 6-7, 2009

1 Introduction Offering software and consultancy services to innovateaudit practices and audit software firms • Since 2003: Company - Canada, Netherlands • 1988-2003: Deloitte. with intermezzo at Bakkenist Management Consultants, sold to Deloitte. • 1990-1996: PhD Computational Auditing - Principal, chief architect & inventor of Smart Audit Support - Smart Audit Support: since 1994 key in Deloitte’s worldwideaudit practice. Currently integrated in ‘The Deloitte Audit’- System blueprint in chapter 5 of … - PhD in Mathematics & Computing Science, on Financial Auditing - In parallel to Smart Audit project, 30% part-time, Vrije Universiteit- Directly after appearance awarded with the biennial Alfred Coini Prize for the best publication in Auditing The Dutch Tax Office used Computational Auditing in 2001-2003 as frame of reference to compare Big 4 planning and decision-support models & systems to investigate how to improve audit productivity (57 page report); considers Smart Audit Support ‘leader of the pack’ ComputationalAuditing.com

2 AgendaAutomatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation • Web platform for audit support: “What is the content?” • Web platform for audit support: “How to use that content?” • Managing the use of aggregation & classification • Aggregation mechanisms: quantitative, qualitative & confidence • Royal NIVRA: ‘Golden opportunity for the audit profession’, Identify a way to contribute to systemic risk anticipation ComputationalAuditing.com

3 Web platform for audit support:What’s the content?by auditors, for auditors Audit repository: data, scripts for analytics (CM), findings • CaseWare Open Engagement & CaseWare IDEA • ACL AuditExchange (AX 2), Business Assurance Platform Working paper templates & scripts, DMS & KMS, partially organized per type of industry (website building system) • Audit support architecture of a big audit firm, or of a shared back-office of a group of smaller audit firms Platform of audit packs* with check lists & audit planning templates,organized per type of industry • Deloitte’s ‘Builder Player Platform’-architecture All mentioned + capturingcontext to offer guidance in determining & configuring scripts for data analysis, addressing the key questions: - “When to do which test?” - “What to do with the test results?” Interactive Audit Documentation * Audit pack: a bundle of interrelated forms, specific for an industry, or sector ComputationalAuditing.com

‘Correctness by Construction’ Proven Architecture 4 Deloitte’s Smart Audit Support: Interactive Audit Documentation published in Word and browsers,World’s Strongest Audit Support* Player p.334 Builder Specified Audit Methodsdrive integralPlanning, Execution & Documentation Conditional Relevancy Flexible Questionnaire integrated in Web Forms: By making explicit what is needed to answer “When to do which audit test?” & “What to do with the test results?” you articulate a body of multiple-choice questions, tables, etc., connected by choice-labeled relevancy links, embodying an approach, a method, or even, if possible, a workflow process, to guide how to achieve assurance Effective: don’t miss relevant issue Drives & Captures the ‘Story of the Audit’ Efficient: no access to less relevant issues Adequate Instantaneous ComputationalAuditing.com Optimal mitigation of litigation risk p.337 * Dutch Tax Office

All planning docs are smart forms All planning docs are smart forms All planning docs are smart forms All planning docs are smart forms All planning docs are smart forms All planning docs are smart forms with built-in Conditional Relevancy Deloitte’s approach 5 Smart Audit Support’s document index related to Deloitte’s International Audit Approach (1990’s) Example audit pack Yearly ROI guess: 20K man-yrs/yr x $10K cost reduction/man-yr = $200M • In addition to $200M yearly cost reduction ROI is: • Relevant Doc & Planning, no more no less • Comfortable & stringent way to get it p.336 ComputationalAuditing.com p.62

6 Interactive Audit Documentation:DedicatedFunctionalities for theAudit Team • Documents and guides: • “What has been done?” & “What has to be done?” • “What information has been found?” & “What’s the impact on the audit?” Functionalities for audit workflow operators Filling out a web-based questionnaire with multiple-choice questions: • Activates relevant, more detailed questions & de-activates irrelevant • Aggregates audit risk/audit evidence, according to a prescribed processing scheme, as captured in risk summarization tables • Plans and configures audit tasks to constitute an audit plan, for example, based on accumulated risk: • To be able to rely on a specific assertion level control • To further investigate the risk by planning substantive procedures • Shows when to stop investigating an account, a process or an assertion • Sets a risk classification to ‘significant inherent risk’ • Activates dedicated support to indicate how to: • Specify a norm for an entity-level control • Specify a fraud risk, including a description of who is able & how to do it • Specify a norm for initial numerical analysis; when within norm, no extra tasks • Specify or configure a script for a data analysis tool • Decide to involve an external specialist in your audit team (e.g. forensic, EDP) Capturing the ‘Story of the Audit’,ISA 315.122 “The Auditor’s New Clothes”, 2008, Tom Koning & the ‘Audit Navigator’, translation into English is pending ComputationalAuditing.com

8 Web platform for audit support: How to use that content? ‘business wise’by auditors, for auditors • Building & uploading by fee-earning expert auditors Interactive audit documentation & business positioning: • Downloading & use by fee-paying engagement teams • Broker-fee for the hosting platform provider Successfully positioned within Deloitte • Trade in audit packs between member firms • External auditors develop tailored packs & on-line services for client’s internal audit department.Why? Marketing strategy of ‘vendor lock-in’ • Professional bodies of CPAs and standard setters upload high-level guidance packs à la ISA & strict forms à la Tax.Basis to be refined upon, but not overruled “Audit Software: From Bench Warmer to Star Player”, Royal NIVRA, “de Accountant”, March 2009, pp. 12-18, Annotated translation into English by Dutch-American Translations & ComputationalAuditing.com ComputationalAuditing.com

9 Web platform for audit support: How to use that content? ‘society wise’ by auditors, for auditors • Uploading by content providing expert auditors,using a dedicated content builder Interactive audit documentation & ‘open pack’-platform: • Downloading by engagement teams,using a generic player to apply content • Content is certified, published & hosted by • an audit firm’s global and national office (layers) • a professional body of auditors • a standard setter or regulator each granting access rights to their members, ideally with ‘content overlaying’ (A on top of B, B on top of C) “Audit Software: From Bench Warmer to Star Player”, Royal NIVRA, “de Accountant”, March 2009, pp. 12-18, Annotated translation into English by Dutch-American Translations & ComputationalAuditing.com Invitation to CaseWare & ACL: do you want to contribute to proposing a tailored version to AICPA & CICA? ComputationalAuditing.com

10 Recap ‘Builder Player Platform’-architecture Goal of the Builder Goal of the Player Goal of the Platform Support in capturingaudit methods Support in applying audit methods Support in classifying audit methods Builder Player “What keeps audit leaders up at night?”, ACL, 2008 “How to get the data?” is not the challenge anymore. Today, audit analytics fully focuses on: “How to use the data?” & “How to manage that use?” Aggregation & classification are key methods of using data, so let’s have a look into how to manage aggregation & classification “Audit Automation as the Foundation of Continuous Auditing”, Michael Alles, Alexander Kogan, Miklos Vasarhelyi & Donald Warren, 16th WCAS, 2008 ComputationalAuditing.com

Aggregation scheme for risk assertions (cf 20) 12 Managing the use of aggregation & classification Risk summarization tables capturing assertion-based aggregation schemes Builder What do the arrows mean?E.g. Table A1.2.1 accumulates risks regarding the assertion ‘Systems that retain …’ based upon underlying feeding questions such as E1.6 & classifies & propagates the accumulated risk to Table A1.2 & A1 to contribute to driving the configuring, via table S2, of audit tasks constituting the audit plan Yahoo! SiteBuilder + own plug-ins to specify, visualize & interact with aggregation links (W3C SVG) Expressible, in a similar way, in Deloitte’s Smart Audit Support, see: ‘Computational Auditing’, p.328 Experiments with Adobe Flex, MXML & Google Open Docs, considering CaseWare’s Open Engagement Website Building System ComputationalAuditing.com The arrow is an Audit Workflow operator

A B D C 13 Aggregation, Process Mining & Workflow Managing the use of aggregation & classification Input: event log with journals, e.g. SAP Output: smart flowchart Analyzing 3232 cases, classi-fying casualties (red arrows): A. Invoice receipt without prior approval (2537x) B. Approval acquired after purchase completion (261x) C. Purchase order established for rejected request (9x) D. Handled order status skip- ping receipt (875x), etc. Design-time workflowvs. run-time workflow Based on: “Towards a Computer-Assisted Audit Analysis of Business Processes: Process Mining as Tool for IT Auditors”, Maria Bezverhaya, Emiel Caron & Piet Goeyenbier, ‘de EDP-Auditor’, NOREA, 2009 Pull signal from audit practitioners & IT audit educators Computational Auditing: - focus on discovery of supercycle- framing ‘stand alone’ workflows - connecting to 80 years of audit theory Push signal from Technical University of Eindhoven, ProM, Fluxicon & Anne Rozinat ComputationalAuditing.com

F m d Static: State Balance Item S 225 25 D f t C m M D F M 500 L f 25 D D s t 500 225 L F C L f t C b f t A t 100 400 1,000 S L F A L F 400 400 B F F t A S t 200 25 B F A t B f t B f t A 20 P A A t P t 20 A P P 20 A t P t P 20 W P t A Dynamic: Transaction Profit & Loss Item T W t W W t 14 Ernst & Young’s Smart Flowchart Pilot Study Top-level is Supercycle, or Top-cycle. Connects traditional cycles Case by Hans Verkruijsse & EY team, 2005-2006 EY’s evaluation report: - Clarifying. Refreshing. - Systematic framework guides input preparation process (2009: new style) - Quantitatively motivated process decomposition Approach: Powerful and easy system to support practice, founded in theory World’s strongest ‘business process’-oriented auditing theory: classical Dutch auditing theory (80+ years) & its best-fitting rigorous process theory: Petri nets tailored to the auditing domain Managing the use of aggregation & classification Fit recognized by Jagdish Gangolly, 2007-2008 New in 2009: Process mining; pilots by a Big 4, UvA.nl & CWI.nl  Focus on top-cycle discovery Agent Legend M: Majority Owner-ManagerS: Sales department B: Buy/Purchase departmentF: Financial department T: IT department W: Warehouse managerL: Labor/salary accountsP: Planning departmentC: Creditor accountsD: Debtor accounts A: Application Input: event log Output: 1. ‘As Is’ diagram (‘Ist’)2. Identify ‘To Be’ (‘Soll’)3. Built-in audit analytics Agent’s access is associated to: 1. Transactions 2. States 3. Flows More on integrated audit analytics: “Enterprise-level Process Documentation incorporating Automatic Audit Analytics”, 2008 Deloitte/KU Symposium & follow-up with Raj Srivastava & EY CARAT Capital letter: authorized, legitimate access Small letter: illegitimate access Case in Efrim Boritz’ CAATTs class, 2007-2008 ComputationalAuditing.com

15 Typology of Top-cycles Managing the use of aggregation & classification Top-cycle: normative backbone of the ‘business process’-oriented audit approach Typology/classification of top-cycles: ordered by the strength of the backbone Top-cycle concept & typology: Central result of integral evolution. Of ‘business process’-oriented Auditing Theory, Auditing Practice & Auditing Education. Over 60-80 years Unfortunately hardly translated into English Scientific foundation: rationally rigorous. With mathematical & computational formalization. Superbly suited for the digital age. Recognized as such in accelerating pace. Easy by new tech previous slide: example supercycle Limperg, Starreveld, Frielink, Blokdijk & Veenstra ComputationalAuditing.com

16 ‘Industry classification’-based auditing concepts, norms & methods Managing the use of aggregation & classification Frielink et al. Supercycle-backbonedAudit Approach Volumes 1, 2a, 2b, etc. Starreveld et al. Typology of Top-cycles Decisive advantage of these concepts, norms & methods: no need to prove again in practice, since practice was part of the evolution process ComputationalAuditing.com

18 Mechanism for quantitative aggregation { At least one noncurrent inventory + = 5 Assets or 2 Receivables 3 Inventories 5 Current Assets All three inventories are current XBRL US GAAP Taxonomy Type Polymorphism: Least Upper Bound in the Taxonomy • Aggregation in XBRL: • Calculation linkbase • XBRL Formula Articulate XBRL Assurance functionality using a dedicated website builder (plug-ins) instead of handcrafting XBRL Formula’s See: “On Positioning XBRL Assurance Business Rules in a Computational Infrastructure for Modern Auditing”, 2009, University of Kansas, Annual International Conference on XBRL Plug-in ‘type polymorphism’ mechanism (transferable) from programming language into XBRL Assurance Builder & Player Domain-Specific Language (DSL) for auditing: Pacioli, developed by Dutch software partner in cooperation with national research center for mathematics and computer science in the Netherlands (CWI) & University of Amsterdam ComputationalAuditing.com

19 How to aggregate weak spots in the Internal Control that are both irreplaceable and indispensable, e.g. weak spots in Segregation of Duties? Mechanism for qualitative aggregation: Irreplaceable in the sense that there is no way for an external auditor to compen-sate its lacking or failing, while it is indispensable for a rationally justifiable approval “X-Raying Segregation of Duties: Support to Illuminate an Enterprise’s Immunity to Solo-Fraud” with discussions & response, IJAIS, June 2008 Method locatingwho has too manyauthorizations inone hand creating a dangerous opportunity for traceless embezzlement, jeopardizing the integrity of financial statements Clarifies why & how weak spots in the SoD require a hot-line direct-top-level aggregation mechanism “Get it right at entry level” Focal point in modern auditing? Launched at Accountant.nl by Jules Muis, Oct. 2009. Directly endorsed by Hans Blokdijk, Marc van Hilvoorde and others. Berry Wammes, CEO Royal NIVRA, directly stated the intent to position “Get it right at entry level” as the theme for the NIVRA spring 2010 debate series Top-of-iceberg solo-frauds: 1. Madoff 2. Stanford 3. Kerviel, etc. For reasons of efficiency: establish a full aggregation as early as possible in the audit process (observation by William Kinney) Solo-fraud free? Design, Implementation & Operation Continuous auditing web service (hosted via external auditor?) intercepts every Authorization Change Request to signal: refuse human intervention required OK Efrim’s proposal (2008): Large-scale introductory study for this science-based method. As for new medicine. New method on top of Dutch auditing theory as incarnated in computational process theory. Collaboration with Canada. Identification of budget doubling when large audit firm steps in. Current status: pilots by Big 4 Dutch member firm ComputationalAuditing.com

Mechanism for confidence-level aggregation(cf 12) 20 Player Fully Automatic Based on:Sun,Srivastava& Mock,2006 “An Informa-tion SystemsSecurity RiskAssessment Model”,pp. 43-48 Semi Automatic Manual This can be realized in Deloitte’s Smart Audit Support with a plug-in for Dempster-Shafer-Srivastava confidence-level computations ComputationalAuditing.com

21 AgendaAutomatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation • Web platform for audit support: “What is the content?” • Web platform for audit support: “How to use that content?” • Managing the use of aggregation & classification • Aggregation mechanisms: quantitative, qualitative & confidence • Royal NIVRA: ‘Golden opportunity for the audit profession’, Identify a way to contribute to systemic risk anticipation “The PCAOB and the Social Responsibility of the Independent Auditor” Douglas Carmichael, Founding Chief Auditor of the PCAOB ‘Golden Opportunity’ Jan Helderman, President Royal NIVRA, Accountant.nl, Sept. 2009 Early Warning System as Killer App for XBRL Assurance & Continuous Auditing: speeding up getting their ‘Place & Future’ into ‘Here & Now’ ComputationalAuditing.com

‘Golden Opportunity’ Royal NIVRA: “Preparing for an Audit Mandate to Contribute to Systemic Risk Anticipation”, magazine, web & adopted in ‘Sharing Knowledge’-project 22 Proposed Solution • An off-the-shelf system for tracking-and-tracing bar-coded products, configured for, and populated by ‘XBRL tagged’ financial products • A regulator-mandated auditor attests internal controls for the XBRL reporting channel to the new governmental systemic risk agency. Allowing for a continuous data stream—further subjected to audit tests, sampling & monitoring—with on-the-fly automatic aggregation into systemic risk indicators (release 1.0: ‘Bookstaber’ indicators) Bailing out inflates moral hazard, early warning deflates More rigor on macro, more rigor on micro: use Dutch auditing Limperg’s Theory of Rationalized Confidence How far away? XBRL Assurance is closer than ever • Instead of expecting more from XML, start expecting more from the builder-based approach to XBRL & continuous auditing • Release 1.0:matter of weeks or months, not years Jumpstart by cooperation of top-specialists Rick Bookstaber, Miklos Vasarhelyi, Raj Srivastava & Charlie Hoffman, and preferably in cooperation with a Big 4 audit firm Small step for XBRL & Continuous Auditing, quantum leap for the financial world Dutch Auditing Day, hosted by Royal NIVRA, November 25, 2009, agenda’s keynote & key discussion: “risk systems & systemic risk” ComputationalAuditing.com

Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation