by Rob Nehmer ( nehmer@oakland ) Hans Weigand ( h.weigand@uvt.nl ) and

1. How Technology Enables a More Cost-Effective Audit Realized by Structuring Analytics to Assertions  by Rob Nehmer (nehmer@oakland.edu) Hans Weigand (h.weigand@uvt.nl) and Philip Elsas (philipelsas@computationalauditing.com) WCARS 44 Sevilla, Es. March 21, 22 2019 1

2. PCAOB Section 329A.02 Analytical Procedures Analytical procedures are an important part of the audit process and consist of evaluations of financial information made by a study of plausible relationships among both financial and non-financial data. Analytical procedures range from simple comparisons to the use of complex models involving many relationships and elements of data. A basic premise underlying the application of analytical procedures is that plausible relationships among data may reasonably be expected to exist and continue in the absence of known conditions to the contrary. WCARS 44 Sevilla, Es. March 21, 22 2019 2

3. PCAOB Recently Identified Deficiencies in Performing Analytical Procedures • Including failure to… • Develop appropriate expectations • Establish threshold for differences without further investigation • Establish threshold for differences to achieve sufficient level of assurance • Test data used in analytical procedure • Investigate unexpected differences WCARS 44 Sevilla, Es. March 21, 22 2019 3

4. AU 329A Continues… • The expected effectiveness and efficiency of an analytical procedure in identifying potential misstatements depends on, among other things • the nature of the assertion • the plausibility and predictability of the relationship • the availability and reliability of the data used to develop the expectation • the precision of the expectation. WCARS 44 Sevilla, Es. March 21, 22 2019 4

5. AU 329A Continues… Investigation and Evaluation of Significant Differences In planning the analytical procedures as a substantive test, the auditor should consider the amount of difference from the expectation that can be accepted without further investigation. This consideration is influenced primarily by materiality and should be consistent with the level of assurance desired from the procedures. Determination of this amount involves considering the possibility that a combination of misstatements in the specific account balances, or class of transactions, or other balances or classes could aggregate to an unacceptable amount. WCARS 44 Sevilla, Es. March 21, 22 2019 5

6. Our proposal • We model the plausible relationships among both financial and non-financial data for substantive testing but also for control assessment • A few non-financial (quasi-)goods are required to be reliably recorded as a control measure • These control measures on (quasi-)goods are tested (assessed) on design, implementation and uninterrupted operation • The client’s reliably monitored (quasi-)goods are used to set expectations for FS amounts to be reported through their relationships with those FS amounts WCARS 44 Sevilla, Es. March 21, 22 2019 6

7. Auditability definition • What is auditing for? • Answer: assurance on financial reporting to owner and potential owner / society at large • What is needed to make auditing more meaningful? • Answer: control infrastructure (CI) • What must be auditable in the first place? • Answer: Financial Statements (FS) • What must be auditable after CI is in place? • Answer: IT system, Information System in Enterprise System (ES) WCARS 44 Sevilla, Es. March 21, 22 2019 7

8. Two cycles • Agency cycle: principal (owner)  enterprise, including management  auditor  owner • CI Design cycle: society (CI req)  auditing professional community (CI design)  company (CI implementation + use) auditor (CI assessment)  society • Two design problems: require require Control infrasructure Audit method Auditable ES + ITS + IS (Basis for) Assurance ES WCARS 44 Sevilla, Es. March 21, 22 2019 8

9. CI as an artefact consisting of several layers Control driver Key controls SoDs reliability based on location based on implementation realized by Industry type blueprint IT infrastructure composed of Internal systems Externally controlled systems External systems Top Value cycle control cycle exchange cycle WCARS 44 Sevilla, Es. March 21, 22 2019 9

10. CI anchor points (within the Enterprise System) • Top value cycle – integration of physical or service flow with financial flow (business process model) • Present in all market Enterprises (but physical flow can be marginal) • Control cycle • Present in all multi-agent Enterprise • Exchange cycle • Embedded in value cycle • Traditionally only partially visible (dependent view). Can become visible with Shared Ledger blockchain or other externally located platforms (e.g. booking site) • 3 cycles used in industry blueprint WCARS 44 Sevilla, Es. March 21, 22 2019 10

11. CI drivers • Self-interest • What can be supposed to be reliable in self-reporting? • Other-interest • What can be supposed to be reliable in the reporting of related agent (next agent in the cycle) • Common interest • 4-eyes principle • No interest • External observer (expensive) • Technology (under responsibility of external party) WCARS 44 Sevilla, Es. March 21, 22 2019 11

12. CI conceptual design • Exploit consistency relationships in the cycles • Check consistency: analytical procedures, assertion-based (during audit) • Design and optimize control infrastructure (during design time): • Select key controls from such that other variables can be derived! • Define minimal Segregation of Duties! • Key controls • Preferably at beginning of physical flow • SoD • Can be validated by fraud scenario analysis (Pacioli) WCARS 44 Sevilla, Es. March 21, 22 2019 12

13. CI technical design • Internal activities, internal control • Authentication/authorization mechanisms, securely supported by e.g. IT controls • Internal activities, external control • Sensors, securely supported by e.g. “heart-beat” mechanisms • Trace good (e.g. tags) • Trace place (e.g. movement sensor; entrance control) • Trace physical activity (e.g. camera) • External activities, external control • Transaction recordings, securely supported by e.g. tamper-proof storage • Trace exchange transactions (e.g. blockchain) • Trace ordering cycle (e.g. external reservation system) • Trace customer use (e.g. barcode sealing) WCARS 44 Sevilla, Es. March 21, 22 2019 13

14. IT System (ITS) components – internal systems Internal systems system access System management applications (processing) storage system network IT key controls (ITS auditability design) e.g. logging network monitoring WCARS 44 Sevilla, Es. March 21, 22 2019 14

15. IT System (ITS) components – externally controlled internal systems Externally controlled systems device (recording) System management IoT service (processing) storage system network or IT key controls (ITS auditability design) e.g. heart-beat mechanism tag sensor/ effector camera WCARS 44 Sevilla, Es. March 21, 22 2019 15

16. AS 2110: Identifying and Assessing Risks of Material Misstatement • Performing Risk Assessment Procedures Includes • Obtaining an understanding of the company and its environment • Obtaining an understanding of internal control over financial reporting • Performing analytical procedures • Inquiring of the audit committee, management, and others within the company about the risks of material misstatement WCARS 44 Sevilla, Es. March 21, 22 2019 16

17. Performing Analytical Procedures • Purpose • Identify areas that might represent specific risks relevant to the audit, including the existence of unusual transactions and events, and amounts, ratios, and trends that warrant investigation •  In applying analytical procedures as risk assessment procedures, the auditor should perform analytical procedures relating to revenue with the objective of identifying unusual or unexpected relationships involving revenue accounts that might indicate a material misstatement, including material misstatement due to fraud. WCARS 44 Sevilla, Es. March 21, 22 2019 17

18. Performing Analytical Procedures • Application: • When performing an analytical procedure, the auditor should use his or her understanding of the company to develop expectations about plausible relationships among the data to be used in the procedure. • When comparison of those expectations with relationships derived from recorded amounts yields unusual or unexpected results, the auditor should take into account those results in identifying the risks of material misstatement. WCARS 44 Sevilla, Es. March 21, 22 2019 18

19. Modified Audit Risk Model • The auditor develops analytics based on the client’s top cycle • After identifying the relevant assertion in the value (money) cycle, such as completeness, the auditor tests for corresponding CI controls in the goods stream (control design, implementation and operation) • Activity levels in these controls/sensors are set at the levels which correspond to external/client-recorded non-financial (quasi-)goods that offer proxy relationships to expected FS values WCARS 44 Sevilla, Es. March 21, 22 2019 19

20. Revenue completeness assertion Control or sensor under an external party with a metric Assertion vs. Analytic

21. Assertion Control metric Smart flow chart 21

22. Developing Analytics from the Value Chain • The auditor can focus on relationships representing the client’s business model under risky conditions • These conditions include: • parameters which historically have experienced relatively large fluctuations • parameters which might be expected to have fluctuations in the current period due to perturbations in the client’s business model or business environment WCARS 44 Sevilla, Es. March 21, 22 2019 22

23. Developing Analytics from the Value Chain • Three conditions can be used to constrain the use of this approach for developing analytic procedures during the planning stage of the audit process: • Looking for conditions where the analytics will hold, • Focus on audit risks and their related assertions either in the clients business model or their business context, and • Focus on associated control parameters which experience variability in their measures in association with the audit risk measure within the audit period or across audit periods. WCARS 44 Sevilla, Es. March 21, 22 2019 23

24. Use in the Audit Process: An Extended Example • Consider a hotel • Say that there is a reservation system which is controlled by one or more external service providers, Booking.com and Hotels.com • There are various sensors in the hotel room: motion detectors, electronic entry, windows sensors, entertainment system, HVAC system etc. • Some of the sensors must capture data that can be retrieved at a later time WCARS 44 Sevilla, Es. March 21, 22 2019 24

25. Assertions, CI and Analytics Reservation System Room Sensors Existence Assertion

26. Use in the Audit Process • Hotel example: • Part of the business model is the collection of sales revenues from paying customers. • The auditor may be concerned that their client could over report revenues. • The assertions involved are the existence of the revenues reported. WCARS 44 Sevilla, Es. March 21, 22 2019 26

27. Use in the Audit Process • Auditors should consider: • The existence of external parties, the booking agencies, which are capturing the quasi-good reservations. • The existence of the records of the room control devices, especially for room rentals which originate directly at the hotel chain itself. • The level of risk (variability) acceptable in their association of the control measures and the revenue assertion. WCARS 44 Sevilla, Es. March 21, 22 2019 27

28. Use in the Audit Process • The auditor can then set up an analytic modelling this situation: • Looking at reservations booked through the external parties • The changes in the behavior of the rooms’ control devices as evidenced by their measures (use of entertainment system, room temperature, etc.) • The reported revenue. • The precision of the estimate (range of acceptability) WCARS 44 Sevilla, Es. March 21, 22 2019 28

29. Use in the Audit Process • In the Planning Stage of the Audit: • The auditor plans to collect evidence from the booking agencies and the suitable control devices during the substantive phase of the audit • The auditor will use this approach to help plan the procedures to perform during the audit • So the three conditions are met: • Looking for conditions where the analytics will hold, • Focus on audit risks and their related assertions either in the clients business model or their business context, and • Focus on associated control parameters which experience variability in their measures in association with the audit risk measure within the audit period or across audit periods. WCARS 44 Sevilla, Es. March 21, 22 2019 29

30. Use in the Audit Process • The Substantive Phase • Existence assertion: • Evidence of accounting transactions exist for all revenues booked by the external agencies. • Appropriate measures collected from the chosen control. • Calculation of the revenue amount from the analytics and comparison with the client reported revenue amount. WCARS 44 Sevilla, Es. March 21, 22 2019 30

31. Use in the Audit Process • Final Stage of the Audit • The importance of the auditor’s evaluation of any difference at this stage will depend on: • The weight attached to the results of the substantive testing stage • Whether any additional procedures were performed to test the existence assertion WCARS 44 Sevilla, Es. March 21, 22 2019 31

32. Summary • We use the plausible relationships among both financial and non-financial data not only for substantive testing but also for control assessment • We showed how this relates to assessment of controls and to analytic procedures in FS audits • We gave an extended example WCARS 44 Sevilla, Es. March 21, 22 2019 32

33. Conclusion • We develop a process to help auditors think about specific control assessments and analytical procedures that they can develop • The process allows auditors to establish controls for reliable expectations to be set for analytical procedures • The process also allows auditors to gauge the level of risk (variability) that they will allow in their expectation WCARS 44 Sevilla, Es. March 21, 22 2019 33

34. Questions? WCARS 44 Sevilla, Es. March 21, 22 2019 34