Continuous Monitoring of Business Process Controls: A Pilot Implementation of a Continuous Auditing System at

1. Continuous Monitoring of Business Process Controls: A Pilot Implementation of a Continuous Auditing System at Michael G. Alles Gerard Brennan Alexander Kogan Miklos A. Vasarhelyi

2. IT-enabled Business Processes (BPs) • A business process is “a set of logically related tasks performed to achieve a defined business outcome,” Davenport and Short (1990). • Modern information technology makes it possible to measure and monitor business processes at the unprecedented level of detail (disaggregation) on the real-time basis. • Continuous auditing (CA) methodology can utilize the IT capability to capture BP data at the source and in the disaggregate and unfiltered form to achieve more efficient, effective and timely audit.

3. Two Approaches Towards Continuous Auditing of BPs • The environment of highly automated and tightly-coupled BPs (e.g., an integrated enterprise system) enables CA based on continuous monitoring of process control settings. • Unfortunately, there are numerous enterprise environments where process controls are either not automated or their settings are not readily accessible. • In environments such as loosely-coupled legacy data processing systems, automated audit procedures of CA have to be data-oriented (tests of details and analytical procedures).

4. Ways of Verifying Existence, Correctness and Functioning of BP Controls • Verifying that the observations of a BP agree with the existence, correctness and functioning of a control; benefit - can be applied even if controls are not directly accessible by the auditor; problem - the observed behavior of the BP may not completely cover the whole range of control functions. • Verifying by executing a prohibited BP behavior that it either cannot happen or is detected and compensated for; problem - auditor has read-only access to the production system and cannot run “penetration testing”. • Verifying that retrieved control settings stored in the enterprise system match the benchmark; problem - relies on the assumption that the programming code of the control in the production system is correct (customization).

5. Conceptual View of Continuous Monitoring of BP Control Settings • The environment of highly automated and tightly-coupled BPs enables CA based on continuous monitoring of business process control settings. • Online access to automated BP control settings (available in ERP systems) from the continuous assurance system. • Enterprise-dependent benchmarks of acceptable control settings. • Frequent (e.g., daily, hourly) comparison of actual settings with the benchmarks. • Automatic generation of alarms in case of critical deviations, such as individual accounts without passwords, aggregation of weaknesses in certain control areas (e.g., segregation of duties).

6. Pilot Implementation of CMBPC Systems by Siemens IT Internal Audit • CA/R/Lab approached by Siemens IT internal audit to explore use of CA methodology to streamline audit of SAP system controls at Siemens USA. • Siemens is uniquely SAP enabled with 60+ applications in the USA alone. • IT internal audit examines sites on a 2 year cycle: labor intensive, costly. • Internal audit needed to find resources to take on 404 implementation without increase in headcount.

7. CA Value Proposition as Seen by Siemens

8. Siemens Pilot Leverages Audit Action Sheets of External Auditor • Key success enabler for pilot is the ability to use AASs provided by KPMG (Germany). • Provides template for CA analytics: 1. No need to reinvent the wheel. 2. Immediate buy in from relevant parties. 3. Satisfies external auditor. • 300+ AASs: need to determine which ones can be CA-enabled. • AASs included a mixture of tasks some of which could only be accomplished by a human, such as: • interviewing the client about their reconciliation procedures; while some others involved well-scripted interactions with the client’s enterprise system such as: • execute a certain SAP R/3 transaction and/or report and verify that its result is as specified.

9. Audit Action Sheet Example

10. AAS Procedure and Rating

11. AAS Formalization

12. Existing IT Audit Procedures at Siemens

13. Steps of Piloting a CA Implementation • Determining the best mode for the continuous monitoring of the chosen BP controls. • Developing a system architecture for this task (either a monitoring and control layer or an embedded audit module). • Designing the interaction and integration between the CA mechanism and the ERP system. • Developing guidelines for the formalization of the AAPs into a computer executable form, including the determination of those AAPs which are automatable and those requiring reengineering. • Creating processes for managing the alarms generated by the automated CA system and putting in place the required set of audit trails. • Formulating a change management plan to move the project from the pilot stage to industrial strength software.

14. System Architecture for Continuous Monitoring of BP Control Settings • Continuous assurance system: Embedded audit modules (EAMs) vs. independent control and monitoring layer (CML). • EAMs are tightly coupled with the enterprise system  • can be triggered by suspicious events, but • are intrinsically more vulnerable to manipulation. • CML relies on remote (read-only) access to the enterprise system  • its code and environment are well-protected, but • can’t query the enterprise system too often, and therefore may miss suspicious enterprise events. • CML implementation is less reliant on the cooperation of the enterprise personnel.

15. Interaction of CA System with ERP System • Modern enterprise systems have a 3-tier architecture (presentation, application, database). • CML can query the system through the application tier using its APIs (e.g., BAPIs in SAP R/3); or EAM can be implemented as a sub-module of the application (e.g., Coded in ABAP in SAP R/3). • CML can query the enterprise database directly (using SQL through ODBC); or EAM can be implemented as a trigger (written in SQL) stored in the database. • The latter approach is strongly resented by enterprise personnel and (in the case of EAM) de facto prohibited by enterprise software vendors.

16. Architecture of Generic CMBPC System CMBPC System Enterprise System Database of System Settings and Audit Trail Presentation Tier Internet Application Tier Program Logic and User Interface Database Tier

17. Pilot CMBPC System at Siemens

18. Siemens Audit Leverage Tool SALT • The SALT pilot was prototyped using the formal Siemens SAP audit process in the basis area (the application layer operating system for SAP) covering the application level controls applicable to any SAP system. • 12 AASs were chosen as representative of challenges in formalizing and reengineering. • SAP E-audit download provided sample data for the model. • The pilot was developed in Visual Basic to serve as a test environment for evaluating technical research questions regarding continuous auditing / assurance.

19. Screenshots from SALT

20. Reengineering and Formalization in CA Implementation • Implementation of CMBPC starting with a clean slate is the most logical and efficient solution, BUT – impractical – clearing it with external auditor presents tremendous complications and delays. • Automation requires formalization of AASs some of which are formalizable while others are not (such as those that require human observation of BPs and interviewing enterprise personnel). • Unavoidability of reengineering stems from the necessity for the formalizable and non-formalizable parts of the audit program to be identified and handled separately. • Formalizable procedures are to be separated out, automated (by two experienced auditors!) and executed with high frequency. • Formalization creates uniformity and assures that every implemented procedure is state of the art.

21. Control and Alarm Hierarchy and Its Management • Organization of controls: identify risk areas, break them down into sub-areas, and develop controls to eliminate or mitigate these risks  enterprise controls form a top-down hierarchy. • Active component of CMBPC– automatically triggered audit alarms; critical – exception conditions to trigger alarms. • Problem associated with the automatic generation of alarms in CMBPC – the alarm flood: either initial flood or overly conservative benchmarks. • The alarms form a hierarchy corresponding to and derived from the control hierarchy  hierarchical alarm management. • “Enabled/disabled” alarm flag; “disabled” in a node overrides the settings in all the children nodes down the hierarchy tree. • The set of alarm recipients specified in a node applies by recursion to all the children nodes down the hierarchy tree.

22. Developers of Continuous Monitoring Software • Three main categories: enterprise software vendors, large public accounting firms, and established audit software vendors. • Enterprise software vendors traditionally provided very limited CM capabilities within their systems (e.g., SAP’s Audit Information System); often quoted reason – lack of demand (assurance does not contribute directly to the bottom line!). • Large public accounting firms have been experimenting with CM software (e.g., KPMG’s KOLA), but remain ambivalent (and question the value proposition, ROI, …).

23. Developers of Continuous Monitoring Software - II • Established audit software (CAAT) vendors have domain knowledge and well-developed libraries of audit tests, and see an opportunity to leverage this intellectual property in the emerging field of CA (e.g., ACL Continuous Controls Monitoring solution for purchase-to-payment cycle). • SarbOx has created a window of opportunity to sell CA software as a Section 404 compliance tool. • Large internal audit shops have been implementing ad-hoc CA-type procedures to mitigate business risks in certain high impact areas, and to achieve labor savings due to automation of audit tasks.

24. Continuing Work • Developing a methodology for formalizing audit procedures. • Extending CA higher up the audit value chain—dealing with auditor judgment. • Managing audit alarms and preventing alarm floods. • Establishing ROI from the pilot. • Scaling up to the production level. • Technology is available, laws and standards are (mostly) in place, time is now. • How to make it worthwhile (trade-off between cost, effectiveness, efficiency and timeliness)?