Automa ção de Auditoría

1. Automação de Auditoría Ryan Teeter, Ph.D. Student, Rutgers University 18th World Continuous Auditing Symposium 6th CONTECSI São Paulo, Brasil – June 4, 2009

2. Outline • Introduction • Continuous Controls Monitoring (CCM) and COSO Guidance • Automating the IT Audit • Evaluating monitoring software platforms • Implementation of CCM at Siemens PLM • Classifying audit requirements into degrees of automation • Creating rules from Audit Action Sheets • Reengineering audit processes • Feedback loop • Preliminary Results • Time and resource commitments • Successes & Challenges • Conclusion

3. 1. Introduction • Continuous Controls Monitoring (CCM) • Evaluating control settings on business processes that provide compliance with regulation and/or internal management objectives. • Proof of concept expands existing research in continuous audit streams (see Brown, Wong, and Baldwin 2007, Alles et al 2006) • COSO “Guidance on Monitoring Internal Control Systems” (2008): • Effective monitoring involves (1) establishing an effective foundation for monitoring, (2) designing and executing monitoring procedures that are prioritized based on risk, and (3) reporting the results, and following up on corrective action where necessary.

4. 2. Automating the IT audit • Why the IT audit? • Cost considerations and effectiveness • Internal audit team spends approximately 70 days manually checking tables, authorizations, and documentation • Vasarhelyi et al (2004) indicate firms are likely to adapt existing internal audit programs • Alles et al (2006) suggest utilizing the expertise of experienced audit professionals • Verifiability of automated controls against results from the manual audit

5. 2.1 Evaluating monitoring software platforms • CCM aids compliance for SOX sec. 201 and 404 • Large accounting firms unable to source CCM • Third-party platforms installed as monitoring and control layer • Minimal impact on performance • See Vasarhelyi, 2004

6. Siemens’ Current SAP Audit Model • Use text file output and transaction checks on line to audit SAP • Report findings and recommendations for remediation • Use follow-up audits to assure appropriate controls are in place and remain in place Company A SAP SYS. PD2 Company B SAP SYS. P88 Company C SAP SYS. P51 Company D SAP SYS. P40 Common –“E -Audit” Extractions on a request basis. Text File Store Text File Store Text File Store Text File Store

7. 3. Implementation of CCM at Siemens PLM • Rules were created in Approva BizRights based on ~300 audit action sheets provided by Siemens • Siemens Corporation wants universally-adaptable sets of rules and control tests for use in different divisions

8. AAS Audit Program Test of Effectiveness: 1. /nSA38 report RSUSR002, user SAPCPIC. 2. Check whether SAPCPIC is used as a dialogue user (>>eAudit: 1.02.060_2 SAP* data in USR02 – last login date, UFLAG <<) 3. Check which profiles have been assigned (>>eAudit: 1.02.060_3 profiles of SAP* in USR04<<)

10. 3.1 Classifying rules by degree of automation • Authorization • users with access to screens or functions • Approx 30% of audit effort • Baseline • Separation of duties • Transaction • Frequency of code use • User Activity Insight • Timeliness and correctness • Configuration • ERP settings • Manual

11. 3.2 Creating rules from Audit Action Sheets • Low-hanging fruit • Authorization requests • Separation of duties checks • Example: See who can create and approve purchase orders • Partial automation • Example: See who has access and whether that is appropriate • Non-automatable • Evaluation of documentation • Interviews with managers

12. 3.3 Reengineering audit processes • Creation of custom rules in Approva InsightStudio • Combination of existing controls tests • Partial automation of manual controls • “Gain an understanding of X process. Verify Y function isn’t allowed.”

13. 3.4 Feedback loop • Rule descriptions were added to aid the audit • Rules were tested and compared to results from the manual audit • Adjustments were made based on results

14. 4. Results • Time and resource commitments • Successes • Challenges • Firm characteristics

15. 4.1 Time and resource commitments • Time commitments: • 70 days for the manual audit • 3 months preparation • Platform installation • AAS classification • Resource commitments • Travel, lodging, etc. • 3-5 researchers – 3 Full-time equivalent • 2 internal auditors at PLM • 2 IT auditors from Siemens • 1 support staff from Approva

16. 4.2 Successes • Initially approximately 63% of controls automated • Rules were used to provide support for the IT audit • Initial evaluation of cost savings (A&D PL specific)1: • For 3 of every 4 years, eliminate ~ 500 man-hours of IT GCC and application control testing (@ $137/hr = $68,750/year) • With system certified, 80% reduction in 500 man-hours of annual external IT audit hours (@ ~$200/hr, $80,000/year) 1 Siemens IT audit pool billing rate is $137/hour; Approx $200/hr Big 4 blended rate.

17. 4.2 Successes

18. 4.3 Challenges • Audit priority • Non-applicable rules ignored because of time constraints • CCM platform issues • Bugs or unimplemented features • Identified when comparing automated with manual results • Vendor vs. auditor priorities • Issues addressed in future releases • Properly functioning controls • Control failure resulted in lack of support for the audit

19. 4.4 Firm characteristics • Siemens PLM • Technology firms generally have better IT controls • Already using SAP R/3 • Degree of success may depend on the amount of IT systems and support.

20. 5. Conclusion • The IT audit is a feasible starting point for CCM implementation • Existing audit plan • Knowledge of experienced auditors • Real-time performance comparison • 63% of audit controls automated • 100% of authorizations, which comprise 30-35% of audit commitment • vs. 75% proposed by Alles et al. (2006)

21. Expanding this paper • Weighting control risk? • (Cushing 1974, Cash et al, 1977, Vasarhelyi 1980, Srindini and Vasarhelyi 1986, Vasarhelyi and Srindini 1989) • Cost savings reallocation to auditing rulebooks?