IS Audit in the Early 21 st Century – A Call for Research

1. IS Audit in the Early 21st Century – A Call for Research Presented at the Information Systems Section of the American Accounting Association 2007 MidYear Meeting Savannah, GA, January 4, 2007

2. Origins • This research report is the result of a PCAOB Research Synthesis Team composed of Mary Curtis, Jean C. Bedard, Donald Deis, Greg Jenkins • Title of this report: Specialty Knowledge and Use of Specialists • Because of the origins of our team, this report specifically addresses the public accounting environment, although many findings are equally relevant to internal audit.

3. What the literature review is: • This summarizes existing literature regarding the practice of IT audit – • what we know about IT and general auditors, • how they perform their jobs, • why and when they perform certain functions, and • how they become proficient at these functions. • What the literature review is not: • This review does not attempt to address the myriad technologies in which IT auditors must be knowledgeable, such as e-com, ERP, XBRL, etc.

4. Existing Standards - What kind of IT knowledge must non-IT auditors possess? • AS No. 2 – An Audit of Internal Control Over Financial Reporting Performed in Conjunction With an Audit of Financial Statements • SAS 94 - Addresses the knowledge that the general auditor should possess if they use IT professional in audit • SAS 80 - Evidential Matter - AU 326.12 • SAS 22 - Planning and Supervision - AU 311 • SAS 109 - Understanding the Entity and its Environment and Assessing the Risks of Material Misstatement- adopted after PCAOB ‘codification’ • ISA No. 315 – Understanding the Entity and Its Environment and Assessing the Risks of Material Misstatement

5. Current standards – When should IT Specialists Be Called into an Audit? • SAS 94 - The Effect of Information Technology on the Auditor’s Consideration of Internal Control in a Financial Statement Audit: • Consider whether a professional possessing specialized IT skills is needed for the audit • SAS 80 - Evidential Matter - AU 326.12 • It may be difficult or impossible for the auditor to acquire adequate evidential matter without using IT specialists. • SAS 108 - Planning and Supervision - adopted after PCAOB ‘codification’ • Considerations in determining the extent of involvement of IT professionals • Identifies the preliminary audit procedures the auditor may assign to an IT specialist

6. Our analyses of the standards and research literature resulted in two primary questions: • Extent of consideration of IT in the audit and IT audit specialist involvement in the financial statement audit: When? What? • What educational issues arise from this involvement? • This research stream is very slim – there are many more questions than answers.

7. Topic 1: Research findings on IT expertise and the use of IT audit specialists on audits • Does current research support the need for the involvement of IT audit specialists in most aspects of the audit engagement? • Computer systems are becoming more complex and computerized controls often the only or best controls (Bell et al. 1998; Collier et al. 1991), yet … • More control problems are identified in today’s more computerized environments, than previously (Messier et al. 2004). • Implication: increased need for auditors knowledgeable in complex IT systems and controls

8. Topic 1: Research findings on IT expertise and the use of IT audit specialists on audits • Does current research support the need for the involvement of IT audit specialists in most aspects of the audit engagement? • Research suggests that auditors are relying more on internal controls than previously, yet are not necessarily using IT auditors more (Janvrin et al. 2006, Bierstaker and Wright 2004, Messier et al. 2004 [contrary view]). • Research suggests that generalist auditors may under-estimate the overall audit risk with complex systems (Bedard et al. 2005, Hunton et al. 2004, Grabski et al. 1987 [contrary view]). • Implication: it appears that generalist auditors may be relying on their own knowledge to do controls testing, yet may not possess the understanding of IT systems necessary to meet this challenge

9. Topic 1: Research findings on IT expertise and the use of IT audit specialists on audits • Does this risk from computerization actually result in greater problems with the financial statements? • One study found that few audit differences were associated in any way with failure in the computerized system (Bell et al. 1998). • One contrary assertion: There are many anecdotal reports that spreadsheet errors have been the primary cause of financial statement errors. • Failed systems implementation has significant impact on going-concern of the company (ex: Hershey)

10. Research findings on IT expertise and the use of IT audit specialists on audits • Is it likely that audit generalists (financial auditors) can develop adequate IT expertise to preclude or at least reduce the need for IT audit specialist involvement in the engagement? • Research suggests that IT audit specialists have a distinctly different way of looking at internal controls and information systems from financial auditors (Biggs et al. 1987, Borthick et al. 2006, Curtis and Viator 2000, Viator and Curtis 1998,). • Implication: These studies imply that it will be difficult for generalist auditors to gain sufficient expertise to perform as well as IT specialists.

11. Research findings on IT expertise and the use of IT audit specialists on audits If current guidance recommends involvement by IT auditors (Yang and Guan 2004) and research supports this recommendation, … Is current guidance in the standards being implemented effectively? • The greater the generalist auditors’ IT expertise, the more appropriate reactions to IT auditor findings and expertise. (Brazel and Agoglia 2006). • Additionally, common knowledge bias (O’Donnell et al. 2000) suggests that knowledge possessed only by the IT specialist may be disregarded by the audit team.

12. Research findings on IT expertise and the use of IT audit specialists on audits If current guidance recommends involvement by IT auditors (Yang and Guan 2004) and research supports this recommendation, … Is current guidance in the standards being implemented effectively? • The greater the generalist auditors’ IT expertise, the more appropriate reactions to IT auditor findings and expertise. (Brazel and Agoglia 2006). • Additionally, common knowledge bias (O’Donnell et al. 2000) suggests that knowledge possessed only by the IT specialist may be disregarded by the audit team.

13. Research findings on IT expertise and the use of IT audit specialists on audits • Finally, in regard to organizational culture and IT audit, IT auditors require specialized training and skills, and may find themselves disadvantaged in firm organizations where the primary career path results from financial audit experience.

14. Potential Research Topics: IT audit and Section 404: • What portion of the 404 audit is primarily IT today? What proportion of the 404 audit could be performed by IT auditors? • What advances in IT audit testing have occurred due to the increase in its use during 404 reviews? • Has increased use led to efficiency or more sophisticated techniques? • Will more rapid adoption of continuous audit be a likely result of 404?

15. Potential Research Topics: Application of 404 testing to financial statement audit • Is audit risk adjusted appropriately for changes in IT? • What is the connection between 404 testing and substantive testing? Has any increase in IT testing for 404 resulted in a significant reduction in substantive testing? Where have efficiency gains not been achieved and what factors impact this?

16. Potential Research Topics: • What are the impediments to the use of IT auditors on the financial statement audit? • General auditor IT knowledge? • Significant deficiencies in non-IT related areas? • Budgetary incongruencies? • Culture? • Inter-personal issues between the general and IT audit groups?

17. Potential Research Topics: • What are the culture implications for firms seeking to develop strong IT audit staffs? • Career paths • Training • Rewards

18. Potential Research Topics: • What is the impact of inadequate IT knowledge on the part of the generalist auditor? • Risk assessment? • Budgeting? • Audit program design? • Task assignment between IT auditor and generalist auditors? • Conclusions they draw?

19. Topic 2: Guidance and research regarding the education of generalist auditors and IT audit specialists on IT audit issues: Guidance: • The International Federation of Accountancy (IFAC) guidance regarding the education of accountants: International Education Standards for Professional Accountants (IES 8 and IES 11). • Standard IES 8 relates to general competency requirements and IES 11 specifically to Information Technology for Professional Accountants • AICPA issued report on the implementation of IES 11 in the U.S. The report states that all students should study IT from the perspective of its usefulness, application and impact, and all educators should be encouraged to integrate the study of technology with the study of accounting. • Section on professional training

20. Guidance and research regarding the education of generalist auditors on IT audit issues: • Standards (IES 11 and SAS 94) require that audit generalists possess a certain level of IT knowledge, even when computer audit specialists are involved. Research supports this need. • No research found assessing the educational preparedness of U.S. accountants in regard to IT knowledge and competencies. • Implication: Academic institutions might consider whether their audit curriculum is adequate for providing the IT knowledge of their students training to be audit generalists. • Implication: Firms may consider assessment methods for evaluating auditors’ knowledge of relevant IT issues and what training is necessary for continuing knowledge growth as technology changes.

21. Guidance and research regarding the education of IT audit specialists on IT audit issues: • While significant commentary was published early in the history of the IT audit profession regarding the types of education and training needed by IT auditors, little has been published in the last 15 years

22. Potential Research Topics • Are there gaps between knowledge needed and knowledge possessed by public accountants? If so, what factors contribute to current knowledge gaps? Possible causal factors include: • Pace of change, • lack of coverage in university accounting programs, • effectiveness of training methods, • CPE ineffectiveness, • lack of interest or commitment to gain expertise by practitioners. • Research could examine the precedents for specifying certain topics for CPE, and whether those strategies have been effective. This research literature review has not discovered any inroads into this question (beyond PwC 2003 report).

23. Potential Research Topics • There is the big issue of generalists versus specialists. Possible questions include: • If specialty knowledge areas are difficult for the generalist to acquire, it may be more efficient to use specialists appropriately. • How much should the generalist auditor know about these knowledge areas, in order to recognize the need for specialist help? • How accurate are auditors' perceptions of their own knowledge related to these topics? • Do current budgeting procedures and other behavioral motivators inhibit generalists from calling in specialists when they might be needed?

24. Potential Research Topics Any additional research ideas from the audience? Thank You