Students perceptions of the usefulness of an audit simulation Len Steenkamp

1. Students� perceptions of the usefulness of an audit simulationLen Steenkamp

2. Background Objectives The simulation Constraints to the simulation Previous studies Toyrama results Conclusion

3. Background International calls for change Students lack experience Learn by doing South African Institute of Chartered Accountants (SAICA) syllabus Introduction Internationally, many have argued in favour of changing the manner in which Accounting programmes are delivered to students (Accounting Education Change Commission 1990; Stewart & Dougherty 1993:9, Masey, Poli & Proctor 2002:1). These calls for change include suggestions that Information Technology should be integrated into the Accounting and Auditing curricula (Johnson, Baird, Caster, Dilla, Earley & Louwers 2003:242) and should more actively involve students in the learning process (Adler & Milne 1997:192; Hassall, Lewis & Broadbent 1998:327). Albrecht and Sack (2000:43) argues that Accounting courses do not adequately prepare students for the �ambiguous business world they will encounter upon graduation� due to the absence of concrete experience (Hoyle 1975:230; Siegel, Omer & Agrawal 1997:217). Students generally have not been exposed to the business world and are not familiar with transactions and the usual activities which form the basis of Auditing (Siegel et al 1997:218).This makes it difficult for students to understand the field of Auditing With this in mind Accounting education is increasingly being required to integrate Information Technology skills with professional subjects, such as Financial Accounting, Management Accounting, Taxation and Auditing with students being required to illustrate the ability to apply theoretical knowledge to �real-world� scenarios In an effort to achieve the above goals, the Department of Accounting at Stellenbosch University developed an audit simulation whereby Information Technology skills are used to perform the audit of a hypothetical company selling toys. This study reports on the perceptions of students on the usefulness of this audit simulation for the development of skills and knowledge. Introduction Internationally, many have argued in favour of changing the manner in which Accounting programmes are delivered to students (Accounting Education Change Commission 1990; Stewart & Dougherty 1993:9, Masey, Poli & Proctor 2002:1). These calls for change include suggestions that Information Technology should be integrated into the Accounting and Auditing curricula (Johnson, Baird, Caster, Dilla, Earley & Louwers 2003:242) and should more actively involve students in the learning process (Adler & Milne 1997:192; Hassall, Lewis & Broadbent 1998:327). Albrecht and Sack (2000:43) argues that Accounting courses do not adequately prepare students for the �ambiguous business world they will encounter upon graduation� due to the absence of concrete experience (Hoyle 1975:230; Siegel, Omer & Agrawal 1997:217). Students generally have not been exposed to the business world and are not familiar with transactions and the usual activities which form the basis of Auditing (Siegel et al 1997:218).This makes it difficult for students to understand the field of Auditing With this in mind Accounting education is increasingly being required to integrate Information Technology skills with professional subjects, such as Financial Accounting, Management Accounting, Taxation and Auditing with students being required to illustrate the ability to apply theoretical knowledge to �real-world� scenarios In an effort to achieve the above goals, the Department of Accounting at Stellenbosch University developed an audit simulation whereby Information Technology skills are used to perform the audit of a hypothetical company selling toys. This study reports on the perceptions of students on the usefulness of this audit simulation for the development of skills and knowledge.

4. Objectives Was the audit simulation valuable in developing certain competencies? Did the audit simulation contribute to students� studies in different ways? What were the most important benefits derived from the simulation? What were the major constraints to the simulation? The primary objective of the study is to investigate student perceptions on the usefulness of an audit simulation as a tool to develop knowledge and skills conducted as part of their Information Systems studies at third year level. The study seeks to answer to the following questions:The primary objective of the study is to investigate student perceptions on the usefulness of an audit simulation as a tool to develop knowledge and skills conducted as part of their Information Systems studies at third year level. The study seeks to answer to the following questions:

5. The simulation Audit of stock only: Toyrama Focus on Information Technology techniques Large Microsoft Excel files Working papers Assessments The audit simulation reported on in this study, has been conducted for the past two years as part of the Information Systems 372 course in the third year of students studying towards an Accounting degree. Two groups complete this module, namely Bachelor of Commerce (B Comm) Management Accounting and Bachelor of Accounting (B Acc) students. The objective of the B Comm degree is to write the Chartered Institute of Management Accountants (CIMA) examination after completing a post-graduate degree, while the B Acc degree has as goal (after completing a post-graduate course) the Qualifying Exam of the South African Institute of Chartered Accountants (SAICA), conferring the right of the CA(SA) designation upon registration. One of the focus areas of the B Acc degree is Auditing. However, it is not the core focus of B Comm, but it forms part of the program in the students� second year. It was felt appropriate that both groups of students are exposed to the audit simulation, as it gives valuable insight into the workings of both external and internal auditors, and the basic skill of manipulating data (especially in large databases) in order to make meaningful conclusions. The project on which this article is based consisted of an audit simulation of a hypothetical company which sells toys. The objective of the simulation is to integrate theoretical Auditing knowledge with practical Information Technology skills and to make Auditing more practical. The students were provided with background information on the purpose of the simulation as well as the hypothetical company. The simulation can be divided into three stages: Students were required to prepare the pre-engagement and planning working papers. The majority of the information for this stage of the simulation was provided by class discussions. The significant part of the simulation consisted of a requirement to complete working papers for the substantive procedures on the company�s stock, together with a report to management indicating problems and errors identified during the audit of stock. Students were provided with a stock list, stock count list and a price list with selling prices in Microsoft Excel. All three these databases exceeded 3,500 lines in Excel, thereby forcing students to make use of Information Technology and providing the chance for illustrating the relevance of Information Systems as subject to the students. The databases included various errors and omissions to illustrate a number of Auditing and Information Technology concepts, skills and methods. Five lectures were spent on explaining, discussing and illustrating the procedures to be performed on this data. The last section of the simulation consisted of a lecture relating to the additional procedures that would be performed during the conclusion stage of the audit. The assessment of the simulation consisted of the working papers for the pre-engagement and planning stages of the audit, working papers for the substantive procedures on stock and the report to management on problems and errors in stock. Due to limited availability of time and pressure on students to start preparing for examinations, students were not required to hand in working papers for the completion stage of the audit. An electronic test was also completed by students where new stock lists (similar to the ones used in the simulation) were provided to students, which students had to manipulate as they did in the simulation in order to answer a number of questions. This test assessed whether students were able to perform the necessary procedures in Excel themselves. The audit simulation reported on in this study, has been conducted for the past two years as part of the Information Systems 372 course in the third year of students studying towards an Accounting degree. Two groups complete this module, namely Bachelor of Commerce (B Comm) Management Accounting and Bachelor of Accounting (B Acc) students. The objective of the B Comm degree is to write the Chartered Institute of Management Accountants (CIMA) examination after completing a post-graduate degree, while the B Acc degree has as goal (after completing a post-graduate course) the Qualifying Exam of the South African Institute of Chartered Accountants (SAICA), conferring the right of the CA(SA) designation upon registration. One of the focus areas of the B Acc degree is Auditing. However, it is not the core focus of B Comm, but it forms part of the program in the students� second year. It was felt appropriate that both groups of students are exposed to the audit simulation, as it gives valuable insight into the workings of both external and internal auditors, and the basic skill of manipulating data (especially in large databases) in order to make meaningful conclusions. The project on which this article is based consisted of an audit simulation of a hypothetical company which sells toys. The objective of the simulation is to integrate theoretical Auditing knowledge with practical Information Technology skills and to make Auditing more practical. The students were provided with background information on the purpose of the simulation as well as the hypothetical company. The simulation can be divided into three stages: Students were required to prepare the pre-engagement and planning working papers. The majority of the information for this stage of the simulation was provided by class discussions. The significant part of the simulation consisted of a requirement to complete working papers for the substantive procedures on the company�s stock, together with a report to management indicating problems and errors identified during the audit of stock. Students were provided with a stock list, stock count list and a price list with selling prices in Microsoft Excel. All three these databases exceeded 3,500 lines in Excel, thereby forcing students to make use of Information Technology and providing the chance for illustrating the relevance of Information Systems as subject to the students. The databases included various errors and omissions to illustrate a number of Auditing and Information Technology concepts, skills and methods. Five lectures were spent on explaining, discussing and illustrating the procedures to be performed on this data. The last section of the simulation consisted of a lecture relating to the additional procedures that would be performed during the conclusion stage of the audit. The assessment of the simulation consisted of the working papers for the pre-engagement and planning stages of the audit, working papers for the substantive procedures on stock and the report to management on problems and errors in stock. Due to limited availability of time and pressure on students to start preparing for examinations, students were not required to hand in working papers for the completion stage of the audit. An electronic test was also completed by students where new stock lists (similar to the ones used in the simulation) were provided to students, which students had to manipulate as they did in the simulation in order to answer a number of questions. This test assessed whether students were able to perform the necessary procedures in Excel themselves.

6. Constraints to the simulation Simulated Time One element audited Information Systems There are certain limitations to the simulation. It was not possible to provide the full scope in experience that a real financial statement audit would provide, nor could a full set of financial statement, such as in the study by Hoyle (1975), be audited. This was compounded by the time limitation, therefore only one element, namely stock, was addressed. It was felt that the manipulation of the stock records would provide ample opportunity to illustrate certain Information Technology and Auditing tools and principles. An actual stock count could not be performed and invoices were not provided to students as in the study by Felix et al. (1985). It should also be borne in mind that the simulation was conducted as part of the Information Systems course, with dual goals of (1) making Auditing more practical and (2) integrating Auditing and Information Technology techniques and skills, consequently considerable time was therefore spent on the Information Technology aspect of the simulation.There are certain limitations to the simulation. It was not possible to provide the full scope in experience that a real financial statement audit would provide, nor could a full set of financial statement, such as in the study by Hoyle (1975), be audited. This was compounded by the time limitation, therefore only one element, namely stock, was addressed. It was felt that the manipulation of the stock records would provide ample opportunity to illustrate certain Information Technology and Auditing tools and principles. An actual stock count could not be performed and invoices were not provided to students as in the study by Felix et al. (1985). It should also be borne in mind that the simulation was conducted as part of the Information Systems course, with dual goals of (1) making Auditing more practical and (2) integrating Auditing and Information Technology techniques and skills, consequently considerable time was therefore spent on the Information Technology aspect of the simulation.

7. Previous studies Arens, May & Dominiak 1970 Hoyle 1975 Siegel, Omer & Agrawal 1997 Gelinas, Levy & Thibodeau 2001 Limited literature on students� perceptions A review of international studies on audit simulations and students� perceptions of these simulations are presented in this section. The literature on audit simulations is limited In 1970, Arens, May and Dominiak developed a simulated case for audit education because they argued that it is �difficult to relate the important ideas [of Auditing] to students because they have not clearly established in their own minds an adequate frame of reference to analyze and understand Auditing concepts� (Arens et al. 1970:573). They attributed this to the students� lack of exposure to accounting systems, source documents and evidence accumulation. The simulation Arens et al. (1970:574) developed differs from the one in this study, in that it focused on discussions of factors relevant to the simulated audit case, rather than the performance of audit procedures. Moreover, the simulated case was constrained by the technology of the time and limited use of personal computers. Hoyle (1975:232) reports on his use of an audit simulation on the whole set of accounts of a company, not only one account or sub-set (stock) as is used in this study Siegel et al. (1997) report on a video simulation of an audit, The CableCo Chronicles. The set of four videos, supported by written materials, takes the students through the audit process from acquiring a new audit client to the concluding stages of the audit. These videos were developed by the Coopers & Lybrand Foundation in the USA as part of their �Excellence in Audit Education� programme in 1985. results showed that The CableCo Chronicles �provide an effective integration, helping students form an understanding of the complete process of Auditing� (Ward 1992:12-13). Siegel et al. (1997:233) did however note that �the videotape does not teach Auditing. Other classroom activities must still be carried out in order for students to understand the development of an audit.� An audit simulation would therefore �supplement rather than replace classroom work� (Hoyle 1975:231). Norwood Office Supplies, Inc is an audit simulation developed by Gelinas, Levy and Thibodeau (2001) that uses Audit Command Language (ACL) audit software. The simulation covers risk assessment to performing audit procedures with ACL. The goal of the simulation is to address technological, strategic and critical-thinking skills. A review of international studies on audit simulations and students� perceptions of these simulations are presented in this section. The literature on audit simulations is limited In 1970, Arens, May and Dominiak developed a simulated case for audit education because they argued that it is �difficult to relate the important ideas [of Auditing] to students because they have not clearly established in their own minds an adequate frame of reference to analyze and understand Auditing concepts� (Arens et al. 1970:573). They attributed this to the students� lack of exposure to accounting systems, source documents and evidence accumulation. The simulation Arens et al. (1970:574) developed differs from the one in this study, in that it focused on discussions of factors relevant to the simulated audit case, rather than the performance of audit procedures. Moreover, the simulated case was constrained by the technology of the time and limited use of personal computers. Hoyle (1975:232) reports on his use of an audit simulation on the whole set of accounts of a company, not only one account or sub-set (stock) as is used in this study Siegel et al. (1997) report on a video simulation of an audit, The CableCo Chronicles. The set of four videos, supported by written materials, takes the students through the audit process from acquiring a new audit client to the concluding stages of the audit. These videos were developed by the Coopers & Lybrand Foundation in the USA as part of their �Excellence in Audit Education� programme in 1985. results showed that The CableCo Chronicles �provide an effective integration, helping students form an understanding of the complete process of Auditing� (Ward 1992:12-13). Siegel et al. (1997:233) did however note that �the videotape does not teach Auditing. Other classroom activities must still be carried out in order for students to understand the development of an audit.� An audit simulation would therefore �supplement rather than replace classroom work� (Hoyle 1975:231). Norwood Office Supplies, Inc is an audit simulation developed by Gelinas, Levy and Thibodeau (2001) that uses Audit Command Language (ACL) audit software. The simulation covers risk assessment to performing audit procedures with ACL. The goal of the simulation is to address technological, strategic and critical-thinking skills.

8. Findings from previous studies Valuable Better understanding Better performance �Hit the ground running� Criticism Time From the above discussion it appears that there are certain benefits that may be derived from using an audit simulation, inter alia a better understanding of Auditing concepts and real-world audit complexities. The main concern students raised relate to the time needed to complete the simulation. From the above discussion it appears that there are certain benefits that may be derived from using an audit simulation, inter alia a better understanding of Auditing concepts and real-world audit complexities. The main concern students raised relate to the time needed to complete the simulation.

9. Results from Toyrama questionnaire

10. Respondent profile Response rate: 81.5% From of the total population of 394 students, 321 useful responses were obtained, giving a response rate of 81.5%. The majority of the class (74.6%) is represented by B Acc students, also reflected by 75.4% of the respondents. 19.9% of the respondents studies B Comm (20.0% of the class in total) and B Accounting LLB 4.7% (5.4% of the total class). The respondents are therefore representative of the class. From of the total population of 394 students, 321 useful responses were obtained, giving a response rate of 81.5%. The majority of the class (74.6%) is represented by B Acc students, also reflected by 75.4% of the respondents. 19.9% of the respondents studies B Comm (20.0% of the class in total) and B Accounting LLB 4.7% (5.4% of the total class). The respondents are therefore representative of the class.

11. Usefulness for teaching competencies Insight into the practical operations of an audit 84% Interpret large volumes of data (defined as the ability to understand and decipher data) 83% Relating theory in Auditing to real-life situations 81% Ability to summarise data 74% Integrating understanding of the different components of Auditing courses 73% Encouraging students to apply their theoretical knowledge to new and unique situations 71% Students were provided with 29 questions testing their opinion on the usefulness of the audit simulation in teaching various competencies, compared to lectures and traditional learning methods. They were able to answer on a four point Likert scale varying from �Not at all [useful]� to �Extensively [useful]�. The most important competencies learned by students from the simulation in order of usefulness (with the percentage of students perceiving the particular item to be either considerably or extensively useful in brackets) are as follows (in rank order): Providing insight into the practical operations of an audit (84%); The skills to interpret large volumes of data (defined as the ability to understand and decipher data) (83%); The skill to relate theory and technical knowledge in Auditing to real-life situations or practical problems (with complexities and ambiguous information) (81%); The ability to summarise data (74%); The ability to integrate the understanding of the different components of Auditing courses (73%); Encouraging students to apply their theoretical knowledge to new and unique situations (71%). The results of the 29 questions indicate that the main aims of the simulation, to integrate Auditing with Information Systems and making Auditing more practical were therefore realised.Students were provided with 29 questions testing their opinion on the usefulness of the audit simulation in teaching various competencies, compared to lectures and traditional learning methods. They were able to answer on a four point Likert scale varying from �Not at all [useful]� to �Extensively [useful]�. The most important competencies learned by students from the simulation in order of usefulness (with the percentage of students perceiving the particular item to be either considerably or extensively useful in brackets) are as follows (in rank order): Providing insight into the practical operations of an audit (84%); The skills to interpret large volumes of data (defined as the ability to understand and decipher data) (83%); The skill to relate theory and technical knowledge in Auditing to real-life situations or practical problems (with complexities and ambiguous information) (81%); The ability to summarise data (74%); The ability to integrate the understanding of the different components of Auditing courses (73%); Encouraging students to apply their theoretical knowledge to new and unique situations (71%). The results of the 29 questions indicate that the main aims of the simulation, to integrate Auditing with Information Systems and making Auditing more practical were therefore realised.

12. Potential benefits Students were given the opportunity to list the three most important benefits or advantages arising from using the audit simulation as a teaching and learning tool. It was an open-ended question and the results were summarised in similar groupings The other benefits (Table 4) mentioned by 22.7% of the respondents consisted of a wide variety of items, for example increased decision-making and time management skills and having fun. Students were given the opportunity to list the three most important benefits or advantages arising from using the audit simulation as a teaching and learning tool. It was an open-ended question and the results were summarised in similar groupings The other benefits (Table 4) mentioned by 22.7% of the respondents consisted of a wide variety of items, for example increased decision-making and time management skills and having fun.

13. Potential benefits (continued) Summary: Practical exposure to realistic audit situations and the practical application of theory Enhanced utilisation of Information Technology tools Integration of different subjects The benefits listed focused around three main areas: practical exposure to realistic audit situations and the practical application of theory; enhanced utilisation of Information Technology tools; and integration of different subjects. These benefits were the stated objectives of the course and students clearly felt that these were addressed. It is noteworthy that this was an open-ended question and students therefore gave their own opinions without prompting. This therefore validates the use of the audit simulation as a teaching tool. The benefits listed focused around three main areas: practical exposure to realistic audit situations and the practical application of theory; enhanced utilisation of Information Technology tools; and integration of different subjects. These benefits were the stated objectives of the course and students clearly felt that these were addressed. It is noteworthy that this was an open-ended question and students therefore gave their own opinions without prompting. This therefore validates the use of the audit simulation as a teaching tool.

14. Constraints Students were asked to list the three most significant constraints or obstacles that arose from the simulation and the results are summarised in the above table Unlike the benefits students perceived which focused around three benefits, students provided a wide variety of drawbacks. As discussed above, the major drawback of the simulation is the perceived amount of time invested in the audit simulation, with 120 (37.4%) of the respondents feeling that this is the major constraint of the simulation, confirming findings by Hoyle (1975:232). Another concern raised is that the objectives of the audit simulation are not clear and that insufficient background information is provided. The reason for this is unclear, as 50 out of the 63 respondents citing this as a problem indicated that they did read the instructions and course learning objectives before commencing with the simulation and that they had a clear understanding of the objectives of the simulation. It might very well be that the problem lies in the transference of theoretical knowledge to a practical scenario. Students were asked to list the three most significant constraints or obstacles that arose from the simulation and the results are summarised in the above table Unlike the benefits students perceived which focused around three benefits, students provided a wide variety of drawbacks. As discussed above, the major drawback of the simulation is the perceived amount of time invested in the audit simulation, with 120 (37.4%) of the respondents feeling that this is the major constraint of the simulation, confirming findings by Hoyle (1975:232). Another concern raised is that the objectives of the audit simulation are not clear and that insufficient background information is provided. The reason for this is unclear, as 50 out of the 63 respondents citing this as a problem indicated that they did read the instructions and course learning objectives before commencing with the simulation and that they had a clear understanding of the objectives of the simulation. It might very well be that the problem lies in the transference of theoretical knowledge to a practical scenario.

15. Constraints (continued) �Other� Repetition of work previously covered Too much Auditing knowledge required Class attendance important to know how to approach project Waste of time / boring / not difficult enough All the information is given in class - too little chance to take initiative Lack of practical experience Each situation in practice is different and unique Database too large and impractical Lack of Excel skills Class attendance did not appear to be the main problem as the percentage of students citing this as a problem that attended 50% or more of the theory and tutorial classes were only six percentage points less than the percentage for the group as a whole. Students (23.1%) listed a variety of constraints, shown as �other� in Table 6. These relate to administrative matters (e.g. language (Afrikaans) used in lectures) and do not warrant discussion. Class attendance did not appear to be the main problem as the percentage of students citing this as a problem that attended 50% or more of the theory and tutorial classes were only six percentage points less than the percentage for the group as a whole. Students (23.1%) listed a variety of constraints, shown as �other� in Table 6. These relate to administrative matters (e.g. language (Afrikaans) used in lectures) and do not warrant discussion.

16. Conclusion Goals of simulation met Valuable teaching tool Benefit students The results of the questionnaire discussed in section 6 validate the findings of similar studies as discussed in the literature review (section 3). It also adds various new insights into students� perceptions and experiences of the audit simulation in a South African context. A large portion of the students developed or improved their skills in interpreting and summarising large volumes of data. This is related to the derived benefit of enhancing their Information Technology skills. The simulation made a valuable contribution to students� studies, assisting them in conceptualising Auditing and made Auditing theory more practical, thereby addressing the main aims of the simulation. The students agreed that the simulation is an appropriate means of teaching and felt that it should be extended to other subjects. The time required to complete the simulation was noted as a major constraint. This is worth considering in developing similar study aids. Students were not always clear on the objectives of the simulation.The results of the questionnaire discussed in section 6 validate the findings of similar studies as discussed in the literature review (section 3). It also adds various new insights into students� perceptions and experiences of the audit simulation in a South African context. A large portion of the students developed or improved their skills in interpreting and summarising large volumes of data. This is related to the derived benefit of enhancing their Information Technology skills. The simulation made a valuable contribution to students� studies, assisting them in conceptualising Auditing and made Auditing theory more practical, thereby addressing the main aims of the simulation. The students agreed that the simulation is an appropriate means of teaching and felt that it should be extended to other subjects. The time required to complete the simulation was noted as a major constraint. This is worth considering in developing similar study aids. Students were not always clear on the objectives of the simulation.

17. lsteenkamp@sun.ac.zaQuestions?