Data Testing Techniques

1. Data Testing Techniques • Auditor obtains data file and uses program to examine the contents of the file and to produce reports • Common functions of computer software • Reading the contents of electronic data files • Calculation and summarization • Comparison - fields, records or files • Sorting and reorganizing files • Selection/Extraction data based on attributes • Selection of rep. samples - using stat sampling • Printing

2. GAS - generalized audit software • Only limited by lack of auditor familiarity • Most are easy to use • Read data files, convert to std format file for processing • Edit for unacceptable output and display control totals, browse data and analyze • Advantages • cheaper than custom • better than manually • Disadvantages • reliability must be proven • equipment/file constraints

3. Embedded audit modules • Build audit routines or modules into regular processing • e.g., extend a program that prints aged a/r listing to do confirmations • Good for paperless systems, tags transaction for analysis • Need to be involved in original design of system

4. Utility and custom written programs • Utility • Special function programs that can be performed by computer manufacturer library of programs • Main limitation is that not designed for audit • Include: sort, merge, copy, print, backup (speed backups), edit, recover, compression, virus detection, file transfer • Custom written by auditor • can perform exactly what is needed • have to be written specifically for certain hardware • Disadvantage - time cost and expertise to develop program, limited use and higher risk of error due to insufficient testing

5. Expert Systems • Computer programs that emulate problem solving knowledge and skill of human experts • Three components • knowledge base - conceptual equivalent of the data base, made up mostly of rules invoked by pattern matching (thousands) • inference engine - "heuristics”...rules of good judgement that characterize expert-level decision making in the fieldactivated by patterns in the knowledge base... this enables or fires some rules and disables others...problem processor finds the enabled rules and decides which one applies to obtain a solution...problems arise that every possible path is traced • user interface - how information is asked for and given

6. Neural networks • Try to solve problems by modeling the trial and error process of learning from experience • They can be trained to solve certain problems or identify a set of specific patterns • Learning algorithm adjusts the network by adjusting the weights among network components until it can correctly classify all presented facts (i.e., it becomes trained) • Major advantages • ease of system development, no need for if…then logic conditions • an easier alternative to rule based expert systems where pattern recog’n and classification involved • Disadvantages - • no facilities to determine how conclusion reached

7. Plan audit objectives and execution steps Select an appropriate technique Analyze accounting systems, data retention practices Arrange processing facilities and personnel Obtain and analyze data files Reformat / sort files if needed Select, summarize or extract data based on criteria Perform calcs on some/all of the data Select representative sample from sub-population Sort selected data into logical print sequence Print reports Review results for reasonableness Document results Profile of data testing application

8. Analytical Procedures • Ratio analysis • Statistical ratio analysis - id unusual and systematic fluctuations in ratios outside normal range of fluctuations; sets a confidence interval • stat ratio - calculate expected value and std deviation using past data; confidence interval based on acceptable range of variation • limitation - reliance on a normal dist'n as reference; if not normally distributed results may be incorrect or indefensible

9. Analytical Procedures • Pattern analysis - use several ratios to signal problem, construct confidence levels around each and determine whether significant fluctuations represent pattern • better screens out chance of fluctuations to avoid inappropriate conclusions • limitation • reliance on normal distributions • poor choice in pattern may prevent systematic shifts from getting to auditors att'n or lead to excessive investigation • Regression analysis - used to estimate relationships based on known values

10. Control Testing • Stat. selection from transaction stream for subsequent checking of attributes (approvals etc.); lots of manual time involved • Could use integrated audit facilities to provide evidence that key prog. procedures (edit and validation etc.) are being done

11. Detailed Testing • Key item testing • Cut-off work • Calcs etc. • A/R - select confirms, ageing , cash receipts • Inv - counts/price test selection, clerical accuracy, key items, negatives, cutoff • A/P - confirmation, footing, cutoff etc.

12. Def'n, design, execution and control of CAATs • Feasibility Considerations • significance of audit effort and time required • more efficient? • may be the only way to achieve audit objective • availability and sequence of data files • make sure available in readable form, • watch retention policies

13. Def'n, design, execution and control of CAATs • Feasibility Considerations • availability of facilities • make sure enterprise facilities are available, data files are readily usable, compatibility of software • availability of qualified staff • appropriate experience is needed • economic considerations - • need to be able to control audit costs, develop cost estimates, consider life cycle of the CAAT

14. Def'n, design, execution and control of CAATs • Definition, design, execution and control of CAATs • define audit objectives • what to prove - indicate type of test and test data to be used • selection of appropriate technique • based on objective, degree of audit assurance required, techniques that are available, cost/benefit factors • consider training, support, time constraints • design of input, processing and output requirements • expand conceptual ideas into detailed descriptions of application features (flowcharts, etc.)

15. Def'n, design, execution and control of CAATs • Design of input, processing and output requirements • audit software application flowcharts • understanding of inputs, file interaction and outputs • details of application logic - define before coding • reports and other output requirements • code tables - describe general purpose of codes, list each code and meaning and indicate procedures performed if an undocumented code is encountered • control points - make sure audit software works, control totals etc.

16. Def'n, design, execution and control of CAATs • Formulate test plans to make sure it works • critical step • use desk checking - detects logic problems • computer testing using prior or audit data to review and compare • Review and challenge application design

17. Execution and Control of CAATs • Select and arrange facilities and resources • determine whether audit software can be readily installed on client computer • id version of software needed • determine availability of supplies (disks etc.) • Desk Checking and Testing • coded specs should be tested • determine data availability • Running the applications • auditor should control processing and control outputs • reconcile control totals etc.

18. Execution and Control of CAATs • Audit documentation • update working papers to show final status • most GAS is self documenting to a degree • Application summary report • summarize whether objectives are met, significant problems encountered, special software features and other comments • File retention • client data should be maintained for at least year to permit year to year comparison

19. Audit Control over CAATs • Maintain to reduce errors such as • data records not match documentation • dropping intermediate files before finished processing • audit software not working properly • application logic not correct • Auditor should ensure results of the application will be reliable by choosing adequate controls to guarantee reliability • Control techniques selected will have a direct impact on work plans

20. Factors Affecting Control Environment • Nature of CAAT - • integrity of software, methodology for testing is well documented • watch reliance on any auditee software • concern over manipulation • test and do alternative testing to ensure integrity • Inherent risk • more stringent testing for higher risk items • Internal control • will impact extent of computer audit procedures that can be used • Assurance to be derived • more reliance on primary evidence  more stringent CAAT controls

21. Bonus Program Background Audit Objectives Documentation Company Policy Software Flowcharts Input (record layout) Process (black box) Output Organizing Audit Software Approach How to run the program When to stop What to hand in IDEA Case Study