INTRODUCTION TO NONSTATISTICAL SAMPLING FOR AUDITORS

1. INTRODUCTION TO NONSTATISTICAL SAMPLING FOR AUDITORS Jeanne H. Yamamura CPA, MIM, PhD

2. SITUATION • You are auditing the Dept. of Admissions & Records for Micronesia College. • One of your objectives is to verify that student records are being updated correctly and timely. • You decide to select a sample of grades posted from the most recent semester completed.

3. SITUATION • What would you normally document about this sample? • Sample size • Selection method • Population • Procedures to be performed • Purpose of test • What kind of test is this?

4. OBJECTIVES • Review of sampling concepts • Types of sampling - overview • Nonstatistical attribute sampling • Steps in applying • Additional coverage of: • Sampling methods • Compliance auditing

5. Applicable Professional Standards • SAS 39 Audit Sampling • SAS 111 Amendment to SAS 39 Audit Sampling • ISA 530 Audit Sampling

6. AUDIT SAMPLING • Application of an audit procedure to less than 100% of the items in a population • Account balance • Class of transactions • Examination “on a test basis” • Key: Sample is intended to be representative of the population. • Objective: To reach a conclusion about the population based on the sample items tested.

7. SAMPLING RISK • Possibility that the sample is NOT representative of the population • As a result, auditor will reach WRONG conclusion • Decision errors • Type I – Risk of incorrect rejection • Type II – Risk of incorrect acceptance

8. TYPE I – RISK OF INCORRECT REJECTION • Internal control: Risk that sample supports conclusion that control is NOT operating effectively when it really is • AKA – Risk of underreliance, risk of assessing control risk too high • Substantive testing: Risk that sample supports conclusion that balance is NOT properly stated when it really is

9. TYPE II – RISK OF INCORRECT ACCEPTANCE • Internal control: Risk that sample supports conclusion that control is operating effectively when it really isn’t • AKA – Risk of overreliance, risk of assessing control risk too low • Substantive testing: Risk that sample supports conclusion that balance is properly stated when it really isn’t

10. WHICH RISK POSES THE GREATER DANGER TO AN AUDITOR? • Type I - Risk of incorrect rejection • Efficiency • Type II - Risk of incorrect acceptance • Effectiveness • Auditor focus on Type II • Also provides coverage for Type I

11. NONSAMPLING RISK • Risk of auditor error • Sample wrong population • Fail to detect a misstatement when applying audit procedure • Misinterpret audit result • Controlled through • Adequate training • Proper planning • Effective supervision

12. SAMPLE SIZE FACTORS • Desired level of assurance (confidence level) • Acceptable defect rate (tolerable error) • Historical defect rate (expected error)

13. CONFIDENCE LEVEL • Complement of sampling risk • 5% sampling risk, 95% confidence level • How much reliance will be placed on test results • The greater the reliance and the more severe the consequences of Type II error, the higher the confidence level needed • Sample size increases with confidence level (decreases with sampling risk)

14. TOLERABLE ERROR AND EXPECTED ERROR • “Precision” – the gap between tolerable error and expected error • Expected population error rate = 1% • Auditor’s tolerable error rate = 3% • AKA Allowance for sampling risk • Sample size increases as precision decreases

15. WHEN DO YOU SAMPLE? • Inspection of tangible assets, e.g., inventory observation • Inspection of records or documents, e.g., internal control testing • Reperformance, e.g., internal control testing • Confirmation, e.g., verification of AR balances

16. WHEN IS SAMPLING INAPPROPRIATE? • Selection of all items with a particular characteristic, e.g., all disbursements > $100,000 • Testing only one or a few items, e.g., automated IT controls, walk throughs • Analytical procedures • Scanning • Inquiry • Observation

17. WALKTHROUGHS • Designed to provide evidence regarding the design and implementation of controls • Can provide some assurance of operating effectiveness BUT • Depends on nature of control (automated or manual) • Depends on nature of auditor’s procedures to test control (also includes inquiry and observation combined with strong control environment and adequate monitoring) • Walkthough = sample of 1

18. STATISTICAL VS NONSTATISTICAL SAMPLING • Statistical sampling • Statistical computation of sample size • Statistical evaluation of results • Nonstatistical sampling • Sample sizes should be approximately the same (AU 350.22) • Sample sizes must be sufficient to support reliance on controls and assertions being tested

19. WHEN IS SAMPLING NONSTATISTICAL? • If sample size determined judgmentally • If sample selected haphazardly • If sample results evaluated judgmentally

20. TYPES OF SAMPLING • Attribute sampling • Monetary unit sampling • Classical variables sampling

21. ATTRIBUTE SAMPLING • Used to estimate proportion of a population that possesses a specific characteristic • Most commonly used for T of C • Can also be used for dual purpose testing (T of C and Substantive T of T)

22. MONETARY-UNIT SAMPLING • AKA probability proportional to size (PPS) sampling, cumulative monetary unit sampling • Used to estimate dollar amount of misstatement

23. CLASSICAL VARIABLES SAMPLING • Uses normal distribution theory to identify amount of misstatement • Useful when large number of differences expected • Smaller sample size than MUS • Effective for both overstatements and understatements • Can easily incorporate zero balances

24. STEPS IN NONSTATISTICAL ATTRIBUTE SAMPLING APPLICATION • Planning • Determine the test objectives • Define the population characteristics • Determine the sample size • Performance • Select sample items • Perform the auditing procedures • Evaluation • Calculate the results • Draw conclusions

25. STEP 1: DETERMINE THE TEST OBJECTIVES • Objective for T of C: To determine the operating effectiveness of the internal control • Support control risk assessment below maximum (FS audit) • Identify controls to be tested and understand why they are to be tested

26. TESTS OF CONTROLS • Concerned primarily with • Were the necessary controls performed? • How were they performed? • By whom were they performed? • Appropriate when documentary evidence of performance exists

27. SUBSTANTIVE TEST OF TRANSACTIONS • Objective for S T of T: To determine whether the transactions contain monetary misstatements • Alternatively, to determine whether the system is operating as designed • Identify transactions to be tested and understand why they are to be tested

28. STEP 2: DEFINE THE POPULATION CHARACTERISTICS • Define the sampling population • Can be defined however desired BUT must include entire population as defined • Test population for completeness • Define the sampling unit • Determined by available records • Based on definition of population and audit objective • Define the control deviation conditions

29. STEP 3: DETERMINE THE SAMPLE SIZE • Consider desired confidence level, tolerable deviation rate, and expected population deviation rate • Judgmentally determine sample size • NOTE: Check against statistical sample size tables to verify adequacy

30. TOLERABLE RATE GUIDELINES

31. TOLERABLE RATE GUIDELINES

32. ESTIMATE OF POPULATION ERROR RATE Prior year results Preliminary sample Should be low – 0, 1% Higher rates increase sample size

33. STEP 3: DETERMINE THE SAMPLE SIZE • Guidelines for nonstatistical sample sizes for tests of controls • If any errors found, increase sample size or increase control risk (Probably not applicable to Public Auditor)

34. SMALL POPULATIONS AND INFREQUENTLY OPERATING CONTROLS

35. STEP 4: SELECT SAMPLE ITEMS • Random sample • Systematic sample (with random start) • Haphazard selection

36. RANDOM SELECTION • Every possible combination of population items has an equal chance of being included in the sample • Random number tables • Computer generation of random numbers

37. SYSTEMATIC SELECTION • Interval calculated and items selected based on size of interval • Interval = Population / Desired Sample Size • Starting point is random number within interval • Need to consider if bias present due to patterns in data

38. HAPHAZARD SELECTION • Selection by auditor without any conscious bias • If you select large, risky, or unusual items, it is NOT haphazard selection and it is NOT audit sampling. Instead – targeted or directed selection • Still desire representative sample • Avoid unusual, large, first or last • Useful for certain situations • Example: Tracing credits from AR to CR/other sources looking for fictitious credits • Less costly and simpler

39. STEP 5: PERFORM THE AUDITING PROCEDURES • Conduct planned audit procedures • What if? • Voided documents - if properly voided, not a deviation; replace with new sample item • Unused or inapplicable documents – replace with new sample item • Inability to examine sample item – deviation • Stopping test before completion – large number of deviations detected

40. STEP 5: PERFORM THE AUDITING PROCEDURES • Deviations observed • Investigate nature, cause, and consequence of every exception • Unintentional error? Or fraud? • Monetary misstatement resulted? • Cause – misunderstanding of instructions? Carelessness? • Effect on other areas?

41. STEP 6: CALCULATE THE RESULTS • No computed upper deviation rate (per table in statistical sampling) • Compute Calculated Sampling Error = Tolerable Error Rate – Sample Error Rate.

42. STEP 7: DRAW CONCLUSIONS • Control not effective (system not working as designed) if • Calculated Sampling Error too small • Depends on sample size used • Sample Error Rate > Tolerable Error Rate • Sample Error Rate > Expected Population Error Rate

43. COMPLIANCE AUDITING • Performance of auditing procedures to determine whether an entity is complying with specific requirements of laws, regulations, or agreements • Governmental entities and other recipients of governmental financial assistance • Compliance with laws and regulations that materially affect each major federal assistance program

44. COMPLIANCE AUDITING OF FEDERAL ASSISTANCE PROGRAMS • Definition of population for testing of an internal control procedure that applies to more than one program • Define items from each major program as a separate population, OR • Define all items to which control is applicable as a single population • Second choice usually more efficient

45. COMPLIANCE AUDITING - EXAMPLE • Federal financial assistance for Island City • Three major federal financial assistance programs • Four nonmajor programs • Control: Transaction review to ensure that only legally allowable costs are charged to each program

46. COMPLIANCE AUDITING - EXAMPLE • More efficient to select one sample from population of all transactions (major and nonmajor programs) • Confidence level = 95% • Tolerable deviation rate = 9% • Expected population deviation rate = 1% • Sample size: 51 • 1 allowable deviation

47. T of C versus S T of T • Test of Control • Verifies that a control is operating effectively • Substantive Test of Transactions • Verified that a transaction does not contain a misstatement

48. ASSERTIONS FOR CLASSES OF TRANSACTIONS • Occurrence: Transaction actually occurred and pertains to the entity (existence/validity) • Completeness: All transactions have been recorded • Accuracy: Amounts and other data have been recorded correctly

49. ASSERTIONS FOR CLASSES OF TRANSACTIONS • Cutoff: Transactions have been recorded in the correct accounting period • Classification: Transactions have been recorded in the proper accounts

50. CALCULATED SAMPLING ERROR • Tolerable error rate – Sample error rate = Calculated sampling error • Sample error rate = Population error rate due to sampling error • Auditor must evaluate calculated sampling error to see if it is big enough (sufficiently large to allow for sampling error in population)