Chapter 9

1. Chapter 9 Audit Sampling

2. Overview Audit sampling is defined as applying audit procedures to less than 100 percent of a population in order to estimate some characteristic about that population Typically, auditors sample to determine whether • A control procedure is operating effectively (test of controls) • An account balance is presented fairly (substantive test) • Fraud exists

3. Overview (continued) In some cases, sampling may not be the best approach Some audit procedures do not provide sufficient evidence when applied on a sample basis • Example: auditors read minutes of all BOD meetings to identify related party transactions • Reading the minutes of a sample of BOD meetings would not be sufficient Audit procedures that provide high quality evidence at low cost may be applied more extensively simply because its cheaper to test all items rather than sampling • Example: auditors typically confirm all bank account balances Account balances that are immaterial (or where the potential misstatement is immaterial) may not be worth sampling • Such accounts may be audited more efficiently with analytics

4. Overview (continued) From the results of sampling, the auditor makes an inference about the underlying population For this inference to be valid, the sampling units tested must be representative of the underlying population The auditor needs to make four important decisions to ensure the sample is representative and to control against making an incorrect inference: • Which population should be tested and for what characteristics? (population) • Sample size? • Which items should be included in the sample? (selection) • What inferences can be made from the sample? (evaluation)

5. What is non-sampling and sampling risk? When auditors draw an erroneous inference from sampling, the cause is either non-sampling or sampling risk Non-sampling Risk • Occurs when auditor does not appropriately carry out audit procedures or misinterprets results • Results from human error • Cannot be quantified • CPA firms try to minimize through quality control practices Sampling Risk • Occurs when sample is not representative of the underlying population • Can be controlled through sample size - as sample size increases, sampling risk decreases • If the sample is 100% of the population, sampling risk is zero; however, this is often not practical

6. Discuss Sampling Risks Related to Tests of Controls If the sample is not representative of the population, the auditor may draw an incorrect conclusion about the effectiveness of a control: Auditor assesses control risk too high: • Sample indicates control is worse than it really is • As a result, the auditor does not rely on the control and does more substantive testing than necessary • Assessing control risk too high does not directly affect audit quality, but does lead to audit inefficiencies Auditor assesses control risk too low: • Sample indicates control is better than it really is • As a result, the auditor relies on an ineffective control (without realizing it's unreliable) and substantive testing is not rigorous as it should be • This increases the risk that material misstatements are not found and an incorrect audit opinion issued

7. Review Sampling Risks Related to Substantive Testing If the sample is not representative of the population, the auditor may draw an incorrect conclusion about whether an account balance is presented fairly: Incorrect acceptance • Sample indicates account balance is not materially misstated when it is • Auditor may issue unqualified opinion on materially misstated statements • Because of the potential costs associated with incorrect acceptance, auditors control for this risk

8. Review Sampling Risks Related to Substantive Testing (Continued) Incorrect rejection • Sample indicates account balance is materially misstated when it isn't • There are things that reduce this risk • Before telling client to adjust its books, auditor usually performs additional tests • If client believes account balance is correct, client will ask auditor to perform more tests • These increase probability that incorrect rejection will be discovered • Incorrect rejection affects the efficiency of the audit, but does not affect the fairness of the audited financial statements

9. Discuss Selecting a Sampling Approach Auditors use both statistical and non-statistical sampling techniques Non-statistical sampling • Auditor judgment used to determine sample size, sample selection, and evaluate sample results • Does not provide objective way to control and measure sampling risk • Because its subjective, results are less defendable in legal proceedings • -ay take less time to perform • Frequently used in audits of small clients

10. Discuss Selecting a Sampling Approach (Continued) Statistical sampling • Allows auditor to statistically design an efficient sample, measure sufficiency of evidence, and evaluate sample results • Provides quantified measures of control procedure failure rates, amount of error in account balances, and sampling risk • Requires precise definitions of acceptable risk and sample objectives • Requires knowledge of statistical sampling methods • Efficient method for testing large populations

11. Review Testing Controls and Compliance If an auditor believes a control is effective and plans to rely on that control, s/he must test the control to see if it is operating effectively Attribute estimation sampling and discovery sampling are the statistical methods frequently used to test controls In this context, an attribute is the characteristic that indicates the control is working effectively • Example: the organization requires all sales on account be approved by the credit manager • Approval is evidenced by the manager's initials on the sales invoice • The manager's initials are the attribute • The auditor would examine sales invoices and look for the initials

12. Explain Attribute Estimation Sampling The appropriate sample size depends on a number of factors including: Statistical Risk (Risk of assessing control risk too low) • Risk of concluding controls are effective when, in fact, they are not • Means auditor relies on an ineffective control without realizing it • The lower the risk, the larger the sample size

13. Explain Attribute Estimation Sampling (continued) Tolerable failure rate • Failure rate at which auditor will determine the control is not operating effectively • Based on the importance of the control • If a control is crucial, the tolerable failure rate is set at low level • The lower the tolerable failure rate, the larger the sample size Expected failure rate • Based on auditor's experience with the client • The higher the expected failure rate, the larger the sample size

14. What is attribute estimation sampling as an audit objective? The steps to implement an attribute estimation sampling plan are: • Identify the attribute to be tested and define conditions of failure • Define the population to be tested including the period covered by the test, sampling unit, and ensuring population is complete • Determine appropriate sample size • Determine effective and efficient method of selecting the sample • Select and audit sample items • Evaluate sample results and reach conclusion on audit objectives • Document all phases of the sampling plan

15. Explain Attribute Estimation Sampling - Sample Size The appropriate sample size depends on a number of factors including statistical risk, and the tolerable and expected failure rates Other issues: Multiple Attributes • Auditors frequently test several attributes using the same set of source documents • While the sampling risk should be the same, the tolerable and expected failure rates may differ between controls • The result is a different sample size for each control • There are several approaches to select items for the sample Small Populations (Appendix) • - If the sample is a large portion of the population, auditor may be able to reduce the sample size • - Use a finite adjustment factor

16. Review Attribute Estimation Sampling - Sample Selection Once the appropriate sample size has been determined, the auditor must decide how to select sample Random-based methods eliminate the possibility of unintentional bias in the selection process and help ensure the sample is representative • - Random number - efficient selection method if there is an easy way to relate random numbers to the population • Examples: sales invoice number, purchase order number • Computer programs typically used to generate random numbers

17. Review Attribute Estimation Sampling - Sample Selection (continued) • Systematic selection - selects every nth item in the population from a randomly selected starting point • Sampling interval (n) is determined by dividing population size by desired sample size • To use this method, auditor must be sure there is not a systematic pattern of failures in the population • Haphazard selection (non-statistical method) • Arbitrary selection • Not random based • Judgmental sampling (non-statistical method) • Auditor may use judgment to select sample • Not random based

18. Comment on Attribute Estimation Sampling - Evaluate Sample Results The auditor projects the results of sampling to the population before drawing a conclusion If the sample failure rate is no greater than the expected failure rate, the auditor can conclude the control is as effective as expected If the sample failure rate exceeds the expected failure rate, the auditor must determine whether the projected maximum failure rate is likely to exceed the tolerable failure rate • To do this, the auditor must determine the upper limit of the potential failure rate in the population • The upper limit is based on the sample failure rate and sample size and is adjusted upward for sampling error

19. Attribute Estimation Sampling - Evaluate Sample Results(continued) If the upper limit exceeds the tolerable failure rate, the internal control process has deficiencies The auditor should either • Test a compensating control (if available) • Increase the rigor of the subsequent substantive testing The auditor should also evaluate • The nature of the control procedure failures (pattern of error) • The effect of such failures on potential financial statement misstatement

20. Attribute Estimation Sampling - Evaluate Sample Results (Continued) When control failures are found, they should be analyzed qualitatively as well as quantitatively Auditor should try to determine whether the failures • Were intentional or unintentional • Were random or systematic • Had a direct dollar effect

21. Discuss Searching for Fraud Discovery sampling may be used to help identify potential fraud • Tolerable rate is set very low and expected rate is set at zero percent • Results in large sample size At any point, if evidence of just one potential fraud is found, the auditor stops sampling and starting investigating to determine if fraud actually occurred

22. Basic steps: Specify audit objective of the test Define misstatement Define population (and sampling units) Choose sampling method Determine sample size Select sample Audit selected items Evaluate sample results Perform follow-up work as necessary Document sampling procedure and results Sampling to Test for Account Balance Misstatements (Substantive Testing)

23. Define Specify Audit Objective Sampling always relates to one specific procedure usually testing one specific assertion Specifying the audit objective determines the population to test • For example: • If objective is to determine existence, the sample should be selected from recorded information • - On the other hand, if the objective is to determine completeness, the sample should be selected from a complementary population such as source documents

24. Define Misstatements Misstatements should be defined before sampling to • Preclude auditor from rationalizing away misstatements as isolated events • Provide guidance to the audit team Misstatement is usually defined as difference that affects the correctness of the overall account balance

25. Define the Population Group of items in an account balance that the auditor wants to test • Does not include: • Items the auditor has decided to examine 100% • Items that will be tested separately Important to properly define the population: • Sample results can be projected only to the group from which the sample is selected • The population must be directly related to the audit objective

26. Define the Sampling Unit • Sampling units are the individual auditable elements that make up the population • Example: sampling units for confirming accounts receivable could be the individual customer's balance or individual unpaid invoices

27. Identify Individually Significant Items Many account balances are comprised of a few large dollar items and many smaller items Dividing a population into two or more subgroups based on dollar amount can increase audit efficiency • Items in excess of a specified dollar amount (top stratum items) are examined 100% • Items less than the specified amount (lower stratum items) are sampled This process (stratification) allows the auditor to examine a significant portion of an account balance even though s/he examines a relatively few items

28. Discuss Choosing a Sampling Method There are a number of sampling methods an auditor may use Non-statistical Probability proportional to size (PPS) Classical sampling methods (not covered in this text) • Mean-per-unit • Ratio estimation • Difference estimation

29. More Depth on Choosing a Sampling Method The sampling methods differ in a number of ways: Measure of sampling risk • Statistical methods provide an objective measure of sampling risk • Non-statistical methods do not provide such a measure Tests for account balance • PPS is designed to test for overstatement of an account balance • Classical methods test for both overstatement and understatement Statistical estimates • PPS provides an estimate of the amount of misstatement in the account • Classical methods provide an estimated range of the account balance Sample selection • PPS is a dollar-based approach; each dollar is a sampling unit • Classical samples are selected using a variety of sampling units

30. Even More on Choosing a Sampling Method Use of PPS would be appropriate if • Auditor is testing for overstatements in an account balance • A dollar-based sampling approach increases the probability of selecting overstated items • Few or no misstatements expected • Individual book values (like a subsidiary ledger) are available One of the classical methods would be appropriate if the auditor • Is concerned about understatements in an account balance • Expects numerous misstatements • Is examining an account balance based on estimates rather than a total of individual items • Is trying to estimate an account balance

31. Explain Determining Sample Size, Selecting Sample, Evaluating Results Sample size, method of selecting the sample, and the approach to evaluating sample results all depend on the sampling method used Whichever sampling method is used, consideration must be given to the risk of misstatement, sampling risk, and the auditor's assessment of tolerable and expected misstatement • Tolerable misstatement • Maximum misstatement an auditor will accept before deciding the recorded account balance is materially misstated • Expected misstatement • Based on results of other substantive tests and auditor's prior • experience with the client • Expected misstatement should be less than tolerable misstatement

32. What is non-statistical sampling? Determine sample size • All significant items should be tested • No way to mathematically control sampling risk Select the sample • Sample must be representative of population • Could use random-based method or haphazard selection Evaluate sample results • Project misstatements to the population • Consider sampling error • Make judgment as to whether account is likely to be materially misstated

33. Define Probability Proportional to Size (PPS) Sampling • Dollar-based sampling approach where the population is the number of dollars in the account balance examined • Using dollars as sampling units means larger dollar items in the account balance are more likely to be selected in the sample • PPS is an effective sampling approach when the auditor is testing for overstatements • Appropriate when few misstatements are expected and individual book values are available

34. What is probability proportional to size (PPS) sampling - TD risk? • To use PPS, the auditor must determine the allowable risk of the sample failing to detect a material misstatement (test of details risk) and tolerable and expected misstatements for the account balance • Test of Details Risk • Detection risk is the risk that the substantive audit procedures will fail to detect material misstatements • There are two types of substantive audit procedures - those that use sampling, and other (non-sampling) substantive procedures • Test of details (TD) risk is the part of detection risk related to sampling; the risk that substantive sampling procedures will fail to detect a material misstatement • Other substantive procedures risk (OSPR) is the risk that the non-sampling procedures will fail to detect a material misstatement

35. Probability Proportional to Size (PPS) Sampling The relation between TD risk and inherent and control risks and OSPR is inverse • High inherent risk means the auditor is examining transactions that are susceptible to misstatement • High control risk means the client controls are weak • High OPSR means the non-sampling audit procedures are not effective in detecting material misstatements In each of these situations, the auditor would want to be more careful with his/her sampling procedures • The auditor would want lower TD risk; less chance of failing to detect material misstatements with sampling procedures Lower TD risk means the auditor wants a lower risk of sampling procedures failing to detect material misstatements To achieve this lower risk of failing to detect, the sample size must increase

36. Probability Proportional to Size (PPS) Sampling - Sample Size • PPS samples are usually selected using a fixed interval sampling approach • The sampling interval (I) is calculated as I = TM - (EM x EEF) RF TM = Tolerable misstatement EM = Expected misstatement EEF = Error expansion factor RF = Reliability factor • Error expansion and reliability factors are based on TD risk • Sample size (n) is computed by dividing the account book value by the sampling interval n = Population Book Value Sampling Interval

37. Probability Proportional to Size (PPS) Sampling - Sample Selection Sample items are often selected using a fixed interval approach • Every Ith dollar after a random start • A random start is required to give every dollar in the population an equal chance to be included in the sample The first sample item is the one that first causes the cumulative total (cumulative book value + random start) to equal or exceed the sampling interval Successive sample items are those first causing the cumulative total to equal or exceed multiples of the interval Sample composition: • All top stratum items will be included in the sample • Lower stratum items will be sampled

38. Discuss Probability Proportional to Size - Zero or Negative Balances Items with zero balances have no chance of being selected using PPS • If evaluation is necessary, zero balance items should be audited as a different population Two approaches to deal with population items with negative balances: • Exclude them from the selection process and test them as a separate population • Include them in the selection process and ignore the negative sign

39. Review Probability Proportional to Size - Sample Evaluation Based on sample results, the auditor computes the upper misstatement limit Upper misstatement limit (UML) • Maximum dollar overstatement that might exist in the population • Given the misstatements detected in the sample • At the specified TD risk level • UML is the sum of three components: • Basic precision • Most likely misstatement • Incremental allowance for sampling error.

40. Review Probability Proportional to Size - Sample Evaluation Evaluation: • If the UML is less than the tolerable misstatement, the account balance is considered fairly presented • If the UML exceeds the tolerable misstatement, the account balance is not fairly presented