Audit testing and Sampling methodology
Sampling is the process of selecting data from a population. By analyzing samples, characteristics of the entire population can be identified. Sampling is performed when it is not feasible to study the entire population due to time and cost constraints. Therefore, samples are a subset of the population.
Sampling is an integral part of audit execution as it allows auditors to efficiently evaluate the overall effectiveness of processes without the need to review every single item.
Sampling Types
This is a very important topic from a CISA exam perspective. Two or three questions can be expected on this topic. A CISA candidate should have an understanding of the sampling techniques discussed in the next subsections.
Statistical Sampling
This is an objective sampling technique. This is also known as non-judgmental sampling. It uses the laws of probability, where each unit has an equal chance of selection. In statistical sampling, the probability of error can be objectively quantified, and hence the detection risk can be reduced.
For example, suppose the total population is 100 and the auditor wants to select 10% as a sample. In statistical sampling, the auditor will use random sampling to select 10 accounts. This ensures that every account has an equal chance of being selected, minimizing selection bias.
Non-Statistical Sampling
This is a subjective sampling technique. It’s also known as judgmental sampling. The auditor uses their experience and judgment to select the samples that are material and represent a higher risk.
Attribute Sampling
Attribute sampling is the simplest kind of sampling based on certain attributes; it measures basic compliance. It answers the question, How many?. It is expressed as a percentage—for example, 90% complied. Attribute sampling is usually used in compliance testing.
Variable Sampling
Variable sampling offers more information than attribute sampling. It answers the question, How much?. It is expressed in monetary value, weight, height, or some other measurement—for example, an average profit of $25,000. Variable sampling is usually used in substantive testing.
Stop-or-Go Sampling
Stop-or-go sampling is used where controls are strong and very few errors are expected. It helps to prevent excess sampling by allowing the audit test to end at the earliest possible moment. Stop-or-go sampling is generally applied where controls are automated such as auto patch updates.
Discovery Sampling
Discovery sampling is used when the objective is to detect fraud or other irregularities. If a single error is found, the entire sample is believed to be fraudulent/irregular.
The following table summarizes the use cases for each sampling type:
|
Sampling Type |
When to Use |
|
Statistical |
When the question is about how the probability of error can be objectively quantified |
|
Non-Statistical |
When the question is about a technique where the experience and judgment of the auditor are required |
|
Attribute |
When the question is about the technique for compliance testing |
|
Variable |
When the question is about the technique for substantive testing |
|
Stop-or-Go |
When the question is about the technique to use when few errors are expected |
|
Discovery |
When the question is about the technique used to detect fraud |
Table 2.3: Different types of sampling
Note
Remember the term AC-VS—attribute sampling for compliance testing and variable sampling for substantive testing.
Sampling Risk
Sampling risk refers to the risk that a sample is not a true representation of the population. This implies that the conclusion drawn by analyzing the sample may be different from the conclusion that would have been drawn by analyzing the entire population.
Other Sampling Terms
A CISA candidate should be aware of the following terms related to sampling.
The Confidence Coefficient
A confidence coefficient, or confidence level, is a measure of the accuracy of and confidence in the quality of a sample. The sample size and confidence coefficient are directly related. A high sample size will give a high confidence coefficient.
Look at the following example:
|
Population |
Sample Size |
Confidence Coefficient |
|
100 |
95 |
95% |
|
50 |
50% |
|
|
25 |
25% |
Table 2.4: Example of confidence coefficient
In the case of poor internal controls, the auditor may want to verify 95 samples (sample size) out of a total population of 100. This gives a 95% confidence coefficient.
In the case of strong internal controls, the auditor may be satisfied with only 25 samples out of the total population of 100. This gives a 25% confidence coefficient.
Level of Risk
The level of risk can be derived by deducting the confidence coefficient from 100. For example, if the confidence coefficient is 95%, then the level of risk is 5% (100% – 95%).
Expected Error Rate
This indicates the expected percentage of errors in procession that may exist. When the expected error rate is high, the auditor should select a higher sample size.
Tolerable Error Rate
This indicates the maximum error rate that can exist without the audit result being materially misstated.
Sample Mean
The sample mean is the average of all collected samples. It is derived by adding all the samples and dividing the sum by the number of samples.
Sample Standard Deviation
This indicates the variance of the sample value from the sample mean.
Compliance versus Substantive Testing
A CISA candidate should be able to differentiate between compliance testing and substantive testing. They should be able to determine which type of testing is to be performed under different scenarios.
The Differences between Compliance Testing and Substantive Testing
The following table differentiates between compliance and substantive testing:
|
Compliance Testing |
Substantive Testing |
|
Compliance testing involves the verification of the controls of a process |
Substantive testing involves the verification of data or transactions |
|
Compliance testing checks for the presence of controls |
Substantive testing checks for the completeness, accuracy, and validity of the data |
|
In compliance testing, attribute sampling is preferred |
In substantive testing, variable sampling is preferred |
Table 2.5: Differences between compliance testing and substantive testing
Essentially, verifying whether a control is present or not is compliance testing. Meanwhile, verification of the complete process by testing the data/transaction to “substantiate” that the process is working is substantive testing.
Examples of Compliance Testing and Substantive Testing
The following examples will further help you understand the different use cases of compliance testing and substantive testing:
|
Compliance Testing |
Substantive Testing |
|
Checking for controls in router configuration |
Counting and confirming the physical inventory |
|
Checking for controls in the change management process |
Confirming the validity of inventory valuation calculations |
|
Verification of system access rights |
Counting and confirming the cash balance |
|
Verification of firewall settings |
Examining the trial balance |
|
Reviewing compliance with the password policy |
Examining other financial statements |
Table 2.6: Differences between the use cases of compliance testing and substantive testing
The Relationship between Compliance Testing and Substantive Testing
A CISA candidate should understand the following points about the relationship between compliance testing and substantive testing:
- Ideally, compliance testing should be performed first and should be followed by substantive testing.
- The outcome of compliance testing is used to plan for a substantive test. For instance, if the outcome of compliance testing indicates the existence of effective internal controls, then substantive testing may not be required or limited testing may be carried out. However, if the outcome of compliance testing indicates a poor internal control system, more rigorous substantive testing is required. Thus, the design of substantive tests is often dependent on the result of compliance testing.
- The attribute sampling technique is useful for compliance testing as it indicates that a control is either present or absent, whereas variable sampling will be useful for substantive testing.
Key Aspects for the CISA Exam
The following table covers important aspects from the CISA exam perspective:
|
Questions |
Possible Answers |
|
Which sampling technique should be used when the probability of error must be objectively quantified? |
Statistical sampling |
|
How can sampling risk be mitigated? |
By using statistical sampling |
|
Which sampling method is most useful when testing for compliance? |
Attribute sampling |
|
In the case of a strong internal control, should the confidence coefficient/sample size be increased or lowered? |
The confidence coefficient/sampling size may be lowered |
|
Which sampling method would best assist auditors when there are concerns of fraud? |
Discovery sampling |
|
How can you differentiate between compliance testing and substantive testing? |
The objective of compliance testing is to test the presence of controls, whereas the objective of substantive testing is to test individual transactions. Take the example of asset inventory:
|
|
What are some examples of compliance testing? |
|
|
What are some examples of substantive testing? |
|
|
In what scenario can the substantive test procedure be reduced? |
The internal control is strong/the control risk is within acceptable limits |
|
When is stratified sampling useful? |
Stratified sampling involves dividing the population into subgroups (strata) and then taking a sample from each subgroup. This approach is most appropriate when you want to focus on specific groups within the population. |
Table 2.7: Key aspects for the CISA exam
Apart from the appropriate sampling technique, another important aspect of the audit process is using appropriate evidence-gathering techniques. Audit evidence should be collected properly to establish its reliability. Details on the reliability of audit evidence and collection techniques are covered in the next section.
