The effect of control risk assessment on the use of CAATs : evidence from a big 4 firm

(1)

The effect of control risk assessment on the use of CAATs:

evidence from a big 4 firm

Name: Aatish Jageswar Student number: 11371250

Thesis supervisor: Edo Roos Lindgreen Date: June 25, 2018

Word count: 10463

MSc Accountancy & Control, Accountancy track

(2)

Statement of Originality

This document is written by student Aatish Jageswar who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(3)

Abstract

Computer-aided audit tools (CAATs) is believed to enhance the efficiency of the auditing tasks. Recent research indicates that the adoption of CAATs is still low nowadays (Subaryani binti Zainol et al., 2017; Smidt, Van der Nest & Lubbe, 2014; Mustapha & Jin Lai, 2017; Ahmi et al., 2016). This is despite the rise of complex ERP systems and transformation of IT-environments, which enables auditors to apply CAATs more easily (Vasile-Daniel, 2010; Chen et al., 2014; Vogel et al., 2017). Existing research also indicates that auditors’ use of CAATs tend to differ by control risk assessment. Auditors tend to apply CAATs more in low control risk conditions (Janvrin, Bierstaker & Lowe, 2009). To gain more insights in the determinants to use CAATs, this research will assess whether auditors’ application of CAATs (after 10 years) still differ by control risk assessment. I hypothesize that auditors still tend to apply CAATs different in various control risk conditions and auditors therefore gravitate away from manual procedures (Janvrin et al., 2009) despite the indications of recent research. By measuring the perceived usefulness of CAATs with a survey within a Big 4 auditing firm in the Netherlands, this research will provide (empirical) evidence that the application of CAATs (to the extent that perceived usefulness leads to actual appliance) ten years later still vary by control risk assessment. Post-hoc analysis indicated that auditors even tend to apply computer-related audit procedures with CAATs more in high control risk situations. Which is in contrast with the previous findings from Janvrin et al. (2009). This research provides evidence that the relationship between control risk assessment and the use of computer-related audit procedures with CAATs has developed to a relationship where auditors prefer to apply CAATs more in high control risk situations.

(4)

Acknowledgement

This master’s thesis serves as the crown on my work of the past 5 months. In this period I learned a lot, on a personal level and especially in the scientific area. In this acknowledgement I want to mention the actors that played a significant role in bringing this thesis to a successful end.

My internship supervisor and buddy from the big 4 services firm where I conducted my research provided grateful guidance in the set up and distribution of the survey that is used in this research. They ensured that the right and proper amount of respondents participated in the survey in order to get a robust statistical analysis.

At last I want to give a special thanks to my thesis supervisor Edo Roos Lindgreen from the Amsterdam Business School for his supervision over the thesis process. He provided grateful comments and guidance in the periodically feedback sessions which enabled me to bring this thesis to a successful end.

(5)

1 Introduction

New technologies gained rapid popularity within companies and this trend hasn’t come unnoticed for audit practices. Computer-aided audit tools (CAATs) are often argued as the means for improvement of the efficiency and effectiveness of audit practices. Certain standards even encourage the use of CAATs like SAS no.99 (AICPA, 2002b, p. 1734) and HRA standaard 315 (NBA, 2017). Since technology continues to play a more significant role in businesses (Altahtooh & Emsley, 2014; Flyvbjerg & Budzier, 2011), the complexity of those technologies will rise additionally. Which will force auditors to reform their audit gathering techniques, whereby CAATs will play a prominent role in enforcing the audit efficiency (Shamsuddin et al., 2015).

The purpose of this research is to investigate the relationship between control risk assessment and the way audit procedures are performed. In 2009 it became known that there was a relationship between control risk assessment and the use of computer-related audit practices (Janvrin, Bierstaker & Lowe, 2009). Auditors tend to make more use of computer-related audit procedures (with CAATs) when the control risk is assessed as low. These authors proposed that this relationship would develop over time since businesses’ IT-environments also would became more complex over time. This would thus mean that auditors gravitate away from manual (traditional) procedures. Present research already indicated that businesses’ IT-environments have developed over time (Vogel et al., 2017). This research will examine whether auditors’ application of computer-related audit procedures with CAATs, ten years later and in different (more complex) IT-environments, still vary by control risk assessment. If the application of computer-related procedures with CAATs still vary, this would indicate that auditors at least in one control condition gravitate away from traditional audit procedures. Present research indicates that auditors increase their audit fees in response to higher levels of control risk (Hogan & Wilkins, 2008). To the extent that these fees are a proxy for audit effort (Han et al., 2016), this suggest that auditors increase their effort when the control risk is assessed higher. These findings are in line with the findings of Janvrin et al. (2009), auditors tend to apply more traditional audit procedures when the control risk is assessed higher. Traditional audit procedures require more manual tasks and are therefore more time-consuming causing higher audit fees and effort.

Although CAATs can automate these traditional audit procedures (Darono & Panggabean, 2015), the adoption of CAATs is still low in general (Subaryani binti Zainol et al., 2017; Smidt,

(7)

Van der Nest & Lubbe, 2014; Mustapha & Jin Lai, 2017; Ahmi et al, 2016). Notwithstanding, some argue the opposite (Jordan Lowe et al., 2018). This research aims to gain more insights into the determinants for auditors to make use of CAATs nowadays. Existing research indicates that performance expectancy is one of the main determinants in order to adopt CAATs within the workspace of auditors (Pedrosa, Costa & Laureano, 2015; Mansour, 2016; Bierstaker, Janvrin & Jordan Lowe, 2014). Since the adoption of CAATs is still low while new technologies gained rapid popularity, this might indicate that auditors prefer the traditional approach and don’t gravitate away from manual procedures as proposed by Janvrin et al. (2009). This led to an examination in this paper whether the use of CAATs still vary by assessed control risk levels. Existing research already made clear that perceived usefulness is a root construct of performance expectancy (Venkatesh et al., 2003). These findings and assumptions engendered the following research question:

RQ: Does the auditors’ perceived usefulness of Computer Assisted Audit Techniques (CAATs) vary by control risk assessment?

(8)

2 Literature review

Braun and Davis (2003) defined CAATs as any use of technology to assist in the completion of an audit. With CAATs, auditors can utilize various computer tools and techniques such that an auditor can automatically perform certain audit routines (Darono & Panggabean, 2015). CAATs can be classified in 7 types of features as identified by Pedrosa & Costa (2014), namely: “database queries, ratio analysis, audit sampling, digital analysis, data mining: regression/ANOVA, data mining: classification and working papers on auditing” (p. 2). CAATs are “always used for data analysis, data acquisition and operational analysis” (Ghani, Azizi Ismail & Zabedah Saidin, 2016, p. 37). New types of features are also emerging nowadays, like the use of drones to evaluate inventory existence (Appelbaum & Nehmer, 2017). CAATs have gained a significant role in financial audit practices (Pedrosa & Costa, 2012). Therefore more (empirical) insight is needed in the determinants of auditors to make use of CAATs.

2.1 Computer-related audit procedures with CAATs

Previous research identified computer-related audit procedures within the auditing standards (Janvrin et al., 2009). Within these computer-related audit procedures, certain audit procedures can be performed with CAATs as mentioned by the US standard setters, but those procedures are not compulsory. Most of these standards are also covered in the Dutch accounting standards (HRA):

Table 2.1 Computer-related audit procedures with CAATs within the auditing standards

Use CAATs to: Reference in SaS. Reference in HRA

Evaluate Fraud risks AU 316.52 HRA 240, par. A37

Identify journal entries and other adjustments to be tested AU 316.62 HRA 315, par. A94 Check accuracy of electronic files AU 104.38 HRA 330, par. A16 Re-perform procedures (i.e. aging of accounts receivable) AU 104.39 Not present Select sample transactions from key electronic files AU 327.19 HRA 330, par. A16 Sort transactions with specific characteristics AU 327.19 HRA 330, par. A16 Test an entire population instead of a sample AU 327.19, 327.61 HRA 330, par. A16 Obtain evidence about control effectiveness AU 327.27 HRA 330, par. A27 Evaluate inventory existence and completeness AU 316.54 Not present Evaluate transactions with new related entities Not present HRA 550, par. A36 Evaluate physical observed inventory records Not present HRA 240, par. A40 bijlage 2

(9)

The research of Janvrin et al. (2009) indicated that these (optional) computer-related audit procedures with CAATs are more used when the assessed control risk is below the maximum. This would might indicate that auditors find the use of CAATs not reliable or useful to find material financial misstatements and that they do rely on a traditional approach in situations of heightened risks of material misstatements. Vasile-Daniel concluded in 2010 that indeed a significant number of financial auditors conduct their audits with a traditional approach despite the rise of complex ERP systems.

2.2 Perceived usefulness of new technology

A lot of research about the related variables is present in the area of accepting new technologies among the perceived users (Thompson et al, 1991; Davis et al., 1992; Moore & Benbasat, 1991 and Compeau et al., 1999). All these findings stood on the basis regarding the development of a comprehensive technology acceptance model, named as the Unified Theory of Acceptance and Use of Technology model (UTAUT) (Venkatesh et al., 2003). In this unified model, performance expectancy appeared to be the strongest predictor of intention to use the new technology. Performance expectancy refers to the degree to which an individual believes that using the tool will help him, or her better achieve desired outcomes (Venkatesh et al., 2003). This predictor consists of five root constructs, namely perceived usefulness, extrinsic motivation, job-fit, relative advantage and outcome expectations. Out of these root constructs, perceived usefulness appeared to be the strongest construct of performance expectancy. In the extension of this technology acceptance model (UTAUT 2), performance expectancy remained a significant effect on the determination of accepting new technologies (Venkatesh et al., 2012).

According to Davis (1989), perceived usefulness can be interpreted as: “the degree to which a person believes that using a particular system would enhance his or her job performance.” (p.320). Recent research also indicates that perceived usefulness of a particular IT system has a positive relationship with actual use of that same IT system (Barnett et al., 2015). Park et al. (2014) examined the acceptance of (new) teleconferencing systems among employees, perceived usefulness seemed to be the key predictor of intention to use the system. Existing evidence also suggests that performance expectancy is significant with the behavioral intention to adopt a new technology (Martin & Herrero, 2012). This research will provide empirical evidence of the perceived usefulness of CAATs, by measuring the perceived usefulness of

(10)

certain prescribed audit procedures to be performed with CAATs in different assessed control risk situations.

2.3 Adoption of CAATs

Nowadays, basic accounting tasks can already be performed highly automated (Nassauer, 2016). Technology is already developed to a level that tasks beyond these basic bookkeeping tasks can be automated (Huerta & Jensen, 2017). E-business practices (IT-environments) are nowadays also developed into service-orientated practices, which also led to innovation in information system usage (Vogel et al., 2017). This innovation enables auditors to rely on internal controls within the environment (Chen et al., 2014). This transformation in IT-environment should enable auditors’ CAATs usage (Alles, Kogan & Vasarhelyi, 2008; Janvrin et al., 2009).

Braun and Davis (2003) studied the perception from auditors of certain types of CAATs, their research indicated that generalized audit software was the most frequently used technique, and the auditors seemed to perceive the potential benefits that came by with using CAATs. Although generalized audit software was then the most frequently used technique, in 2012 this type of CAATs was still not widely used when conducting audits (Ahmi & Kent). Further research also indicated that the use of CAATs while performing auditing work is still low nowadays (Subaryani binti Zainol et al., 2017; Smidt, Van der Nest & Lubbe, 2014; Mustapha & Jin Lai, 2017; Ahmi et al., 2016). Auditors tend to feel uncomfortable with issuing an audit opinion on financial statements based on audit evidence generated with CAATs (Oladele & Agochukwu, 2016), indicating that CAATs appeared to be lacking performance within the auditors’ workspace. Although, findings also exist that nowadays auditors’ overall IT use of auditors have increased the past ten years (Jordan Low et al., 2018).

New emerging features of CAATs has new challenges to overcome for auditors. According to Pedrosa and Costa (2014), present research indicates that techniques relevant to data mining are not as expressive as their apparent relevance on auditors’ work could predict. Those new features also have challenges to overcome in order to be properly adopted by chartered accountants, for example the lack of standardization of the data which would take too much time to provide the promised audit efficiency. Because of the retained utilization of CAATs, more recent research is needed in order to evaluate the performance expectancy of CAATs within the auditors’ workspace.

(11)

Mansour (2016) identified the two key drivers (factors) of external auditors to adopt CAATs using the earlier discussed UTAUT model by Venkatesh et al. (2003, 2012). Those two key drivers are performance expectancy and facilitating conditions. These findings are in line with the findings of Pedrosa, Costa and Laureano (2015) who identified that performance expectancy and firm influence are the two main motivations to adopt CAATs within the workspace of auditors. Furthermore, since auditors tend to feel uncomfortable with issuing an audit opinion on financial statements based on audit evidence that is generated with CAATs (Oladele & Agochukwu, 2016), this also might indicate that performance expectancy plays a significant role in adopting CAATs. Since perceived usefulness is a root construct of performance expectancy (Venkatesh et al. 2003), this research will provide insights in auditors’ performance expectancy of CAATs, by measuring whether auditors perceive the usefulness different in various control risk conditions.

2.4 Control risk assessment

A commonly used model for assessing audit risk is the audit risk model. This model proposes that the audit risk is a function of the inherent risk, control risk and detection risk (AICPA, 1983). Inherent risk and control risk are determined by an assessment of the client (Hogan & Wilkins, 2008). Inherent risk is the possibility that serious misstatement in financial statements is present in conditions regarding absence of internal control (Chang et al. 2008). Control risk is the risk that the internal control of the auditee could not immediately prevent or detect the risk of serious errors (Chang et al. 2008). Detection risk needs to be reduced to an acceptable level when the inherent risk and control risk appear to be significant in this model. The audit risk model is expressed as 𝐴𝐴𝐴𝐴 = 𝐼𝐼𝐴𝐴 ∗ 𝐶𝐶𝐴𝐴 ∗ 𝐷𝐷𝐴𝐴. Where AR is the audit risk, CR is the control risk and DR represents the detection risk.

As can be seen in the model, audit risk needs to be held at an acceptable level. When the control risk appears to be high, detection risk needs to be reduced to keep the audit risk acceptable. Detection risk can be reduced with the use of substantive testing (Hogan & Wilkins, 2008). Since CAATs can optimize substantive testing procedures (Darono & Panggabean, 2015), this would reduce the total effort needed to conduct the audit. This would mean that CAATs will more likely to be used when the control risk is assessed as high. Although existing research indicates that CAATs is more used in low control risk conditions (Janvrin et al. 2009). As mentioned in paragraph 2.3, the performance expectancy of CAATs is still low. The lack of performance from CAATs is a potential explanation about why auditors prefer to use

(12)

CAATs more in low control risk conditions. After all, a material misstatement is less likely to be found when the control risk is assessed as low. As mentioned before, present research already indicated that auditors are insecure about drawing results on audit evidence generated with CAATs (Oladele & Agochukwu, 2016), this would cause auditors to perform more manual tasks when the control risk is assessed as high in order to feel more secure about the generated audit evidence.

(13)

3 Research method

This research will measure the perceived usefulness of certain audit procedures that prescribe the use of CAATs in the Dutch accounting standards (HRA), similar to the research of Janvrin et al. (2009) who applied the audit procedures that prescribe the use of CAATs in the Statements on Auditing Standards (SaS). With the use of a survey held within a big 4 auditing firm, this research will examine whether auditors’ CAATs usage vary by control risk assessment.

3.1 Hypothesis development

Whereas Janvrin et al. (2009) measured the actual use of the audit procedures, this research will measure the perceived usefulness in order to find an answer on the following implication for future research by Janvrin et al.:

“Moreover, if auditors tend to rely more on client controls in complex IT environments, future

research could investigate if the relationship between control risk assessments and audit procedures has strengthened over time. Auditors may increasingly gravitate away from manual audit procedures and reconsider the balance of manual and computerized audit procedures that is most appropriate for each client.” (pp. 12-18)

Recent research indicated that CAATs are still not widely used (Subaryani binti Zainol et al., 2017; Smidt, Van der Nest & Lubbe, 2014; Mustapha & Jin Lai, 2017; Ahmi et al., 2016) despite the fact that IT-environments have developed over the past years (Vogel et al., 2017) and auditors rely more on client controls in their IT-environments (Chen et al., 2014). This would indicate that auditors are not gravitating away from manual procedures and that the particular relationship has been vanished over time. By measuring perceived usefulness, this research will provide evidence whether the use of computer-related audit procedures with CAATs still vary by control risk assessment or not. Client IT-environments nowadays enable CAATs usage more (Chen et al., 2014; Vogel et al., 2017). Thus the presence of a relationship between control risk assessment and the use of computer-related procedures with CAATs nowadays would indicate that auditors have gravitated (or are still gravitating) away from manual procedures (according to the assertions of Janvrin et al. 2009) in at least one control condition, despite indications from existing research.

To determine whether the variable computer-related audit procedures with CAATs has an influence on the variable control risk assessment, auditors’ perceived usefulness of applying

(14)

those procedures will be measured. Davis’ interpretation of perceived usefulness already indicated that it can be interpreted as: “the degree to which a person believes that using a particular system would enhance his or her job performance.” (1989, p. 320). Recent research also indicates that perceived use of a particular IT system has a positive relationship with actual use of that same IT system (Barnett et al., 2015). Since perceived usefulness is the strongest construct of performance expectancy (Davis, 1989; Venkatesh et al., 2003) and performance expectancy is significant with the behavioral intention to adopt a new technology (Martin & Herrero, 2012), I hypothesize the following:

H1: The perceived usefulness of computer-related audit procedures with CAATs vary by control risk assessment.

The stated hypothesis (H1) contains the research (alternative) hypothesis. Thus in this research, the null hypothesis would be that the perceived usefulness of computer-related audit procedures with CAATs doesn’t vary by control risk assessment.

3.2 Variables

In order to investigate whether perceived usefulness of certain computer-related audit procedures with CAATs differ by the assessed level of control risk, the following variables will be measured:

Dependent Variable: Perceived usefulness of computer-related audit procedures with CAATs

Perceived usefulness of computer-related audit procedures with CAATs (PU) is the independent variable that will be measured, this will be measured with the use of a survey based on the identified computer-related audit procedures with CAATs in the regulatory standards (section 2.1) after control risk is assessed. The perceived usefulness of the audit procedures that are part of responding to the assessed risks (HRA, chapter 330) and identifying transactions with related entities (chapter 550) will be measured. These procedures will be measured for perceived usefulness on a 5-point Likert-scale.

Independent Variable: Assessed control risk of the audit

The assessed control risk of the audit is the dependent variable (CR). Control risk will be measured differently than the research of Janvrin et al. (2009). This research will measure control risk with the definition of the risk that control policies and procedures that are present in the client environment wouldn’t detect or prevent an error of misstatement as a whole

(15)

(Bhattacharjee, Maletta & Moreno, 2016). Whereas Janvrin et al. (2009) measured control risk as a consequence of the assessed client IT complexity. Perceived usefulness of computer-related audit procedures with CAATs in this research will be measured in three levels of control risk (low, intermediate and high). I hypothesize that auditors’ perceived usefulness of computer-related audit procedures still differ by control risk assessment.

3.3 Measurement of variables

The audit procedures that will be used to measure the independent variable (PU) are procedures in the Dutch accounting standards that specifically advice making use of CAATs. For this research, only the procedures where the control risk is already assessed will be surveyed in order to determine whether auditors’ perceived usefulness of CAATs differ by control risk assessment. To compare the results of this research with the findings of Janvrin et al. (2009), the procedures mentioned in their (peer-reviewed) research and that are present in the Dutch accounting standards will be applied. After control risk is assessed, the following audit procedures (table 3.1) as identified by Janvrin et al. (2009) are present in the Dutch accounting standards that advice the use of CAATs (and one procedure identified in this research that’s only present in the Dutch accounting standards as shown in section 2.1):

Items Reference in standard

1 Check accuracy of electronic files HRA 330, par. A16 2 Select sample transactions from key electronic files

HRA 330, par. A16

3 Sort transactions with specific characteristics

HRA 330, par. A16

4 Test an entire population instead of a sample

HRA 330, par. A16

5 Obtain evidence about control effectiveness HRA 330, par. A27 6 Evaluate transactions with new related entities HRA 550, par. A36

Table 3.1 Audit procedures that advice the use of CAATs in Dutch standards (after control risk is assessed)

These procedures are present in three different paragraphs in de standards of the Dutch accounting board (NBA), namely “de Handleiding Regelgeving Accountancy” (HRA): HRA 330, par. A16

(16)

CAATs when there are signals present that indicate a material misstatement. The auditor can check for the accuracy of the electronic files over a larger population when CAATs will be used (item 1). Also, the auditor can select sample transactions from key electronic files which will help to investigate those signals (item 2). Furthermore, this standard advices to sort transactions with specific characteristics in order to test the data more thoroughly (item 3). When there are signals present that indicate a material misstatement, the auditor can make use of CAATs to sort transactions that have those characteristics that can cause the material misstatement. The possibility to test an entire population instead of a sample is also

mentioned, to rule out undetected material misstatements (item 4). HRA 330, par. A27

This standard mentions the use of CAATs to test the effectiveness of internal controls (item 5). When there’s no documentation present about the working of the internal controls (for example the process of assigning authorization), the auditor can make use of CAATs to determine whether certain transactions took place with the proper authorization. HRA 550, par. A36

When the entity that is audited has a (new) relationship with another entity, the auditor needs to test the transactions made with that entity. In order to test those transactions, the standard mentions that CAATs can help the auditor with testing (item 6).

3.4 Research setting

The hypothesis will be tested through an online survey within the auditing department of a Big 4 auditing firm in the Netherlands. The use of CAATs is highly facilitated within this firm; online training and manuals are available within the intranet of the firm. For every audit activity they perform, the auditor needs to file whether CAATs have been used. The auditors will be surveyed through a set of questions that will take them between 3 to 5 minutes to complete. In order to stimulate the auditors to participate in the survey, for every valid survey a donation will be made to the World Wide Life Fund. The questionnaire is included in Appendix A. Since this questionnaire is held within a Big 4 services firm, I expect that auditors consider themselves well trained to make use of CAATs according to the findings of Janvrin et al (2009). Since auditors will be asked to fill in their perceived usefulness for the three control risk conditions and the responses are on an interval measurement scale (Boone & Boone, 2012), responses to the questionnaire will be analyzed with the use of repeated-measures ANOVA.

(17)

3.5 Preliminary testing

The questionnaire consists of the same computer-related audit procedures with CAATs as was tested in the peer-reviewed research of Janvrin et al. (2009). This research will use the same procedures but will measure perceived usefulness as a scale which will lead to the actual appliance of those computer-related audit procedures (Venkatesh et al., 2003). To ensure the questionnaire for internal reliability, the questionnaire is pilot tested for reliability among a focus group with the same background as the testing group (active in the accounting field). The focus group consisted of 11 participants, who were asked to fill in the survey online. Due to the evaluation of the first three respondents, the survey was modified by adding a hyperlink to the relevant accounting standard in the questions and adding a clarification of the conception of control risk. The survey was again completed by these three respondents and further distributed among the pilot group. From the pilot group, 3 participants were part of the sample group, the other participants were not employed by the firm but are active in the audit profession. Among the three control risk conditions (Low, Intermediate and High), the Cronbach’s alpha was calculated to ensure measurement reliability of the variables.

Table 3.2 Pilot group – Internal consistency

Control Condition Cronbach's α N of items (Audit procedures)

Low control risk 0.913 6

Intermediate control risk 0.849 6 High control risk 0.889 6

For the three control conditions (Low, Intermediate and High), the results of Cronbach’s α is displayed in panel A. According to Gliem and Gliem (2003), internal consistency is excellent when Cronbach’s α is higher than 0.9 and good when it is between 0.8 and 0.9. For low control risk, the internal consistency is excellent (α ≥ 0.9) which means that for every audit procedure, the perceived usefulness that is measured for low control risk conditions are internally consistent. The internal consistency of the control conditions intermediate and high control risk are good (0.9 > α ≥ 0.8). This means that for every audit procedure, the perceived usefulness that is measured for intermediate and high control risk conditions are internally consistent. These results indicate that the survey consists of internally consistent questions.

(18)

4 Analysis

The survey has been sent out to two business units of the audit firm, auditors were asked to fill in the questionnaire by e-mail. The e-mail contained a link to the online questionnaire where participants could answer the questions anonymously. In order to determine a reliable analysis, auditors with less than one year experience and responses where all statements were answered with 3 (neither useless nor useful) will be excluded of the analysis (in order to eliminate response bias). The total valid responses collected is 83.

4.1 Demographics, experience and training

Demographics are displayed in table 4.1, statistics about experience and training are displayed in table 4.2. The respondents consisted of 58 males (69.9%) and 25 females (30.1%). The most of these respondents, 72,3 % (n = 60) fall in the age range of 25-34. The mean of experience as an external auditor is 6 years. The statement whether auditors considered themselves well trained to make use of CAATs is answered with a mean of 4 (somewhat agree). Within the respondents, the most respondents had an (highest) educational level of post master’s degree or higher (44). The mean group of educational level is a master’s degree (university). The group of respondents that have a certification consisted of 36 respondents, where 47 of the respondents had no certification. Of these respondents with a certification, 35 have a CPA certification or similar (of which is also the mean group).

Table 4.1: Demographics

Gender Frequency (%) Age Frequency (%)

Male 58 (69.9) 18-24 9 (10.8)

Female 25 (30.1) 25-34 60 (72.3)

Rather not say 0 ≥ 35 14 (16.8)

Total 83 (100) Total 83 (100)

Table 4.2: Experience & Training

Experience & Training Groups (frequencies) Mean

Years of experience as an external auditor • 1-2 years (15) • 3-5 years (34) • 6-10 years (16) • 10 years or more (18) 6 Years

(19)

4.2 Descriptive statistics

Participants indicated their perceived usefulness of the identified computer-related audit procedures with CAATs on a scale from 1 to 5, where a score of 1 indicates that they perceive the procedure as extremely useless and a score of 5 as extremely useful. For every audit procedure that is questioned, the mean is calculated in low, intermediate and high control risk conditions. The descriptive statistics are displayed in table 4.3.

Table 4.3: Descriptive statistics

Control conditions N Mean Median Mode St. dev

LOW_CR 83 3.4 4 4 0.7

INT_CR 83 3.6 4 4 0.7

HIGH_CR 83 3.8 4 4 0.8

When the control risk is assessed as low (LOW_CR), perceived usefulness of applying the CAATs procedure has a mean of 3.4 (median and mode of 4). Indicating that auditors perceive the usefulness of computer-related audit procedures with CAATs useless nor useful. In intermediate (INT_CR) and high control risk (HIGH_CR) conditions, perceived usefulness has a mean of respectively 3,6 and 3,7. Both these conditions have a median and mode of 4, indicating that auditors perceive the usefulness in the control risk conditions as slightly useful.

Well trained to make use of CAATs

• Strongly disagree (2) • Somewhat disagree (8)

• Neither agree nor disagree (11) • Somewhat agree (48) • Strongly agree (14) Somewhat agree Education level • Bachelor’s degree, applied sciences (6) • Bachelor’s degree, university (6) • Master’s degree, applied sciences (0) • Master’s degree, university (27) • Post master’s degree

or higher (44)

Master’s degree (University)

Certification • Yes (36)

• No (47) No

Type of certification • • CPA or similar (35) Other than CPA (CISA, RE, CMA, etc.) (1)

(20)

Overall, the usefulness of computer-related audit procedures with CAATs has a mean of 3.6 (with a mode and mean of 4), indicating that auditors perceive computer-related audit procedures overall slightly useful.

Table 4.4 shows the Pearson Correlation Matrix, coefficients indicated with an asterisk (*) are significant with a level of 1 percent (2-tailed).

Table 4.4 Pearson Correlation Matrix

EXP AGE EDU CERT GEN TRAIN_ CAATS PU_ LOW PU_ INTER PU_ HIGH

EXP 1 AGE 0.684* 1 EDU 0.328* 0.275 1 CERT -0.582* -0.460* -0.039 1 GEN 0.081 0.093 -0.136 -0.061 1 TRAIN_CAATS 0.010 -0.113 0.057 -0.112 0.021 1 PU_LOW 0.077 0.034 -0.088 -0.068 -0.240 -0.037 1 PU_INTER 0.068 0.103 0.002 0.003 -0.205 -0.090 0.727* 1 PU_HIGH 0.069 0.148 0.070 0.022 -0.158 -0.180 0.395* 0.872* 1 Proxy Meaning EXP Experience AGE Age

EDU Educational level

CERT Certification

GEN Gender

TRAIN_CAATS Training to make use of CAATs

PU_LOW Perceived usefulness when control risk is assessed as low

PU_INTER Perceived usefulness when control risk is assessed as intermediate

PU_HIGH Perceived usefulness when control risk is assessed as high

Large significant (negative) correlation exists within the age of the auditors (AGE) and experience of the auditors (EXP). Indicating that older auditors have more auditing experience. There’s also a large significant correlation within perceived usefulness in low control risk conditions (PU_LOW) and intermediate control risk conditions (PU_INTER), indicating that auditors tend to apply CAATs the same in these both conditions. Within perceived usefulness in intermediate control risk conditions (PU_INTER) and high control risk conditions (PU_HIGH) an even larger significant correlation exists, indicating that auditors also tend to apply CAATs the same in these conditions.

(21)

Table 4.5 shows the scores per item (audit procedure). For every item, the mean, median and mode of the perceived usefulness score is calculated over the three control risk conditions (low, intermediate and high).

Item N Mean Median Mode St. dev Reference in _standard 1 Check accuracy of electronic _files 83 3.9 4 4 0.8 HRA 330, par. A16 2 Select sample transactions _{from key electronic files} 83 3.8 4 4 0.8 HRA 330, par. A16 3 Sort transactions with specific _{characteristics} 83 4.1 4 4 0.8 HRA 330, par. A16 4 Test an entire population _{instead of a sample} 83 3.4 4 4 1.2 HRA 330, par. A16 5 Obtain evidence about control _{effectiveness} 83 3.1 3 3 1.2 HRA 330, par. A27 6 Evaluate transactions with _{new related entities} 83 3.2 3 4 1.1 HRA 550, par. A36

Table 4.5 Scores per item

Items 1, 2 and 3 appear to score higher (slightly useful) than the other items (4, 5 and 6). All these procedures are part of the standard HRA 330, par. A16. Thus, these mean scores indicate that auditors perceive the appliance of CAATs within this standard more useful than the other standards (with the exception of item 4, which is also part of the same standard).

4.3 Hypothesis testing

In order to examine whether the perceived usefulness of computer-related audit procedures with CAATs vary by control risk assessment, repeated-measures ANOVA will be applied. Repeated-measures ANOVA will test whether auditors perceive the usefulness of computer-related audit procedures different in the three levels of control risk conditions that are surveyed. In this analysis, the three control risk conditions are recoded into 1 (low), 2 (intermediate) and 3 (high). The scores of the perceived usefulness in the different control risk conditions will be analyzed in SPSS. Before the results can be interpreted, the variance between the groups needs to be equal (Sphericity). To test whether the scores violates this assumption, Mauchly’s Test of Sphericity will be applied (with a confidence level of 95%). Results of Mauchly’s Test of Sphericity are displayed in table 4.6.

(22)

Table 4.6: Mauchly’s Test of Sphericity

With χ(2) = 0.197, results indicate a significant p-value (p < 0.05), which means Sphericity is not present among the scores. Next step in the analysis is to determine whether the scores of perceived usefulness differ in the three control risk conditions (CR_condition).

Tests of Within-Subjects Effects

source

Type III sum of

squares df Mean Square F Sig.

Partial Eta squared

CR_condition Sphericity assumed 4.687 2 2.343 12.145 0.000 0.129 Greenhouse-Geisser 4.687 1.110 4.224 12.145 0.000 0.129

Huynh-Feldt 4.687 1.114 4.208 12.145 0.000 0.129

Lower-bound 4.687 1 4.687 12.145 0.001 0.129

Error Sphericity assumed 31.646 164 0.193

(CR_condition) Greenhouse-Geisser 31.646 90.983 0.348

Huynh-Feldt 31.646 91.329 0.347

Lower-bound 31.646 82 0.386

Table 4.7: Tests of Within-Subjects Effects

Results of the Within-Subjects Effects tests are shown in table 4.7. Mauchly’s test already indicated that Sphericity was not present among the scores, therefore Greenhouse-Geisser and Feldt tests will be applied. When the p-value of the Greenhouse-Geisser and Huynh-Feldt tests appear to be significant, variation within the perceived usefulness scores in the various control risk conditions can be assumed and the research hypothesis can be accepted. Greenhouse-Geisser test indicates a p-value of 0.000 with F(2, 164) = 12.145. Huynh-Feldt test indicates also a p-value of 0.000 with F(1.114, 91.329) = 12.145. Both tests appear to have a significant p-value, therefore H1 is accepted. Auditors’ perceived usefulness indeed differs by control risk assessment. Partial Eta squared (ηp2) indicates a value of 0.129. Which means that the variance in the perceived usefulness scores is for 12,9% explained by the difference in control risk conditions.

Mauchly’s Test of Sphericity Within subjects

effect Mauchly’s W Approx. Chi-Square df sig. Greenhouse-Geisser Huynh-Feldt Lower-bound

(23)

4.4 Post-hoc analysis

Hypothesis testing already concluded that auditors perceive the usefulness of computer-related audit procedures with CAATs different by control risk level. A post-hoc analysis will be applied in order to gain more insights into the differences of the perceived usefulness between the assessed control risk conditions. Pairwise comparisons would indicate in what control risk conditions auditors perceive the appliance of computer-related audit procedures with CAATs more useful. Results of the pairwise comparisons are shown in table 4.8.

Pairwise Comparisons

95% confidence interval for difference

(I)

CR_condition (J) CR_condition Mean Difference (I-J) Std. Error Sig. ** Lower bound Upper bound

1 2 -0.187* 0.057 0.004 -0.325 -0.048 3 -0.335* 0.094 0.002 -0.564 -0.107 2 1 0.187* 0.057 0.004 0.048 0.325 3 -0.149* 0.045 0.004 -0.257 -0.040 3 1 0.335* 0.094 0.002 0.107 0.564 2 0.149* 0.045 0.004 0.040 0.257

* Significant at the 0.05 level

**Adjustment for multiple comparisons: Bonferroni

Table 4.8: Pairwise Comparisons

Pairwise comparisons show significant mean differences across the three control risk levels (95% confidence). Results show that the mean difference of CR_condition level 1 (low) is negative with both level 2 (intermediate) and level 3 (high). This indicates that auditors perceive the usefulness of computer-related audit procedures more in intermediate and high control risk conditions than in low control risk conditions. The mean difference of

CR_condition level 2 with level 3 is -0.149. This means that auditors perceive the usefulness of computer-related audit procedures more in high control risk conditions than in

intermediate control risk conditions. 4.5 Robustness analysis

In order to assure that the results are robust, an analysis is made whether there is a presence of multicollinearity. There’s a high Pearson correlation between PU_INTER, PU_LOW and PU_HIGH (table 4.4), which might indicate the presence of multicollinearity. To test for

(24)

multicollinearity, the variance inflation factor (VIF) is calculated. When the VIF is higher than 10 (O’Brien, 2007) and the tolerance value is lower than 0.1 (Maas, 2009), this would indicate that serious collinearity is present within the variables. Results are shown in table 4.9.

Table 4.9: Collinearity statistics

Collinearity Statistics

Proxy* Tolerance VIF Proxy** Tolerance VIF Proxy*** Tolerance VIF PU_LOW 0.472 2.118 PU_HIGH 0.240 4.169 PU_LOW 0.844 1.185 PU_INTER 0.472 2.118 PU_INTER 0.240 4.169 PU_HIGH 0.844 1.185

* Dependent variable is PU_HIGH **Dependent variable is PU_LOW ***Dependent variable is PU_INTER

Collinearity statistics show a variation inflation factor of 2.118 (tolerance is 0.472) when the dependent variable is PU_HIGH. When PU_LOW is the dependent variable, PU_HIGH and PU_INTER have a variation inflation factor of 4.169 (tolerance is 0.240). A variation inflation factor of 1.185 (tolerance is 0.844) is shown when the dependent variable is PU_INTER. Collinearity statistics indicate that there is no presence of multicollinearity among the variables. 4.6 Results

Descriptive statistics indicate that auditors perceive the appliance of computer-related audit procedures with CAATs overall as slightly useful (mean of PU_LOW is 3.4, PU_INTER is 3.6 and PU_HIGH is 3.8). Furthermore, auditors seem to perceive the computer-related audit procedures of item 1, 2 and 3 more useful than the rest of the items (4, 5, and 6).

The hypothesis as mentioned in chapter 3 is tested with the use of repeated-measures ANOVA. After the test for Sphericity is conducted, results of repeated-measures ANOVA conclude that the research hypothesis is accepted with a p-value of 0.03 (confidence level of 95%). Thus, auditors indeed tend to apply computer-related audit procedures with CAATs different in various control risk conditions. Furthermore, the model indicates that 12.9% of the variance in the perceived usefulness scores is explained by the variable control risk conditions. As a rule of the thumb, this is a medium effect (Pierce, Block & Aguinis, 2004).

The Post-hoc analysis is made with the use of pairwise comparisons. Pairwise comparisons conclude that auditors perceive the usefulness of computer-related audit procedures the most in high control risk conditions and less in intermediate and low control risk conditions. These differences are all significant with a confidence level of 95%.

(25)

The results appear to be robust, no multicollinearity is detected among the variables despite the high observed Pearson correlation coefficient between PU_HIGH and PU_INTER (0.872), and PU_LOW and PU_INTER (0.727). Collinearity statistics indicate a variation inflation factor of 2.118 (tolerance is 0.472) when the dependent variable is PU_HIGH, when the dependent variable is PU_LOW the variation inflation factor is 4.169 (tolerance is 0.240) and when the dependent variable is PU_INTER the variation inflation factor is 1.185 (tolerance is 0.844).

(26)

5 Discussion

In this research, I examined whether auditors perceive the usefulness of computer-related audit procedures with CAATs different in various control risk conditions. Previous research from Janvrin et al. (2009) indicated that auditors applied computer-related audit procedures with CAATs more in low control risk conditions. The results of that research also indicated that the appliance of CAATs overall was still low. This research examines assertions made in that research in order to gauge the perception from auditors of CAATs nowadays.

Within a big 4 auditing firm in the Netherlands, a survey was held among external auditors where their perceived usefulness of CAATs was questioned for certain procedures in the Dutch auditing standards that mentioned the use of CAATs in different control risk conditions. Based on previous literature (Subaryani binti Zainol et al., 2017; Smidt, Van der Nest & Lubbe, 2014; Mustapha & Jin Lai, 2017; Ahmi et al., 2016), indications arose that auditors still prefer the traditional approach and thus auditors not gravitate away from manual audit procedures. These indications are contrary to the assertions made in the research by Janvrin et al. (2009), which predicted that auditors would gravitate away from manual audit procedures if the relationship between control risk assessment and computer-related procedures would develop over time. I hypothesized that the particular relationship was still present, based on the assertions made by Janvrin et al. (2009).

Analysis of the results accepted the research hypothesis, auditors perceive the usefulness of computer-related audit procedures with CAATs different in the various control risk conditions. The existence of this relationship indicates that to the extent that perceived usefulness leads to actual usage) auditors tend to gravitate away from traditional audit procedures (in high control risk conditions). The results also appeared to be robust. The post-hoc analysis concluded that auditors tend to apply computer-related audit procedures with CAATs more in high control risk conditions rather than low control risk conditions. These results are in contrast with the research of Janvrin et al. (2009), who indicates that auditors prefer to apply computer-related audit procedures with CAATs more in low control risk conditions. Results of this research indicate that auditors are now more comfortable with expressing an audit opinion based on audit evidence generated with CAATs, which is in contrast with the assertions made by Oladele and Agochuckwu (2016). Auditors seem to depend on CAATs to find material misstatements according to the tension of auditors to apply CAATs more in high control risk conditions.

(27)

Also, the overall score of perceived usefulness of computer-related audit procedures with CAATs has a mean of 3.6. Indicating that auditors perceive the appliance of computer-related audit procedures with CAATs nowadays slightly useful. Thus, this research indicates that the appliance (to the extent that perceived usefulness leads to the actual appliance) of CAATs has risen in popularity. To my knowledge, this is the first research that provides this particular evidence.

With the findings, one can say that the adoption of CAATs nowadays is prosperous after the slow adoption in the early stages. This is in line with recent research of Jordan Lowe et al. (2018), who concluded that overall IT use of auditors has increased in the past ten years. Based on these findings, it seems that the adoption of CAATs is prosperous after (at least) 15 years (Braun & Davis, 2003; Ahmi & Kent, 2012). This timespan seems to be relatively long comparing to technology adoption within other sectors. For example in mobile e-commerce, evidence exists that consumers adopted the technology within ten years (Einvav et al., 2014). Also, social media usage among internet increased significantly within a decade (Perrin, 2015). Remarkably, ERP systems were already widely adopted in 2012 (El Amrani & Saint-Léger, 2013). Where ERP systems enable the opportunity to apply CAATs more easily (Darono & Panggabean, 2015; Vasile-Daniel, 2010), and are essential to evaluate the internal control framework within the audit client. The lack of alignment between auditors’ perceived importance and their knowledge of the technologies (Azizi Ismail & Zainol Abidin, 2009; Jordan Lowe et al., 2018) might be a potential explanation why the adoption of CAATs within the auditors’ workspace did not follow the same trend as ERP adoption.

Employees of audit firms still believe their firm needs to innovate more to remain competitive (CPA.com, 2015). CAATs are now improving and techniques like artificial intelligence and data analytics would enable auditors to apply continuous auditing more easily (Kokina & Davenport, 2017). It would be advised for audit firms to adopt these new techniques early in order to diffuse later innovations (Compagni, Mele & Ravasi, 2015). Evidence exists that the implementations from early adopters of new techniques constantly delivers technical and/or economic benefits. Whereas late adopters are more likely to innovate as a sequence of isomorphic pressures (Compagni et al., 2015).

(28)

6 Limitations and implications for future research

Although the results of this research appeared to be robust in the analysis, limitations of the study are present. My research measures perceived usefulness instead of actual usage of computer-related audit procedures with CAATs. Existent research indicates that perceived usefulness is a strong determinant of the actual use of that technology, but this is not the only determinant (Venkatesh et al., 2003). Further research can investigate whether other determinants are also dependent on control risk assessment, which is not unthinkable because of the medium effect size in this research (12,9%).

The survey used in this research defined control risk in three levels (low, intermediate and high). Because of the high Pearson correlation coefficients between low and intermediate control risk conditions, and between intermediate and high control risk conditions, the possibility exists that participants do not cope all these levels the same. Even though, this would be in contrast with the audit risk model as discussed in section 2.4. For example, one might perform the same audit tasks for low and intermediate control risk situations, where others might alter their intensity.

Also, this survey questioned only audit procedures where the use of CAATs is mentioned in the auditing standards. The possibility exists that auditors perform other audit procedures with CAATs that are not measured in this research. Further research can investigate whether other audit procedures are also used with CAATs.

Results also indicated that auditors seem to perceive the computer-related audit procedures of items 1, 2 and 3 more useful than the other items that were questioned. Further research can investigate why auditors seem to perceive the usefulness of those computer-related audit procedures more. Since the standard HRA 330 par. A16 provides guidance about the procedures to perform against identified fraud risks, auditors may prefer the use of CAATs when fraud is involved.

Furthermore, this research is held within a big 4 services firm in the Netherlands. Thus the results of this research may not be generalizable. Even though recent research indicates that big 4 auditors’ IT use not significantly differ from non-big 4 auditors (Jordan Lowe et al., 2018). Possibility also exists that the results are not generalizable to auditors from another big 4 services firms. Further research can investigate whether these results are also applicable to auditors from other (non-)big 4 services firms.

(29)

References

Ahmi, A. and Kent, S. (2012) The utilisation of generalized audit software (GAS) by external auditors, Managerial Auditing Journal, 28(2), pp. 88-113.

Ahmi, A., Saidin, S. Z., Abdullah, A., Ahmad, A. C. and Ismail, N. A. (2016) State of Information Technology Adoption by Internal Audit Department in Malaysian Public Sector, International Journal of Economics and Financial Issues, 6(S7), pp. 103-108 Altahtooh, U. A. and Emsley, M. W. (2014) Is a challenged project one of the final outcomes

for an IT project?, 2014 47th Hawaii International Conference on System Sciences, 1, pp. 4296-4304.

Alles, M. G., Kogan, A. and Vasarhelyi, M. A. (2008) Audit automation for implementing continuous auditing: Principles and problems, Ninth International Research Symposium

on Accounting Information Systems Paris, France.

American Institute of Certified Public Accountants (2002) Statements on auditing Standards. Retrieved from https://www.aicpa.org/research/standards/auditattest/sas.html

Appelbaum, D. and Nehmer, R. A. (2017) Using Drones in Internal and External Audits: An Exploratory Framework., Journal of Emerging Technologies in Accounting, 14(1), pp. 99-113.

Barnett, T., Pearson, A. W., Pearson, R. and Kellermans, F. W. (2015) Five-factor model personality traits as predictors of perceived and actual usage of technology, European

Journal of Information Systems, 24(4), pp. 374-390.

Bhattacharjee, S., Maletta, M .J. and Moreno, K. K. (2016) The Role of Account Subjectivity and Risk of Material Misstatement on Auditors' Internal Audit Reliance Judgments.

Accounting Horizons. 30(2), pp. 225-238.

Bierstaker, J., Janvrin, D. and Jordan Lowe, D. (2014) What factors influence auditors' use of computer-assisted audit techniques?, Advances in accounting, 30(1), pp. 67-74.

(30)

Braun, L. and Davis, H. E. (2003) Computer-assisted audit tools and techniques: analysis and perspectives, Managerial Auditing Journal, 18(9), pp. 725-731.

Boone, H. N. and Boone, D. A. (2012) Analyzing Likert Data, Journal of Extension, 50(2), pp. 1-5.

Chang, S. I., Tsai, C. F., Shih, D. H. and Hwang, C. L. (2008) The development of audit detection risk assessment system: Using the fuzzy theory and audit risk model, Expert

Systems with Applications, 35, pp. 1053-1067.

Chen, Y., Smith, A. L., Cao, J. and Xia, W. (2014) Information Technology Capability, Internal Control Effectiveness, and Audit Fees and Delays, Journal of Information

Systems, 28(2), pp. 149-180.

Compagni, A., Mele, V. and Ravasi D. (2015) How Early Implementation Influence Later Adoptions of Innovation: Social Positioning and Skill Reproduction in the Diffusion of Robotic Surgery, Academy of Management Journal, pp. 1-70.

Compeau, D. R., Higgins, C. A. and Huff, S. (1999) Social Cognitive Theory and Individual Reactions to Computing Technology: A Longitudinal Study, MIS Quarterly, 23(2), pp. 145-158.

CPA.com (2015) Innovation in Public Accounting Survey, The Digital CPA Conference, pp. 1-14. Retrieved from http://media.aicpastore.com/Publication/cpa/Innovation-In-Public-Accounting-Survey_dec2015.pdf

Darono, A. and Panggabean, T. (2015) Examining the Adoption of Computer-Assisted Audit Tools and Techniques for Tax Audits in Indonesia: An Institutional Logics Perspective,

2015 International Workshop on Computer Auditing Education.

Davis, F. D. (1989) Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology, MIS Quarterly, 13(3), pp. 319-340.

Davis, F. D. (1992) Extrinsic and Intrinsic Motivation to Use Computers in the Workplace,

(31)

Einvav, L., Levin, J., Popov, I. and Sundaresan, N. (2014) Growth, Adoption, and Use of Mobile E-Commerce, American Economic Review: Papers & Proceedings 2014, 104 (5), pp. 489-494.

El Amrani, R. and Saint-Léger, G. (2013) Etat des lieux de la recherché ERP francophone,

Systémes d’information & management, 2 (13), pp. 111-160.

Flyvbjerg, B. and Budzier, A. (2011) Why Your IT Project May Be Riskier than You Think,

Harvard Business Review, 89(9), pp. 601-603.

Ghani, R., Azizi Ismail, N. and Zabedah Saidin, S. (2016) Adoption of Computer-Assisted Audit Tools and Techniques (CAATTs): An Exploratory Study in Audit Firms,

Proceedings of the International Conference on Accounting Studies (ICAS) 2016, pp.

35-40.

Gliem, J. A. and Gliem, R. R. (2003) Calculating, Interpreting, and Reporting Cronbach’s Alpha Reliability Coefficient for Likert-Type Scales, 2003 Midwest Research to

Practice Conference in Adult, Continuing, and Community Education pp. 82-88.

Han, S., Rezaee, Z., Xue, L. and Zhang, J. H. (2016) The Association between Information Technology Investments and Audit Risk, Journal of Information Systems, 30(1), pp. 93-116.

Hogan, C. E., & Wilkins, M. S. (2008). Evidence on the Audit Risk Model: Do Auditors Increase Audit Fees in the Presence of Internal Control Deficiencies? Contemporary

Accounting Research, 25(1), pp. 219-242.

Huerta, E. and Jensen, S. (2017). An Accounting Information Systems Perspective on Data Analytics and Big Data, Journal of Information Systems, 31(3), pp. 101-114.

Janvrin, D., Bierstaker, J. and Jordan Lowe, D. (2009) An Investigation of Factors Influencing the Use of Computer-Related Audit Procedures, Journal of Information

systems, 23(1), pp. 1-22.

Jordan Low, D., Bierstaker, J. L., Janvrin, D. J. and Gregory Jenkins, J. (2018) Information Technology in an Audit Context: Have the Big 4 Lost Their Advantage?. Journal of

(32)

Kokina, J. and Davenport, T. H. (2017) The Emergence of Artificial Intelligence: How Automation is Changing Auditing. Journal of Emerging Technologies in Accounting., 14 (1), pp. 115-122

Maas, V. S. (2009) A Concise guide to quantitative data analysis for MSc students,

Amsterdam Business School, pp. 1-59.

Mansour, E. (2016) Factors Affecting the Adoption of Computer Assisted Audit Techniques in Audit Process: Findings from Jordan, Business and Economic Research, 6(1), pp. 248-271.

Martin, H. S. and Herrero, A. (2012) Influence of the user’s psychological factors on the online purchase intention in rural tourism: Integrating innovativeness to the UTAUT framework

Moore, G. C. and Benbasat, I. (1991) Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation. Information Systems

Research. 2(3), pp. 192-222.

Mustapha, M. and Jin Lai, S. (2017) Information Technology in Audit Processes: An

Empirical Evidence from Malaysian Audit Firms. International Review of Management

and Marketing, 7(2), pp. 53-59.

Nassauer, S. (2016) Walmart to cut 7,000 back-office store jobs. Wall Street Journal, 2. Nederlandse Beroepsorganisatie van Accountants (2017) Handleiding Regelgeving

Accountants. Retrieved from https://www.nba.nl/tools/hra-2017

O’Brien, R. M. (2007) A Caution Regarding Rules of Thumb for Variance Inflation Factors.

Quality & Quantity, 41(5), pp. 673-690.

Oladele, K. O. and Agochukwu, B. O. (2016) An Empirical Study of Task-Technology Fit, Audit Firms and Clients Innovations on CAATs Usage in Nigeria. International Journal

of Management Science, 1(1), pp. 90-102.

Park, N., Rhoads, M., Hou, J., and Lee, K. M. (2014) Understanding the acceptance of teleconferencing systems among employees: An extension of the technology acceptance model. Computers in Human Behavior, 39, 118-127.

Pedrosa, I. and Costa, C. J. (2012) Financial auditing and surveys: How are financial auditors using information technology?: An approach using expert interviews, ISDOC

'12: Proceedings of the Workshop on Information Systems and Design of Communication, pp. 37-43.

(33)

Pedrosa, I. and Costa, C. J. (2014) New trends on CAATTs: what are the Chartered

Accountants’ new challenges?. Proceeding ISDOC '14 Proceedings of the International

Conference on Information Systems and Design of Communication, pp.138-142.

Pedrosa, I., Costa, C. J. and Laureano, R.M.S. (2015) Use of information technology on statutory auditors' work: New profiles beyond Spreadsheets' users. 10th Iberian Conference on Information Systems and Technologies (CISTI).

Perrin, A. (2015) Social Media Usage: 2005-2015. Pew research center, 1, pp. (1-11)

Shamsuddin, A., Rajasharen, L., Maran, D., Feyas Mohamed Ameer, M. and Maratha Muthu, P. (2015) Factors Influencing Usage Level of Computer Assisted Audit Techniques (CAATs) by Internal Auditors in Malaysia. Kuala Lumpur International Business,

Economics and Law Conference 6, 1, pp. 123-131.

Smidt, L., Van der Nest, D. P. and Lubbe, D. (2014) The use of sampling and CAATs within internal audit functions in the South African banking industry. 9th Iberian Conference on Information Systems and Technologies (CISTI)

Subaryani binti Zainol, S., Suhaily binti Samsuri, A., Rahimah binti tunku Arifin, T., Binti Hussin, S. and Syazwani binti Othman, M. (2017) Determinants of Computer Assisted Audit Techniques (CAATs) Adoption. A Study in Small and Medium Practices in Malaysia. European Journal of Business and Social Sciences, 6(2), pp. 135-150. Thompson, R. L., Higgins, C. A. and Howell, J. M. (1991) Personal Computing: Toward a

Conceptual Model of Utilization. MIS Quarterly, 15(1), pp. 124-143.

Vasile-Daniel, C. (2010) How financial auditors use CAATs and perceive ERP systems?,

Annals of the University of Oradea: Economic Science, 1, pp. 490-495.

Venkatesh, V., Morris, M. G.., Davis, G. B. and Davis, F. D. (2003) User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), pp. 425-478. Venkatesh, V., Thong, J. Y. L. and Xu, X. (2012) Consumer Acceptance and Use of

Information Technology: Extending the Unified Theory of Acceptance and Use of Technology. MIS Quarterly, 36 (1), pp. 157-178.

Vogel, D., Zhao, J., Zhu Z., Guo, X. and Chen, J. (2017) Introduction to the special issue on service-oriented E-business development. Electronic Markets, 27(3), pp. 197-198.

(34)

Appendix A: Overview questionnaire

Perceived usefulness of CAATs while conducting certain audit procedures

Q1 Please specify how many years of experience as an external auditor you have

o

Less than 1 year (1)

o

1 year (2)

o

2 years (3)

o

3 years (4)

o

4 years (5)

o

5 years (6)

o

6 years (7)

o

7 years (8)

o

8 years (9)

o

9 years (10)

o

10 years (11)

o

More than 10 years (12) Q2 Please specify your age

o

Under 18 (1)

o

18 - 24 (2)

(35)

o

35 - 44 (4)

o

45 - 54 (5)

o

55 - 64 (6)

o

65 - 74 (7)

o

75 - 84 (8)

o

85 or older (9)

Q3 Please specify your highest education level currently studying or achieved

o

High school diploma (VMBO, HAVO) (1)

o

Intermediate vocational education (MBO) (2)

o

Bachelor's degree (applied sciences, HBO) (3)

o

Bachelor's degree (university, WO) (4)

o

Master's degree (applied sciences, HBO) (5)

o

Master's degree (university, WO) (6)

o

Post master's degree or higher (HBO, WO) (7)

Q4 Do you currently have or have had a certification (e.g. Registeraccountant or Certified Public Accountant)?

o

Yes (1)

o

No (2)

Q5 Please specify your certification (multiple selection possible)

▢

Certified Public Accountant (CPA, RA or similar) (1)

(36)

Q6 Please specify your gender

o

Male (1)

o

Female (2)

o

Rather not say (3)

Q7 Please answer the following statement:

I consider myself well trained to make use of CAATs (Computer Assisted Audit Techniques)

o

Strongly disagree (1)

o

Somewhat disagree (2)

o

Neither agree nor disagree (3)

o

Somewhat agree (4)

o

Strongly agree (5)

In the following questions control risk can be interpreted as:

The risk that the control policies and procedures that are present in the client environment wouldn't detect or prevent an error of material misstatement.

(37)

Q8 I find it useful to apply the following (optional) auditing procedure:

Select sample transactions from key electronic files with the use of CAATs to assess identified fraud risks (HRA 330, A16) when:

Extremely useless (1) Slightly useless (2) Neither useful nor useless (3) Moderately useful (4) Extremely useful (5) Assessed control risk is Low (1)

o

Assessed control risk is Intermediate (2)

o

Assessed control risk is High (3)

o

Check electronic transactions and accuracy of electronic files with the use of CAATs to assess identified fraud risks (HRA 330, A16) when:

o

(38)

Sort transactions with specific characteristics with the use of CAATs to assess identified fraud risks (HRA 330, A16) when:

o

Test an entire population instead of a sample to assess identified fraud risks with the use of CAATs (HRA 330, A16) when:

o

(39)

Obtain evidence about control effectiveness with the use of CAATs (HRA 330, A27) when:

o

Evaluate transactions with new related entities with the use of CAATs (HRA 550, A36) when:

o

We thank you for your time spent taking this survey. Your response has been recorded.

The effect of control risk assessment on the use of CAATs : evidence from a big 4 firm