THE ADOPTION OF DATA ANALYTICS IN INTERNAL AUDITING:
A REVIEW OF INFLUENTIAL FACTORS
by
P.B. van Haagen, BSc - 10575863
A thesis submitted in partial fulfillment of the requirements for the degree of
Master of Science
Information Studies: Business Information Systems
University of Amsterdam Faculty of Science Final version: 18-04-2016
Supervisors: Signature:
Prof.dr ir. H.A. Reijers, VU (first examiner)
2
I.
Preface
Before you lies my thesis “The adoption of Data Analytics in Internal Auditing: A Review of Influential Factors”. It has been written to fulfill the graduation requirements of the master Information Studies at the University of Amsterdam. I was engaged in researching and writing this thesis from July 2015 till April 2016.
I would like to thank professor Hajo Reijers for his support, input, patience and time. Also I would like to thank Dr. Heinhuis for being my second supervisor. Special thanks go to Reynold ten Hoor for providing me the opportunity to conduct my research at Rabobank and for guiding me throughout my internship.
Finally, I would like to thank Joris for his great support during the last 6 years. Yes, I have switched and quit studies multiple times, but thanks to your support I have always felt the pressure to continue.
3
II.
Abstract
In 2014 Audit Rabobank Group (ARG) experimented with advanced data analytics and process mining in their internal audits. They believed that these innovative techniques could help them to do their auditing work more efficiently and more effectively. However, the majority of the auditors have still not used process mining or advanced data analytics in their audits.
The goal of this research is to acquire knowledge about what factors are of influence on the adoption of new technology, and more in specific data analytics tools, by internal auditors. This goal leads to the following main research question:
What factors influence the adoption of data analytics in internal auditing?
The UTAUT model has been selected as a basis to measure the factors that influence the Behavioral Intention of someone to use a certain technology. This model is expanded by a new factor,
Innovation Awareness. This expansion is based on findings in literature of Jans et al. (2013) where they state that the acceptance of process mining in audits depends on the “focus on what is new and unique about process mining”.
To be able to advice the ARG and to test the new model, a questionnaire has been send out to all auditors of ARG and four interviews have been conducted at two external organizations.
The following hypotheses are tested in this research:
H1 There is a positive correlation between Performance Expectancy and the Behavioral Intention to use data analytics in internal auditing.
H2 There is a positive correlation between Effort Expectancy and the Behavioral Intention to use data analytics in internal auditing.
H3 There is a positive correlation between Social Influence and the Behavioral Intention to use data analytics in internal auditing.
H4 There is a positive correlation between Innovation Awareness and the Behavioral Intention to use data analytics in internal auditing.
Based on the questionnaire, only H4 was rejected, because no significant relation between Innovation Awareness and Behavioral Intention has been found.
The Behavioral Intention to use data analytics in internal auditing is related to the Performance Expectancy, Effort Expectancy and Social Influence factors. Social influence has the strongest relation with Behavioral Intention. For each point scored higher on Social Influence, the Behavioral Intention goes up by 0,659. For Effort Expectancy this number is 0,317 and for Performance Expectancy it is 0,253. Based on these results and the interviews, eight recommendations for ARG were made.
4
Table of Contents
I. Preface ... 2 II. Abstract ... 3 1. Introduction ... 5 2. Theoretical framework ... 7 3. Research design ... 15 4. Research results ... 23 4.1 Questionnaire results ... 23 4.2 Interview Results ... 28 5. Discussion ... 31 6. Conclusion ... 33 III. References ... 35 IV. Appendices ... 375
1. Introduction
In 2014 Audit Rabobank Group (ARG) experimented with advanced data analytics and process mining in their internal audits. They believed that these innovative techniques could help them to do their auditing work more efficiently and more effectively. A standard approach was written, a couple of auditors were trained in the use of tools and were instructed to introduce their new way of working in their audits. Nevertheless, when looking at the amount of audits where data analytics tools were used, it can be stated that the majority of the auditors have not used process mining or advanced data analytics in their audits.
The traditional way of auditing is to take a sample of a complete dataset and check each of the samples for deviations of the accepted business process. Data analytics, a concept that means “process mining and advanced data analysis” in this thesis, could be used to change the way auditors work. Instead of working with samples, modern technology enables auditors to go through enormous datasets. Log files of all transactions in a certain period can be analyzed to be able to assure the state of a certain process at all time.
This technological improvements sounds quite impressive, but for some reason, internal auditors do not use process mining regularly. Jans et al. (2013) find it remarkable that compared to other focus areas like healthcare and business process improvement, process mining still needs to be
implemented in auditing methods. It may be clear that the adoption of these technologies is not only influenced by the technical improvements and stories about the potential performance
improvements. This thesis tries to find out what is influencing internal auditors to adopt data analytics in their way of working. In general this thesis tries to contribute to the research field of IT adoption.
The findings at Rabobank in 2014 are more or less confirmed by research of Javrin et al. (2009). In their research they concluded that less than 40 percent of the auditors use any form of data analysis in their audit work. Please take into account that the 40 percent also consists of people who perform a simple Excel analysis. These results were published 7 years ago and technology made a significant transition during that period, but it suggests that auditors in general are not the fastest in adopting new IT.
1.1 Research question
The goal of this research is to acquire knowledge about what factors are of influence on the adoption of new technology by internal auditors. This goal leads to the following main research question:
What factors influence the adoption of data analytics in internal auditing? To answer this question the following questions need to be answered first:
1. What models are used to describe the adoption of new technology? 2. What factors play a role in the adoption of new technology?
6
After the theoretical framework that is presented in chapter 2, hypotheses are formulated in section 3.3. These hypotheses describe the relations between the factors that are found in the literature and the intention to use data analytic tools in internal auditing.
1.2 Scope
This master thesis is executed at Rabobank Nederland within the ARG department. This department is responsible for the internal audits at the Rabobank Group and consists of more than 190
employees. They execute audits within the Dutch part of Rabobank, but also in the international branch of the organization.
This research focusses on adoption of IT theories and the factors that influence an individual to adopt or not adopt a certain technology. The adoption of IT by organizations as a whole and the effects on the organizations and their way of working is not in scope.
To be able to advice the ARG, a questionnaire has been send out to all auditors of ARG and four interviews have been conducted at two organizations. The first organization is the Central
Government Audit Service, the largest internal audit service of the Netherlands. This organization was selected because their organizational size is comparable to the ARG. The second organization is a Dutch consulting and audit organization which is specialized in using data analysis and process mining in audits. This organization was selected on their experience with the implementation of process mining in audit departments of big Dutch organizations.
1.3 Research outline
Chapter 2 provides the theoretical framework that forms the basis of the rest of the research. Chapter 3 explains the research method that has been conducted. It gives details on how the data was gathered and analyzed. In chapter 4 the research results are presented and important statistical outcomes are given. The results of the interviews and the recommendations to ARG can also be found in this chapter. The conclusion and discussion are provided in chapter 5 and 6.
7
2. Theoretical framework
The purpose of this chapter is to ensure the understanding of the central concepts of this research project. First there is a small introduction about the concepts Audit and Process Mining, including the advantages of Process Mining. Secondly, adoption and acceptation models are discussed and
compared. At the end of this comparison a choice is made for a specific model that is going to be used for this research. The factors of this model are discussed, including findings about relations between the different factors in earlier research. At the end of this chapter, the literature that combines all aspects: audit, process mining, adoption and the factors that influence the adoption, is discussed.
Process Mining and Audits
Van der Aalst et al. wrote in 2010 that “Auditors can use process mining techniques to evaluate all events in a business process, and do so while it is still running” and that these new process mining techniques will enable a new way of auditing that will change the jobs of auditors.
However, despite that the automation and analysis of audit processes could really help, not much research has been done on the use of process mining in audit. Jans et al. (2013) state that “.. there have been only a handful of papers in accounting that have discussed process mining, and moreover, they have been essentially technical in character, focusing more on the methodology of process mining than on its specific application to accounting.” For example, scientific research about the effects on the role or efficiency of auditors after implementing process mining is not available.
What is audit?
An audit is an independent confirmation from an auditor about the organization’s financial position and the processes which lead to this statement. Information about the financial position is most useful when it is “timely and free from material errors, omissions, and fraud” (Chan & Vaserhelyi, 2011). Due to the time and labor-intensive manual audit procedures, it is very common to audit a process only once a year. In the period between two audits, the organization runs a risk of material errors, omissions and fraud. Besides audits performed by an external auditor, large organizations have their own internal auditing organization as well. Internal auditors perform audits on the internal processes and financial position to ensure high quality processes and minimal risk.
By automating audit methods and procedures the time and labor of an audit can decrease significantly. By automating the audit of a process, the risk can be assessed at all times instead of monthly or yearly. This implementation leads to continuous auditing and continuous assurance (Vasarhelyi et al., 2012). Although automating audit methods has advantages, it costs time and money to develop an internal audit organization which is ready to use more innovative ways of auditing.
What is process mining and what can it do?
Information technology changed the way companies have organized their processes and information. The digital economy has resulted in a major adoption of Enterprise Resource Planning tools (ERP) to handle information streams and business processes. Most of the customers of big accountant firms
8
have implemented such systems (Kanellou et al., 2011). Besides ERP, companies nowadays use Customer Relationship software, Workflow tools, Supply Chain optimization systems, etc. Al these types of systems are used to register and process data. All activities are registered in an event log, which is also called an audit trail (Jans, 2013). Because the audit trail is filled with all activities in the system, and cannot be adjusted by the end user, it can be used to check the history of a certain transaction. The events in the log can be transformed into a visual representation of the transaction. This transformation and representation is called process mining.
Van der Aalst et al. (2004) define process mining as “the method of distilling a structured process description from a set of real executions”. Because real executions form the basis of process mining, this intelligent method for analysis can be of great value for the work of an auditor (Jans, 2013). According to Van der Aalst (2011) process mining can be divided into three types of mining:
1. Discovery 2. Conformance 3. Enhancement
The discovery technique is used without the presence of an existing model or form of representation of the process. Real processes are revealed based on an event log, but no comparison is made, because there is nothing to compare to. The second form of mining is conformance mining. In this situation process mining is used to compare two sets of event logs of the same process or a model versus event logs. The last type of mining is enhancement. The idea of enhancement is that the event logs are combined with other information about the process from another source.
Figure 1: Three types of process mining (Van Der Aalst, (2011))
Process conformance can be used in an audit to check whether business processes match the
theoretical processes. Theoretical processes are the processes that the organization has described on paper or in a tool. However, this does not always match the real life situation. The conformance check gives an auditor the opportunity to perform his statutory duty to test if control measures are
9
followed as intended or if security rules are breached (Jans et al., 2013). An event log that is combined with more data than activities and timestamps, like product details, order details or user details gives even more audit possibilities.
Although process mining is a relatively young research topic, one of the first publications was already published in 1998 (Cook and Wolf, 1998). They wrote how the discovering of models of software processes from event-based data was possible with process mining techniques. Jans et al. (2013) state that process mining is a good addition to auditing, because it enables the possibility to compare the actual process against the theoretical process. It enables the use of the whole dataset instead of a sample. However, they find it remarkable that compared to other focus areas like business process improvement and healthcare, process mining still need to be implemented in auditing methods.
Adoption and acceptation models
The new process mining techniques will enable a new way of auditing. However, not every human reacts the same on changes in their work, life, environment et cetera. Some persons like to keep things the same and have a natural resistance to change. When talking about the adoption of innovations, Rogers (2010) distinguishes 5 groups, from slow adopters to really fast adopters. The levels that he distinguishes are: laggards, late majority, early majority, early adopters and innovators. The adoption of data analytics in internal auditing can be seen as an innovation. Some people will be innovators, others offer more resistance and can be seen as laggards. Please note that this is not a value judgement. People respond different, but what is the reason behind these differences and is there scientific research about it?
Rogers (2010) distinguishes the research on innovation into three main topics. The first main topic is “domestication” research. Domestication research is about the integration of technologies into daily practices and the effect of the adaption of technologies on the society itself. The second topic is “diffusion” research and this field is researching on differences between groups of people and their level of adoption, like the laggard-innovator scale. The last research stream is called adoption research and this stream focusses on the why and how of individuals instead of society or certain groups. This thesis mainly uses theories from the adoption research stream, because the focus lies on influential factors for an individual auditor to adopt or not adopt a certain technology or way of working.
At the moment no empirical research on the adoption of process mining or advanced data analytics by internal auditors has been conducted. However, certain models for predicting adoption of other (IT) innovations have been widely used and validated (Jeyaraj et al., 2006). The use of a highly validated and widely accepted model as a basis for further research, could help to get valid and reliable results. The following table gives an overview of the most referenced theories and their constructs. It enables a quick comparison of the differences and similarities between the models.
10
Social Cognitive Theory (SCT) of Bandura (1977) Constructs SCT assumes that people learn by observing others. The cognition,
environment and behavior of the individual has an effect on the result of the observation. SCT states that people remember the sequence of certain behavior and the effect of it and that they learn from this. SCT states that behavior is influenced by self-efficacy and the expectations for a certain outcome. These two factors are in their turn influenced by the individual’s prior behavior.
1. Subjective Norm
2. Attitude Toward Behavior
Technology Acceptance Model (TAM) of Davis (1989) Constructs The Technology Acceptance Model focusses on the adoption and use of IT and
what factors are influencing this. Davis identified three factors that influence the Behavioral Intention of the individual. The perceived ease of use and perceived usefulness have an effect on the attitude towards IT adoption and use which on his turn influences the Behavioral Intention, which may result in actual system use.
1. Perceived Ease of Use 2. Perceived Usefulness 3. Subjective Norm
Innovation Diffusion Theory (IDT) of Rogers (2010) Constructs Another model that focusses on adoption of innovations, is the IDT model of
Roger (2010). This theory state that adoption of innovation is affected by the following four, most important elements: the innovation itself, time,
communication channels and the social system. These elements are affected by the constructs that are mentioned in the right column.
1. Relative Advantage 2. Visibility 3. Image 4. Ease of Use 5. Voluntariness of Use 6. Result Demonstrability 7. Compatibility
Unified Technology Acceptance and Use Theory (UTAUT) of Venkatesh et al. (2003)
Constructs The UTAUT model of Venkatesh is based on the principle that there is a
relation between Behavioral Intention and Usage Behavior. Behavioral Intention is influenced by Performance Expectancy, Effort Expectancy and Social Influence, while the Behavioral Intention itself and Facilitating Conditions are influencing the Usage.
1. Performance Expectancy 2. Effort Expectancy 3. Social influence 4. Facilitating conditions
Table 1: Four adoption and acceptance theories
Most models are conceptualizing the Behavioral Intention or intention to use as the output variable that is effected by a selection of other variables. To be able to select the right model that fits the research question, more information about the differences between the models is needed.
Venkatesh et al. (2003) state that most of the models that measure the Behavioral Intention are set up like shown in figure 2. This “basic” model can be found in all four discussed models.
11
Because the TAM and UTAUT model both have a construct that measures the subjective norm or social influence, these two models are selected for a deep-dive. ARG wants to know whether auditors feel some kind of peer pressure or not and how that can be used to stimulate adoption. The following sections describe more details about the TAM and UTAUT models.
TAM Model
The Technology Acceptance Model (TAM) focusses on the intention to use and use of an IT system. A schematic view of the model can be found in the figure below.
Figure 3: The TAM model of Davis (1989)
The following definitions are used by Davis (1989) for each of the constructs:
1. External Variables: External variables
2. Perceived Ease of Use: The degree of ease associated with the use of the system
3. Perceived Usefulness: The degree to which an individual believes that using the system will help
him or her to attain gains in job performance
4. Attitude Toward Using: Individual's positive or negative feeling about performing the target
behavior (e.g., using a system)
5. Behavioral Intention to Use: The degree to which a person has formulated conscious plans to
perform or not perform some specified future behavior
6. Actual System Use: The degree to which a person actually uses the system
Research of Davis et al. (1989) state that Perceived Ease of Use and Perceived Usefulness are the strongest predictors for acceptation of IT systems. Four years later they concluded that Perceived Usefulness is a 1.5 times stronger predictor compared to the Perceived Ease of Use (Davis et al., 1993). This implies that the degree to which an individual believes in the performance improvements has more impact on the attitude than the degree of ease that is associated with the system use. The TAM model was expanded by Venkatesh and Davis (2000) and named the TAM2 model. It uses TAM as a starting point, but tried to incorporate some extra constructs like Social Influence by for example peers/coworkers/et cetera. Because this thesis is about the adoption of IT in an
organizational environment, the effect of management and peers of their attitude towards the adoption needs to be taken into account. Curtis and Payne (2008) state that auditors are more likely to implement innovative technologies when the managing partner has a positive attitude towards
12
this adoption. Not only the TAM2 model is taking the Social Influence into account, the UTAUT model has the same construct.
UTAUT Model
The UTAUT model has distilled constructs out of eight models that earlier researchers had validated. The model tries to explain the use behavior to use a specific technology system. Venkatesh, Thong and Xu (2012) state that “about 70 percent of the variance in Behavioral Intention to use a
technology is explained by the UTAUT model”. This high level of explained variance has led to frequent use of the UTAUT model in research about IT adoption and acceptance in organizations (Bierstaker et al., 2014).
Figure 4: The UTAUT model of Venkatesh et al. (2003)
The following definitions are used by Venkatesh et al. (2003) for each of the constructs:
1. Performance Expectancy: The degree to which an individual believes that using the system will
help him or her to attain gains in job performance
2. Effort Expectancy: The degree of ease associated with the use of the system
3. Social Influence: The degree to which an individual perceives that important others believe he
or she should use the new system
4. Facilitating conditions: The degree to which an individual believes that an organizational and
technical infrastructure exists to support use of the system
5. Behavioral Intention: The degree to which a person has formulated conscious plans to perform
or not perform some specified future behavior.
6. Use Behavior: The degree to which a person actually uses the system
7. Voluntariness of Use: The extent to which potential adopters perceive the adoption decision to
13
Comparable to the relationships in the TAM model, the Performance Expectancy is according to Venkatesh et al. (2003) the strongest predictor of Behavioral Intention.
Model selection
Both the TAM and the UTAUT model provide clear questionnaires that can be used to determine which factors influence the adoption of process mining or advanced analytical tools for internal auditing purposes. However the UTAUT model explained in the research of Venkatesh (2003) 70% of the variation in the user’s intention to accept technology. Research of Ligris et al. (2013) states that the variance explanation of the TAM model is far lower than that. They claim that TAM and the other version TAM2 together, only account for 40% of the variance of a technological system’s use.
Because of the higher score of the UTAUT model and its proven track record, the UTAUT model is selected.
The constructs of the UTAUT model will be discussed during the next chapter. The original model will be adjusted and the Innovation Awareness construct will be added. The following paragraph explains the concept of Innovation Awareness.
Innovation Awareness
Relative advantage is seen as the degree to which the individual believe that the innovation is better than the system or method used before. However this definition does not state anything about if the individual believes the new way of working or the new system can be called an innovation.
According to Jans et al. (2013) the acceptance of process mining in audits depends on the “focus on what is new and unique about process mining”. The focus on what is new and unique is in their opinion the following list of points:
1. Look at the entire population and not at sample data 2. Available meta data
3. A more effective way of implementing an audit risk model, helped by analytic procedures 4. A new way of discovering insights in data and processes, which were not possible without
process mining tools.
Even though Performance Expectancy can be related to the degree to which an individual believes that the new method/tool is new and unique, no research has been performed about this
“Innovation Awareness” and the effect of it on adoption or acceptance by internal auditors. The following literature has been found on the combination of audit, data analytics and the acceptance and adoption of it.
The acceptance and adoption of data analytics in auditing
Gonzales et al. (2012) did a study with 210 internal auditors from all over the world about the status of the adoption of continuous auditing in their firms. They used a survey to measure the UTAUT model factors and concluded that no significant relation exists between Performance Expectancy and Behavioral Intention. This is remarkable since Venkatesh et al. (2003) state that Performance
Expectancy is the strongest predictor of Behavioral Intention. No reasons were given for the lack of a significant relation.
14
Bierstaker et al. (2014) did research on the adoption of computer-assisted audit tools and techniques within accountant firms in the United States of America (USA). 181 auditors took part in this study where not only the Behavioral Intention was measured, but also the actual usage. This is unique since most other studies focus only on the Behavioral Intention or the usage behavior and not both. A significant relation between the Performance Expectancy and the Behavioral Intentions was found. In the research of Curtis and Payne (2014) on 75 accountant managers of the Big-4 accountant firms (PWC, EY, KPMG and Deloitte) a significant relation between Performance Expectancy and Behavioral Intention was found. They selected managers instead of the accountants that perform the checks themselves, because they did assume that managers are more long term focused instead of short term. With this more long term focus, Curtis and Payne tried to prevent other constructs like Effort Expectancy and a possible drop of declarable hours would interfere with the Performance
Expectancy.
The UTAUT model is used by Mahzan and Lymer (2014) to research what influences the use of data analysis by auditors. They used 10 semi structured interviews with auditors with a higher
management position to see how adoption of data analysis took place within their firm. Mahzan and Lymer conclude that Facilitating Conditions and Performance Expectancy both are great predictors of the eventual usage of the new techniques. Most interviewees use the techniques because of the improved efficiency because routine work can be easier automated. The Facilitating Conditions are very important because without proper software, hardware and instructions, but also the availability of practice material it can become more difficult to adopt a certain way or working.
Loraas et al. (2006) did research on the willingness of potential IT system users to invest in their education and training to be able to use the new system. They used a variant of the TAM theory of Davis and concluded that there is a significant relation between the willingness of someone to invest and the Behavioral Intention. Loraas et al. (2006) see the willingness to learn as an important
predictor for Behavioral Intention. After all if someone has no intention to put any effort in learning something new, it can be doubtable of that person has the intention to use the new system at all.
Conclusion
After discussing the alternative models for acceptation and adoption, the UTAUT model has been selected. The model is used a lot in scientific research and validated multiple times. Its unified view and theoretical strong basis, which is due to the fact that it is a combination of eight other models, makes it the best fit for this research. Compared to the other models the high explanation of the variance (70%) gives confidence for a high explained total variance when the model is expanded by the new construct, Innovation Awareness.
15
3. Research design
To analyze what factors influence the adoption of data analytics in internal auditing a mixed methods approach is used. Mixed research provides a better understanding than a qualitative or quantitative method alone (Creswell, 2009) and the combination minimizes the weaknesses of both. Research of Johnson and Onwuegbuzie (2004) state that many research questions are best and most fully answered through mixed research solutions.
The quantitative and main part of the research consists of a questionnaire based on the UTAUT model. This model is used to identify influential factors in the adoption of data analytics in internal auditing. To be able to advice ARG, interviews form the qualitative part of the research. Interviews take place at the internal organization and two external organizations. The results of the interviews enable the comparison of adoption strategies and validation of the adoption model and are
discussed in chapter 4.2.
3.2 Research model
This research follows the UTAUT model of Venkatesh et al. (2003). The model explains the influential factors of the acceptance and use of technology. There are three constructs that influence the ‘Behavioral Intention’. These constructs are ‘Performance Expectancy’, ‘Effort Expectancy’ and ‘Social Influence’. Behavioral Intention influences the ‘Use Behavior’ in the model of Venkatesh.
Because the ARG has just started to use data analytics in audits, the main focus on this research is on finding influential factors on Behavioral Intention and not of Use Behavior. The reason behind this is that ARG estimates that at this moment less than 10 percent of the employees have experience (= Use Behavior) with data analytics in their audits. With this amount of experienced respondents of a population of only 168 auditors, it is very hard or even impossible to find statistically significant differences between two sample groups.
Gender, Age, Experience and Voluntariness of Use are measured and used in the UTAUT model. In this research they are measured too, but are only used as control variables. The reason behind this is that they can influence the Behavioral Intention, but they are not of primary interest to the research. There are no hypothesis based on gender, age and the other control variables formulated.
The model has been extended by a fourth independent factor named “Innovation Awareness”. Based on research of Jans et al. (2013) where they state that in the process of accepting process mining in audits is accelerated when there will be focused on “what is new and unique”. Innovation awareness is something different than the other constructs. It focusses on thoughts of the individual about how modern and new the techniques are, instead of the added value to their performance or the amount of time or energy it costs to learn to use the new techniques. Although it does not seem logical to want to use something solely based on the fact that it is innovative, there are people who are sensitive for it. For example, even before people knew the specifications and capabilities of
innovative products like a Google Glass or Smartwatch, they were already enthusiastic and showed ‘Behavioral Intention’. Innovativeness is defined by Sethi et al. (2001) as the extent to which a new product provides meaningfully unique benefits. The ‘meaningfully’ part of this definition suggests a
16
possible correlation between Innovation Awareness and Performance Expectancy. This leads to the research model that can be found in figure 5.
Another construct of the UTAUT model is Facilitating Conditions. Because this construct influences the Use Behavior directly and not the Behavioral Intention, it is left out of the research model. It is an important factor when measuring the Use Behavior, but because Use Behavior is not in scope, there is no added value for this research.
3.3 Hypotheses
Based on the assumptions of the UTAUT model and the extension with the Innovation Awareness factor based on Jans et al. (2013) the following four hypotheses were assessed:
H1 There is a positive correlation between Performance Expectancy and the Behavioral Intention to use data analytics in internal auditing.
Venkatesh et al. (2003) concluded that “The Performance Expectancy construct within each individual model is the strongest predictor of intention and remains significant at all points of measurement in both voluntary and mandatory settings.” No signals have been found both in
literature as in desk research that the strong prediction rate of Performance Expectancy should differ in the case of data analytics in internal auditing.
H2 There is a positive correlation between Effort Expectancy and the Behavioral Intention to use data analytics in internal auditing.
Effort expectancy is the second strongest predictor of Behavioral Intention in the original UTAUT model. The concept of Effort Expectancy consists of the perceived ease of use, ease of use and complexity (Venkatesh et al., 2003).
H3 There is a positive correlation between Social Influence and the Behavioral Intention to use data analytics in internal auditing.
The Social Influence construct is defined by Venkatesh as “the degree to which an individual perceives that important others believe he or she should use the new system.” This construct is the third strongest predictor of the model and the last original construct from the UTAUT model that is used in this research model.
H4 There is a positive correlation between Innovation Awareness and the Behavioral Intention to use data analytics in internal auditing.
Jans et al. (2013) state that auditors tend to adopt process mining tools as advanced data analytics tools earlier when the focus is more on what the new and unique possibilities are. H4 is tested to see
whether Jans et al. are right and the perceive of innovativeness is a predictor of Behavioral Intention. The above hypotheses were tested using correlation- and regressionanalysis in SPSS and AMOS.
17
Figure 5: The research model with hypothesis
3.4 Instruments
This section explains the instrument that measures the constructs of the research model. The model consists of five constructs: Performance Expectancy, Effort Expectancy, Social Influence, Innovation Awareness and Behavioral Intention. With the exception of Innovation Awareness all the constructs are of the original UTAUT model of Venkatesh (2003). The constructs with their definitions can be found in the table below.
Construct Definition
Performance expectancy The degree to which an individual believes that using data analytics in internal auditing will help him or her to attain gains in job performance. Effort expectancy The degree of ease associated with the use of data
analytics in internal auditing.
Social influence The degree to which an individual perceives that important others believe he or she should use data analytics in internal auditing.
Innovation awareness The degree to which an individual perceives that data analytics in internal auditing is new, unique and innovative.
Behavioral intention The degree to which a person has formulated conscious plans to use or not use data analytics in internal auditing
Table 2: Constructs of the research model
Based on the original questions in the UTAUT model questionnaire, new items are formulated to measure the individual constructs. Only the subject of each original question was changed. For example, the original question “People who influence my behavior think that I should use the system” was changed in “People who influence my behavior think that I should use data analytics in audits”. The tables on the following page give an overview on the questionnaire items per construct. The complete questionnaire can be found in appendix I.
18
Performance expectancy items
PE1 I would find data analytics useful in my job.
PE2 Using data analytics in audits enables me to accomplish tasks more quickly. PE3 Using data analytics in audits increases my productivity.
PE4 If I use data analytics in audits, I will increase my chances of getting a raise.
Effort expectancy items
EE1 My interaction with data analytics tools would be clear and understandable. EE2 It would be easy for me to become skillful at using data analytics in audits. EE3 I would find data analytics tools easy to use.
EE4 Learning to operate data analytics tools is easy for me.
Social influence items
SI1 People who influence my behavior think that I should use data analytics in audits. SI2 People who are important to me think that I should use data analytics in audits. SI3 The senior management has been helpful in the use of data analytics in audits. SI4 In general, the organization has supported the use of data analytics in audits.
Innovation awareness items
IA1 Data analytics in audits gives the organization new and unique possibilities. IA2 I am aware of the new and unique possibilities of data analytics in an audit. IA3 The use of data analytics in audits gives me a new way of discovering insights in
data and processes, which were not possible without these tools.
IA4 I appreciate the new and unique possibilities of data analytics in internal auditing.
Behavioral intention items
BI1 I intend to use data analytics in audits in the next three months. BI2 I predict I would use data analytics in the next three months. BI3 I plan to use the data analytics in audits in the next three months.
Table 3: Questionnaire items
Besides the items that measure the main constructs of the model, four extra general items have been used in the questionnaire. These four items are age, gender, experience and voluntariness of use. They are used as control variables. The items can be found in the table below.
Control variables items
GENDER Your gender? Male / Female / Other AGEGROUP What is your age? -25 / 25-40 / 40-55 / 55+
EXP Have you used advanced data analytics tools in an audit?
VOLUNT Do you feel any kind of force or pressure from the organization to use data analytics in an audit?
19
3.5 Procedure
The procedure for gathering data consisted of acquiring a list with all the email addresses of all the auditors and audit managers. People who are not actively auditing anymore or had a staff function were excluded. The goal was to get a response of at least 118 respondents, because of the required minimum sample size. Due to this high number of required responses, personal looking invitations have been sent to the respondents.
First, for every potential respondent it was decided which invitation text they needed to receive. Four different texts were made for four different groups. One for managers, one for known
colleagues, one for unknown colleagues and one for colleagues who heard about the research during a course about Process Mining in Internal Auditing. The main difference in the invitations was in the introduction about the reason they received the invitation. For example, the colleagues who went to the course about Process Mining received an invitation with a sentence like “As I told in the course from last week, I’m doing research about..”.
Secondly, all invitations started with “Dear FIRST NAME”. The variable first name is replaced by the name of the colleague. The own name combined with the personal introduction were used to prevent that people saw it as a bulk mail and that they would ignore the questionnaire. People were asked to give a short reply after they finished the questionnaire. This was done for two reasons. First it prevents sending a reminder to colleagues who already responded and secondly it tells people that there is a list of people who did cooperate and who did not.
After one week the questionnaire was closed and the results were extracted, cleaned and imported into SPSS. An exploratory factor analysis was performed and iterated until a pattern of factors, with associated questions from the questionnaire, was found. Both convergent and discriminant validity as well as reliability was checked to make sure the data matches the model.
A confirmatory factor analysis was performed in AMOS. After a rough model was made, configural and metric invariance tests were performed. After the tests, the validity and reliability checks were performed again, to make sure the model fits. To make sure the constructs have no multicollinearity issues, a multicollinearity check was carried out. The results are presented in the next chapter.
3.6 Validity and Reliability
The construct validity was checked with factor analysis in SPSS and was performed by a Oblimin rotation. Construct validity is the degree to which the questions of the test measure what they claim to be measuring. Questions that behave different than the theory says a measure of a certain construct would behave, can be removed from the research to improve the construct validity. The Oblimin rotation was selected, because an oblique rotation is more representative of the reality, because it allows the factors to be correlated with one another. The results of the factor analysis led to the removal of PE4, EE3 and SI4.
Item PE4 was removed because it was not measuring the right construct. The explanation can be that it is very uncommon to get a raise or bonus at the Rabobank since the credit crisis. Item EE3 got a very low communalities score, which suggests that the question did measure something different.
20
Communalities give an indication of the amount of variance in each variable that is accounted for. Maybe the question is too broad, because the majority of the sample used a data analytics tool within the ARG. SI4 is removed because of a low communalities score. An explanation can be that respondents found ‘the organization’ a too vague and ambiguous term.
To examine to what extent all items of a concept in the questionnaire are measuring the same concept, the reliability of the construct can be calculated. The outcome of this test gives the
Cronbach’s α of the concept. SPSS calculates the Cronbach’s score of the construct, but also gives the Cronbach’s α of a construct when a certain item is removed. For the Social Influence factor, the removal of SI3 led to a higher Cronbach’s α. A higher reliability is an improvement, so SI3 was removed. This led to the following Cronbach’s α’s.
Construct Cronbach's α Number of items
Performance Expectancy 0,744 2 Effort Expectancy 0,660 3 Social Influence 0,837 2 Innovation Awareness 0,681 3 Behavioral Intention 0,955 3 Table 5: Cronbach’s α
According to George and Mallery (2003) Cronbach’s α above .9 is excellent, between .8 and .9 is good, between .7 and .8 is acceptable and between .6 and .7 is questionable. Although both Effort Expectancy and Innovation Awareness are below .7, they are close to 0.7. The increasing of α is partially dependent upon the number of items per scale. Because the number of items is quite low and Cronbach’s α is near .7, the results were accepted for further analysis of the model.
3.7 Sample
The study was conducted amongst auditors and audit managers from various departments within ARG. The total amount of auditors and audit managers within ARG is 168 (n=168). The acceptable margin of error is 5% and the reliability level for this study is 95%. For a representative sample of the study population a required sample size of 118 respondents needs to be achieved.
The questionnaire was checked by two persons without a background in audit and one person from within ARG (n=3). No substantial issues related to the questionnaire were found, so no corrections were made. Only some minor spelling issues in the invitation email and introduction text were found and corrected.
From the 168 in the population, 125 employees (n=125) filled in the questionnaire in time. There were 10 persons not available during the research period, 2 persons who did respond after statistical analysis was done and 31 who did not respond at all. As a result, the response rate was 74,4%. Table 6 provides an overview of the sample.
21
Variable Answer # of respondents % of respondents
Gender Male 88 73%
Female 30 25%
Other 2 2%
Total 120 100%
Age Less than 25 0 0%
25-40 47 39% 40-55 64 53% 55 and over 9 8% Total 120 100% Experience No 57 48% 1 time 18 15% 2 or more times 45 38% Total 120 100%
Pressure No, I do not 67 56%
Only a little 27 23%
Yes, I do 26 22%
Total 120 100%
Table 6: Overview of the sample
3.8 Data cleaning
The data must be screened in order to ensure that the data is valid, useable and reliable for further testing. The data screening did focus on removing cases that had too much missing data points, replacing missing data in cases with only a few missing items and removing cases that had only a low variance. A very low variance appears when a respondent responds with the same value to all questions. For example, if someone selects the 3rd answer on all questions. The lack of variance or
very low variance, results in a respondent that does not contribute to the overall variance of the sample and only increases the sample size.
Three cases are removed from the sample due to a large amount (>10) of missing values. There were 10 cases remaining with 1 or 2 missing values. These missing values have been replaced by the mean of the other cases on that item. This did not affect the variance and prevents the unnecessary removal of valuable data points. Two cases where removed due to a very low variance. The first case did not have any variance and the second case had a variance of only 0.2. Of all the questions, only 1 item had been answered differently than the rest. With such a low variance, it is doubtful if the respondent was seriously. To make sure the other results are not influenced they are removed from the sample. The result is a sample size of 120 cases (= 71% of the population).
No outliers were found in the dataset, because all responses where multiple choice or on a five-point Likert scale. For all items Kurtosis was calculated. The Kurtosis is a measurement that tells to which extent a distribution is more flat-topped or more peaked than the normal curve. None of the items
22
had Kurtosis issues, which implies a normal distribution of the data: there is sufficient and normal variance. All Kurtosis scores were between -2 and +2.
23
4. Research results
This chapter consists of the results of the questionnaire and the results of the interviews. The interview questions are based on the outcome of the questionnaire and the analysis. The outcome is used to check what is done now by Rabobank to stimulate the Behavioral Intention and what is done by other organizations to reach that goal. The interviews are used to formulate eight
recommendations that can be found in paragraph 2.
4.1 Questionnaire results
This chapter presents the results of the statistical analysis in SPSS and AMOS of the data that has been gathered via an online survey among 120 internal auditors of ARG. In the first three chapters of this thesis a theoretical model has been developed, an explanation about how the data was gathered and how missing data was handled was given and now it is time for the results.
The first part of the results will mainly focus on the exploratory factor analysis and in the second part the confirmatory factor analysis will be discussed. The first part was done in SPSS and the second part in AMOS.
Exploratory Factor Analysis
For the Exploratory Factor Analysis (EFA) rotation was used to enable the loadings of the factor do be more differentiated. This helps in facilitating interpretation. The Oblimin rotation was chosen, because it does not force results to act orthogonally. This means that the factors are allowed to be correlated. The EFA is conducted with Maximum Likelihood to see if the questions/variables grouped together as expected. Besides that it is important that the observed variables met the validity and reliability standards. Maximum Likelihood was chosen to determine correlation between factors and unique variance among the different items.
Adequacy:
Adequacy is measured by the Kaiser-Meyer-Olkin (KMO) measure and Bartlett’s test for sampling (Beavers et al., 2013). KMO is a statistic that indicates the variance in the variables that might be caused by common underlying factors. A high value, close 1.0, indicates that a factor analysis can be useful. A low value, below 0.5, indicates that a factor analysis, won’t be very useful. The score of .734 is considered as “middling”. The Bartlett’s test is used to verify whether there is enough variance across the sample. This is a requirement to retrieve meaningful results from the EFA. The Sig. outcome of the Bartlett’s test is 0.000 which is significant (Sig. < 0.05), this implies that a meaningful analysis is possible. The Chi-Square value and degrees of freedom (df) are calculated by SPSS based on the data and are used to calculate the Sig. level.
KMO and Bartlett's Test
Kaiser-Meyer-Olkin Measure of Sampling Adequacy. ,734
Bartlett's Test of Sphericity Approx. Chi-Square 757,498
df 78
Sig. ,000
24
Additionally, the communalities for each variable were in almost all cases sufficiently high. Only two below 0.300 and most higher than 0.400. This indicates that the chosen variables correlate in the right way for a factor analysis.
Communalities Initial Extraction BI1 ,824 ,855 BI2 ,834 ,863 BI3 ,880 ,947 EE2 ,367 ,600 EE4 ,346 ,423 IA1 ,436 ,827 IA4 ,356 ,388 PE2 ,416 ,407 PE3 ,466 ,999 SI1 ,603 ,977 SI2 ,603 ,598 EE1 ,250 ,286 IA3 ,232 ,240
Extraction Method: Maximum Likelihood.
Table 8: Communalities of the sample
Reliability:
In the table below the Cronbach’s alphas for the factors are shown. The alphas were almost all above the 0.70 level which is seen as the minimal needed alpha. Although Effort Expectancy and Innovation Awareness did not reach 0.70 their value is very close to it. Because no exclusion of items increases the constructs Cronbach’s value, the alphas are accepted.
Construct Cronbach's α Number of items
Performance Expectancy 0,744 2
Effort Expectancy 0,660 3
Social Influence 0,837 2
Innovation Awareness 0,681 3
Behavioral Intention 0,955 3
Table 9: Cronbach’s α values of the constructs
Validity:
The Pattern Matrix shows the load of all the items to all the factors. Because a 0.350 is seen as the minimum load (Hair, 2010) a viewing threshold of 0.350 was chosen. This means that whenever an item loads less than .350 to a factor, the box remain blank.
25 Pattern Matrixa Factor 1 2 3 4 5 BI3 ,955 BI1 ,924 BI2 ,885 SI1 1,004 SI2 ,642 IA1 ,936 IA4 ,592 IA3 ,375 PE3 ,999 PE2 ,618 EE2 ,767 EE4 ,627 EE1 ,463 Extraction Method: Maximum Likelihood.
Rotation Method: Promax with Kaiser Normalization.a a. Rotation converged in 5 iterations.
Table 10: Pattern matrix
The factors demonstrate a sufficient validity, because their loadings exceed the 0.350 threshold. Besides that the factors also show that there are no cross-loadings, which means that there are no items that load to more than one factor.
According to the EFA this five-factor model (that corresponds with the theoretical model) explains 65% of the variance, with 4 out of 5 factors having eigenvalues that are greater than 1 and one that is close to 1. Beavers et al. (2013) write that “the majority of papers state that 75 to 90% of the
variance should be accounted for…; however some indicate that as little as 50% of the variance explained is acceptable”. A score of 65% is in the middle and is not great, but well above “acceptable”.
Total Variance Explained
Factor
Initial Eigenvalues
Extraction Sums of Squared Loadings Rotation Sums of Squared Loadingsa Total % of Variance Cumulative % Total % of Variance Cumulative % Total 1 4,090 31,462 31,462 1,819 13,991 13,991 3,402 2 2,209 16,995 48,457 2,818 21,680 35,670 2,473 3 1,390 10,692 59,149 1,307 10,055 45,726 1,794 4 1,270 9,767 68,916 1,493 11,486 57,212 1,854 5 1,010 7,771 76,688 ,973 7,481 64,693 1,741 6 ,678 5,215 81,902 7 ,625 4,807 86,709
26 8 ,479 3,682 90,391 9 ,463 3,564 93,955 10 ,336 2,588 96,542 11 ,231 1,779 98,321 12 ,140 1,078 99,400 13 ,078 ,600 100,000
Extraction Method: Maximum Likelihood.
a. When factors are correlated, sums of squared loadings cannot be added to obtain a total variance.
Table 11: Total variance explained
Confirmatory Factor Analysis
After EFA a Confirmatory Factor Analysis (CFA) is the next step to determine the structure of the factors in the dataset (Hair, 2010). During the EFA the factor structure is explored and CFA is about confirming this extracted structure.
Model fit
Because of discrepancies between the estimated and the proposed model, regression lines are placed between all of the observed and latent variables. Although the Gender, Age, Voluntariness of Use and Experience constructs from the UTAUT model were only used as control variables, AMOS concluded that regression lines between Voluntary and ExpCat were needed to be able to fit the model. This means that there exists relations between these variables that needs to be taken into account. After adding the regression lines the estimated model does fit the proposed model. The model fit figures can be found in the table below. The metrics are quite specific calculations to determine the goodness of fit. All observed values are in the recommended area of the model as described by Hair (2013).
Metric Observed value Recommended
cmin/df 1,201 Between 1 and 3
CFI 0,981 >0.950
RMSEA 0,041 <0.050 PCLOSE 0,559 >0.050 SRMR 0,023 <0.090
27
Figure 6: The model in AMOS
Regression Weights
Based on the model the regression weights can be calculated. For each regression line the Estimate and P value are calculated. The estimate value stands for the correlation between the two factors. For example, for the Performance Expectancy the estimate is 0,253 which means when Performance Expectancy goes up by 1, the Behavioral Intention goes up by 0,253 and this is significant because the P value is below 0.05. The table with the regression weights can be found below.
Regression Weights: (Group number 1 - Default model)
Estimate P BI <--- PE 0,253 0,042 BI <--- EE 0,317 0,035 BI <--- SI 0,659 *** BI <--- IA -0,24 0,236 BI <--- GENDER -0,013 0,925 BI <--- AgeCat 0,114 0,397 BI <--- Voluntary 0,178 0,212 BI <--- ExpCat 0,666 ***
*** = is significantly different at the 0,001 level (two-tailed)
Table 13: Regression weights
The P values of the regression between Innovation Awareness and Behavioral Intention is 0,236 and therefore the regression is not significant. In other words the prediction of BI is not significantly different from zero at the 0.05 level.
Because the UTAUT model has multiple control variables, these variables were also taken into account for the regression weights. The control variables had no impact on Behavioral Intention accept for the Experience (=ExpCat) variable. Internal auditors with more experience in data analytics have a higher Behavioral Intention to use data analytics.
28
What can be learned from this analysis?
The table above described the relations between the different factors, with the influence of the gender, age, voluntariness and experience factors are taken into account. There is a significant relation between Behavioral Intention and the following factors: Performance Expectancy, Effort Expectancy and Social Influence.
The Estimate column of the table shows the weights of the regression. The higher the estimate, the stronger the influence is. For example, for Social Influence, an estimate of 0,659 means that when the value of Social Influence goes up by 1 standard deviation, the value of the Behavioral Intention goes up by 0,659 standard deviation.
4.2 Interview Results
The second part of this results section consists of the interview results and the recommendations for ARG which are based on these interviews. ARG provided the initial problem that the adoption of data analytics is not going very fast and that they were curious which aspects influence this adoption. Because a more practical view was needed on this topic, 5 interviews were held.
One interview was held with an audit manager of Rabobank. Two interviews were held with auditors of the Central Government Audit Service and two interviews were held with an audit and consulting firm which is specialized in the use of data for audits. The Central Government Audit Service is the biggest internal auditing department of The Netherlands, while the consulting firm exists more than 10 years and has advised big company’s how to adopt data analysis in their audits. Besides that they are also reseller of data analytics tools and they give lots of training to auditors and internal auditors. The transcriptions of three of the five interviews can be found in appendix II. Two of the participants found it no problem to take part in the interview, but they preferred that no recording device was used. Their answers were used to check consistency with the other interviews. The
recommendations are not solely scientific, but a combination of results from the questionnaire, the interviews, literature and own interpretation of impressions during the internship. Based on the activities that were conducted from the interview, 8 recommendations have been formulated.
Recommendations
Based on this thesis the below three factors, in this specific order, have been identified to influence on the Behavioral Intention of the internal auditors of Rabobank:
1. Social Influence 2. Effort Expectancy 3. Performance Expectancy
For each recommendation it is described on which factors they may have a positive influence and what the background of that recommendation is.
1. Put the topic as high as possible on the management agenda
As described by one of the partners that was interviewed “it all starts with top management support”. Without this support employees do not feel the freedom to experiment with new
29
technology. This can be in terms of budget, time, but also on the possibility to make mistakes or “waste” time. When they do not feel this freedom, not only they will not experiment, but they are also no example for their peers. It is not said that it is not on the agenda now, but it can always be better and more visible. A manager with experience in data analytics can be a great promotor.
2. Strengthen visibility of Data Analytics ambassadors
The existing team of data analytics employees needs to be as visible as possible. Social Influence has the strongest relation with Bevioral Intention, so it is important that colleagues who have did audits on a data analytics way, show this as much as possible to employees who did not.
3. Select 3 successes and let the auditors present it with visualizations
Let this group of Data Analytics ambassadors select 3 audits which were handled by use of data analytics and let them present this in small groups to colleagues. In this way, the colleagues have the change to ask their coworkers whatever they want to know. This helps estimating the amount of effort it will cost to learn the tools and way of working. It also shows them the possibilities of data analytics tools and some concrete examples of how they can use it in a similar situation.
4. Link Data Analytics ambassadors to a small group of employees
People do not always like big groups and presentations where there is little room for questions. Especially with a topic where, as one of the partners said you need “vlieguren” it is very important that you do not only see a presentation, but you can learn directly from doing. In small groups of a maximum of 5 employees, the group leader can still easily manage the progress of the group and it is easier to plan small sessions during the week. This will stimulate the Social Influence.
5. Help a group of auditors plan how their “Data Analytics year” will look like
Instead of leaving it to the employee themselves, the organization can take some responsibility in guiding employees. Data analytics is something that you really need to do regularly. It is quite useless to send everyone on a basic training without setting any individual targets for the coming year. Auditors must be guided to write a plan what they want to do with the data analytics skills that they are going to obtain in they are interested in the topic and are interested to go to an expensive training. This does not focus on a certain factor, but more on stimulating people in general.
6. Include Data Analytics topic in yearly assessments
Every employee has yearly conversations and assessments to see whether progress is been made. However at the moment data analytics is not something that is scored. For employees who choose to change their way of working and become a data analytics auditor, their yearly assessment must be adapted to this situation. This is a control mechanism for the organization to check whether an employee meets the expectations, but also a guide for the employee during the year. People do not like to fail, so it can be stimulating for people if their progress is measured.
7. Let recruitment focus on technical and analytical skills
Recruitment must select more on technical and analytical skills. If operational and IT audits become much more data related, and that movement is seen at other organizations, literature and also in the
30
organization itself, than HR must adopt. At the Rijksoverheid they search for auditors with feeling for IT, even when they are applying for a financial or operational audit position. Long term focus on HR policy is needed. This is also something which will help the auditors who are already working at ARG. If they get new colleagues with experience in data analytics, and they see what others can do with it, the Social Influence factor will be stimulated.
8. Create training material and the possibility to train with easy examples in a sandbox
Employees need to have their own sandbox environment where they can train with data analytics. Although the gathering of data is a serious difficulty of data analytics, the techniques itself can be practiced in a sandbox. Training material is needed to give people the opportunity to train in their own time, on their own speed and in their own way. This helps stimulating the Performance Expectancy and Effort Expectancy factor. People will see the possibilities and can better estimate how much time they need to spend on training and other effort.
31
5. Discussion
5.1 Interpretation of the results
The conclusion of this research is that no relation between Innovation Awareness and Behavioral Intention was found in the data. The other three factors of the model, Performance Expectancy, Effort Expectancy and Social Influence all have a statistical significant influence on Behavioral Intention.
The fact that the three factors out of the original UTAUT model have a significant relation was expected. The UTAUT model have been widely used for the last 13 years and these factors were found to be of influence in a lot of studies, which were discussed in the theoretical framework. The fact that Innovation Awareness was not found to be related could have something to do with the personality of auditors. Auditors are people who are into facts and less about “the outside” of things. The fact that something is new and gives new possibilities is not always an improvement. This thesis showed that auditors are not affected by the idea itself that something is new.
5.2 Limitations of the research
The reader should bear in mind that this study is based on research within one group of persons within one organizations at one moment in time. The results may give direction of the outcome in other similar organizations, but the data itself was only gathered at ARG.
Although the sample size of 120 was big enough to form a representative sample of the study population, a bigger sample would possible give more accurate results. The conclusions that were drawn are however statistical valid.
The used research model has Behavioral Intention as the influenced factor. Although a relation between Behavioral Intention and Usage Behavior does exists in the UTAUT model and most papers that are based on the UTAUT model, strictly seen this research is only about the Behavioral Intention. Based on the figures of ARG itself, only a small group of employees have used Data Analytics in their work. But the survey results show that 45 respondents state that they used Data Analytics at least 2 times in their work. This can be a warning signal for the validity because it can be that, although the term is used a lot at ARG, Data Analytics is an ambiguous term for some respondents. On the other hand it can also be that the figures that Rabobank provided were outdated.
In general the limitation of a survey is that no extra questions can be asked to elaborate more on certain remarkable answers. One respondent did send an e-mail that he answered all the questions of the survey, but that he works for a specific group that focusses more on policy audits. Process mining is less relevant for these kind of audits, because no data fields, except text, is available in most cases. A shift on forehand to exclude auditors that perform that kind of audits could improve the research quality.
5.3 Future work
Like already mentioned in the limitations, this research did not take the actual usage of data analytics into consideration. Future research could try to get the bigger picture by perform research on the
