Task model applied : intention to use artificial intelligence throughout the IT-auditing process

(1)

Task model applied: Intention to use

artificial intelligence throughout the

IT-auditing process

Track: MSc BA – Digital Business

Student: Khatera Djadran

Student no: 11420707 University of Amsterdam

Faculty of Economics and Business

Supervisor: Dhr. Prof. Dr. P.J. Peter van Baalen

University of Amsterdam

Faculty Economics and Business Version: final

(2)

2 | P a g e

Statement of Originality

This document is written by student Khatera Djadran who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document are original and that no sources other than those mentioned in the text and its references have been used in creating it.

The faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(3)

3 | P a g e

List of figures

Figure 1: Clustering ... 13

Figure 2: Unified concept TAM ... 14

Figure 3: Conceptual model ... 18

List of tables

Table 1: Elements of General IT Control ... 16

Table 2: Interviewees ... 21

Table 3: Results... 28

(5)

5 | P a g e

Abstract

In this study the task model of Levy & Murnane (2003) is applied to examine in what extent artificial intelligence can be integrated throughout the IT-auditing process. It is expected that proactive and reactive AI are preferred to be used in tasks that are high in routine and low in cognitive thinking. Conversely, informative AI is preferred to be used in tasks that are non-routine and high in cognition. The qualitative interviews are conducted between a period of 5 months in a big four company in the Netherlands. The findings of the study imply informative AI is chosen when tasks are nonroutine and require cognition; and that proactive AI is preferred to be used when tasks are cognitive – routine; reactive AI is preferred to be used by IT-auditors when a task is manually or cognitive performed following pre-set rules only if the task requires professional judgement. This implies that cognitive routine tasks can also be complementary in contrast to the first proposition where AI is substitutionary. Additionally, the findings suggest that when a task involves personal interaction and creative thinking none of the forms of AI is preferred to be used.

1. Introduction

The theory of artificial intelligence (AI) can be drawn back in 1959 by Arthur Samuel (Puge, 2016). Arthur Samuel (1959) defined machine learning as: ‘’field of study that gives computers the ability to learn without being explicitly programmed’’. However, there was only theory about machine learning accessible to the attention of researchers and little knowledge about processing AI algorithms. In the recent years the development of the World Wide Web and the availability of data, such as documentations and texts files, there is a huge desire of collecting data by organizations in order to act upon it (Puge, 2016). Machine learning algorithms are able to process the large amount of information that is in the documentation and text files. More importantly, computers can calculate, coordinate, communicate the information in an efficiently and effectively than a human could ever do, and act upon the information retrieved from the databases (Bierstaker, Burnaby, & Thibodeau, 2001). Eventually, this will lead to supplementing or augmenting the human work tasks (Levy & Murnane, 2013). This study examines to what extent AI can be integrated throughout the IT-auditing process. Progressed machine learning algorithms are able to perform better at tasks that require simple and critical thinking (Hastie, Tibshirani, & Friedman, 2009). Also, with the use of machine learning current work activities can be done more efficient and smarter, in some cases beyond human capabilities and scalability.

(6)

6 | P a g e Frey and Osborne’s (2013) infer that the employment of the accounting and auditing professions will become to its end due to the extended development of artificial intelligence and the increasing threat of computerized jobs in the near future. In contrast to this, Richins, Stapleton, Stratopoulos and Wong (2017) predict that professional services such as accountant and auditors can establish itself in the world of big data by delivering value-creating services through data-driven innovations. Furthermore, Brown-Liburd, Issa, and Lombardi (2015) and Earley (2015) argue that professional accounting companies can face rivalry from non-auditing firms that are highly competent with data techniques and data analysis. So, if auditing companies do not explore and exploit technological opportunities against the threats, existing tech companies such as FinTech start-ups can catch the opportunity to infiltrate the industry accountants and auditors.

Rogers (1995) defines technological innovation as ‘’an idea, practice, or object that is perceived as new by an individual or other unit of adoption’’.

Moreover, the use of technology and internet have found their way into our everyday lives, influencing the way we do our work, influencing the way we live and even influence the way we want to entertain ourselves (Adams, 2017). The technology of artificial intelligence (AI) has really sept its way into our lives. AI has three major distinctive categories: informative AI, reactive AI, and proactive AI. Informative AI has the objective to provide useful information, it reduces data to store the most important and right information, and if you search for the information, it will provide you the right information (Frank, 1994).

Reactive AI is about the system that is reacting to events, rather than acting first to prevent something. Good examples of reactive AI are the personal assistants of Apple’s Siri (2011) and Amazon’s Alexa (2014) which are intended to help in very particular parts of the everyday lives. Proactive AI is a more complex and underlying technology that is being used in suggestive search and autonomously self-driving cars. The definition of pro-active AI is taking action by causing change or taking action to prevent something.

AI is a neural network that can learn on its own by making connections, gets smarter by improving itself on past iterations and enhance its knowledge and capabilities (German, Tani, Cornelius, & Wertmer, 2017), without depending on pre-defined algorithms (Adams, 2017).

(7)

7 | P a g e Even though companies like Amazon, Facebook, Google and Apple are trying to find a revolutionary way of reaching out to artificial intelligence techniques, there are still a lot of unclear answers how AI can be used by different businesses operating in different industries.

In particular, IT-auditors test software’s and applications to identify risks, controls and evaluate whether the IT-landscape of an entity is effective.

Furthermore, IT-auditors try to identify and control risks that have to do with the use of IT in an entity. Most of the time this process is done manually to obtain sufficient audit evidence with respect to aspects in the financial statements (International standard on auditing 550 related parties, 2009). One component of the risk assessment is the general IT controls. As described in ISA (2009) General IT control are policies and procedures that relate to one or more IT applications and support the effective functioning of application controls by helping to ensure the continued proper operation of information systems, it applies to mainframes, client server systems, file servers, end-user computing environments, or combinations thereof. From the perspective of ISA (2009) an IT system is effective when the system maintains the integrity of information and the security of the data such systems process, and include effective application controls and general IT controls.

According to the International Standard on Auditing (ISA) IT benefits the entity in various ways, some examples are: enhancement of the timelines, enhancement of accuracy and availability of information, facilitation of additional analysis of information, enhancement of the ability to achieve effective segregation of duties by implementing security controls in IT applications, databases, and operating systems. On the other hand, IT brings also risks affecting the internal control of an entity, such as processing inaccurate data, bringing unauthorized changes to data, bringing unauthorized changes to systems or programs, and the possibility of IT personnel gaining access privileges beyond those necessary to perform their assigned duties thereby breaking down segregation of duties, are illustrations of these risks related to IT (International standard on auditing 550 related parties, 2009).

This study is related to two literatures. The conceptual model builds on the acceptance of technology (Venkatesh, Morris, Davis, & Davis, 2003) and expands by including the task model (Autor, Levy, & Murnane, 2003). Due to the lack of understanding to what extent recent

(8)

8 | P a g e developments of AI can be integrated throughout the IT-auditing, the present study will give an answer to the following research question:

‘’To what extent can artificial intelligence be integrated throughout the IT-auditing process to ensure activities that have been occurred are authorized, accurately recorded and processed?’’

Nowadays, auditors perform almost all of the tests that are done using conventional sampling techniques. By using artificial intelligence techniques such as text mining information can be extracted from documents and files to support the auditing process. Using such a system as the text mining machine learning software the evidence will be more reliant since the software can examine the whole population instead of a sample. In line with the scope of this thesis, activities refer to the activities of general IT controls that occur in the elements of: access to Programs and Data, Program Changes, Program Development and Computer Changes (see table I).

Since a lot of previous research has been conducted on the appearance of technology advancements, still little is known about the human qualitative side of the set of tasks machines can perform. Therefore, this thesis will focus on the different tasks that are currently performed by employees of the IT-audit department, and to what extent artificial intelligence can be used throughout the IT-audit process. This contribution is an important turnaround since previous research has highly focused on the functions and computerization of tasks, neglecting the human input.

(9)

9 | P a g e

2. Literature review

Innovation is the idea of practice, an object that is considered as new by a group of people or by individuals (Rogers, 1995). In this paper we will discuss technological innovations as defined by Rogers (1995) ‘’an idea, practice, or object that is perceived as new by an individual or other unit of adoption’’.

Rogers (1995) has identified five characteristics of innovation that determines whether an innovation will be adopted by its members that are engaged in a joint problem. The five characteristics of innovation are relative advantage, compatibility, complexity, trial ability and observability. Relative advantage is explained as ‘’the degree to which the innovation is perceived as better than the idea it overrules’’. Relative advantage in the case of innovation refers to the extent whether an innovation is perceived to be more productive, cost efficient, or an improvement in another way. Compatibility has to with the socially acceptance of innovations. It is defined as ‘’the degree to which the innovation is perceived as being true to existing values, taking in charge past experiences and the needs of potential adopters. On the other hand, observability is defined as ‘’the degree to which the innovation is clear and noticeable to the rest’’. Lastly, trial ability is defined as ‘’the degree to which the innovation may be investigated with on a limited basis’’. In fact, this describes that in order to adopt an innovation it is more convenient for the group of individuals to cut innovations into pieces and test it in small parts.

Moreover, the innovation decision process as described by Rogers (1995) has five stages. This process is about the different stages an individual will encounter once (s) he enters an innovation-decision process. With this process the individual creates attitude and forms an approach towards the innovation, whether to adopt or reject the new idea. The different stages are classified as:

1) Knowledge: the innovation is exposed to the individual and the individual gets an

understanding of how the innovation functions;

2) Persuasion: the individual forms an positive or negative attitude towards the innovation; 3) Decision: in this stage the individual is engaging in activities that leads to adopting the

innovation or in activities that leads to rejecting the innovation;

(10)

10 | P a g e 5) Confirmation: the individual seeks support from the innovation-decision that already has

been made but may lead to reversion if the individual is exposed to contradicting information about the innovation.

Technology innovation has a big influence in the auditing process. The Big Four auditing companies use the business processes and the technology systems that are being used in the client company in order to understand and interpret the financial statement balances (Bell et. All, 1997). The auditor set some queries and analyzes the data to determine controls that are used or are missing in the business process. This makes it possible to address and identify risks.

A lot of companies follow the idea of a risk-based audit approach and acquired software that helps the auditor in understanding how internal and external risk affect the process. The software packages help also in the process of selling risk management and/or risk identification to potential and existing clients (Bierstaker, Burnaby, & Thibodeau, 2001).

According to (Helms & Mancino, 1998) there are three major technologies identified in the process of auditing, these are: security, electronic commerce and internet. The first group, security, deals with procedures and policies to assure that data, software and equipment are constricted to authorized users. Electronic commerce involves in technologies like automated teller machines, electronic funds transfers and other information technologies. The major audit interests are that the only legitimate transactions are received and transmitted, and importantly that every transaction is unique and not copied. As massive transactions are handled over the internet, auditors needs to acknowledge control issues to assure acceptable security protocols and procedure.

One of the forms of firm’s capacity is the intellectual asset which can be managed and improved by using technology. Many firms have invested in wide range database of firm documents and assigned one individual who is responsible for the data management. The objective of the database is to avert copies of documents such as spreadsheets, presentations and so on. This will reduce the time needed for the professional services to complete these documents (Bierstaker, Burnaby, & Thibodeau, 2001).

Auditors use technology by assessing risks in the key business processes of their client company. In order to understand the internal control and establish evidence the auditor needs to analyze the

(11)

11 | P a g e business process. This needs to be performed in an environment where there are aspects of enterprise wide computing. (Bierstaker, Burnaby, & Thibodeau, 2001)

There are more than thousands companies using some form of enterprise computing software such as SAP and Oracle. It is been predicted by (Gibss, 1998) that the use of enterprise computing will grow to 35% per year. For auditors this means that they need to be aware of this growth and how to understand these systems, since auditing requires the auditors to audit the technology that the client company is using to process data (Gibss, 1998). Testing complex controls such as firewalls, sensitive information, passwords, and authentication of electronic images may be necessary when dealing with electronic evidence (Helms & Mancino, 1998). Enterprise-wide computing platforms makes it available to audit and evaluate internal controls. Audit software has enormous possibilities to make the audit process more effective and efficient. Some of the possibilities are: inspecting, analyzing and selecting immense amount of data, making recalculations, scanning unusual transactions, and continuously monitoring (Bierstaker, Burnaby, & Thibodeau, 2001). Keeping in mind that big data is growing day by day enormously with a fast rate, implies that organizations are storing more and more data. Although big data is beneficial for businesses and society, it also brings challenges to the world. Challenges as protecting sensitive data, how to organize the data, and understanding the amount of data in order to say something about the data. One way of managing big data and making it useful for the organization is using the technique of artificial intelligence (AI). (Wani & Jabin, 2018)

2.1. Artificial intelligence

AI has two major distinct forms: strong AI and weak AI. Strong AI is a self-consciousness computer that can act and think independently, just like a human being, which is contrary to weak AI. Strong AI learns from examples and are data driven. Even the tiniest relationship among the data is captured by strong AI. (Lee & Shin, 2018)

According to Lee & Shin (2018) they describe weak AI as ‘functional AI’ which refers to a computer without self-consciousness and has the main purpose to improve productivity for a specific task and to contribute to the fields where the human being is limited. Most of the AIs that are implemented are of the latter form (Lee & Shin, 2018). Even more, Deis and Giroux (1992)

(12)

12 | P a g e predict that the digitization of work will increase and requires changes in different dimensions across organizations such as competencies of employees, workspaces, and work equipment. Moreover, AI is able to be a self-organized neural learning model which can make predictions. Trough incrementally processing perceptual cues and classifying input that has been brought into the computer (German, Tani, Cornelius, & Wertmer, 2017). In other words before the computer can make any predictions or carry out a task, it needs training to make predictions or to carry out tasks. The training of a strong AI computer is the process of establishing the arc weights which are the basis of AI. All the knowledge that is stored and learned by the network happens in the arcs and nodes in types of node biases and arc weights (Zhang, Patuwo, & Hu, 1998).

Auditors get a lot of data from their clients to analyze. Analyzing the data is a difficult and work intensive duty that can take hours. The three different forms of AI techniques (informative, reactive or proactive) can be used to make this process more efficient and effective.

2.1.1 Supervised learning

With supervised machine learning one has access to example cases that can be used during training phases to determine the exact input and output that needs to be produced. A common used example for supervised learning is as following, the computer is shown a sum of figures of handwritten digits and labels the handwritten digits. After this, the computer learns the different patterns that are related to unique labels. The problem with supervised learning is that we can predict patterns to a certain level that is known but we are not able to put unknown patterns into the algorithm since we don’t know these patterns. So, to some degree we want the algorithm to be capable enough to detect known as well as unknown anomalies and potential fraud patterns. (Fraser, 2017)

2.1.2 Unsupervised learning

With unsupervised learning the machine is looking to structure the given data by finding patterns and cases that are similar to each other (Hastie, Tibshirani, & Friedman, 2009). In this paragraph two most used techniques of unsupervised learning are presented.

(13)

13 | P a g e

2.1.3 Data clustering

Any form of data can consist of hidden patterns and features that are unknown for humans to classify it into labels or cannot be labelled manually (Frank, 1994). Therefore, the technique of data clustering can be applied into cases to identify hidden patterns and features by storing the most important

information instead of storing the raw or actual dataset to identify a structure in large amounts of data. In other words, the objective of clustering is finding patterns in unlabeled dataset that are similar or dissimilar to label a set of unlabeled data (Mishra, 2017).

2.1.4 Expert systems

Expert systems are designed with artificial intelligence technology that combines set rules and experience of an engine in order to apply this into a specific situations to solve problems (Turban, 1992). The strength of an expert system is the specific knowledge about a domain where the system is storing all the information about. An expert system can be enhanced by adding more knowledge and rules into the system. The artificial intelligence capability allows the system to improve previous performances of the system based on prior experience (Rouse & Petersen, sd). An expert system can be used as a decision maker or as an assistant to decision makers (Turban, 1992). According to Turban (1992) expert systems can be applied in variety of situations like: diagnosing anomalies from the observed data; classifying data based on components, monitor data from observed data to prescribed data or action, generations of alternative outcomes and possible solutions to specific problems.

2.2. Technology acceptance model

However, the use of technology and growing impressive possibilities of technology are expanding. For technologies and innovations to be adopted, they must be used and accepted by employees of the organization. Some studies have focused on the possibilities of technology and how to implement it in the organization, without saying anything about the adoption of technology (Leonard-Barton and Descamps, 1988).

(14)

14 | P a g e Davis (1993) explains which mechanisms are needed in technology that influence user acceptance. The identified mechanisms are perceived usefulness of the technology, perceived ease of use and the system design features mediates the effect on usage.

On the other hand, Campeau and Higgins (1995) found that individual acceptance of technology depends on different individual variables which are self-efficacy, performance and prior performance. Self-efficacy is about to influence judgments as well how to execute formally asked behavior. Prior successes or failures, which is called prior performance, have an influence on self-efficacy which in turns influences performance.

In 1985, Fred Davis, the founder of the Technology Acceptance Model (TAM) explains the causal relationship between technology acceptance and perceived ease of use, attitude toward using, perceived usefulness and actual usage behavior. This model is based on psychology principles, adopted from Azjen & Fischbein (1977), which has a great focus on behavior-relevant variables of human attitudes, discern between one’s beliefs and one’s attitudes and how external variables have a causal link to one’s beliefs, attitudes and behavior.

In addition to this Venkatesh, Morris, Davis & Davis (2003) conducted a research to identify a unified theory of technology

acceptance as shown in figure 2. Furthermore, Venkatesh et.al (2003) found that user acceptance depends on performance expectancy, effort

expectancy, social influence and facilitating conditions.

According to the study of Venkatesh et. Al (2003) performance expectancy is defined as ‘the degree to which an individual believes that using the system will help him or her to attain gains in job performance’. Effort expectancy is defined as ‘the degree of ease associated with the use of the system’. Social influence is defined as ‘the degree to which an individual perceives that important others believe he or she could use the new system’. Facilitating conditions is defined as ‘the degree to which an individual believes that an organizational and technical infrastructure exists to support the use of the system’.

(15)

15 | P a g e

2.2.1. Task model

As described before, there are various activities involved in the IT-auditing process and there are numerous techniques that can be applied to work more efficiently. Levy and Murnane (2003) developed the task model which describes which tasks can be performed by computer based systems. A computer can suit as substitutable or complementary on workplaces. Levy and Murnane (2003) made a task model and they identified four main categories; routine, non-routine, cognitive and manual tasks. Routine tasks refer to tasks that can be completed by following pre-set rules and are highly repetitive, whether manual or cognitive. Non-routine tasks according to this model (2003) are tasks that cannot be completed following pre-set rules because this kind of tasks are not conventional. Moreover, non-routine cognitive tasks are tasks that require problem-solving capabilities, creativity, and complex communication.

2.3. IT-Audit

Furthermore, IT-auditors try to identify and control risks that have to do with the use of IT in an entity. Most of the time this process is done manually to obtain sufficient audit evidence with respect to aspects in the financial statements. (International standard on auditing 550 related parties, 2009). One component of the risk assessment is the general IT control whereby the IT-auditor obtains an understanding of the IT-landscape in the entity. Elements of General IT control that are in scope of this thesis are: Access to Programs and Data (access management), Program Changes, Program Development and Computer Changes (change management) (see table 1). According to the International Standard on Auditing (ISA) IT benefits the entity in various ways, some examples are: enhancement of the timelines, enhancement of accuracy and availability of information, facilitation of additional analysis of information, enhancement of the ability to achieve effective segregation of duties by implementing security controls in IT applications, databases, and operating systems.

On the other hand, IT brings also risks affecting the internal control of an entity, such as processing inaccurate data, bringing unauthorized changes to data, bringing unauthorized changes to systems or programs, and the possibility of IT personnel gaining access privileges beyond those necessary to perform their assigned duties thereby breaking down segregation of duties, are illustrations of these risks related to IT. (KPMG, 2017)

(16)

16 | P a g e In table 1 the general IT controls are described.

Table 1: Elements of General IT Control

2.4. Thesis contribution

Since a lot of previous research has been conducted on the appearance of technology advancements, still little is known about the human qualitative side of the set of tasks machine learning can perform. Also, with the rise of machine learning and its applications this study aims to investigate to what extent artificial intelligence can be used. Therefore, this thesis focus on the different tasks that are currently performed by employees of the IT-audit department, and investigate to what extent artificial intelligence can be used throughout the IT-audit process. This contribution is an important turnaround since previous research has highly focused on the functions and computerization of tasks, neglecting the human input.

Elements of general IT control

Description

Access to programs and data (Source: ISA 315 Appendix 1.9)

Controls established by management to reduce the risk of unauthorized/unappropriated access to the relevant information systems related to financial reporting and prevent individuals from perpetrating and concealing an error or irregularity.

Program changes

(Source: ISA 315 Appendix 1.9)

Controls established by management to determine that changes to existing systems/IT applications are authorized, tested, approved, properly implemented, and documented.

Program

Development (Source: ISA 315 Appendix 1.9)

Controls established by management to determine that new systems/IT applications which are developed or acquired are authorized, tested, approved, properly implemented and documented.

(17)

17 | P a g e

3. Research model

This chapter describes the research model that is crucial for conducting this study.

The task-model variables of Levy & Murnane (2003) are applied as moderators in the conceptual model. This model explains routine tasks as tasks that follow pre-defined set of manual and cognitive repetitive activities that can be achieved attending exact rules that are provided. Therefore the proposition are as following:

P1A: Routine (manual, cognitive) tasks positively influences the relationship between pro-active AI and intention to use AI.

P1B: Routine (manual, cognitive) tasks positively influences the relationship between reactive AI and intentions to use AI.

P1C: Routine (manual, cognitive) tasks negatively influences the relationship between informative AI and intention to use AI.

Levy & Murnane (2003) determined non-routine tasks as activities that are more complex and involve problem-solving tasks that cannot be described or set in programmed rules. Non-routine tasks are in contrast to routine tasks often not repetitive. Non-routine tasks according to this model (2003) are tasks that cannot be completed following pre-set rules because of this kind of tasks are not conventional. Moreover, non-routine tasks are tasks that require problem-solving capabilities, creativity, and complex communication. In addition to this, Polanyi (1966) suggested that non-routine tasks can be recognized as tasks where the pre-defined rules are not sufficient enough for a machine to be understood. Therefore the proposition is that non-routine tasks have a negative influence on pro-active AI and intention to use, but controversially have positive influence on informative AI and intention to use.

P2A: Non-routine (manual, cognitive) tasks negatively influences the relationship between pro-active AI and intention to use AI.

P2B: Non-routine (manual, cognitive) tasks negatively influences the relationship between reactive AI and intention to use AI.

(18)

18 | P a g e P2C: Non-routine (manual, cognitive) tasks positively influences the relationship between informative AI and intention to use AI.

These five propositions are formulated to be give a comprehensive answer to the research question ‘’To what extent can artificial intelligence be integrated throughout the IT-auditing process to ensure activities that have been occurred are authorized, accurately recorded and processed?’’

Intention to use is defined as the likelihood that employees will use the intended technology (AI). Proactive AI

Reactive AI

Forms of Artificial intelligence

Intention to use Artificial Intelligence throughout the IT-audit process — Routine — Non-routine — Manual — Cognitive Informative AI Task model

(19)

19 | P a g e

4. Research methodology

This chapter describes the chosen research methods. The first section is about the chosen data and the second section provides background information about the interviewees.

The objective of this research is to attain an understanding to what extent artificial intelligence is intended to be used in an IT-auditing firm. Moreover, this research takes a qualitative approach to examine the human side separately from the technological advancements and possibilities. Moreover, qualitative research design makes it possible to understand the meaning of the given answers because of context one gives an answer to. Since there is relatively a small number of participants (N=10) involved in this study, specific situations study can be analyzed and how these situations shape individual experiences (Maxwell, 1992).

The overall research approach is a combination of deduction and induction. The process of deductive research is defined as ‘’research approach which involves the testing of a theoretical proposition by using a research strategy specifically designed for the purpose of its testing’’ (Saunders & Lewis, 2016). This study has gathered information together about the potential relationship between the different forms of AI, which variables can strengthen or weaken this relationship and the intention to use AI. On the other hand, the start point of an inductive research approach is gathering data together to examine and subsequently develop a new theory based on the gathered data (Saunders & Lewis, 2016). This study also employed an inductive approach by collecting qualitative data to acquire additional perceptions about the subject.

4.1. Method

This section describes how the research method were conducted and describes the background of the interviewees.

During my internship I attended an internal meeting with IT-auditors whereby I informed the consultants for the first time about participating in the study. After this meeting, I have sent them an email with further details of the interview and made an interview-schedule (appendix I). Malsch and Salterio (2016) outline in their study about qualitative research design, that a high N is most desirable. Yet, conducting a qualitative research approach is not about conducting statistical significant findings. Rather the power of qualitative research lies on conducting a field

(20)

20 | P a g e specific understanding by taking interviewees with experts that have the knowledge and experience about the phenomena (Malsch & Salterio, 2016).

4.1.2 Semi-structured interviews

For this study, 10 IT-audit consultants were interviewed from a big four company, with various experiences and expertise. The participants work on different levels ranging from director level to junior consultant level. With the consent of the interviewee the interviews were recorded, and on basis of the records the interviews could be further transcribed. The interviewees were held in Dutch and subsequently translated into English.

The interviews were held on face-to-face basis during a period of five months. The data has been collected between February 2018 and June 2018. The questions of the interviews were semi-structured conducted to ensure some degree of comparability and also to understand the context of the answers. The interview questions can be found in appendix II. I have chosen for semi-structured interview method because this structure allows to set up a general structure for the interviews that needs to be covered by several fixed questions that are arranged based on the literature review. At the same time, the semi-structure method allows a decent degree of flexibility for the interviewer and interviewees (Drever, 1995). Also, this form of interview structure supports the research question; to gain an understanding which form of AI is most likely to be accepted by IT-consultants. Furthermore, to make the interviews as convenient as possible for the interviewees, the interviews took place at the offices of the interviewees. The duration of the interviews ranged from 8 minutes to 36 minutes. The average duration was 21 minutes.

4.1.3 Coding

The interviews are transcribed and coded with the software NVivo version 10. The codes were extracted from existing literature. The extra nodes that are added during the coding procedure are derived to gain in-depth understanding of the context of an IT-auditor. The coding scheme can be found in appendix IV.

4.1.4 Target group

The respondents of this study are IT-audit consultants (N=10). The participants work on different levels ranging from management level to junior consultant level and are between 24-48 years old.

(21)

21 | P a g e In table 2 an overview is given of the interviewees for this study, including their function, duration of employment, age, gender and duration of the interview.

Table 2: Interviewees

Interviewee Function Duration of employment

Age Gender Duration of interview Director Director 22 years 48 M 32 minutes

Manager A Manager 10 years 33 M 23 minutes

Manager B Manager 6,5 years 32 M 18 minutes

Senior consultant A

Senior consultant 6 years 34 M 21 minutes

Senior consultant B

Senior consultant 4 years 28 F 20 minutes

Senior consultant C

Senior consultant 4 years 27 M 36 minutes

Senior consultant D

Senior consultant 3,5 years 26 F 16 minutes

Consultant A Consultant 2 years 26 M 23 minutes

Consultant B Consultant 11 months 26 M 8 minutes

(22)

22 | P a g e

5. Evaluation of research quality

Qualitative interviews can decrease the construct validity of the research. Construct validity refers to the extent a researcher is measuring the things (s) he wants to measure. To mitigate this threat, I tried to assure consistently that the interviewees had the same understandings as I had on the different elements. In addition to this, during the interviews every interviewee was accommodated a description about the different forms of AI and the different tasks within the IT-auditing process to provide clarification about the terms.

(23)

6. Results

This section deals with the results of the research. The results are conducted from the qualitative interview data set to examine which form of AI is most likely to be used during the IT-auditing process. During the interview 7 main tasks of the IT-auditing process has been questioned and which form AI (proactive, reactive, informative) the IT-auditors would like to use at every task.

The different tasks have different levels of routinization and cognitive requirements. The respondents have overlap in their answers in relation to the degree of routinization and cognitive requirements of a particular task. The task model, whether a task is performed on routine basis or requires critical thinking, determines which form of AI is chosen by IT-auditors.

P1A: Routine (manual, cognitive) tasks positively influences the relationship between pro-active AI and intention to use AI. P1B: Routine (manual, cognitive) tasks positively influences the relationship between reactive AI and intentions to use AI P1C: Routine (manual, cognitive) tasks negatively influences the relationship between informative AI and intention to use AI.

The three propositions are plausible to be true based on the qualitative data set. As can be seen in table 3 proactive AI is mostly used when a task is described as routine based, standardized, not requiring professional judgment. The following quotes support this finding:

‘’A lot can be done proactively by the system itself. Especially in very routine based components of some of the tasks like checking whether the required evidence is received. It is literally comparing row A with row B. At the moment, this happens manually, which is very time consuming and not cognitive challenging at all’’ (manager A). Moreover, senior-consultant B said: ‘’By using proactive AI the system will be able to extract a sample that is 100% representative of the population which is more reliable than when I, as a human, is taking a sample which is for example 95% representative for the population.

Another respondent said: “Sometimes, even when a task is very standard and routine based, I would prefer to use reactive AI in order

to have control over the whole process and to have an understanding what is really happening and why it is happening. Especially, tasks where there can be a lot of exceptions. If a system is performing all the tasks, as an IT-auditor it will be very hard to have an

(24)

in-24 | P a g e

depth understanding about the situations and develop e a professional judgment about the situation’’ (consultant B). In addition to this,

consultant A said: ‘’Even though, this task is not cognitive challenging, it requires sometimes professional judgment about the

documentation we receive from the client. I would choose for reactive AI because sometimes we get documentation from the client which is not exhaustive or faultless. I know whether a documentation is exhaustive or complete because of my professional judgment. Also, by bringing in the input for the system that it needs to perform a particular task, I will be in control’’.

The data from the qualitative interview with the director indicated that he was willing to adopt AI throughout the IT-auditing process. Out of the 7 identified tasks, the director choose 4 times for using a pro-active form of AI. The director mentioned: “These tasks are

very easy to automate, since it does not require any critical thinking or complex thinking. It is very routine based, therefore a pro-active system can perform them, which will be time and cost efficient’’.

The director is almost similar to the managers in their preference for an AI form throughout the IT-auditing process, except for the task ‘audit documentation from selected sample’. The director choose for using an informative form of AI for this task and the managers choose for a proactive/reactive form of AI. The director said about using informative form of AI by auditing the documentation: ‘’I

won’t use pro-active AI by this task because it is more than just a check mark whether it is valid or invalid. As an IT-auditor you need to think why a document is valid or is not valid because you need to justify it to your client. You want to know the core reason why a certain document is valid or is not valid, you need to know the context of it’’. Manager B said about auditing the documentation: ‘’It is very easy and it is routine-based, especially for very easy tasks like; auditing how many people left the organization in a particular year and how many people entered the organization in a particular year. But I am willing to take a further step in this; I would use proactive AI also for very complex processes because the control will be the same, whether a task is complex or very easy. For example: every change integrated in the organization has a standardized procedure. First, there needs to be a request for a change, the next step is whether the change has been approved by the super user and the last step is whether the release manager has approved to implement the change in the production. This is always the same procedure you are testing (interview manager B). The qualitative data set with

(25)

25 | P a g e the director and the managers indicated that if a task is largely routine based, proactive or reactive AI is most likely to be used and if a task is non-routine based and involves cognition informative AI is most likely to be used.

P2A: Non-routine (manual, cognitive) tasks negatively influences the relationship between pro-active AI and intention to use AI. P2B: Non-routine (manual, cognitive) tasks negatively influences the relationship between reactive AI and intention to use AI. P2C: Non-routine (manual, cognitive) tasks positively influences the relationship between informative AI and intention to use AI.

The three propositions are plausible to be true based on the qualitative data set. Nine out of ten respondents indicated to use informative AI by the task ‘understanding of IT’. The majority of the respondents argued that the output of understanding of IT determines the further developments in the IT-audit process. An informative AI based tool can serve as a base tool for the interview of understanding of IT. The director of IT-audit mentioned: ‘’It would be very nice if the informative AI tool sends a questionnaire to the client about IT

systems that are in use in the organization, which kind of IT governance they have and so on. Subsequently, I want to receive a short description from the AI tool which IT systems are in use by the organization. As a consequence of this, I will be able to dive in during the interview with the client on the most important questions or subjects. This will make the process understanding of IT more efficient’’.

In addition to this, manager B mentioned: ‘’I would use informative AI to prepare myself for the interview of understanding of IT. I don’t

think that an AI robot can replace an IT-auditor during the interview, since an interview is not only about questioning and giving answers. It is a social interaction between two humans where emotion recognition is involved. Also, during an interview you can down scope to certain subjects if I need more clarification’’.

As can be seen in table 3, the four senior consultants are very consistent in their answers. For the task ‘write management letter’ the senior consultants indicate different forms of AI to use. Four out of four senior consultants indicate to use informative form of AI. Senior consultant A mentioned that using proactive AI will not result into a personalized management letter that acknowledges soft findings as

(26)

26 | P a g e well, such as the experienced ambiance and noticed emotions during social interactions. Senior consultant D added to this that choosing for informative AI will also support the next step which is sharing the management letter with the client in a personal meeting. Also choose for informative AI. Senior consultant D states: ‘’I want to use informative AI by writing my management letter for the client

because the system can help me to give signals what I need to write down, what are the most important findings and so on. But I want to write the management letter by myself because I can be in lead of the words I am choosing to write down. It is very important to write very carefully all of the findings, to acknowledge the emotions of the client. This will support to maintain the relationship between me and the client’’. In contrast to the senior-consultants, two out of three consultant choose to use proactive AI by writing the management

letter. Consultant B mentioned that writing the management letter is routine based and is actually a summary of the findings and procedures that are already written in eAudit. He (consultant B) suggests to implement a proactive AI tool that can extract the findings and procedures form eAudit and write a management letter from it. Moreover, consultant A mentioned: ’’I would like to outsource the

task’ writing a management letter’ to a proactive AI tool. But, I would insert a review step by myself after the proactive system has written down the management letter in order to verify the system has included everything I would include in the management letter’’.

Furthermore, something noticeable are the tasks ‘understanding of IT’ and ‘share with the client’, where IT-auditors unanimous intend to use informative AI and do not intend to use any kind of AI. A clarification for this is the personal interactions involved in the meetings and to maintain long term relationships with the client through personal contact. Also, 10 out of 10 respondents indicate that often clients are very skeptical about the management letter and during a personal meeting with the client the management letter can be reasoned. Manager A indicated: ‘’AI can improve a lot of tasks that we are doing nowadays manually and tasks that do not require a lot of critical

thinking. However, I do not expect that AI is able to computerize tasks with a personal contact component on it. Since an interview in person is also about maintaining the relationship with the client, asking questions about the things you could not find in the documentations’’.

(27)

27 | P a g e For the second task ‘Receive documentation about access/change management from client’ seven out of 10 respondents choose to use proactive AI due to the standardized characteristic of the task. Only one consultant prefers informative AI. Consultant C mentioned:

‘’Sometimes I do get documentation from the client but the documentation does not provide enough information for me as a consultant. I do not trust whether the system is capable enough if it can recognize whether a given information is enough or not. That is why I am choosing for informative AI’’. This indicates that consultant C does have concerns about the perceived usefulness of the technology. In

contrast to this, the seven respondents that choose for proactive AI argue that the second tasks is repetitive and has low risk concerns. Two respondents choose for reactive AI. Senior-consultant C mentioned: ‘’I want to be in control of the requested documentations that

the client needs to provide. This is because of the reason that I am able to judge which documentations are needed based on the profile of the client. After I have indicated which documentations need to be requested, the system can run a documentation check’’. For the

task ‘Identify sample size based on risk of failure and frequency of control’ seven out of ten respondents indicated to use proactive AI due to the cognitive standardization of the task.

For the task ‘audit documentation from selected sample’ five out of ten respondents indicated to use proactive AI. The respondents mentioned that auditing the documentation does not require critical thinking or professional judgment. Also, auditing the documentation is based on some hard rules that can be programmed easily in an algorithm. The director argued that he would be able to use informative AI by auditing the documentation. The director mentioned: ‘’I won’t use pro-active AI by this task because it is more than just a check

mark whether it is valid or invalid. As an IT-auditor you need to think why a document is valid or is not valid because you need to justify it by your client. You want to know the core reason why a certain document is valid or is not valid, you need to know the context of it’’.

Next, the IT-auditor needs to register the findings and procedures in eAudit. Eight out of ten respondents indicate to use proactive AI during this task. The director mentioned: ‘’by auditing the documentation the professional judgment of the IT-auditor is really necessary,

But when entering the procedures and findings, it can be easily done by a proactive tool. Currently, a lot of the auditors are doing this manually by copy – pasting what they have already written’’. Senior consultant A indicated to use reactive AI, he mentioned: ‘’in the

(28)

28 | P a g e

previous step I chose for proactive AI, so the task is fully computerized. In this step I want to use reactive AI so that I can review the output the from the previous steps that proactive AI has ran’’.

Overall, regardless which management level an employee has, their perceptions are generally similar. Proactive AI is more intended to be used when a task is on routine basis and standardized, whereas reactive AI is more intended to be used in cases a task requires the perception of a professional before it can be performed. Lastly, informative AI is intended to be used in cases where routinization of a task is low. Table 3: Results Function Understanding of IT Receive documentation about access/change management from client Identify sample size based on risk of failure and frequency of control Audit documentation from selected sample Register in eAudit about procedures & findings Write management letter Share with client

Director Informative Proactive Pro-active Informative Pro-active Proactive Nothing

Manager A

Informative Pro-active Pro-active Reactive Pro-active Proactive Nothing

Manager B

(29)

29 | P a g e Function Understanding of IT Receive documentation about access/change management from client Identify sample size based on risk of failure and frequency of control Audit documentation from selected sample Register in eAudit about procedures & findings Write management letter Share with client Senior-consultant A

Informative Pro-active Pro-active Pro-active Reactive Informative Nothing

Senior-consultant B

Informative Pro-active Pro-active Pro-active Pro-active Informative Nothing

Senior-consultant C

Informative Reactive Pro-active Pro-active Pro-active Informative Nothing

Senior-consultant D

(30)

30 | P a g e Function Understanding of IT Receive documentation about access/change management from client Identify sample size based on risk of failure and frequency of control Audit documentation from selected sample Register in eAudit about procedures & findings Write management letter Share with client Consultant A

Informative Pro-active Pro-active Reactive Proactive Pro-active Nothing

Consultant B

Informative Pro-active Reactive Pro-active Pro-active Pro-active Nothing

Consultant C

(31)

As a summary, table 4 provides an overview of the tasks that are routine –cognitive and tasks that are non-routine - cognitive. It is most likely to use proactive AI for the tasks that are routine – cognitive and informative AI or no AI for non-routine – cognitive tasks. Reactive AI is preferred to be used by IT-auditors when a task is manually or cognitive performed following pre-set rules only if the task requires professional judgement. This implies that cognitive routine tasks can also be complementary in contrast to the first proposition where AI is substitutionary. Additionally, the results imply that when a task is involved in personal interaction, creative thinking and perception none of the forms of AI is preferred to be used. This is also due to the characteristics of the tasks according to Frey and Osborne (2013) (e.g. perception and manipulation elements, creative intelligence, or tasks that require social intelligence).

Table 4: Routine - cognitive

Cognitive

Routine

Routine – manual Routine – cognitive

Receive documentation about access/change management

Identify sample size based on risk of failure and frequency of control

Audit documentation from selected sample Register in eAudit about procedures & findings

Non routine – manual Non routine – cognitive

Understanding of IT Write management letter Share with client

(32)

32 | P a g e

7. Discussion

This qualitative research contributed to the scientific literature to what extent artificial intelligence can be integrated throughout the IT-auditing process in order to ensure activities that have been occurred are authorized, accurately recorded and processes. There are limitless studies that have been conducted on the appearance of technology and technological advancements. Also, a lot of research has been done on computerizing tasks and ignoring the human side of these developments. The research question of this research is: To what extent can artificial intelligence be integrated

throughout the IT-auditing process to ensure activities that have been occurred are authorized, accurately recorded and processed?

This research points out that throughout the IT-auditing process there are different kinds of AI preferred to use by IT-auditors, moderated by the task model. The results of this research indicates that if a task is performed regularly following pre-defined rules, on a routine basis, proactive AI is more likely to be used than informative AI. On the other hand, when a task requires critical thinking, and professional judgement informative AI is more likely to be used. Moreover, the qualitative data indicates that if a task involves personal interaction with the client none of the AI forms is preferred to be used by IT-auditors.

Moreover, as can be seen in table 3 there are a lot of cognitive tasks that are repetitive and can be automated throughout the IT auditing process. In the model of Levy and Murnane (2003) it can be classified into routine cognitive. However, the majority of the respondents consider that the competitive advantage of an IT-auditor is due to their professional judgment and their social and emotional intelligence. According to Frey & Osborne (2013) also the cognitive – non routine tasks can be computerized due to the availability of big data. Machine learning algorithms are able to manage complex and enormous amounts of data. Specifically, machine learning makes it possible to also computerize to some extent professional judgment which is required at some activities? Frey and Osborne (2013) discuss that the output of an algorithm can serve as an input for the human decision maker. However, machine learning is free of biases and is better in scaling large amounts of data in comparison with humans. It will be likely that eventually machine learning will computerize more cognitive non-routine tasks. Consequently to the computerization of these tasks, the IT-auditor will have more time to spend on other value adding tasks than cannot be easily computerized. According to Frey and Osborne (2013) tasks that cannot be computerized easily

(33)

33 | P a g e include perception and manipulation tasks, creative intelligence tasks and social intelligence tasks which are not very likely to be taken over by a computer in the next decades. Perception and manipulation tasks refer to recognizing objects in unstructured data or data that is not standard or daily handled. Creative intelligence tasks are ‘’the ability to come up with ideas or artifacts that are novel and valuable’’; social intelligence tasks refer to social interaction activities between human beings such as negotiating, taking care of and degree of convincing the other. As social interaction also include recognizing the emotion of the other one, and the ability to respond to the emotions it is not likely that a computer can take over these tasks. In accordance with this and the results shown in table 3 the tasks understanding of IT, writing management letter and share with client are concerned with the perception and manipulation of the IT-auditor, creative intelligence and social intelligence skills. As this tasks are involved in unregularly environments, creative solutions thinking, and some aspects of social interaction it is not likely that it will be computerized in the next decades (Frey & Osborne, 2013).

Researchers have developed a theory about digitalizing the human brain, it is called brain emulation (Calimera, Macii, & Poncino, 2013). In theory this technology of brain emulation is socially intelligent and is able to interact in social contexts. However, at this moment it is only a theoretical framework.

P1A: Routine (manual, cognitive) tasks positively influences the relationship between pro-active AI and intention to use AI.

Proactive AI is preferred by IT-auditors when a task is manually or cognitive performed following pre-set rules. The IT-auditors indicated that at this moment some of the tasks that are performed are time consuming and do not require any professional judgement. If the system can proactively computerize the tasks that do not require professional judgment, the IT-auditor will be able to do other things that are value adding (Frey & Osborne, 2013).

P1B: Routine (manual, cognitive) tasks positively influences the relationship between reactive AI and intentions to use AI.

Reactive AI is preferred to be used by IT-auditors when a task is manually or cognitive performed following pre-set rules only if the task requires professional judgement. This implies that cognitive

(34)

34 | P a g e routine tasks can also be complementary in contrast to the first proposition where AI is substitutionary (Autor, Levy, & Murnane, 2003).

P1C: Routine (manual, cognitive) tasks negatively influences the relationship between informative AI and intention to use AI

IT-auditors are least willing to adopt informative AI when the task is cognitive routinized. This is because of that it does not help to improve their current work of state in terms of quality or efficiency.

P2A: Non-routine (manual, cognitive) tasks negatively influences the relationship between pro-active AI and intention to use AI.

P2B: Non-routine (manual, cognitive) tasks negatively influences the relationship between reactive AI and intention to use AI.

In cases where the rules of a task cannot be predefined or programmed proactive and reactive AI are least preferred to use by IT-auditors. The reason behind this is because these tasks require to some extent professional judgement, or in cases of personal interview, they require social interactions which are capabilities that cannot be described in a code (Autor, Levy, & Murnane, 2003; Frey & Osborne, 2013).

P2C: Non-routine (manual, cognitive) tasks positively influences the relationship between informative AI and intention to use AI.

Informative AI is preferred to use when a task is concerned with professional judgment, perception and social/creative intelligence of the IT-auditor. In these tasks informative AI augments the cognition of the IT-auditor to process information (Frey & Osborne, 2013).

(35)

35 | P a g e

8. Conclusion

The aim of this study is to investigate to what extent artificial intelligence can be integrated throughout the IT-auditing process. Specifically, the objective of this study is to give an answer to the following research question:

To what extent can artificial intelligence be integrated throughout the IT-auditing process to ensure activities that have been occurred are authorized, accurately recorded and processed?

The objective of the study is accomplished. To accomplish the objective different fundamental theories were studied about technology acceptance (Davis, 1993;Venkatesh, Morris, Davis, Davis, F, 2003; Levy & Murnane, 2003) and about how innovations are diffused in organisations (Rogers, 1995). As a result of the literature study, the conceptual model was developed. Subsequently, the conceptual model is validated by conducting a qualitative study. The data has been gathered from ten employees who are working for a big four company in the Netherlands, from different management levels. The interviews were semi-structured. Afterwards the interviews were transcribed and coded using the software of NVivo. After the interviews had been conducted, six out of six propositions are plausible to be true. Informative AI is most intended to be used when a task is nonroutine and requires professional judgment or critical thinking. When a task is routine and cognitive, it is performed on predefined set of rule that determine what the outcome is. In this case, proactive AI is mostly intended to be used. When a task is routine and cognitive, but requires the perception of an IT-auditor reactive AI is mostly preferred to be used by the IT-auditors. Additionally, the results imply that when a tasks involves personal interaction, creative thinking, and perception of an IT-auditor none of the forms of AI is preferred to be used. Frey and Osborne (2013) argue that work tasks that are involved in recognizing human emotions, and require the ability to respond in a human intelligent manner are still a challenge to computerize. Even more, Frey and Osborne (2013) indicated that text interactions that occur over the internet and involve human emotions, an algorithm cannot judge in a human way how to process the information. They (2013) found that this has to do with the large amount of information that humans possess to make ‘’common sense’’.

The technology of artificial intelligence is not a new concept, but it is developed in the recent years to a more accessible technology for organizations. With the rise of World Wide Web and expanding usage of documentations, text files organization are collecting amounts of data.

(36)

36 | P a g e Organizations are finding a way to organize, analyze, and react properly upon the big data they have. AI techniques can serve in different ways to supplant or augment information processing activities (Levy & Murnane, 2003). In this study, we investigated in to what extent artificial intelligence could be integrated throughout the IT-auditing process taking the three forms (proactive, reactive, and informative) of AI into account. The tasks receive documentation about access/change management, identify sample size based on risk of failure and frequency of control, audit documentation from selected sample, register in eAudit about procedures & findings are classified as routine-cognitive in the model of Levy & Murnane (2003). These tasks consist a predefined set of rules to achieve a certain outcome. Proactive and reactive AI can be used to supplant human cognition in these tasks. The cognitive tasks that come with complex cognitive communication or creativity skills are limited in these cognitive routine tasks (Levy & Murnane, 2003; Frey & Osborne, 2013). In contrast to this, the following tasks understanding of IT, write management letter and share with client have a high degree of demand for flexibility, social interaction, complex cognitive thinking. For this tasks it is more feasible to use informative AI, as it can augment the human cognition for processing information.

8.1 Contribution to theory

There is little academic research done in the field of the three different forms of AI and the acceptance of it using the task model of Levy & Murnane (2013). This study combined the technology acceptance model with the task model to investigate to what extent AI is likely to be integrated in the IT-auditing process. Furthermore, the results of the study build upon the study of Frey and Osborne (2013). Additionally, this study included the relationship between the three different forms of AI and the task model.

8.2 Contribution to management practice

Based on the findings of this research discussed in the previous chapters and the theoretical construct as well, this sub-paragraph discusses the contribution for managers. For managers it is important to identify which processes belong to which domain in the task model of Levy & Murnane (2003). Also, it is important to identify what the characteristics of the processes are in order to understand the possibilities of the particular technology. The implication of the discussion is that coworkers can focus more on nonroutine-cognitive tasks that involve social intelligence, creative thinking, and the perception judgment of the professional. The tasks that are cognitive –

Task model applied : intention to use artificial intelligence throughout the IT-auditing process