Robotic Process Automation in Internal Auditing: Motivating factors for usage

(1)

Robotic Process Automation in Internal Auditing:

Motivating factors for usage

Lars Hisken

l.hisken@student.rug.nl

s2740397

Thesis supervisor: Dr. A. Bellisario

January 20th, 2020

University of Groningen

Faculty of Economics and Business

MSc BA – Management Accounting and Control

Word count: 12.904

(2)

2

Abstract

The internal audit profession is under pressure by the digitalization of the business environment to adapt its practices. The paradigm shift in internal auditing makes the adoption of automatization tools a necessity. However, internal auditors struggle to adopt these tools in practice. Prior research lacks in explaining usage considerations for the computer-assisted audit techniques and tools (CAATTs) used by internal auditors. This study examines CAATTs adoption by internal auditors through the lens of robotic process automatization (RPA). In developing an understanding of the factors motivating CAATT adoption, this research theorizes a contingency approach for extending the Unified Theory of Acceptance and Use of Technology (UTAUT). Using maximum variation sampling, eight interviews were conducted. The findings postulate fifteen dynamic motivational factors to the pre-adoption stage of RPA in internal auditing. Most influential are the market environment, digital strategy, knowledge sharing, and competency related factors. This paper contributes to contemporary literature by reducing the knowledge gap on motivational factors of CAAT adoption and prevalence in internal auditing. It also contributes to the expansion of UTAUT model research by theorizing dynamic contextual pre-adoption effects and cross-dimensional influences on technology use outcomes by extending the contingency theory approach to internal auditing.

(3)

3

1. Introduction

Attempting to take advantage of the deluge of data, organizations are using novel analytical tools that employ the concepts of business analytics (BA), data analytics (DA), and big data and data analytics (BDA) to support processes throughout the organization. New tools are used for providing new insights relevant to decision-making (George, Haas, & Pentland, 2014; Shollo & Galliers, 2016) and higher quality accounting information (Gepp, Linnenluecke, O’Neill, & Smith, 2018). This potentially leads to improvements in decision-making capabilities (McAfee & Brynjolfsson, 2012), the creation of more valuable products and services (Fosso Wamba, Akter, Edwards, Chopin, & Gnanzou, 2015), and higher quality audits (Pincus, Tian, Wellmeyer, & Xu, 2017; Salijeni, Samsonova-Taddei, & Turley, 2019). In line with this trend, Big Four clients are increasingly utilizing novel tools (Appelbaum, Kogan, & Vasarhelyi, 2017).

In contrast, the auditing profession appears slow in capitalizing on the opportunities that recent developments entail (Alles, 2015; Appelbaum et al., 2017; Gepp et al., 2018; Smidt, Ahmi, Steenkamp, der Nest, & Lubbe, 2019). To allow for efficient and effective audits (Smidt et al., 2019), the Big Four need to optimize workflows with digitalization in mind (Salijeni et al., 2019) and introduce incentives to promote the use of new tools over conventional auditing techniques (DeFond & Zhang, 2014). These tools allow for the enhancement of the audit process by improving workflow, audit judgment and audit quality (Brown-Liburd, Issa, & Lombardi, 2015; Moffitt, Rozario, & Vasarhelyi, 2018). Additionally, organizations must decide on appropriate auditing tools (Brown-Liburd et al., 2015; Moffitt et al., 2018) and deal with factors influencing the utilization of such computer-assisted auditing techniques (CAATs) (Bierstaker, Janvrin, & Lowe, 2014; Curtis & Payne, 2008; Li, Dai, Gershberg, & Vasarhelyi, 2018; Mahzan & Lymer, 2014; Smidt et al., 2019). The absence of motivating factors in an organization impedes CAAT adoption (Li et al., 2018; Mahzan & Lymer, 2014), making research on motivating factors for CAAT adoption relevant for audit professionals.

(5)

5

internal auditing (Moffitt et al., 2018). Therefore, this study focuses on RPA adoption in internal auditing (IA), answering the call for research on CAAT prevalence in auditing through the following research question: “What factors motivate the usage of robotic process automation in internal auditing?”.

(6)

6

2. Theoretical background

2.1 Internal audit in the digital era

According to the 2017 International Standards for the Professional Practice of Internal Auditing (IPPF) of the Institute of Internal Auditors (IIA), internal audit activity adds value to organizations when it “provides objective and relevant assurance, and contributes to the effectiveness and efficiency of governance, risk management, and control processes”. Internal audit activities are thus important for providing assurance to the executive management, board of directors and other internal stakeholders. IA activities include, but are not limited to, evaluating controls, evaluating risks, analysing operations, confirming information, and supporting other assurance providers (Chartered Institute of Internal Auditors, 2019).

Given the many activities of IAFs (Roussy & Perron, 2018), the IA profession is certain to be affected by the rise of new business models and will be compelled to adapt its practices in order to best serve their clients. Accordingly, technological innovations such as e-commerce and online payment processing have led to increasing volumes of electronic audit evidence (Brown-Liburd et al., 2015; Smidt et al., 2019; Vasarhelyi, Kogan, & Tuttle, 2015). With big data applications becoming essential to client strategies, IAs will progressively be required to process and analyse more extensive volumes of complex data to provide adequate and independent assurance to organizational stakeholders (Bos et al., 2017; Vasarhelyi et al., 2015). This is troublesome because audit procedures are traditionally labour and time-intensive due to the many manually performed audit procedures (Chan & Vasarhelyi, 2011). Internal auditors must thus adapt their added value proposition in order to maintain credibility (Alles, 2015; Farkas & Hirsch, 2016) and legitimacy (Salijeni et al., 2019).

The advent of big data has had a direct impact on the internal auditing profession, with recent revisions to how they collect audit evidence to ensure efficiency and effectiveness (Smidt et al., 2019). Standard 1220.A2 of the IPPF 2017 for internal auditors by the IIA requires IAs to consider using technology-based tools in exercising due professional care (The Institute of Internal Auditors, 2016). It is then puzzling why, despite the characteristics of contemporary data and the IIAs guidance, IAs are struggling to utilize novel tools in practice (Li et al., 2018; Mahzan & Lymer, 2014; Smidt et al., 2019).

(7)

7

2019). Fully automated audits are hence unlikely due to the necessity for professional judgment of what, for example, comprises satisfactory and appropriate audit evidence in the changing data environment of organizations (Appelbaum et al., 2017; Brown-Liburd et al., 2015; Brown-Liburd & Vasarhelyi, 2015; Issa et al., 2016; Richins, Stapleton, Stratopoulos, & Wong, 2017). Nevertheless, using automatization tools that thrive in this emerging environment can help auditors in exerting their judgment by providing more substantive arguments to their clients (Salijeni et al., 2019).

The automation of repetitive accounting and auditing tasks is related to the general trend of workforce automation (Moffitt et al., 2018), in which businesses are looking at the tasks humans must perform because computers cannot (Moffitt, Richardson, Snow, Weisner, & Wood, 2016). Technology-based audit techniques may enable auditors to overcome information processing limitations, better determine information relevance and aid in pattern recognition (Brown-Liburd et al., 2015). As a consequence, auditor activities could become less repetitive and more focused on adding value to the business of clients (Richins et al., 2017). Accordingly, Chan & Vasarhelyi (2011) propose that the traditional audit paradigm has become outdated in the new economic environment and conclude the necessity of automation technology in developing the shift to real-time assurance.

Prevailing automation tools in IA are computer-assisted auditing techniques (CAATs) which potentially allow for effective and efficient audits in the new economic environment (Smidt et al., 2019). While in general not as complex as advanced BDA tools, CAATs can help IAs in performing repetitive and simple tasks, arising from their broad range of activities in, for example, fraud investigations, financial and operational testing, security testing, information retrieval, continuous monitoring and risk analysis (Li et al., 2018; Mahzan & Lymer, 2014). Using CAATs as an automation technology to aid in enabling real-time assurance is a reasonable step in pursuing the organizational data maturity levels and innovative audit methodologies required for achieving continuous auditing and continuous monitoring (Chan & Vasarhelyi, 2011; Smidt et al., 2019) and may aid IAs in providing assurance over voluminous and complex datasets (Salijeni et al., 2019; Vasarhelyi et al., 2015). Internal auditing applications for BDA-enabling tools such as CAATs may also improve external audit quality through improving IAF quality (DeFond & Zhang, 2014; Li et al., 2018) and could help IAFs in enhancing internal control effectiveness, leading to higher operational efficiency (Cheng, Goh, & Kim, 2018).

(8)

8

because internal auditors tend to have better access to organizational data (Vasarhelyi et al., 2015) and are less restricted by regulations than external auditors (Li et al., 2018). Furthermore, findings on CAAT adoption in IAFs may be applicable across auditing contexts because some IA activities are similar to external audit activities in the sense that comparable techniques could be applied (Cao, Chychyla, & Stewart, 2015; Li et al., 2018), such that contributions to this domain of research are multidisciplinary. To aid the discussion on CAAT prevalence in internal auditing and to produce insights into the automation tools used by internal auditors, this research considers RPA as a CAAT for internal audit activities in the next section.

2.2 Robotic process automation in internal auditing

As stated in the 2017 IPPF standards of the IIA, technology-based audit techniques include any automated audit tool, such as CAATs. Smidt et al. (2019) observe that literature divides CAATs into five categories; 1) test data, 2) parallel simulations, 3) integrated test facilities, 4) embedded audit modules, 5) generalized audit software (GAS). Commonly used programs such as Microsoft Excel and Microsoft Access are CAATs that require low levels of data maturity. In contrast, well-known GAS software packages such as Galvanize (formerly known as ACL) or IDEA require higher levels of client data maturity (Smidt et al., 2019). They are database tools that can assist IAs in extracting data from unstructured sources without modifying the source data, keeping audit logs of input commands, and allowing users to create scripts for specific tasks.

(9)

9

to deal with large datasets for which they needed to provide assurance. Using RPA could allow them to analyse complete data sets for exceptions, outliers, and trends (Brown-Liburd et al., 2015). If client data is properly structured and the auditor’s judgment is based on quality information, full population testing could terminate audit item associated risks completely through auditing by exception (Brown-Liburd & Vasarhelyi, 2015; Chan & Vasarhelyi, 2011; Richins et al., 2017). A key consideration for the usage of RPA in population testing is how it can solve human bottlenecks for information processing. Human bottlenecks exist when the automation of certain audit procedures can reduce the amount of human labour required or results in a higher output within a shorter timeframe. Automation may then contribute to audit efficiency (Chan & Vasarhelyi, 2011).

Another application for RPA is assisting the IA in interpreting, evaluating and applying the patterns uncovered with BDA tools by performing rule-based analyses automatically in their testing activities (Brown-Liburd et al., 2015; Moffitt et al., 2018). RPA could likewise be used for data stream matching and output formatting (Brown-Liburd & Vasarhelyi, 2015). It can also assist IAs with data gathering from systems that are not integrated and help with data cleaning for testing purposes through e.g. automated checks for completeness of fields, duplicates and input validation. Other example applications for RPA in internal audit are checking log files for proper segregation of duties, KPI monitoring, email testing, email generation, reconciliation testing, internal control testing, detail testing, automated reporting, contract testing and transaction analytics (Moffitt et al., 2018).

Through extensive usage of basic automation tools, auditors can achieve proficiency and become more likely to use advanced audit analytics (Li et al., 2018), thus IA departments may also improve their audit analytics capabilities over time by utilizing RPA. As a result of the scarcity of mature IAFs where audit analytics tools are currently used, RPA usage is defined in this work as both the act of an organization prototyping and experimenting specific audit procedures and daily usage of RPA bots for audit procedure automation. In the former case, RPA usage describes intended usage, while in the latter case it refers to actual usage in practice.

(10)

10

Despite the many advantages associated to the use of CAATs in internal auditing, their use remains low in practice (Li et al., 2018; Mahzan & Lymer, 2014; Smidt et al., 2019). The benefits attributed to RPA bots and their low usage in IA makes it valuable for IA stakeholders to receive insights on factors that motivate their adoption. As Curtis & Payne (2008) propose for external auditing, if CAAT usage improves audit quality and efficiency, there must be factors underlying the low level of adoption in practice. This notion is considered in the next section for CAATs adoption in internal auditing.

2.3 Theorizing CAAT adoption in internal auditing

Mahzan & Lymer (2014) use the Unified Theory of Acceptance and Use of Technology (UTAUT) to examine the adoption of computer-assisted audit tools and techniques (CAATTs) by internal auditors. UTAUT is an established model for the research on acceptance and use of information technology and has been applied in a variety of settings (Venkatesh, Thong, & Xu, 2016). In the traditional UTAUT model, the dimensions of performance expectancy (PE), effort expectancy (EE), social influences (SI) and facilitating conditions (FC) have been found to explain the majority of variance in both behavioural intention to use a technology and technology use (Venkatesh, Morris, Davis, & Davis, 2003; Venkatesh et al., 2016). Regardless, there is still room to improve the UTAUT model through experimental approaches (Bierstaker et al., 2014; Venkatesh et al., 2016). For instance, the UTAUT model does not consider the pre-adoption phase in determining technology use and acceptance (Curtis & Payne, 2008; Venkatesh et al., 2003). Mahzan & Lymer (2014) address this gap in adapting UTAUT to an internal auditing context, replacing behavioural intention and technology use with motivation and successful CAATTs’ adoption, as is shown in figure 1.

Figure 1:UTAUT adapted for CAATTs' adoption in internal auditing, sourced from Mahzan & Lymer (2014)

(11)

11

consideration benefits job performance, seem to be of explicit importance in CAAT(T)s adoption by IAs (Mahzan & Lymer, 2014) and use by external auditors (Bierstaker et al., 2014). Both effort expectancy, which describes the associated perceived ease of use of the system under consideration, and social influence, i.e. the degree to which an individual perceives others to place importance upon the system under consideration, were found to be insignificant in explaining auditor behaviour (Bierstaker et al., 2014; Mahzan & Lymer, 2014). In an audit context, individual technology choices may be of less importance due to effectiveness concerns (Bierstaker et al., 2014) and the voluntary nature of the adoption decision (Curtis & Payne, 2014; Mahzan & Lymer, 2014). This suggests the existence of higher-level influences on UTAUT’s dimensions.

UTAUT arguably lacks in identifying higher-level motivational constructs to explain what role the relationships between the dimensions and the differences in organizational context play in motivating successful CAAT adoption (Bierstaker et al., 2014; Curtis & Payne, 2014; Mahzan & Lymer, 2014), although its dimensions have previously been contemplated in conjunction with extrinsic and intrinsic motivation mechanisms for theory building purposes (Venkatesh et al., 2016). Dagilienė & Klovienė (2019) use a contingency theory approach for formulating a list of motivating factors influencing BDA use in external auditing. Contingency theory attempts to explain how desired outcomes are dependent on the contextual setting (Chenhall, 2003) and the way factors interact with each other (Venkatraman, 1989). Applying contingency theory within the context of this study implicates that the adoption of RPA in IA can only be successful for a specific set of motivating factors that best apply to a certain organizational context. This perfect mix of contingent factors is deemed to be capable of dealing with the uncertainty from this environmental and organizational context, motivating and enabling the adoption of RPA by internal auditors. The contingency theory approach could prove successful in explaining the UTAUT’s interdimensional relationships with contingent contextual factors and may offer further insights into the varying success of CAAT adoption in auditing. This study investigates these motivating contingent factors for RPA adoption in internal auditing.

(12)

12

stakeholders. Strategic orientation influences RPA usage through the strategy the firm’s management decides based upon the current technologies, environment, and organizational vision (Dagilienė & Klovienė, 2019). Size implies that organizational size must be assessed to determine whether it is suitable for using RPA in their business context. In contrast to Li et al. (2018), Dagilienė & Klovienė (2019) finds that corporate size for both the auditor and the auditee influences the adoption of auditing analytics. They find that larger audit firms have more resources and adequate tools at their disposal and that large audit clients will have bigger data volumes that cannot be analysed with traditional audit techniques. Structure relates to the relationships within the organization. The right people must be in a position that suits them best for motivating others.

(13)

13

3. Methodology

3.1 Research design

This paper is exploratory in its nature, attempting to extend the contingency framework of Dagilienė & Klovienė (2019) to internal auditing for the case of RPA adoption. Qualitative research has previously been demonstrated as a valid approach for researching CAAT adoption (Mahzan & Lymer, 2014). It is suitable for the purposes of this study, exploring motivational contingency factors for RPA adoption in internal auditing, because it is well-adapted to examining a phenomenon from a non-dominant theoretical perspective (Bluhm, Harman, Lee, & Mitchell, 2011; Eisenhardt, 1989; Gioia, Corley, & Hamilton, 2013). Appropriately, contingency theory has not previously been considered for formulating contingent higher-level motivational constructs in UTAUT (Venkatesh et al., 2016) nor for internal auditing purposes (Dagilienė & Klovienė, 2019). Moreover, research on RPA use in auditing is rare and still of an explorative nature (Moffitt et al., 2018). In using a qualitative field research approach, this study gains a deeper understanding of the respondents’ experiences with RPA in internal auditing and their interpretation of these experiences using original data (Bluhm et al., 2011; Edmondson & McManus, 2007; Gioia et al., 2013).

3.2 Data collection

(14)

14

Interviewees were selected based on three criteria, following Patton’s (1990) maximum variation sampling strategy for purposeful sampling. In the maximum variation sampling strategy, the picked cases display a wide range of variation to identify common patterns across contexts (Patton, 1990). First, interviewees were expected to have knowledge of RPA and its applications to internal audit. Second, their company was required to either work with RPA in their internal audit department or have explored the usage of RPA in internal audit. Lastly, the sample of companies was picked to reflect a variety of industries and were determined to be information-rich cases.

The seven selected case studies are a Big Four accounting organization both in the Netherlands and the United States, a banking and financial services company, an oil and gas company, a retailer, a fast-moving consumer goods organization, and a technology and communication services company. The Big Four accounting organization was selected as a separate case for the Netherlands and the United States to allow for a limited international comparison, alike the approach of Mahzan & Lymer (2014). Eight interviews were performed with nine participants, providing insights for seven distinct cases. Appendix 4 depicts the selected cases and shows a description of selected respondents. The interview with the technology and communication services company was performed with both a Chief Audit Executive and a Director present. The first five interviews were used to generate the axial codes, while the final three interviews were conducted for confirmation purposes of the axial codes and to provide greater robustness to the findings from the initial five interviews. Appendix 5 depicts the case characteristics regarding RPA usage in the internal audit department.

3.3 Coding and analysis

(15)

15

(16)

16

4. Findings

- Automation assurance

- Automation in IA activities IA practices

Current practices - Benefits - Cost of resources - Emergence - Experience - Risks

- Internal audit quality

General practices

Quality Internal audit

improvement

- Education - Market structure - Regulatory influence

External factors Institutional factors

- Change management - Governance - Competencies - Digital infrastructure - Size Digital strategy Company resources Organizational factors

Figure 2: Code structure

Upon reflection of the open codes and the second-order codes, axial codes were determined. The resulting code structure is depicted in figure 2. Appendix 6 provides examples of quotes for each of the open codes, illustrating the deduced patterns for formulating the second-order codes. Before discussing figure 2, considering contingency theory, it is essential to deliberate on the conditionality of the contextual perfect mix of motivating factors for RPA adoption in IA among the respondents, as is illustrated by the following case at a retailer:

(17)

17

a bot for it. So, we are still in that phase, we do not yet see enough added value to start applying it ourselves. I know that other companies, for example, very easily empty different databases. They, through RPA, even have PDFs scanned with text recognition software and have exceptions that are predefined […] We also do not have a single system within this company which you can use to extract comparable information from all countries. They are other processes; they are other systems. The use case here is still really limited.” [Director Internal Audit, retailer]

In the case of this retailer, RPA is already widely spread throughout the organization but isn’t deemed suitable for the IA department due to digital infrastructure constraints and a lack of perceived benefits to internal audit quality. Surprisingly, their data infrastructure, in general, is quite mature, the organization even has an RPA factory for the creation of bots and deploys bots created there globally in different departments. In contrast, the respondent from a banking and financial services company emphasized their usage of other automation tools in audit that is more appropriate for their organizational environment, while having similar automation characteristics to RPA. In the case of the Big Four accounting organization in the United States, after observing RPA bots in practice, they created a platform based on Python bots:

“I’m not against automation. I’m just all for the right tool for the job. I think there is a lot of hype around bots and not a lot of reality.” [Principal Risk Consulting, Big Four USA]

Regardless of differences in the automation tools currently in use at the selected sample cases and the observed cases where the perfect mix of contingent factors could not be perceived as present due to a lack of RPA adoption in the IAF, the individual motivating factors identified for RPA adoption in internal auditing at the selected cases are relevant to this research due to the non-static nature of contingency theory, allowing the identification of case-specific motivating factors that may be part of the perfect mix. The found motivating factors will be elaborated upon in the following sections.

4.1 Current practices

(18)

18

general emergence of automation tools in the business or influenced by benefits seen in other departments.

4.1.1 IA practices

Internal audit practices relate to either IAs giving assurance over RPA and automation tools used in the organization or the usage of RPA and automation tools for internal audit activities. During the interviews it was identified that, while adopting automation tools for internal activities may prove troublesome for some organizations, giving assurance to stakeholders over their functionality is a plausible first step:

“What we actually see a lot at the moment is that the 3rd_{line, for example, internal audit, will} audit the RPA implementation. They want to get assurance about the robot itself and not so much use it for their own work activities because they actually first want to know whether they can trust the robot or if it does what it is supposed to do. I think that this is the first phase where we see internal audit getting started with RPA. Meaning they will audit the robotic processes with their own team.” [Senior Consultant Robotics, Big Four NL]

Later in the interview the senior consultant at the Big Four accounting organization suggested that this is in line with the three lines of defence model, where the first line functions own and manage risk, second-line functions oversee or specialize in risk management and compliance, and the third second-line functions provide independent assurance through internal audit. According to this participant, it is a natural reaction to increased interest in the business environment. From an auditor perspective, it is a reaction to the automatization of relevant high-risk processes. With these developments, he sees the IA as a connector between the internal environment, giving assurance to internal stakeholders, and the external environment, laying the groundwork for external auditors. He goes on to note how the division between the 2nd_{and 3}rd_{line weakens when setting the standards for RPA. The banking and financial services}

participant acknowledges this notion:

(19)

19

Furthermore, the partner from the Big Four accounting organization confirms this practice and notes the intrinsic curiosity of auditors that drives them to investigate how businesses can remain in control of the RPA bots and automation tools, which is related to automation assurance. The participant from the retailer also mentioned the automation assurance role of IAs:

“[…] we have also audited RPA within the company. It is also considerably being implemented here. Ultimately, it is just a bit of efficiency, saving costs. There have been findings that could be better. In any case, I think that is an important role that we have.” [Director Internal Audit, retailer]

Sketching an ideal situation, he goes on to illustrate how continuous monitoring and continuous auditing could be achieved in conjunction with the second line of defence when processes are sufficiently standardized. However, most organizations lack such maturity for the foreseeable future. For the meantime, the senior consultant robotics proposes that RPA may be used to aid process standardization for either third line or second line purposes. For example, internal audit sample testing could become standardized with an RPA bot and thereby makes it possible to perform such testing more frequently and performing full population tests becomes feasible:

“The high frequency that gives you a higher coverage of your sample is certainly an important use case. […] The question is what you do with the exceptions. Of course, you can have the robot check the entire population for correctness. Then it will produce an exception list as output, and someone will have to work with that exception list.” [Senior Consultant Robotics, Big Four NL]

“Perhaps with the same effort, instead of a partial observation, I can test the entire population. […] I think an auditor should always look for such things, instead of just blindly performing a sample test. That does not mean that there is always a technology component […]” [Partner Business Analytics, Big Four NL]

It remains necessary for IAs to exert their professional judgment over the applicability of auditing technology or the identified exceptions. This is not exclusive to activities automated with RPA, but automation in general, as the banking and financial services case depicts:

(20)

20

He then no longer has to look at three differences, but one.” [Head of Audit Data, banking and financial services]

Yet, internal auditing activities are not necessarily the best use cases for automation tools. Even though IAs may be required to perform activities on high volume datasets, they don’t do this very frequently. This is problematic for both development and thus process standardization. Instead, an alternative approach to RPA is using it throughout the organization:

“So, most of what internal audit does is not what I would consider high volume transactions. They may look at a lot of transactions, but they don’t do that very frequently, like maximum once a day. Where we find our most success with bots, is when people are using a bot to do a thousand times a day the same ‘hey I need to post an invoice’.” [Principal Risk Consulting, Big Four USA]

Moreover, from a Big Four internal auditor’s perspective, it’s very much about what they do with their allotted hours in performing their work for the client. This could be an adverse incentive for sharing their knowledge and pushing automation tools to clients:

“It’s not about value. […] Really where our sweet spot is, is not automating our clients, but automating ourselves.” [Principal Risk Consulting, Big Four USA]

(21)

21

4.1.2 General practices

General practices are defined as the benefits and costs related to RPA and automation tools, the emergence of these tools, experiences with them and their associated risks. The second line of defence is often involved in designing standards and providing guidance when new technology is being adopted. The first line of defence has usually ownership of bots because there tend to be many people performing repetitive work activities. Here, large cost reductions and economies of scale can be achieved using robots. The motivation driving RPA adoption is then simple:

“So, the motivation for our customers is that it is primarily a major cost-benefit. […] A robot license means that you can put a robot to work 7 days a week, 24/7, 365 days a year. If you compare that with the costs of a regular employee, including vacation days and allowances, et cetera, I think you will pay over 40 to 50 thousand euros. That is an employee who works 40 hours a week. Of course, then the business case calculation is made very quickly, and organisations will look for such possibilities.” [Senior Consultant Robotics, Big Four NL]

When RPA can successfully be applied in the organization, its processes become more standardized with fewer exceptions. Moreover, implementing RPA bots can be a quick win in achieving a higher degree of process optimisation and costs reduction. Albeit that quick wins may be risky in the sense that they could prove to be a wasted effort in the long-term:

“The quick wins from the customer’s point of view is that they can achieve benefits in the short term, whatever these benefits are for the organization, is only good. But you have to be very careful upfront in looking at which processes you are going to automate. Because if you robotize a process within an application, for example, that will be removed within your IT landscape next year because they quit using it, then it makes no sense to automate that process” [Senior Consultant Robotics, Big Four NL] RPA adoption could have implications for the human role in organisations. Some human jobs could become redundant due to automation, but in an audit context, it is more probable that the human set of activities will adjust. As is illustrated by the following two quotes below:

(22)

22

“To do the exact same amount of work, through a fully integrated workflow automation environment you can do what you’re doing today in half of the time. So, the question becomes; do I do more stuff, or do I use fewer people. I don’t have an answer for that. Hopefully, it’s more stuff, especially in the area of risk.” [Principal Risk Consulting, Big Four USA]

Within an audit context, achieving cost efficiency through automation is difficult. Rather, an effectiveness-based measure such as the quality of assurance or a higher degree of assurance is more attainable. The partner from the Big Four NL case and the banking and financial services respondent put it the following way:

“That gives you the business case, in terms of efficiency it doesn’t make much of a difference, in terms of effectiveness it does. You have a better basis for the conclusion.” [Partner Business Analytics, Big Four NL]

“[…] it is very quickly about a higher quality of assurance. While I believe that efficiency is also there, but that is still a bit of mindset that we are working on. So yes, the benefits are indeed based on the effectiveness audits, and in fact from at least one in three audits that you can perform data-driven. Efficiency, I really believe it is attainable, that is the goal, but achieving it is still quite difficult.” [Head of Audit Data, banking and financial services]

It follows that achieving a cost efficiency from RPA in internal auditing may not be of primary concern. Another approach of weighing the cost benefits is comparing the cost of RPA alternatives. RPA alternatives are often more extensive tools or programmed scripts. Compared to more extensive automation tools, RPA is relatively cheap and quick to build:

“That is the advantage of RPA. Such a large tool is very expensive. It must be custom-built for each specific organization, which often costs hundreds of thousands of Euros, perhaps millions before you get it up and running. You will be three years in at that point. Building a robot costs approximately ten thousand Euros, you can start using it next week and within two weeks you can show the first results.” [Senior Consultant Robotics, Big Four NL]

(23)

23

“Because when you talk about RPA, you’re basically talking about Macros on steroids. They work on the operating system layer and frankly, you’re limited in the types of use cases that you can deploy.” [Principal Risk Consulting, Big Four USA]

Use cases that are more widely applicable in an organization by far outperform the business case of singular use cases. In turn, having many use cases in your organization bears risks:

“If the bot changes it’s not a big deal, you change it, you have ten bots. When we thought about what we do at our organization, we realized we would have thousands of these things. If every bot broke when you updated software, that would be bad… and we do a lot of software updates.” [Principal Risk Consulting, Big Four USA]

In conclusion, a major driver for RPA adoption is cost benefits. When RPA can successfully be adopted it aids process standardization. The ability of an organization to identify quick wins drives short-term RPA adoption but comes with the risk of wasted resources in the long-term. RPA adoption also influences the work activities of humans in organizations. In an audit context, effectiveness measures may better capture RPA benefits than efficiency measures. RPA is positioned as a cheap alternative to more extensive automation tools. Its adoption is influenced by the number of identifiable use cases and is impeded due to design and organizational constraints. The next section elaborates upon RPA’s role and use cases in internal auditing.

4.2 Internal audit improvement

The axial code of internal audit improvement is based on one second-order concept: quality. When asking the respondents about the role of RPA in internal audit, they indicated that the main contribution of RPA to the internal auditing process is its effect on internal audit quality. Internal audit quality relates to the perceived value added by stakeholders of the auditee and the degree of assurance given over the organization’s processes. To illustrate:

(24)

24

Quality concerns for the 1st_{and 2}nd_{line can also drive knowledge sharing by IAs. When providing assurance}

over 1st_{or 2}nd_{line processes, internal auditors may encounter use cases for RPA that could improve these}

processes someway. They can then share their knowledge of automation tools with upstream lines of defence and in a later phase audit the implemented changes to the process. This is described as follows by a respondent:

“Our goal is of course in particular to help the 1st_{and 2}nd_{line to deliver the quality that must be} delivered. So, if we have found something that you can test or improve, why keep that with us. We would be better off saying, ‘you take it’. Then we will evaluate how we can audit it.” [Head of Audit Data, banking and financial services]

Although RPA isn’t widely adopted yet in internal auditing. It is promising that organizations are starting to experiment with it, as is suggested by the Principal Risk Consulting:

“Clients are playing with it still. We have had a couple of clients that, for instance, they’re trying to build a couple of hundred bots and we’re helping them with that. They’re using it to gather evidence directly from the environment, process it and spit out work papers.” [Principal Risk Consulting, Big Four USA]

Overall, RPA may help internal departments in overcoming capacity concerns. With regards to providing assurance on the processes of the 1st_{and 2}nd_{line, IAs have a role in sharing their automation knowledge}

with upstream lines of defence when detecting room for improvement through automation. Finally, RPA usage is still in the early adoption phase, but there are promising signs. In the next section institutional factors are discussed.

4.3 Institutional factors

(25)

25

“To be able to understand specific risks in the field of IT, the financial auditor should certainly have more knowledge of the IT audit. The reverse is of course also true. […] I understand that this separation exists from the past. I think from the curriculum list that this will converge and that the IT component becomes increasingly important. Such that maybe in the future every auditor should be an IT auditor, or maybe that every financial auditor should have an IT auditor background.” [Senior Consultant Robotics, Big Four NL]

While auditors need to become more IT savvy, they don’t necessarily need to become proficient programmers. Rather, they should be able to identify use cases and communicate them with developers. This is illustrated below:

“Internal auditors need to be able to sort of speak in the language of architects and know what a good use case is and when to use the right tool, but they don’t need to go build it.” [Principal Risk Consulting, Big Four USA]

More specifically, curriculum changes in university education play a role in enabling innovation adoption by auditors. The following quote depicts this notion:

“I think that RA and RE training must fully integrate. I am 100% convinced of that. I also think that a lot of people don't really see the sense of urgency, a lot of RAs or people who study RA or auditors or those who work at internal and don't know anything about data analysis or IT. […] The team that I have has no experience with that. It is very difficult to explain to those people, how does such a development process work, what are the risks there. You know, it takes me a lot of time to run such an audit, so something has to change. You see, these are just things that we have let happen without intervening and we simply have to anticipate more.” [Director Internal Audit, retailer]

It can be difficult for organizations to recruit and hold on to savvy employees. Alternatively, less tech-savvy employees can be trained internally upon recruitment to rely less on the external market for new hires. This way, knowledge may be retained in the organization:

(26)

26

to be able to audit as much as possible with your own capabilities, instead of constantly having to hire everywhere because then you will not be capturing in that knowledge. But where necessary we occasionally source externally.” [Director Internal Audit, retailer]

The market for new hires is related to other market influences on the digitalization of business. Organizations serve clients, giving clients the power to steer organizations through their actions in the market. The director internal audit at the retailer characterizes the influence of client demand in general as strong:

“I think most of it is simply due to developments within the market, that we as a company are of course also forced to be much more of a digital company because customers expect digital convenience and experience and those related developments. Of course, we want to lead the way as a large company. So, it's becoming more of a digital company that also sells products than the other way around.” [Director Internal Audit, retailer]

Another important market force is the price. Pricing of robot licenses drives demand for simple automation and is a typical way for organizations to be competitive to their rivals. At the same time, many new companies are offering the implementation of these robots, such that supply is increasing, further driving the price down. The following quote signals the existence of a strong price effect in this regard:

“The customers we serve are often also served in the consultancy market by us. Price does have an influence on that. You can now see the licenses for robots going down. It is a bit of an opposite movement; demand is only increasing and so does supply. The supply may now even be increasing more than the demand. We can get cheaper and cheaper for our customers who robotize processes because we have more experience because we understand what we do better. Due to competition, there is a certain pressure to offer it cheaper. Because if we ask too much for implementing robots, they will say that they will work with the competitor instead. In that regard, price is a very strong influence on this competition.” [Senior Consultant Robotics, Big Four NL]

(27)

27

“When I look within the audit context, there is just a lot of open discussion. A lot of knowledge is shared, which is very positive. But it’s not so much about competition.” [Head of Audit Data, banking and financial services]

Another external influence is regulations. The partner business analytics mentions a link between auditing activities and regulations. He portrays this relationship the following way:

“I think that regulations are good for indicating how judgement should be performed and what kind of control you have as an auditor. That you also know what happens. Not just judgment of the auditor, even though that is one. But especially also at the organizational level because they often are the first to do something with that kind of technology. […] That you don’t blindly trust everything, such that you know what you can trust.” [Partner Business Analytics, Big Four NL]

Although regulations will eventually adapt to better reflect the new business environment, one respondent urges that legislation is not the correct method for driving innovation. Instead, he proposes that innovation must be driven internally:

“I’m a little cautious in pointing to legislation when we can answer it for ourselves. Legislation should be used to remove externalities and level the playing field.” [Principal Risk Consulting, Big Four USA] To conclude, education plays a role in supplying more tech-savvy auditors to businesses, enabling increased digital innovation. Alternatively, organizations can strive to retain knowledge in the organization by sourcing training internally. Client demand plays a role in business innovation by forcing organizations to adopt new technologies by asking for new business experiences. Furthermore, pricing decisions strongly influence the attractiveness of the RPA business proposition. But within an internal audit context, market forces such competition are not as prominent. Finally, regulations should adapt while leaving room for innovation, but must refrain from defining or driving it.

4.4 Organizational factors

(28)

28

4.4.1 Digital strategy

The digital strategy set by the organization is related to the organization’s management of innovations and the degree to which top management exerts influence on organizational processes. The following quote illustrates this:

“It is possible that costs are too high and that they must be lowered. Whether a CFO wants its people to be educated to become more digitally minded, or if they want them to be ready for the future. There is a certain vision or strategy at work there.” [Senior Consultant Robotics, Big Four NL]

Having an organizational strategy is important in realizing a vision for the organization. If such a strategy is missing, there might be a lack of direction to the actions of employees.

“You need to have a certain vision on it from the internal audit perspective because if you don’t have that, you just keep working in the same way and then you will not get that RPA technology within your internal audit function. I see that as a challenge for internal functions, to be proactive […], such that we can also apply RPA within our line of work.” [Senior Consultant Robotics, Big Four NL]

Instead, when a strategy is clearly communicated, employees can become proactive in searching for ways to optimise their work:

“The entire strategy of the company is also based on that. Then, of course, it means that you make your processes efficient, that you implement all the tooling that is available. That in the end, you can quickly respond to such developments. And so, all the data that you also have about your customers end up in a data lake, is analyzed properly, builds smarter models with that, algorithms. That is what the entire company is currently focused on.” [Director Internal Audit, retailer]

An addition to the necessity of having an organizational strategy is that it must also incorporate a change management policy. This is illustrated below:

(29)

29

Change management relates to how you manage innovations as an organization. For instance, the earlier identified quick wins can be tricky to deal with. Not knowing how to evaluate them may be harmful to the organization in the long run:

“There are, however, various considerations as to whether it is smart to proceed with such quick wins because you have to anticipate future developments to determine whether it is sustainable to do so.” [Senior Consultant Robotics, Big Four NL]

Eventually, the strategy is set is to cultivate and govern a certain type of behaviour throughout the organization. The top management has an influence on the decision-making process regarding innovations this way. In some cases, top management is open to change. In others, they need some convincing when other motivating factors are present. These cases are characterized by the following two quotes:

“I think that we want to be known as a digital company, that is of course what my team mainly stands for.” [Partner Business Analytics, Big Four NL]

“But if you look at our customers you certainly have to deal with management that thinks people are not mature enough yet to start using this technology.” [Senior Consultant Robotics, Big Four NL] In sum, the organization’s strategy plays a crucial role. The top management has a direct influence on organizational innovation by setting a visionary strategy. In addition, organizations must pay attention to change management to ensure sustainable change.

4.4.2 Company resources

For organizations to adopt new technologies, the respondents suggest that employees need to have the required competencies to work with the tool. One solution is training new employees upon recruitment when their education or new experience isn’t sufficient:

(30)

30

Regarding the type of training, respondents note that auditors do not need specialist knowledge. Rather, they need to recognize use cases in their internal audit activities and then work together with a specialist. The following quote describes this relationship:

“So that you are a conversation partner. You don’t have to be a specialist because internal auditors are not always intrinsically digital specialists of their own. What you actually see is that an internal auditor, they must have the competence to be able to talk to a data scientist, to be able to work together. So actually, the competence is that he finds out that he can no longer perform the work on his own. That is an important aspect.” [Partner Business Analytics, Big Four NL]

Building a team with more specialist competencies can be tough for internal audit departments. As is illustrated in this quote:

“For internal audit, it is exceedingly difficult to build such a capability yourself because it is often a very small team. The moment a robotics team exists somewhere else within the organization, it could very well be that an employee or a few employees of the internal audit team can train with the robotics team, gain knowledge, and then apply robotics within his own team. I know a handful of organizations that are experimenting with his.” [Senior Consultant Robotics, Big Four NL]

The above quote proposes letting another team within the organization train other teams. However, this solution only works when this is possible with regards to robotics team capacity. The IA team must be big enough to allow one team member to temporarily go in training and the robotics team needs to have one member available for teaching the auditor. It is likely that the size of the organization warrants whether this is possible. As is depicted in the following quote:

(31)

31

“You get that critical mass when, for example, you put processes together in a shared service centre. If a company already has a shared service centre, then robotics might be interesting. So, you can already look at that, via pricing. If prices are low then you may need less critical mass.” [Partner Business Analytics, Big Four NL]

Shared service centres relate to the organization’s data infrastructure. As illustrated above, when the data infrastructure is more extensive, then the RPA business case may also become more positive. Therefore, the organization must have proper data infrastructure to allow for the adoption of RPA. The retail respondent illustrates this:

“We just have a very good process. We have a good efficient process for building the bots. We have a good efficient process for connecting IT. Even when something in the backend changes, RPA is connected with it. […] What impact do changes have is important to identify to prevent them from failing We have a good service model to maintain those things.” [Director Internal Audit, retailer]

(32)

32

5. Discussion

The findings suggest multiple motivational factors for RPA adoption in internal auditing, answering the research question of “What factors motivate the usage of robotic process automation in internal auditing?”. Based on the qualitative data, two key groups of motivators are determined that influence RPA adoption in IA directly or indirectly. These factors are displayed in table 1. The relationship between extant literature and the identified factors will be discussed in this section.

Table 1: Identified motivating factors Motivating factors for RPA

adoption in IA

Description

Organizational motivators

Automation assurance Knowing the pitfalls of building bots or usage of bots motivates RPA adoption

Relationship between the IAF and the other lines of defence

Increased interaction across the lines of defence encourages knowledge sharing

Perceived benefits compared to costs

Positive business cases are an important factor in the decision-making process, benefits vs. costs are a traditional paradigm, relates to the organization’s business model

Efficiency versus effectiveness For internal audit activities, higher perceived importance of effectiveness measures motivates RPA adoption

Ability to identify quick wins When sustainable quick wins can be identified for achieving a pre-specified goal

Ability to identify

organization-wide use cases

Organisations capable of identifying a multitude of use cases can employ RPA more sustainably

Availability of tech-savvy internal professionals

Adoption is conditional on the competency to utilize the bots efficiently and effectively

Digital strategy The adoption of RPA must be in line with the organization's digital strategy and the organization must have sufficient experience with sustainable change

Willingness to source capabilities externally

If not all required capabilities can be developed internally, the organization must be willing to source them

Size Some form of critical mass is necessary to enable RPA adoption, monetarily, in terms of human resources, or in terms of data volume Digital infrastructure The digital infrastructure must be adequate to support RPA adoption

Institutional motivators

Market environment Lower vendor and implementation fees influence the business case for adoption positively

Regulations Applicable regulations shouldn't be driving nor restricting innovation Education Education institutions provide professionals with skills, curriculum

changes towards digitalization drives future innovative thinking, Client/stakeholder demand Clients demanding a more digital business experience forces and

(33)

33

Organizational motivators are found to be more numerous than institutional motivators, similar to Dagilienė & Klovienė (2019). In comparing the findings to their study, six distinct novelties for the internal auditing context are formulated.

First, the practice of automation assurance from an IA perspective is emerging as an internal counterpart to the external IT auditor. Automation assurance activities are more specific to the organization than IT auditor activities and serve as both a knowledge-sharing and assurance activity. Knowledge-sharing in this sense may contribute to the internal auditor’s perception of RPA. Secondly, increased interaction across the lines of defence results in a similar knowledge-sharing effect. Issa et al. (2016) suggested such a conceptual change to the definition of lines of defence. Furthermore, the found knowledge-sharing effects are closely related to Li et al.’s (2018) suggestion of audit analytics proficiency effects. Third, conditional to RPA adoption is the need for internal discussion on organization-specific measures to accommodate the efficiency vs. effectiveness debate. Using effectiveness measures in evaluating the business case makes more sense from an internal audit perspective than efficiency measures, given the typically associated costs with a higher degree of assurance. In turn, the business case outcome is found to be strongly related to the organization’s identified use cases, digital strategy and the organization’s digital infrastructure. Fifth, the organization’s ability to identify quick wins in the short run aids in building a positive business case. Correspondingly, an organization’s ability to identify sustainable use cases should be regarded as the major driving force behind organizational RPA usage in the long run. Finally, the client or stakeholder demand is found to be an important institutional factor. Organizations may adopt and innovate their processes to meet the demands of their clients and stakeholders.

Explicitly in contrast to Dagilienė & Klovienė (2019), competition among auditors is not established as a motivating factor for RPA adoption in internal auditing. Rather, internal audit departments are found to regularly confer with each other to further the knowledge of the profession. The effects of competition related to RPA adoption in IA does not originate from the internal auditors themselves but from the competition amongst firms positioning themselves for RPA implementation for clients or vendors providing the RPA tools. The market environment for RPA in an internal audit setting is affected by these competitive pricing decisions and microeconomic supply and demand factors. Consequently, the market environment influences the perceived benefits compared to costs factors from an efficiency perspective.

(34)

34

and applicable to the CAAT adoption context. The identification of three organizational motivators that depict IA competencies is in line with Smidt et al. (2019), Li et al. (2018), and Mahzan & Lymer (2014), who all link internal auditor competencies to CAAT adoption in internal auditing. Moreover, digital strategy is found to motivate RPA adoption through the influence of top management, comparable to Smidt et al. (2019) and Curtis & Payne (2008). Size depicts the required critical mass of an organization’s resources as an influence on CAAT adoption and is closely related to the organization’s digital infrastructure (Bierstaker et al., 2014; Smidt et al., 2019; Vasarhelyi et al., 2015). The found relevance of size contrasts with Li et al. (2018), while the influence of digital infrastructure on motivation confirms Vasarhelyi et al.’s (2015) perception of the internal auditors’ unique access to organizational data. Regulatory interventions can be made to influence the audit practice (Appelbaum et al., 2017; Brown-Liburd et al., 2015; DeFond & Zhang, 2014), but they may have a smaller impact on internal audit practice since internal audit regulations are more flexible than regulations for external auditors (Li et al., 2018). With respect to RPA, it is not yet clear how and whether audit standards should change (Moffitt et al., 2018). Interview data suggests it is not desirable for regulations to enforce change when it comes to internal audit innovation efforts. Instead, the regulations should accommodate and guide internal audit departments in their innovation efforts.

The influence of individual motivational factors on RPA adoption is found to differ for each organizational context. Market environment, digital strategy, knowledge sharing, and competency related factors are found to be the most influential across contexts. In addition, the factors derived from the interviews are dynamic and therefore subject to change until the event of adoption. Following the logic of contingency theory, RPA adoption in a specific organizational internal audit context occurs when a perfect mix of these factors with differing influences is achieved for a given context at a certain point in time. For example, when a mixture of motivational factors relevant to a specific organization’s context is present but not yet sufficient, all else equal, a decrease in vendor pricing may lead to a positive business case and subsequent RPA adoption.

(35)

35

client/stakeholder demand. Mahzan & Lymer (2014) implies the factor perceived ease of use to depict the effort expectancy dimension. However, the perceived ease of use wasn’t established as a motivational factor in the interviews. This is surprising since Li et al. (2018) find that ease of use is the most important factor for tools usage by internal auditing.

(36)

36

6. Conclusion

The new digitized business environment requires internal auditors to adapt their practices (Chan & Vasarhelyi, 2011). IIA guidance has been adjusted to reflect this paradigm shift (The Institute of Internal Auditors, 2016), but CAATs remain lacking in usage (Li et al., 2018; Mahzan & Lymer, 2014; Smidt et al., 2019). The literature lacks consideration of RPA in auditing, with only the paper by Moffitt et al. (2018) proposing the utilization of RPA in auditing and little research on the IT tools used by internal auditors (Roussy & Perron, 2018). Hence, this research adds to the extant literature by examining the specific case of RPA adoption in internal auditing. Furthermore, this study is the first to link UTAUT and contingency theory for exploring higher-level motivational factors to the UTAUT dimensions. In addition, the emergence of these factors within an organization is proposed to be dynamic, extending the traditional UTAUT model by reducing its staticness (Venkatesh et al., 2016). This research thereby contributes to a broad range of research domains.

Through the understanding of motivational factors of CAAT adoption, organizations and stakeholders can take steps to accommodate their usage in practice. When the internal audit department itself struggles with using RPA for its own activities, it may learn to better recognize use cases by performing audits on operational RPA bots in other departments. In evaluating internal audit use cases, efficiency measures may be insufficient in capturing added value for specific contexts such as banking. In such contexts, assurance-based effectiveness measures may be more adequate in evaluating business cases. Internal audit consultants working for Big Four organizations should actively search for opportunities to use new techniques at clients, despite their time-constraints, cultivating awareness for use cases of automation tools. The inferred weakening of the lines of defence model implicates the need for the IIA to address new concepts such as RPA in their revision of the three lines of defence model. In turn, managers must re-evaluate how the lines of defence interact, ensuring knowledge sharing activities across the lines of defence to promote the usage of automation tools. The automatization of internal audit activities may change the role of internal auditors, with increased importance on their professional judgment in a highly digital environment. Educational and regulatory institutions must accommodate this shift, while managers need to ensure that their employees have the required competencies for working in a highly digital environment.

(37)

37

(38)

38

References

Alles, M. G. (2015). Drivers of the use and facilitators and obstacles of the evolution of big data by the audit profession. Accounting Horizons, 29(2), 439-449. doi:10.2308/acch-51067

Appelbaum, D., Kogan, A., & Vasarhelyi, M. A. (2017). Big data and analytics in the modern audit engagement: Research needs. Auditing: A Journal of Practice & Theory, 36(4), 1-27. doi:10.2308/ajpt-51684

Bierstaker, J., Janvrin, D., & Lowe, D. J. (2014). What factors influence auditors' use of computer-assisted audit techniques? Advances in Accounting, Incorporating Advances in International Accounting, 30(1), 67-74. doi:10.1016/j.adiac.2013.12.005

Bluhm, D. J., Harman, W., Lee, T. W., & Mitchell, T. R. (2011). Qualitative research in management: A decade of progress. Journal of Management Studies, 48(8), 1866-1891. doi:10.1111/j.1467-6486.2010.00972.x

Bos, P. W., Boersen, D., Van Ark, P., & Van Kleef, M. (2017). Analytics: Good practices for (smaller) IAFs. A field

study into experiences and succes factors. Netherlands: IIA Netherlands.

Brown-Liburd, H., Issa, H., & Lombardi, D. (2015). Behavioral implications of big data's impact on audit judgment and decision making and future research directions. Accounting Horizons, 29(2), 451-468. doi:10.2308/acch-51023

Brown-Liburd, H., & Vasarhelyi, M. A. (2015). Big data and audit evidence. Journal of Emerging Technologies in

Accounting, 12(1), 1-16. doi:10.2308/jeta-10468

Cao, M., Chychyla, R., & Stewart, T. (2015). Big data analytics in financial statement audits. Accounting Horizons,

29(2), 423-429. doi:10.2308/acch-51068

Chan, D. Y., & Vasarhelyi, M. A. (2011). Innovation and practice of continuous auditing. International Journal of

Accounting Information Systems, 12(2), 152-160. doi:10.1016/j.accinf.2011.01.001

Chartered Institute of Internal Auditors. (2019). What is internal audit?

Cheng, Q., Goh, B. W., & Kim, J. B. (2018). Internal control and operational efficiency. Contemporary Accounting

Research, 35(2), 1102-1139. doi:10.1111/1911-3846.12409

Chenhall, R. H. (2003). Management control systems design within its organizational context: Findings from contingency-based research and directions for the future. Accounting, Organizations and Society, 28(2), 127-168. doi:10.1016/S0361-3682(01)00027-7

Curtis, M. B., & Payne, E. A. (2014). Modeling voluntary CAAT utilization decisions in auditing. Managerial Auditing

Journal, 29(4), 304-326. doi:10.1108/MAJ-07-2013-0903

Curtis, M. B., & Payne, E. A. (2008). An examination of contextual factors and individual characteristics affecting technology implementation decisions in auditing. International Journal of Accounting Information Systems,

9(2), 104-121. doi:10.1016/j.accinf.2007.10.002

Dagilienė, L., & Klovienė, L. (2019). Motivation to use big data and big data analytics in external auditing.

Robotic Process Automation in Internal Auditing: Motivating factors for usage