Data analytical procedures; a case study about its challenges
Name: Tim Koelewijn Student number: 11080493
Thesis supervisor: Prof. Dr. B.G.D. O’Dwyer Date: June 26, 2017
Word count: 20601, 0
MSc Accountancy & Control, specialization Accountancy Faculty of Economics and Business, University of Amsterdam
Statement of Originality
This document is written by student Tim Koelewijn who declares to take full responsibility for the contents of this document.
I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.
The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.
Abstract
This research tries to understand the way auditors ensure obtaining assurance from the new audit procedures resulting from the implementation of data analytics within the audit practice. Data analytics has become a hot-item in the accounting practice, but only a little focus has given to the challenges with regards to the implementation. For auditors, these challenges might exist in the fields of data quality or technological feasibility. Due to the highly regulated practice, auditors must overcome these challenges as otherwise data analytics won’t provide assurance but will cost the organization costly time. By looking at the actions undertaken in a case study at one of the Big-Four professional practice firms in The Netherlands during the institutionalization process, this research will look at the influence of these actions on the outcome of this process and determine what actions the auditors exactly had undertaken to ensure obtaining assurance.
Contents
1 Introduction ... 7
1.1 Background of subject ... 7
Big Data ... 7
Audit transformation ... 7
Challenges in the audit transformation ... 8
1.2 Research contribution ... 8 1.3 Research focus ... 9 1.4 Research question ... 9 1.5 Thesis structure ... 10 2 Theoretical framing ... 11 2.1 Auditing process ... 11 Audit evidence ... 11 Analytical procedures ... 12 2.2 Big data ... 12
Definition of Big Data ... 12
Characteristics of (big) data ... 12
2.3 Applying Big Data in Audit Procedures ... 13
Application of big data within the audit procedures ... 13
Challenges arising from the implementation ... 14
Waves of audit transformation ... 14
2.4 Institutionalization process ... 16
Institutionalization work ... 16
Forms of institutionalization ... 17
Phases of institutionalization ... 23
3.1 Research description ... 26
3.2 Research set-up ... 26
3.3 Interviews ... 27
3.4 Other data sources ... 28
3.5 Interviewees ... 28
3.6 Coding of the results ... 29
4 Case narrative ... 32
4.1 Big data in the financial audit practice ... 32
First application ... 32
Definition ... 32
Different applications ... 33
Risks covered ... 34
4.2 Process of the implementation ... 34
Reasons for the implementation ... 34
Audit transformation ... 35
Support of specialists ... 36
Technological implementation ... 37
Internal guidelines ... 37
Moving to the next phase: executing the new procedures ... 38
Top-Down strategy ... 38
Training of the employees ... 39
Supportive materials ... 39
Technological malfunction ... 40
Form of execution ... 40
Data validation ... 40
Industry specific characteristics ... 41
Failed strategy ... 42
Bottom-up strategy ... 43
Change in actions ... 44
Lack of internal guidelines ... 44
Client difficulties ... 45
Maturity of implementation... 45
Regulatory (External)... 46
Continuation ... 46
Recognition of institutionalization actions ... 47
4.3 Outcome of the implementation process ... 49
People ... 49
Technological difficulties ... 50
Guidelines (internal) ... 50
Application in the audit practice ... 51
Client benefits ... 51
Audit approach/Strategy... 52
Management’s assertions audited ... 52
Assurance considerations ... 53 Regulatory approval ... 53 5 Discussion ... 55 6 Conclusion ... 58 6.1 Conclusions ... 58 6.2 Limitations... 59
6.3 Suggestions for further research ... 59
7 References ... 61
1 Introduction
1.1 Background of subject
Big Data
Big data is nowadays a so-called hot item and raising in popularity under professionals and academics. In their article, Setty and Bakhshi (2013) note that there has been a significant growth of the volume of data generated over the past years. These types of data were not available or not even considered in the past, according to the authors. They also notice that there are a lot of new tools coming available to enable the auditors to connect all the new records in all databases. This will enable to structure the available data.
The newspaper, “Het Financieele Dagblad”, published an article (Rolvink Couzy, 2017) about the upcoming market of (big) data assurance. Big Four professional practices are massively stepping into these new markets as the potential of this new market is even larger as the financial statement audit market. Added to that the expectation of the disappearance of the traditional financial statements and transforming into a continuous audit of the (accounting) data recorded. (Remmerswaal, 2016)
According to EY (2015) the term big data describes “the massive portfolio of data that is available”. They describe the analytics of this data as “the process of analyzing data with the objective of drawing meaningful conclusions”. In terms of audit procedures this will mean that the procedures will include an analysis of the data to draw a conclusion upon the audit object.
Audit transformation
The use of analytical procedures increased after the mandatory use in planning and review purposes and after increased concerns about audit efficiency and effectiveness. (Hirst & Koonce, 1996) However, a few years ago big data analytical procedures were still limitedly applied in external audits in testing procedures. (Cao, Chychyla, & Stewart, 2015) Implementing big data in the audit procedures provide usually provide unique and timelier audit evidence. (Yoon, Hoogduin, & Zhang, 2015, p. 434) Implementation of big data within audit procedures can increase the change of detecting accounting misstatements. (Larcker & Zakolyukina, 2012; Issa & Kogan, 2014)
Driven by regulation and the willing to prevent for further financial accounting scandals and enabled by the technological developments and changes in audit approaches by audit firms, it is found that there is an increase of analytical procedures within the audit procedures. There is a limited increase in relying on analytical procedures and therefore decreasing the amount of testing.
(Trompeter & Wright, 2010, p. 669) Due to the fact real-time reporting will be made possible as XBRL is implemented, traditional documentation of business processes becomes electronic files which requires different audit procedures and therefore also causes changes in the audit procedures applied in the practice. (Rezaee, Elam, & Sharbatoghlie, 2001)
As example, EY (2015) describes the transformation as that the audit procedures no longer will be sample-based but will include the entire population. Today’s (or Traditional) audit procedures use a sample of the population to draw a conclusion upon the entire population. The use of this method is statistically proofed. With data analytical procedures, the use of samples is no longer needed and the analysis can be performed on the entire population.
Challenges in the audit transformation
Despite the popularity and started transformation, it can be doubt whether this new type of audit procedures will truly provide more or even assurance for the auditor. Main challenges mentioned in different literature are among other things about the data quality, source validation, processing and scalability. These challenges are mainly due to the input of the data, storage of the data or the processing of the data. (Najafabadi, et al., 2015; Cuzzocrea, Song, & Davis, 2011; Zhang, Pawlicki, McQuilken, & Titera, 2012; Cao, Chychyla, & Stewart, 2015)
Also, the professional associations might decide to not allow the new audit procedures to fully rely on them as audit evidence. This will already limit the innovation in the audit process beforehand. (Greenwood, Suddaby, & Hinings, 2002) The prominent level of regulation in the audit practice might therefore be a challenge for obtaining assurance from analytical procedures instead of performing new procedures besides the old traditional ones.
1.2 Research contribution
The contribution of this research to the existing research on the subject is about the focus on the assurance gained from the new procedures. In her article, Early (2015) discusses some challenges which come with the implementation, but she notices that there is almost no research on the challenges of this implementation. This research will be mainly focused on the challenges of the implementation and how these are controlled in the audit practice to gain actual assurance.
Furthermore, the focus of this research on the audit regulation part on big data, also contributes largely to the existing literature on the subject. This is mainly due to the novelty of the subjects in case of the regulation. Therefore, the regulation has (to my best knowledge) not yet been recognized as a challenge in implementing big data within the audit process by existing
Since the research will be performed as a qualitative case study, the case company needs to be representative. The case company is a Big-Four professional services company located in The Netherlands. Since the company is one of the Big-Four and a leader on the field of data analytics in the assurance practice, the company will be a representative case company. Furthermore, there is no reason to assume that results of the study will not count for other cases since they are not that case specific.
1.3 Research focus
The research focuses on the level of assurance obtained from the use of big data in the audit process. The definition of the audit process includes all phases of an audit as recognized in the international auditing standards. Since the level of assurance depends, among other things, on the way the big data is institutionalized within the audit process, there will be a focus on the institutionalization process of big data. However, the research won’t focus too deep on the how big data is institutionalized, but more what challenges are faced and how these treated in order to obtain assurance are. Moreover, the role of players within the field during this process won’t be researched and the challenges regarding big data are most likely obtained from existing research.
There will be a special focus on the regulation on the use of big data in the audit process. Since auditing is a highly regulated profession, this require special attention during the institutionalization process. When the regulation is not adjusted to the new institution, it might that there is no assurance obtained as there is no allowance to rely on and therefore the big data is only additionally to the traditional process. Furthermore, this research is not meant to investigate all possibilities created with the implementation of big data within the audit process. I will make use of the existing or future implementation of big data within the case company to determine the level of assurance obtained. This level of assurance obtained by big data will be separately observed per phase of the institutionalization process.
1.4 Research question
During the research, I will research whether the degree of the assurance given by the data analytical procedures changes by the way they are implemented in the overall procedures. The research question of this research proposal is:
How do auditors seek to enhance assurance practice using data analytic procedures?
The above research enables to research whether the auditor truly has more assurance based achieved from the performed analytical procedures. During the interviews, I focused on the
old/new situations in being able to see a difference in the answers. Otherwise, it might become hard to see a difference in level of assurance.
1.5 Thesis structure
The thesis continues in the following structure. First the theoretical framework will be discussed in order to understand the empirical results. In chapter 3 the research methodology will be discussed including the selection of interviewees and how the interviews are performed. The case narrative is included in chapter 4 split into the institutionalization process and the outcome of this process. Chapter 4 ends with an analysis of the influence the institutionalization process had on the outcome. The discussion on the subject will be conducted in chapter 5. In chapter 6 the conclusion will be made based upon the results. Finally, the reference list and appendices can be found in chapter 7,8 and 9.
2 Theoretical framing
In this chapter, the theory to understand the empirical results of the research is defined. As described in the introduction part, Liddy (2015) described that there is a notable change going on within the performed audit procedures. She describes this as a shift from sample based procedures to population based procedures. To make this shift towards population based procedures, auditors make more and more use of Big Data obtained by data mining processes. To describe the results about the change in procedures, the institutionalization theory used to understand the results.
At first, understanding the auditing process is important to explain the fundamental change coming with the implementation of big data and will therefore be defined paragraph 2.1. In order to understand the challenges of using big data, the definition of big data is defined in paragraph 2.2. As last, the institutionalization theory will be defined in paragraph 2.4, split into the forms (2.4.2) and the phases (2.4.3) of the institutionalization process.
2.1 Auditing process
Arens & Elder (2016, p. 28) describe auditing as “the accumulation and evaluation of evidence about information to determine … on the correspondence between the information and established criteria”. The definition includes two separate parts, namely the task and object used by auditing. The object of an audit consists in most cases of historical financial information prepared by the client in accordance to local regulation.
In the definition of auditing, the task is described as “accumulation and evaluation of evidence”. This says that the auditor needs to get and evaluate the evidence used in his audit. There is not a limited base on the way audit evidence might obtained or evaluated. However, there are international auditing standards which define the minimum level of performed audit procedures to prepare an opinion. Auditing is a highly regulated profession which causes dependence of regulators and regulation for new innovative audit methods.
Audit evidence
As Arens & Elder (2016) describe, there are eight types of audit evidence that can be obtained. Analytical procedures are described as “evaluations of financial information through analysis of plausible relationships among financial and nonfinancial data”. Audit evidence is evaluated based on two criteria: sufficiency and appropriateness. The appropriateness is determined on the relevance and reliability of the audit evidence. Both criteria are evaluated based on professional judgment by the auditor.
To make auditing cost efficient, auditors use sampling techniques in their procedures. These sampling techniques causes the auditor to apply procedures on a limited sample to form an opinion about the entire population. In historical perspective, it was indeed difficult to see all transactions for an audit, but with new technological developments it is possible to perform procedures on the entire population of transactions in a cost-efficient way. As one of the main enablers, technological developments has provided a large contribution to the change in audit procedures. (Trompeter & Wright, 2010)
Analytical procedures
Analytical procedures can be used to identify significant fluctuations or relationships that are inconsistent with other relevant information say Hayes, Wallage and Gortemaker (2014, p. 284). Any further steps taken by the auditor need to precede by inquiries of management as the author’s note that the further audit procedures will be based on the answers received from these inquiries. The purpose of analytical procedures can therefore be seen as being able to see significant fluctuations and relationships that are inconsistent with other relevant information.
The execution of analytical procedures might have a significant impact on the use of the outcomes as audit evidence. This can be influenced by on the one side the data used (See paragraph 2.2) and on the other side the execution by the audit team. The execution of the procedures can be influenced by team skills, lack of employee education or not matching technology.
2.2 Big data
Definition of Big Data
The definition of big data refers to the application of an extensive amount of data. Among the different literature about big data, there are different definitions used of big data. (De Boer, 2015; Gartner, 2013; EY, 2017; KPMG, 2017) Although there are different definitions, all definitions share a common ground of several aspects including amount, variety and analytics. Connolly (2012) describe big data as all transactions, interactions and observations within an observed field.
Characteristics of (big) data
Big data is being described in different literature with the following characteristics: volume, velocity, variety and veracity. (EY, 2017; DeKroon & Karp, 2013; Gartner, 2013; De Boer, 2015) In their article, DeKroon & Karp (2013) describe these criteria as describing the usability of data. In case of the use of big data within audit procedures, it is important that the data quality is
The volume increase of data used in the procedures is mainly due to exchanging sampling for population based procedures. Also, due to implementation of more extensive accounting information systems, there has been an increase of data recording and output possibilities which increases the flexibility of information generation. (Kanellou & Spathis, 2013, p. 211) The use of more data is made possible as there has not been so much recording of data in such a short time frame in the history as in this century. (De Boer, 2015)
Depending on the type of data used, the velocity can enormously increase. This can be influenced as example by a daily recording or monthly recording of accounting information. Both volume and velocity enables the application of data analytical procedures as they create the foundation for the application, the data itself.
The variety of data depends on the way of recording the data or the source used. It is likely that the variety of data is lower in organization which use integrated accounting systems as they eliminate multiple inputs of accounting information. (Scapens & Jazayeri, 2010, p. 225) Due to this elimination, data is more consistently recorded which increases the use in audit procedures. The use of data from external sources also influences the reliability of data as this might not be managed actively. The veracity of the data mostly depends on the type of data used. When data is used which is influenced by more (external) players, the veracity of the data can be doubted. As stated, the variety and veracity are about the quality of the data. For the auditor, it is important to ensure the quality of the used data as it might endanger the effectiveness of his procedures.
2.3 Applying Big Data in Audit Procedures
Application of big data within the audit procedures
In this research, I will make use of the definition of data analytics applied by the professional accounting professions. (PCAOB, NBA) The use of this definition is consistent with other research on data analytical procedures within the audit practice. (Appelbaum, Kogan, & Vasarhelyi, 2017; Hirst & Koonce, 1996; Liddy CPA, 2015) De Boer (2015) refers to data-analytics as the collecting, processing, transforming and analyzing of data in order to get useful information. A distinction is made by the author between the old data-analytics and the modern version where auditors provide themselves with access to audit evidence in the era of data, digitalizing, standardization, connectivity and unmatched automatized analytical and visualization possibilities.
Within this thesis, I will refer to data analytical procedures as being audit procedures including big data instead of sample selections. Not to be confused with procedures including
(data) analytics since these are only part of the big data procedures and therefore not represent the whole concept of big data applied within the audit procedures.
The application of analytical procedures is required by the international auditing standards in the planning and review phase. The application of analytical procedures as substantive procedures in influence by several factors like a higher inherent risk and a higher control risk. When both risks are assed to be high, the auditor is more likely to emphasize on testing of details rather than on the analytical procedures. (Ameen & Strawser, 1994) This means there is only a very limited reliance on the analytical procedures as the application of big data in audit procedures.
Challenges arising from the implementation
Main challenges mentioned in different literature are among other things about the data quality, source validation, processing and scalability. These challenges are mainly due to the input of the data, storage of the data or the processing of the data. (Najafabadi, et al., 2015; Cuzzocrea, Song, & Davis, 2011; Zhang, Pawlicki, McQuilken, & Titera, 2012; Cao, Chychyla, & Stewart, 2015) In case of an audit the data cannot be received properly, the data might be of inferior quality or the data is not properly processed in the audit process.
In a survey under 1.153 different companies which apply data analytics, employees were asked what the main challenges were they faced. The top five challenges included inadequate training of staff (people), difficulties with the system (technology) and database errors (data). (Russom, 2011) Anderson and Loonce (1998) conclude that auditors are more likely to make a judgment mistake when they only perform a plausibility check and no sufficiency check. In order to avoid making judgment mistakes, business process cycle focused audit software tend to perform better over transaction cycle focused audit software. (O'Donnell & Schultz, 2003)
In this research, it might therefore be important to focus on the checks the company performs during their audit and how their audit software is designed as this will influence the performance of analytical procedures and therefore also the level of assurance gained from these procedures. Also training of the employees is likely to see in the results of this research.
Waves of audit transformation
As Connolly (2012) describes, big data consists of transactions, interactions and observations. Transactions are recorded in an accounting information system and contains structured data. The interactions between people exists of user specific data like communication and other social interactions. The last part exists of observations by as example sensors and GPS chips. It is possible
that there is a limited use of a component of the big data definition as it does not generate usable data. In the following paragraphs, there will be reference made to the components discussed above. Since the introduction of Big Data within the audit practice, audit procedures can be divided into three distinct types of procedures. At the first type of procedures, there is no big data involved in the audit practice, the focus is one the use of transactions. Within this phase the procedures are sample based there is a limited amount of data available. However, the available data is structured which guarantees a higher data quality. In this phase, the volume of the data is low and the velocity might also be low. However, the variety can be at normal level and veracity might be high. The application of analytical procedures is limitedly applied within the engagement in this phase. (Hirst & Koonce, 1996; Anderson & Koonce, 1998; Ameen & Strawser, 1994)
By applying the second type of procedures there is an implementation of the first new data. This new data contains aggregated data and can be described as the data analytical phase. Within this phase new procedures are implemented to extend the analysis on the available data. This will result in population based procedures and semi-structured data to be included. The amount of data is still limited to the recorded data in the accounting information system. (Russom, 2011; Trompeter & Wright, 2010)
As last phase, a new type of data is included in the audit procedures. In this phase, new data is being recorded/used and implemented in the audit procedures. This will result in the use of more unstructured data such as sensor-recorded data. Measuring the level of customer satisfaction increases the understandability of the recorded sales. (Ittner & Larcker, 1998) In this phase, customer satisfaction can be measured by as example using social media data as implementation of unstructured and extensive data.
Element Phase 1 Phase 2 Phase 3
Audit selection Sample based Population based Population based Data amount Limited More extensive Extensive
Data structure Structured Semi-structured Unstructured Types of data Transactions Transactions and
interactions
Transactions, interactions and observations. Table 1
2.4 Institutionalization process
Institutionalization work
The institutionalization work (or: process) is described as “the purposive action of individuals and organizations aimed at creating, maintaining and disrupting institutions” and prioritize the study of actors’ actions in order to “capture structure agency and their interrelations”. (Canning & O'Dwyer, 2016, p. 3) Institutional work traditionally focusses on the macro dynamics that drives the social and economic changes occurring in organizations. (Lawrence, Suddaby, & Leca, 2011, p. 52)
As discussed in previous paragraphs, the current audit transformation is largely due to the technological and regulatory developments. Due to the by nature highly regulated nature and commercial environment of the audit practice, it is important to ensure real assurance is obtained from the new procedures as otherwise the new procedures won’t contribute to the existing audit procedures in regulatory way and on profitability. Actions undertaken during the institutionalization process will therefore be aimed at ensuring the new procedures will provide assurance. This institutional research will look on the influence of these actions on data analytics within the audit practice where the focus will how it has been ensured to gain assurance.
Two main fundamentals for the concept of institutional work. The first concept is about the organizational phenomena that there are “enduring elements … that have profound effect on the thoughts, feelings and behavior of … actors”. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 216) This fundament forms the fundament for the definition of an institution. The second foundation of institutional work describes practices as “embodied, materially mediated arrays of human activity centrally organized around shared practical understanding”. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 218) Understanding how institutions are affected by actions of actors in the field is the main focus of institutional work. The authors describe these actions as intentional and some of the highly visible but mostly invisible. ( (Lawrence, Suddaby, & Leca, 2009, p. 1)
Institutional work can be divided into roughly three key-elements. First it would “highlight the awareness skill and reflexivity of individual and collective actors”. This element highlights the rationality of the actors within their organizational fields and they do not accept everything solely. The second element consists of understanding the fact that the institution is constituted in the actions of the actors in the field. As last element, it is noted that actions undertaken for changing an institution, are institutionalized themselves. This element moreover notes that there is no lack
of innovation but that the actions undertaken are most likely specific for the field. (Lawrence & Suddaby, Institutions and institutional work., 2006; Lawrence, Suddaby, & Leca, 2009)
As auditors are involving new types of procedures from other fields of interest (Big Data), this can be recognized as moving their field boundaries. It is noticed that actor’s boundary work and practice are configured in a cycle of institutional innovation, conflict, stability and re-stabilization. Changes or transitions in the cycle are triggered by “the state of boundaries, the state of practices and the existence of actors with the capacity to undertake the boundary and practice work of a different institutional process”. (Zietsma & Lawrence, 2010) A successful movement of the boundaries, therefore also require the right actors.
The audit transformation consists of different actions undertaken by actors in the field. These actions are intentionally towards the institution (Big Data) undertaken and limitedly visible for all actors. The operating field of actors will enlarge as the implementation of Big Data consists of new procedures. As the actors in the field does not take everything for granted, they do not accept solely that there no acceptance of big data in the audit regulation. Due to undertaken actions by the actors, the institution of big data is constituted. The transformation thus contains multiple elements of the theorized concept of institutional work.
Forms of institutionalization
Lawrence and Suddaby (2006) has described an extensive typology of institutional work describing different forms appearing during the institutionalization process. Below, the different forms will be described per main category: creating, maintaining and disrupting institutions.
Creating institutions
Advocacy The mobilization of political and regulatory support through direct and deliberate techniques of social suasion.
Defining The construction of rule systems that confer status or identity, define boundaries of membership or create status hierarchies within a field.
Vesting The creation of rule structures that confer property rights. Constructing identity Defining the relationship between an actor and the field in
which that actor operates. Changing normative
structures
Re-marking the connections between sets of practices and the moral and cultural foundations for those practices.
Constructing normative networks
Constructing inter-organizational connections through which practices become normatively.
Mimicry Associating new practices with existing sets of taken-for-granted practices, technologies and rules in order to ease adoption. Theorizing The developments and specification of abstract categories and
the elaboration of chains of cause and effect.
Educating The educating of actors in skills and knowledge necessary to support the new institution.
Maintaining institutions
Enabling work The creation of rules that facilitate, supplement and support institutions, such as the creation of authorizing agents or diverting resources.
Policing Ensuring compliance through enforcement, auditing and monitoring.
Deterring Establishing coercive barriers to institutional change. Valorizing and
demonizing
Providing for public consumption positive and negative examples that illustrate the normative foundations of an institution.
Mythologizing Preserving the normative underpinnings of an institution by creating and sustaining myths regarding its history.
Embedding and routinizing
Actively infusing the normative foundations of an institutions in to the participants’ day to day routing and organization practices.
Disrupting institutions
Disconnecting sanctions/rewards
Working through state apparatus to disconnect rewards and sanctions from some set of practices, technologies or rules. Disassociating moral
foundations
Disassociating the practice, rule or technology from its moral foundation as appropriate within a specific cultural context. Undermining
assumptions and beliefs
Decreasing the perceived risks of innovation and differentiation by undermining core assumptions and beliefs.
Creating institutions
Advocacy
Advocacy is described as “the mobilization of political and regulatory support through direct and deliberate techniques of social suasion”. This form of institutionalization is in most cases the starting point of an institution and includes actions like lobbying and advertising. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 221)
Defining
The second form of institutionalization is described as “the construction of rule systems terms that confer status or identity, define boundaries of membership or create status hierarchies within a field. Actions included in this form consists of setting standards or certification of actors within the field. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 222)
Vesting
As third form of institutionalization, vesting is described as “work directed toward the creation of rule structures that confer property rights”. This form includes actions focused on the regulative part between an authority and another actor within the field. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 222)
Constructing identities
Identities within the institutionalization process are about the relationship between the field and the actor. Actions within this form includes most of the time the creation of professions acting in the field. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 222)
Changing normative associations
As a part of the process, reformulation of normative associations is one of the other forms. This form aims at the connections between sets of practices and the moral cultural foundations. Within this form actions like reports about interactions and discussing within the process are written. This form of the process is about the norm-field relationship. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 223)
Constructing normative networks
Within the institutionalization process there will be a need for a peer group. This peer group will be used within the normative networks “with respect to normative compliance, monitoring and evaluation”. The form described the change in the way actors behave with each other in the field. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 225)
Mimicry
During the institutionalization process it might that the actors in the field associate the new procedures/changes with the old situation before the institutionalization. When actors familiarize the new situation with the old situation, the process of institutionalization will be eased since the resistance will be much lower. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 225)
Theorizing
The form of theorizing is described as “the development and specification of abstract categories and the elaboration of chains of cause and effect”. During this form of institutionalization concepts are named and described. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 226)
Educating
As the final form of creating institutions, educating the actors in the field will happen. The educating part is important since “creating new institution often involves the development of novel practices”. This form consists of courses and other training material for the actors within the field. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 227)
The creation of an institution is relevant for the case as the attempt of the institutionalization of big data is being researched. Whether challenges in the adoption phase of the transformation will be overcome, partly depends on the actions undertaken during the creation of the institution. When people are not well enough educated as example, they will counteract to the transformation and will not involve Big Data in the audit procedures. Regardless of the level of assurance obtained, it all starts with a solid foundation to let the people start involving Big Data.
Maintaining institutions
Enabling work
After the creation of institutions, it is required to maintain the institution by among other enabling work. Enabling work can be described as “the creation of rules that facilitate, supplement and support institutions. During this phase, handbooks and guides will be created to enable people working with (in) the new institutions. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 230)
Policing
Besides enabling work, it is important to ensure compliance within the institution. This can be achieved by among other things monitoring of the actors in the field. It might that sanctions are implemented when compliance is not achieved. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 231)
Deterrence
When creating and maintaining institutions it might that there are barriers from existing institutions. These barriers limit the new institution to exist. On the other hand, barriers might preserve the institution being undermined. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 232)
Valorizing and Demonizing
In this form of the institutionalization process the actors are evaluating the behavior of other actors within the field. The evaluation is based on the new moral standards within the institution. In fact, there will be judged on what is going right or wrong. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 232)
Mythologizing
This form of institutionalizing the history of the process is preserved with stories. Actors within the field are preserving stories about the history of the institution. As it seems logical, when there are stories about the history of something, it is an institution that is kept alive. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 233)
Embedding and Routinizing
After creating normative foundations for the new institution, it is needed that these foundations are being routinized and embedded in the actor’s daily practice. After routinizing these foundations, it will be more likely that all procedures are correctly executed in line with the normative foundations. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 233) After the creating of an institution the importance of maintain the institution cannot be overstated. As otherwise people will return to the traditional audit procedures. During this phase, it is furthermore important the new audit procedures are being embedded and routinized within peoples’ work. Also, the first guidance on how to perform the procedures is being created to support people in performing the new procedures. This also involves the first feedback moments
to evaluate each other’s’ involvement of Big Data. Involving regulators at this moment in the process might increase the success rate of the institutionalization.
Disrupting institutions
Disconnecting Sanctions/Rewards
Within this form of the process, there is “work in which state and non-state actors worked through state apparatus to disconnect rewards and sanctions from some set of practices, technologies or rules”. For example, this can mean that actors in the field might have to face jurisdictional sanctions. This form will mainly occur when there are many different actors involved or specific technology as the new institution. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 235)
Disassociating Moral Foundations
As part of disrupting institutions, it is possible that actors “disassociate the practice, rule or the technology from its moral foundation as appropriate within a specific cultural context”. This is mostly done by indirect practices which undermine the moral foundations of the institutions. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 236)
Undermining Assumptions and beliefs
Creating and maintaining institutions comes with costs of “moving actors away from taken-for-granted patterns of practice, technologies and rules”. Thus, when the costs are low the assumption and beliefs of the institutions are most likely to be undermined since the actors might stick to the “taken-for-granted patterns of practice”. (Lawrence & Suddaby, Institutions and institutional work., 2006, p. 237)
As last, Big Data will start to disrupt the traditional procedures as an institution. This is being achieved by elapsing of the time and by the disassociation of moral foundations as the traditional sampling techniques fade away and there are no longer taken-for-granted. For new audit procedures, this also involves adapting the regulation to the new procedures. The regulators need to adapt in order to allow the new procedures in the audit practice. There is a need for a change in assumptions and beliefs which enables to sanction or rewards when the new procedures are not (correctly) applied in the audit practice.
Phases of institutionalization
Canning and O’Dwyer (2016) performed an institutional research about the set/up of a new regulatory body in the accounting profession of Ireland. They recognized five separate phases of the institutionalization process in their research about the institutionalization process. During each of these phases, they describe the dominant forms of institutional work described in the framework of Lawrence and Suddaby (2006).
This research will seek to frame the actions undertaken to institutionalize data analytics within the audit practice in a similar scheme of phases. In this way, all actions undertaken can be linked the interaction between them can be analyzed All forms of institutionalization actions has been described in paragraph 2.4.2. Analyzing the categorization of actions per phase in the process, enables to include the interactions between all undertaken actions in analyzing how auditors ensures to obtain assurance from data analytics.
The first phase in the process of institutionalization is about disruption of existing institutions. These institutions’ assumptions and beliefs are being undermined by disassociating moral foundations and hard advocacy as forms of the institutionalization. Hard-advocacy is defined as “the use of highly direct, explicit, confrontational, and threatening practices of social suasion which mobilize rhetoric and explicit contrasts outlining terrible consequences.” (Canning & O'Dwyer, 2016, p. 7)
Phase two is about setting up the new institution by constructing a normative network and a consensual identity. In this phase, this is achieved by educating of the players in the field, soft advocacy by the disruptors and defining of the identity of the new institution. Soft-advocacy is defined as the use of subtle, largely implicit, unthreatening techniques of social suasion. The construction of a consensual identity “involves efforts to build constructive relationships to be viewed as consensual and responsive.” (Canning & O'Dwyer, 2016, p. 9)
Figure 1 – Different phases of institutionalization (Canning & O'Dwyer, 2016)
As third phase, the confrontational identity is being created by policing and enabling. During this phase, there will be self-mythologizing of the new institution. This confrontational identity is the identify that is shown to the players in the field. Constructing a confrontational identity “is defined as work involving taking firm, non-negotiable, uncompromising positions to be viewed as conflictual and confrontational.” (Canning & O'Dwyer, 2016, p. 11)
Following up the four phases, phase three consists of commencing maintenance work by the same actions as undertaken in phase three, but instead of enabling the construction now deterring of the construction of the commencing identity. Self-mythologizing “work involves work among a community of peers designed to create and sustain myths regarding the community's history and actions in a specific domain.” (Canning & O'Dwyer, 2016, p. 13)
The last phase within the process of institutionalization is about disrupting prior institutionalization work by disassociating moral foundations. This is being achieved by mimicry and hard-advocacy. (Canning & O'Dwyer, 2016, p. 14) Again, the re-definition of hard-advocacy as described in phase one is used in this phase.
Institutionalization in the profession
Since auditing is a highly regulated profession, it is difficult to institutionalize a process without having the profession (including the regulator) included. New processes or techniques need to be fitted within the audit standards or they need to be approved by the regulator(s). (Greenwood, Suddaby, & Hinings, 2002) However, the institutionalization in the profession will be dependent to institutionalization at the companies working in the profession since the regulator will adapt their regulation in response to developments at the companies.
There are signals of regulatory bodies that the application of big data within the audit practice will increases. The AICPA (2014) states that there is a need for research on “how data science and related technologies can be harnessed and tailored into applications for auditors, extending auditing theory to encompass innovative approaches” and “modifying auditing standards where necessary.” These three measures encompass major challenges described in paragraph 2.3.2. In the Netherlands, the Royal Dutch Professional Body for Accountants (NBA) started a knowledge body about Continuous Assurance, Data Analytics and Process Mining. (Bottemanne, 2013)
As the audit procedures are transforming with the involvement of population based techniques instead of solely sample based techniques, the amount of data used increases. This data can be extracted from multiple sources which makes the data quality more insecure. As the involvement of big data is currently going on, three separate phases are recognized with more data included in each phase. Challenges arising from the implementation are about the three main factors people, data and technology. Obtained assurance depends heavily on the regulatory approval with regards to the new procedures. The actions performed to institutionalize Big Data in the audit procedures can be divided into three main categories of actions, namely creating, maintaining and disrupting.
3 Research method
In this chapter, the research methodology will be explained. The chapter starts with a paragraph about the set-up of the research and is followed by a paragraph about the interviewees. Last, the coding of the interview results will be discussed.
3.1 Research description
The audit transformation towards including big data in the practice, is said to improve audit quality through new (additional) assurance. Focusing on both the institutionalization and the outcomes of the process, this research will determine how the big data is institutionalized for the new procedures to provide assurance to the auditor(s). Building on the existing institutionalization theory, the process of the implementation at the case company will be described and analyzed. Secondly, the outcomes of the institutionalization process so far are described and analyzed how they are originated from the process.
The importance of the research is there as the transformation is currently nearing the completion phase as regulators are adopting the new procedures in the regulation and there are alternating signals about whether the new procedures provide assurance or not. Therefore, it might that the institutionalization process has not reached its original goal of the creation of a new institution. Moreover, for smaller audit firms, which traditionally adopt innovating methods later due to budgetary limits, knowing how big data is institutionalized is important to succeed the implementation in their own procedures. By analyzing the current situation, an analysis can be made on how the institutionalization process should be differentiated to get a different outcome.
3.2 Research set-up
As noted earlier, the research is performed as a qualitative study. Due to the explanatory (How) nature of the research question, this suits a qualitative research method (Marshall & Rossman, 1999) with a case study setting. Qualitative research tends to “build rich descriptions of complex circumstances” and “show relationships between events and the meaning of these relationships.” This research will try to analyze processes that occur around data analytics and analyze the relationship between the actions and the assurance obtained, Gephart (2004, p. 455) described analysis of processes as one of the emphasizes of qualitative research. Studying a “phenomenon with its real-world context” and collecting the data in the natural setting, a case study is favored as setting.
The data access, which sometimes might be difficult with a qualitative research, had been ensured with my thesis internship at the case company. I will partly use existing data like publications by professionals in the field or the firm itself. To expand the existing data, I will also perform interviews with employees of the case-company. Below, a schedule of all used data sources can be found. In the next paragraphs, it is explained how these sources were used within the research.
Source description Specific source Notes
Interviews (See: Appendix A – List of interviewees) Internal webpages - Audit transformation website
- Data analytical webpage - [Department A] support page - [Department A] learning page Guidelines - Guidelines Health care industry
- Guidelines Public sector - Data extraction guides - Overview of analysis
Regulation - [Newsletter A] about methodology - [Newsletter B] about Weekly News
- Internal methodology guide through [software A]
3.3 Interviews
All interviews have been conducted by myself. During the interviews, I did not make use of a fixed list of interview questions. The three main subjects to be discussed were the development over the past year, the experience with data analytics and the vision on obtaining assurance. For interviewees of management level and higher I also discussed the subject of external regulator as lower level employees are not involved in those discussions.
During the interviews, the definition of data analytical procedures has been determined based on the use of the organization. In the theoretical part of the thesis, this definition will be linked to the definition in the literature and differences between both definitions will be explained. This is also important for the use of the results of this research in other cases.
After the interview, a transcript of the whole interview I recorded had been made in Dutch. The transcripts of the interviews have been send to the interviewees for their approval of the included statements. I did not receive any rejection of a transcript, only an addition from one of the interviewees. One of the interviewees also included a requested to soften one of his opinion about a client. This change had no influence on the interview results. After making and receiving the approval, all conclusions and other interesting parts said in the interview has been coded based on the coding scheme.
3.4 Other data sources
Within the organization, there are multiple sources of information about data analytics. Most of them are only internally published. Due to the anonymity of the case-company, I have not used any externally published material. As the amount of internally published material includes is bigger, it can be assumed no essential information is missing within the analysis.
Documents and webpages had been listed on beforehand and categorized on two types of content: guidance or statements. The materials containing statements had been analyzed and the statements within the materials are used within the research to support the interview results. Guidance materials are used to support the results of the interviews and the materials containing statements of the organization. Both categorizations of materials are analyzed with the coding scheme as for the interview results.
3.5 Interviewees
In the previous paragraph it is explained that part of the data will be gained by interviewing professionals working at company Z. The selection of the employees will be based on their function (example: staff or manager), the department they are working for and industry specialization. The interviewees of senior (S) level had been selected based on their experience for reviewing the audit procedures and S2 was also selected based on his experience in the health care industry. Interviewee M2 and M3 were selected on their role as Data Champion at the local office and M1 was selected on his role as manager on an audit team. The selection of P1 was one as his role as engagement partner.
In the company, there is a specific department for data analytical related work. By including people working for this department in the selection of interviewees, a better insight had been gained from non-auditors in the actual (right) implementation of the data analytical procedures. Therefore, M4 was included as manager at [Department A].
Ref. Department Motivation for selection Function/Rank S1 Assurance department Experience with DA (Review) Senior Staff B1 Assurance department Experience with DA (Preparing) Staff P1 Assurance department Role of signing partner on engagements Partner B2 Assurance department Experience with DA (Preparing/Review) Staff
M1 Assurance department Experience with DA (Management) Senior Manager M2 Assurance department Role of Data Champion Senior Manager S2 Assurance department Industry specific knowledge (Healthcare) Senior Staff B3 Assurance department Experience with DA (Preparing) Staff
B4 Assurance department Role of Data Champion Staff
M3 Assurance department Experience with DA (Management) Manager M4 Data analytic
department Specialist on supporting department Senior
3.6 Coding of the results
After making all the transcripts of the interviews, the results are coded based on the coding scheme developed. As mentioned by O’Dwyer (2004, p. 391), coding of the results has three main purposes: data reduction, data display and data interpretation. Data reduction will be less important for this research as the number of interviews is somewhat lower and the interviewees are choosier selected. Therefore, coding of the transcripts was mainly done in order display the data and to interpret the data.
The coding scheme distinguishes three various categories of results: results about the actions during the institutionalization process, results about the outcome of the process and results about possible contradictions. An additional code has been created for the definition of data analytics as that definition needed to be compared with the definition used in the literature. The following part of the coding scheme is the results:
Code Description
The first set of results about the transition in procedures (T) are ordered based on the time-series analyses. In this way, it is possible to recognize the phases of institutionalization and see in what order the actions had been undertaken. The coding in this set of results has been done on chronology order after performing a substantive part of the interview and this chronology could be determined reliably. Code Description T1 General T2 Training T3 Change in procedures T4 Publication T5 People T6 Software (client) T7 Software (own) T8 Guidance T9 Benefits T10 Strategy T11 Warranty
The results about obtaining assurance with data analytics (A) of the process are coded in a pattern-matching way. In this way, it is able to analyze the pattern with the expected pattern after the undertaken actions. Additionally, the pattern-matching way enables it to unveil opposite results on subjects. These subjects have been selected after performing a substantive part of the interviews.
Code Description A1 Regulation
A2 Level of Assurance A3 Audit future A4 Guidance
To ensure assurance from the new data analytics, it is important that for at least all (experienced) challenges an action needs to be undertaken. Otherwise, the level of assurance might end up being lower as requested. For all results with regards to the challenges (C) the following coding scheme has been created. These results can be matched with the results about the transition towards the implementation.
Code Description C1 General C2 Skills C3 Data C4 Software (client) C5 Software (own) C6 Audit Quality C7 Client Skills C8 People
To describe the current situation, a coding scheme was made for the results about the situation (S). The results about interpretation of interviewees are not included. All codes have been ranked on pattern matching order to be compared with the other results.
Code Description
S1 Performing procedures double S2 Guidelines in practice
S3 Benefits and disadvantages S4 Difference p/k
S5 Client benefits
S3 Year after year difference
During the interviews or other meetings, employees provided remarks about their experience (E) with data analytics. As their experiences are likely to provide contradictions to the theoretical situations as described under the other coding results, a separate coding scheme was created to capture the reality in practice separately.
Code Description E1 Comparable with E2 Examples E3 Action/reaction E4 Difficulties E5 Exchange of
4 Case narrative
This case narrative will start with the definition of big data within the audit practice of the organization. Through describing the differences with existing literature, the analysis will be made whether the organization uses the same definition of data analytics. As second part, the process of implementation existing of the actions performed by involved people will be described. After describing the implementation process the current situation about the data analytics will be elaborated. Lastly, an analysis of how the implementation process affected the current situation is made to determine what the case company did to gain assurance from the new procedures.
4.1 Big data in the financial audit practice
The case-company has started a widely spread implementation of data analytics around 2011/2012 after a pre-introduction in earlier years. During the implementation, several actions were undertaken in the working environment to prepare for the implementation of the new procedures. In this paragraph the definition of data analytics in the organization will be elaborated.
First application
A very first implementation of data analytics was initiated by the basic analytical procedures as required by audit regulation. These first analytics existed as example of the recognition of significant movements in comparison to the prior year on both balance sheet and income statement. Any further analytics were not yet initiated nor possible due to the lack of technological possibilities to gain the data and analyze the data. Although advanced analytics became possible, simple analytical reviews are still be performed as these are made mandatory by external regulators. At the end of the 20th century it became possible to implement more advanced data analytics as the amount of data being recorded was raising enormously and new hard- and software made it possible to perform more complex analytics. The organization decided to introduce the new advanced data analytics after the social requests for higher audit quality. The data analytics would enable the organization to improve audit quality in a cost-efficient way.
Definition
Data analytics is described by the company as “capturing and analyzing all of the data rather than relying on sampling techniques”. Performing other analyses on the data set, it will give the audit team different insights (B3, p.1) in the financials, but also in the processes of an audit client by process mining. (M3, p.1) The interviewees made clear that data analytics within the organization
involved all diverse types of analytics involving the entire population. In both the interim as year-end phases of the audit, data analytics are applied. (B3, p.1)
Different applications
During the interim phase of an audit, the auditor will investigate the internal control systems and determine whether this systems ensures it limits the risks of errors in the accounting. B3 (p.1): “Process mining is currently applied sometimes, although more limitedly due to the limited guidance on this subject.” Despite a limited application yet, the auditor will gain additional assurance about the internal control system with the help of process mining. M1 (p.2) mentions that the last part of process walk-throughs can be verified within the accounting data. “This will give you additional assurance that the process really works as it is said by the client.” The process mining is also used to determine the auditors expectations with regards to the process. If the expectation significantly differs from the narrative of the client, the auditor might decide to investigate this further.
For the year-end audit phase, the organization makes use of two separate analytics. The organization applies journal entry testing to cover general risks like management override. Journal entry testing consists of analysis on characteristics of the journal entries like the date of input, responsible user and nature of journal entry. M3 (p.2): “You can see when the entries are made and determine whether it is logical for this client that there are many manual entries made on Saturday and Sunday.” After the analysis, the auditor will focus on the exemptions found.
Secondly, account specific analytical programs are used to analyze these account it relates to. Analysis like a three-way match correlation test are being performed in these programs. The data analytics work sheet programs existed mainly of the match between the expected correlating accounts. For revenue as example, it is expected that all revenue correlates with the trade receivables and that all trade receivables are received by cash. Within the programs this analysis is been done and exemptions are selected for further investigation. The analysis of this accounting data differs from randomly selecting a number of sales invoices and reconciling this with the expected accounting entries. In the new analysis, all data is checked and the exemption are further investigated.
In both current applications the procedures make use of the whole available accounting set. Prior to the implementation, the audit team would randomly select a number of entries in order to order these transactions. These sampling techniques did not focus on the exemptions, but just randomly selected entries. With the use of the whole data set of entries, it has become possible to focus on the exemptions found.
Risks covered
For traditional procedures like a walk-through procedure, audit teams depend heavily on the information obtained from the client. With the new data analytics, the used data exists of all data records. This makes it much hard for the client to hide information for the audit team. Other risks within that are being covered by data analytics are fraud risks (B1, p.1) and management override.
As a signal of possible fraud, journal entries recorded on Saturday and Sunday might raise questions when it is unusual to work on these days at the client. (M1, p.1) Furthermore, manual journal entries can be expected during the end of the period, but many manual entries in the middle of the period are not expected. (M1, p.2)
M1 (p.1): “We also obtain assurance from journal entry testing by covering the risk of management override.” The risk of management override can be covered when an IT control with regards to logical access is effective and the responsible user is included in the data. M1 (p.1): “When everything is correctly recorded, you can see who has made the entry and determine whether this matches to his position.”
Analyzing the application of data analytics within the organization, the definition fits the definition of data analytics used in existing literature as it is described earlier. Thus, the company has no significant different vision about what data analytics are and exists of.
4.2 Process of the implementation
Reasons for the implementation
Reasons for the adoption of data analytics within audit engagements mentioned differ widely from each other. The mostly mentioned reasons are performing an audit more cost-efficient, audit quality and being able to provide additional information to the client.
The organization states at her internal audit transformation webpage, that the audit transformation (e.g. implementing data analytics) focuses on giving the team back their primary aim which consists of providing high quality audits. Data analytics enables audit teams to focus on details that stand out in the analytics instead of analyzing a chart of accounts to comply with audit regulations. (S1, p.1; B1; p.2) Audit quality is thus stated as the main reason for the implementation of the new procedures. However, higher audit quality usually comes with higher audit costs which lowers the profitability of the organization. Therefore, the organization will try to seek ways to improve profitability.
In a best-case description internally published by the organization, it is stated that 2.000 audit hours were saved globally by the successful implementation of data analytics on an audit engagement. Prior to the application of data analytics team members would request general ledger account charts when necessary. B1 (p.1): “Prior to the data analytics, it was required to eyewitness the client obtaining the account chart from the accounting system.” The new procedures therefore enable them to use accounting records more easily, quickly and secure despite the data set verification requirements. B3 (p.1): “Focusing on findings with the analytics will save you lots of time, as the rest won’t contain material errors.
Data analytics also provides analytical outcomes which were interesting for the client and that were communicated to the client during or after the audit. S1 (p.3) mentions that on one of her audit engagements, relevant information was included in the management letter which was send to the client. S1: “The client didn’t even know we could draw this kind of analysis based on their data.” (p.3) Also M1 (p.2) notices that the data analytics sometimes provide more information as the client has. S1 (p.1): “The client especially found it interesting to see we were able to investigate which user made that amount of entries throughout the year.” Also, the information about the correctness of the entries (correlation) has been proven to be valuable to the client, as M1 (p.2) mentions.
Audit transformation
The organization decided to include the implementation of data analytics in a wider audit transformation. By including data analytics in a wider audit transformation, the introduction of data analytics as part of the audit transformation can be marked as a shift within the organization and is therefore likely to get more attention from the employees. Advocating the transformation within the audit practice is necessary to ensure audit quality, the organization started the transformation about 5 years ago.
The advocating was performed through introducing webpages, publishing hand-outs, paying attention to it in newsletters and publishing other materials and had soft-characteristics. These soft-characteristics were mainly due to the fact the organization emphasized on why it was good to implement the data analytics in an engagement. Also, there was no compulsory tone in the published articles.
To make all materials with regards to the data analytics accessible, an intranet section has been created where all materials can be found an accessed. These actions were intended to introduce data analytics towards the employees and define what the data analytics consisted off.
