Data analytics : challenges in financial statement audits

(1)

Data analytics: challenges in financial

statement audits

Name: Sharona Gopal Student number: 11398574 Thesis supervisor: prof. dr. B.G.D. (Brendan) O'Dwyer Date: June 25, 2018

Word count: 28458

MSc Accountancy & Control, specialization Accountancy Faculty of Economics and Business, University of Amsterdam

(2)

Statement of Originality

This document is written by student Sharona Gopal who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(3)

Abstract

This research investigates whether the challenges of using data analytics when performing financial statement audits are consistent with the challenges described in the prior literature and how they will lead to more or less efficiency and effectiveness in performing the financial statement audits. Providing an answer for this research question addresses the problem, which is that it is still a challenge for auditors to work with data analytics with the introduction of tools due to the lack of education and the lack of standards at this moment (Early, 2015). The research is relevant because examining the way data analytics can provide more or less efficiency and effectiveness regarding the challenges, auditors can perform their work better. With the research regarding the new challenges, auditors can focus more on being aware of these issues and solving them. Clients can also be more aware of the challenges auditors face and help auditors, for example, by making their working

environment more structured for running data extraction. This research also makes use of the grounded theory to examine connections without being biased.

In addition, the results from conducting the interviews are summarised, and the

challenges associated with data analytics at one of the big four audit firms in The Netherlands are compared with the challenges described in the prior literature. Interviews are conducted with 10 interviewees from one of the largest audit firms (audit firm X).

Overall, the research suggests that the new self-made categories complement the categories of prior literature, because the challenges contain equations. Improving analytical skills, communication, education and technical skills will lead to more effectiveness of data analytics, but improvements in these areas will have no effect on efficiency, because the auditor must use more actions to explain the data and interpret it. Being analytically strong, being able to make connections, thinking in a certain way and using external sources will not lead to more efficiency, because the auditors will obtain more insight by making connections between multiple objects.

Next, achieving better data quality, determining more exceptions with running data analysis and introducing new tools to keep up with innovations concerning technology will lead to more effectiveness and also to more efficiency. However, because of the learning effect regarding determining how to work with data analysis, it does not yet lead to more efficiency. The auditors must document the results and discuss the results with the client, so it will often take the same amount of time.

(4)

Finally, adjusting the guidelines to data changes of data analytics will help auditors to perform their work effectively, but it will not lead to more efficiency. However, since the audit standards are not adjusted, the guidelines will not help the auditors in performing their job effectively.

(5)

3.2.1 Type of interviews ... 23 3.2.2 Type of roles ... 24 3.2.3 Interview questions ... 25 3.2.4 Interview setting ... 25 3.2.5 Theory ... 25 3.2.6 Result ... 26 4 Case analysis ... 28 4.1 Grounded theory ... 28 4.1.1 Aggregate dimensions ... 30 4.1.2 Categories ... 32 4.1.3 Relationship ... 50

5 Discussion and conclusions ... 53

5.1 Self-made categories... 53

5.1.1 Training and expertise of auditors ... 54

5.1.2 Data availability, innovation and performance ... 55

5.1.3 Guidelines of financial statement users ... 56

5.1.4 New challenges ... 57

5.2 Comparison of challenges ... 58

5.3 Effect of the categories on the financial statement audits ... 59

5.3.1 Effectiveness ... 59

5.3.2 Efficiency ... 60

5.4 Conclusion ... 61

References ... 63

(7)

Appendix A: Interview questions ... 68

Appendix B: Summaries of interview transcriptions ... 69

Appendix C: Basic transcription conventions ... 91

Appendix D: Data structure... 92

Appendix E: Research model ... 100

Appendix F: New challenges ... 101

(8)

1 Introduction

1.1 Background

1.1.1 Big data

Lately, audit firms feel a great deal of pressure to provide reasonable assurance by improving the efficiency and effectiveness with which they accomplish their audits. They search for other methods to improve this efficiency and effectiveness. One way of improving this is by using data analytics (Cao et al., 2015). Section 1.1.2 will explain the definition of data analytics.

Currently, there is no exact definition of the term ‘big data’. It characterizes a very large collection of data, the speed with which data is transferred, and the tremendous growth and availability of data (Fhom, 2015, p. 2). According to Brown-Liburd et al. (2015) there are three features of big data. These features consist of velocity, volume, and variety (Benjelloun et al., 2015). Big data introduces new approaches to designing the audit process as a consequence of emerging technologies, and it influences the tasks completed by accountants and auditors (IFIAR, 2015). Technology is transforming the audit process by changing the way a financial statement audit is done and supporting the use of data analytics. A new approach to designing the audit process is by semi-automating the audits. This involves additional use of hardware and software inputs and computerized documentation (World Bank Group, 2017). For example, Audit Firm X (one of the biggest audit firms) uses ‘Halo’, which is an internal analytical tool, to take over ACL and IDEA. Halo is intended to evolve into a timely software application which analyses and convince data utilising a suite of algorithms (as cited in Salijeni et al, 2018, p. 7). ACL and IDEA are tools that analyse data of an organization and thereby decide whether deviations for analysis are being tracked (Lin and Wang, 2011, p. 777). The tool Halo will be further explained in Section 1.1.3.

1.1.2 Data analytics

Data analytics is the process of inspecting, cleaning, transforming, and forming big data to uncover and communicate useful information and patterns, suggest conclusions, and support the decision making of auditors themselves. This process forces greater visibility and

transparency on their activities (Cao et al., 2015; Earley, 2015). It also helps the audit firms to improve their efficiency and effectiveness in performing their audits. Cao et al. (2015) finds that data analytics yields favorable circumstances to detect exceptions quickly. Data analytics also identifies patterns and trends for analysis purposes.

(9)

These analysis purposes is called data analysis.

Data analytics also presents numerous challenges. For example, for the four largest audit companies Deloitte, PwC, KPMG and E&Y (the ‘big four audit firms’), making use of data analytics results in higher investment in connection with deployment and training (Kwon et al., 2014, p. 391).

1.1.3 Audit literature

Data analytics benefits the audit process by collecting, managing and analysing data more easily. It also imposes greater visibility and transparency on their activities (William, 2013, p. 556). For this reason, data analytics has the power to improve the judgment and decision making of auditors and the way they collect audit evidence (Alles, 2013; Titera, 2013). Also, Hoogduin et al. (2014) shows that auditors can complete their audits more effectively by using data applications like data analytics. This is exactly what the big four audit firms are seeking to comply with the Netherlands Authority for the Financial Markets (AFM).

As mentioned in Section 1.1.1, Audit Firm X uses the tool Halo. Audit firm PwC (2015) states that using algorithms allows the identification of relevant journals based on the criteria of 14 different tests. Halo can run 14 different tests. For example, it can text for work done at unusual times, such as on the weekends. When considering which Halo tests to run, engagement teams map how the proposed tests address the fraud risk factors and testing criteria that were identified during planning and discuss this with the engagement leader. If the proposed tests do not fully address the fraud risk factors identified, then additional testing should be designed. Users select the tests the engagement teams would like to run from the 14 different tests available in Halo. Once the tests have been selected and run, the

engagement team can filter the results. This data analytics tool makes it easier for engagement teams (also called audit teams) to explore and visualize the data, identify journals to test, and start the process of auditing those journals.

In short, data analytics and audit firms are related. Data analytics tools in audit firms are willing, especially in the four biggest audit firms, because of the tremendous amounts of information they handle. Brown-Liburd et al. (2015) discuss that difficulties such as

information overload, which makes it difficult to identify important information, could impair the benefits of applying data analytics. Krahel and Titera (2015) suggest that such

shortcomings could be addressed through the integration of data analytics into auditing standards to eliminate uncertainties.

(10)

Data analytics is likely to make crucial contributions to the auditing field. It aids external auditors by boosting the quality of audit evidence and assisting in the process of fraud detection (Yoon et al., 2015).

1.2 Research question

The purpose of this research is to asses the challenges associated with the use of data analysis by synthesising the academic literature on this subject and comparing this synthesis with the actual challenges faced by one of the big four audit firms. More specifically, this research will investigate whether the challenges of using data analytics when performing financial statement audits are consistent with the challenges described in the prior literature and how they will lead to more or less efficiency and effectiveness in performing the financial statement audits.

As mentioned in Section 1.1.1, technology is transforming the audit process by changing the way a financial statement audit is done, and data analytics is a way of making this audit more or less efficient and effective (Cao et al., 2015; World Bank Group, 2017). By comparing the challenges identified in the literature with the challenges faced in practice by one of the big four audit firms, an overview of new challenges not found in previous literature and challenges from previous literature not recognized in this research will be obtained. A conclusion can then be drawn as whether there are fewer, more, or new

challenges associated with using data analytics to perform financial statement audits. This is important information, because the audit firms need to know the obstacles they face in improving their financial statement audits. It could be that the challenges auditors encounter in practice are the same as described in prior literature, but it could also be that there are additional challenges and/or some mentioned challenges which are not recognized in The Netherlands.

According to the responses to interview questions presented in Appendix A, two examples of challenges identified by prior literature are: (1) data sets containing less quality, and (2) users assume that auditors give 100% assurance that financial statements are settled in an honest manner (Earley, 2015). According to Earley (2015), the lack of quality of data sets means that the information quality resulting from big data is determined by the quality of data collection. There is not enough information to get insight into clients’ processes.

(11)

Earley (2015) also finds that the expectation gap (meaning what other people expect from the auditing field versus what auditors give in return) happens when users rely on that auditors giving absolute assurance (100% certainty) that financial statements are settled in an honest way. In reality, auditors are not providing absolute assurance, because they sample

transactions on a test basis. Taking this into account, this research endeavors to answer the research question:

RQ: ‘How do the categories of challenges associated with data analytics affect financial statement audits in terms of making these audits more or less efficient and effective?’

Providing an answer for this research question addresses the problem, which is that it is still a challenge for auditors to work with data analytics with the introduction of tools due to the lack of education and the lack of standards at this moment (Early, 2015).

1.3 Case-setting

This research will be undertaken in the auditing field, at one of the big four audit firms in the Netherlands (Audit Firm X). This audit firm knows a great deal about their clients and their own capabilities and the conditions within their organization and outside their organization. That is why Audit firm X needs to overcome the challenges associated with data analytics to better serve their clients by advising them in a more proactive and better way.

Another reason for selecting Audit firm X, is because this audit firm attempts for continuous development regarding data, for example, by improving their services and business processes. Audit firm X is also interested in the challenges of big data because they already make use of big data to better understand their clients, for example, if big data enables the audit firm to store and circulate more information about their clients. Acquiring valuable information through data analytics will lead to performing audits better and helping clients in a better way, and therefore the audit firms will eventually gain a greater market advantage.

(12)

1.4 Method

For this research, conducting interviews is the preferred method for analysing the challenges connected to the rise of data analytics in performing financial statement audits. The reason for choosing this class of qualitative method is to test whether the employees of the big four audit firms are aware of the large impact of the challenges associated with data analytics and not only its benefits. Another reason, consistent with Turner III (2010, p. 754), is the desire to evolve the researcher’s understanding, and what they undergo, with qualitative method. It is excelling to apply varied research models for future research. Turner also finds that

interviews provide detailed information about what interviewees live through and what their perspectives are on a specific subject matter (Turner III, 2010, p. 754). These interviews are connected with other designs collecting data to supply the researcher with relevant

information for specific analysis (Turner III, 2010, p. 754). Furthermore, data will be collected internally at Audit Firm X in combination with the use of different databases.

To conduct interviews the grounded theory Lincoln and Guba (1985) will be used. According to Lincoln and Guba (1985) grounded theory is inductive; the theory is developed by gathering data and then developing a theory. A general interview guide approach is preferred as interview design to discuss topics such as how the challenges associated with data analytics affect Audit Firm X, where it is important for the interviewees to express themselves and let them share their own experiences (Gendron and Spira, 2010, p. 279).

To examine whether and how the challenges connected with data analytics have changed in recent years, ten interviews were conducted to gather a sensible amount of

evidence. The general interview guide approach consists of open questions. These interviews were conducted to answer the following research question: ‘How do the categories of

challenges associated with data analytics affect financial statement audits in terms of making these audits more or less efficient and effective?’ Appendix B lists when the interviews were conducted. The interviewees were contacted by the interviewer, by email. The interviewees were big data experts holding positions such as partner, director, manager, senior manager, or senior associate, who are in charge of different business units, for example, assurance, the national audit office and risk assurance.

A location at Audit Firm X was selected to call the interviewees. All the interviews were conducted during working hours at this firm. Audio recordings were made using the interviewer’s laptop, because this method made conducting the interview more efficient while talking with the interviewees by telephone. Each interview took approximately twenty

(13)

1.5 Contribution and relevance

This study is expected to generate new informative and insightful knowledge by investigating (the results of) challenges faced by using big data at one of the big four audit firms by providing empirical insights into such developments from the perspective of the experts on big data themselves. There is some prior research about data analytics by Earley (2015) but not about the effect it has on the auditing field, including the big four audit firms, specifically concerning qualitative research.

Cao et al. (2015) described previous research about the challenges and opportunities of implementing data analytics at the big four companies; however, this previous research concentrated on companies which do not use big data. Audit firm X already uses big data. Furthermore, the prior studies on data analytics were conducted using surveys instead of interviews.

This research complements and extends previous analyses by using recent data which is not publicly available. This study differs in research design from earlier studies because interviews were conducted to investigate this topic and to collect valuable inside information. There has been no previous study done in The Netherlands to investigate the challenges of using data analytics when performing financial statement audits today compared with the challenges faced in previous years. By comparing the challenges from these two different periods, an overview of additional and unrecognised challenges can be developed, leading to a conclusion that there are now fewer, more or new challenges connected with using data analytics when performing financial statement audits. This is important information because audit firms must know the obstacles they face to improve their financial statement audits.

This research continues to investigate the challenges associated with data analytics at one of the big four audit firms, thus generating new knowledge. In addition, the outcomes of this research can prompt clients of audit firms in other fields to make changes by bettering their data in non-traditional ways. Understanding the usefulness of bettering data will eventually result in valuable knowledge for organisations to improve data and lead to boosting profits. This research also investigates three unanswered questions according to Wang and Cuthbertson (2014). One question includes the role of data analytics in the risk analysis process. The second question contains the implications of testing 100% of the data set and the third question is about the consequences of using data analytics.

(14)

A limitation of this research is that it focusses on the big four audit firms, since they deal with the most client information and thus face the biggest challenges. Second, this research is investigated at only one firm. Next, this research can only speak for the situation in The Netherlands. Finally, there are limitations associated with telephone interviews. For example, it is more difficult to establish trust and rapport, the interviews must be shorter, and one may miss non-verbal cues. Another limitation of these particular telephone interviews was that most of the interviews were conducted while the respondent was driving. For this reason, there was not much time to ask questions, and there was a great deal of noise in the background.

The findings of this research demonstrate that the new self-made categories complement the categories of prior literature, because the challenges contain equations. However, four of the nine challenges of prior literature do not appear from the interviews. To be specific, two challenges are 100% not challenges anymore, and of the other two

challenges, results differ whether they are challenges, as mentioned in Section 5.2. There are 13 new challenges, and 3 of them (namely, 2, 3 and 13 of Appendix F) are challenges that already existed. Improving analytical skills, communication, education and technical skills will lead to more effectiveness, but these improvements will have no effect on efficiency. Being analytically strong, being able to make connections, thinking in a certain way and using external sources will not lead to more efficiency, because the auditors will obtain more insight by making connections between multiple objects. It does not lead to more efficiency, because the auditor must use more actions to explain the data and interpret it. Next, achieving better data quality, determining more exceptions with running data analysis and introducing new tools to keep up with innovations concerning technology will lead to more effectiveness and also to more efficiency. However, because of the learning effect regarding determining how to work with data analysis, it does not yet lead to more efficiency. The auditors must document the results and discuss the results with the client, so it will often take the same amount of time. Finally, adjusting the guidelines to data changes of data analytics will help auditors to perform their work effectively, but it will not lead to more efficiency. However, since the audit standards are not adjusted, the guidelines will not help the auditors in performing their job effectively.

The remainder of this paper is organised into five chapters. The next chapter provides a general overview of the definitions of big data and data analytics, followed by a discussion of the challenges that come with using data analytics when performing the financial statement audits. The third chapter first presents an overview of the previous literature and then

(15)

The fourth chapter summarises the results of the interviews and compares the challenges associated with data analytics at a big four audit firm with the challenges described in the prior literature. Finally, the fifth chapter compares the challenges associated with data analytics at one of the big four audit firms with the challenges described in the prior

literature. This chapter also provides a discussion and conclusions about the effect of the new challenges associated with data analytics and the effect these challenges have on the

efficiency and effectiveness of financial statement audits. This chapter ends with offering suggestions for further research.

(16)

2 Literature review and theory

This research consist of different forms of literature. First, it describes the academic literature by defining big data and data analytics. The challenges that come with using data analytics in financial statement audits will be discussed. Second, it relates to the literature on financial statement audits. Finally, it relates to the connection between the challenges associated with data analytics and the literature on audits. Prior research concerning the three streams of literature will be described briefly. Ultimately, the contribution of this research will be explained.

2.1 Definitions 2.1.1 Big data

Yoon et al. (2015) state that the predecessor of big data is traditional auditing. Audit firms use a combination of traditional auditing and big data. Big data plays an important role in auditing because it complements traditional evidence with adequate, reliable, and relevant information. The data consist of a variety of types of traditional structured financial and non-financial data, telephone calls, data from social media, and emails, among others (Earley, 2015).

Integrating big data adds value to traditional auditing due to its special characteristics. For example, big data provides unique and sometimes more timely evidence than traditional sources.

Brown-Liburd et al. (2015) state that the features of big data are velocity, volume, and variety (Benjelloun et al. 2015). According to Katal et al. (2013), velocity discusses enlarging and fluctuating data in a rapid way. Next, Katal et al. (2013) also explain the meaning of variety, such as data contains several formats. Finally, volume of data refers to there being a large, increasing volume of data. As stated by Berman (2013), all Vs should be present in order to be classified as big data. When it seems that, for example one Vs is absent it is not possible to classify it as big data. To describes big data even further, Berman (2013) and Katal et al. (2013) combined six more aspects, expressly vision (the goal of big data), verification (prepared data conform to particular specifications), validation (the aspiration is realised), value (important information that benefits contrasting fields), complexity (it is affronting to analyse big data because of the changing nature of data), and immutability (big data can be stored permanently if it is very well managed).

(17)

2.1.2 Challenges presented by big data

As discussed in Section 2.1.1, after traditional auditing, studies focused on integrating big data with traditional auditing (Benjelloun et al. 2015). As explained by Appelbaum et al. (2017), the nature of big data must be kept in mind. According to Benjelloun et al. (2015), a threat associated with big data is overcoming its inherent complications, such as the three Vs.

Furthermore, Alles (2013) argues that big data users experiences the challenges when they protect their enourmous amounts of data, such as users who are unaware of integrating security without harming the system. It is easy to collect big data, however it is difficult to extract data in performing the financial statement audits. (Alles 2013; Titera 2013; Cao et al. 2015).

Cao et al. (2015) find that big data identifies business patterns and trends, while the predecessor of big data, traditional auditing, is beneficial for conducting a detailed analysis of promising problems.

Contrarily, Yoon et al. (2015) it will make auditing easier with transferring client information and ensure that client information remains private when combining big data with traditional auditing.

2.1.3 Data analytics

Earley (2015) states that ‘Data analytics is the process of inspecting, cleaning, transforming, and modelling big data to discover and communicate useful information and patterns, suggest conclusions, and support decision making’. It helps to achieve reasonable assurance by improving the efficiency and effectiveness with which the big four audit firms perform their audits.

According to Earley (2015), audit firms invest in data analytics to audit more

effectively and efficiently. The essential benefits of using data analytics for audits consist of (1) auditors can analyse an enormous amount of transactions compared to how they deal with it now, (2) increased audit quality of clients’ processes, detecting fraud in an easy way

because of the powerful tools, and (3) providing services and solving problems for clients by utilising data to advise audits. The biggest benefit is the information provided by different forms of data in various quantities. Data analytics are analytical tools which are powerful for working with larger data sets and friendly to unstructured data (Russom, 2011).

(18)

2.1.4 Unanswered questions

According to Wang and Cuthbertson (2014) there are eight unanswered categories of

questions regarding data analytics. These eight categories include the role of data analytics in risk analysis, the actions which may be followed, the implications of testing 100% of the data set, the possible use of external data, the function of internal auditors in the adoption of data analytics, the interpretation of data analytics results, the consequences of using data analytics, and the fact if the auditing field requires a data analytics framework.

In their work, Brown-Liburd et al. (2015) address the skill sets and features of auditors operating data analytics work. They suggest that researchers should focus on auditors’ mental models, expertise development, and information processing issues such as dealing with large amounts of information. This information can then inform how auditors should be prepared for a data analytics environment to comply with firm training and auditing curricula. The information can also be helpful in informing how tools can be increased and/or borrowed from other fields to help auditors in making decisions.

According to Alles and Gray (2014), research encompassing many of these areas. However, they also convey that research outside the internal and external audit field, may be beneficial in advicing how data analytics could influences the financial statement audits.

2.1.5 Types of challenges associated with data analytics

As explained by Earley (2015), the challenges associated with data analytics consist of three categories: (1) the training and expertise of auditors, (2) data availability, relevance and integrity, and (3) the expectations of regulators and financial statement users. This research investigates the challenges associated with data analytics in all three categories.

1. Training and expertise of auditors

It is clear that data analytics presents many challenges for auditors in audit firms and that these challenges can affect the clients if the auditors are not skilled enough. On the website of PwC, they state that senior executives do not have the right skills or expertise to make use of data (PwC, 2018). To better perform financial statement audits, auditors need to overcome the challenges presented by data analytics and, therefore, they need to be trained.

(19)

Brown-Liburd et al. (2015) find that large amounts of non-financial data overpower the capabilities of auditors, who lack skills such as recognizing patterns and evaluating

anomalies, to process information. Auditors who are new to the field are not trained to keep in mind whether a transaction is sensible or to create an understanding of sales which enables those auditors to identify when an anomaly has occurred and how to review once it is

discovered. Regulators have concern about auditors’ lack of skills essential to properly applying data analytics techniques. They also fear that firms will begin extending their advisory services in order to attract and hire data scientists with data analytics skills (Brown-Liburd et al., 2015). Katz (2014) states that firms changing their attention from auditing to advisory services made regulators anxious about audit quality.

2. Data availability, relevance, and integrity

The other major challenge connected with data analytics consists of the availability of data, the ownership of data and the integrity of data.

Clients can also collect data. However, it is the question how much access auditors will gain and what sort of data the clients will approve to share with the auditors. Numerous clients face deficits in skills to extract data in a way that makes it utilisable by the auditor, either the data might include voluminous noise. Absence of access obstructs adjusting data mining for fraud detection to client data. Unfortunately, according to Gray and Debreceny (2014, p. 378), clients are not providing auditors admission to their databases. Even when they do, auditors often rotate off the engagement as a result of loss of the client or required rotation in the EU. Thus, is possible that the auditors are forced to give up access to the data of the clients. This makes it difficult to add information to firm databases or gather more understanding across clients in identical industries.

According to Alles and Gray (2014), Appelbaum (2014), Liddy (2014) and Whitehouse (2014), data is sometimes gladly provided and auditors are granted full access. However, even then auditors must take into account data integrity (used in the paper of Early, 2015).

Given that big data comes from internal sources and external sources, auditors must evaluate if the data comes from a safe source. According to Whitehouse (2014), the Public Company Accounting Oversight Board (PCAOB) is concerned about tampered information in its examinations of public company audits.

Another point of discussion concerning data analytics is completeness of the dataset. Alles and Gray (2014, p. 28) find that in fields other than auditing some ambiguity or deficit in the quality of data sets might be acceptable, although this is not true for auditing.

(20)

3. Expectations of regulators and financial statement users

One last major challenge involves the way investors and regulators view data analytics. Lately, auditing has been dealing with expectations of the auditor that are not consistent with what the standards demand from auditors. This gap in expectations shows up when auditors are assumed to provide absolute assurance of the accuracy of financial statements. In fact, auditors provide a reasonable level of assurance but never 100%. There is a chance that data analytics could aggravate this expectation gap, leading to users eventually wanting 100% assurance. Also, a firm’s board of directors and financial statement users can hold auditors to high norms of detecting fraud and discovering misstatements in the financial statement. For instance, Gray and Debreceny (2014, p. 378) conclude that, with traditional auditing, auditors can defend themselves if they do not find fraud because their sample is free of a ‘smoking gun’, which indicates fraud. By emphasizing the use of data analytics on non-financial information, regulators are concerned about the possibility that “auditors may be paying less attention to auditing their clients and more attention to providing them with non-audit services’’ (Katz, 2014, p. 2).

According to Whitehouse (2014), auditing standards do not take into account data analytics approaches to audits. Standard setters need to figure out how to adjust standards to incorporate brand-new ways. Such as, basing standards on auditors making decisions based on evidence may change to requiring testing 100% of the data set, or standards perhaps need to aim attention at testing the integrity of the data.

2.2 Financial statement audit

2.2.1 Types of auditors

Hayes et al. (2014) states that auditors come in two types: (1) the independent external auditors, and (2) the internal auditors.

Internal auditors (1) focus on the appraisal of internal controls and on operational audits rather than financial audits. It is also possible that they conduct compliance audits. Internal auditors report directly to the board of directors. These auditors are not independent as they are employees of the company they are examining (Hayes et al., 2014).

Independent external auditors (2) are responsible for reviewing the financial statements of publicly traded companies and non-public companies (Hayes et al., 2014) and are certified. Within this research, only external auditors will be discussed.

(21)

2.2.2 Type of audits

Hayes et al. (2014) states audits consist of three types: (1) audits of financial statements, (2) operational audits and (3) compliance audits.

Within this research, only financial statement audits will be discussed. Audits of financial statements (1) are focused on providing an accurate view of the financial statements. Therefore, in the Netherlands, auditors use the international financial reporting standards (IFRS).

The requirements for performing financial statement audits consist of conducting the audit with professional scepticism, being able to recognize when financial statements are materially misstated, utilising professional judgment when conducting the audit, and acquiring adequate and appropriate audit evidence.

There are two goals of the auditor when conducting financial statement audits. One objective is to provide reasonable assurance that the financial statements include no material misstatements, thereby providing an opinion of the say in which the financial statements were prepared. The second objective consists of reporting on the financial statements and

providing information by the International Standards on Auditing (ISA).

Operational audits (2) are focused on measuring the performance of an organization. Such an audit reviews the organization’s operating procedures to evaluate the effectiveness and efficiency of operations. Effectiveness concerns achievement of the goals and objectives of an organisation, while efficiency concerns the way resources are used to accomplish the organisation’s goals.

Compliance audits (3) examine the procedures of an organization to identify whether organizations follow specific procedures, rules and/or regulations. Compliance is measured against established criteria. This type of audit is focused on established laws and regulations or the policies and procedures of organisations.

2.3 Challenges of employing data analytics in financial statement audits 2.3.1 Other studies

The literature includes several studies outlining the challenges and opportunities of using data analytics in a specific field but not specifically using qualitative research to investigate the challenges and opportunities faced by the big four audit firms when implementing data analytics.

(22)

For example, Benjelloun et al. (2015) summarized the challenges, opportunities and benefits associated with big data in various sectors, for example the health sector, the commerce sector, and the tourism sector. However, Cao et al. (2015) did explain how the way big data can be adopt in other fields might be applied to the auditing field.

Yoon et al. (2015) uncovered some challenges in firms that use big data to decrease auditor reliance on client data and desire auditors to become more autonomous. Zhou et al. (2014) also states that it is hard to protect information from disclosure when implementing data analytics. PwC (2015) addressed the challenge of the absence of skills and expertise regarding data analytics in auditing and found that educating students makes auditors capable of performing data analytics effectively.

The research which most closely resembles the present research is the study by Alles (2013) and Titera (2013). They aimed attention at their study on including data analytics into financial statement audits and presented an agenda to recognize particular features of big data that could benefit auditors. The present research is focused on evaluating challenges of incorporating data analytics into the audit process.

2.3.2 Contribution

The purpose of this study is to enhance previous studies and take following actions based on the previously invented results and connections as well as recent knowledge gained from using data analytics in the auditing field. Hence, this research will aid auditors in the big four audit firms to overwhelm big data analytics challenges in the audit process and eventually benefit from the results.

(23)

3 Case description and procedures

This research consists of several phases using various research methods through

triangulation: for example, by employing multiple interviews, collecting data internally at Audit Firm X and consulting prior literature from different databases.

3.1 Literature review

The first phase involves the literature review of Chapter 2. The literature was reviewed concentrating on two topics: defining big data and data analytics, and the challenges of using data analytics in financial statement audits. Prior work on data analytics was studied, the challenges connected with data analytics were identified, and literature about the use of data analytics in audit firms was reviewed. The literature review was conducted through the accession of public data from the databases of the University of Amsterdam, Google Scholar and Audit Firm X.

3.2 Field studies

3.2.1 Type of interviews

In the second phase, field studies were conducted in the form of interviews using the general interview guide approach (Turner III, 2010, pp. 755-756). The term field study mentioned different methods to study, such as case studies to study for example the audit field (Malsch and Salterio, 2015, p. 1).

For the present research, conducting interviews was preferred for exploring the challenges pertaining to the rise of the use of data analytics when performing financial statement audits. The reason for choosing this type of qualitative method is to examine whether the employees of the big four audit firms are aware of the significant challenges presented by data analytics and not only its benefits. Another reason, according to Turner III (2010, p. 754), is the opportunity to grow and expand the researcher’s knowledge and experiences with qualitative design in order to better apply varied research models in future investigations. Turner also finds that interviews provide in-depth information pertaining to interviewees’ experiences and viewpoints on a particular topic (Turner III, 2010, p. 754). The interviews are coupled with other forms of data collection to provide the researcher with adequate information for analysis (Turner III, 2010, p. 754).

(24)

To conduct interviews, there are various forms of interview designs that can be developed. Gall et al. (2003) state that there are three formats for interview design (a) informal conversational interview, (b) general interview guide approach, and (c) standardized open-ended interview (Turner III, 2010, p. 754). First, an informal conversational interview means that the researcher asks semi-structured questions of the interviewee. Thus, the questions asked depend on the interaction of researcher with the interviewees. A general interview guide approach is more structured than the informal conversational but there is still a bit of flexibility. Finally, a standardized open-ended interview is extremely structured (Turner III, 2010, pp. 755-756). A general interview guide approach is preferred to discuss topics like how challenges associated with data analytics affect Audit Firm X and where it is important for the interviewees to express themselves and let them share their own experiences (Gendron and Spira, 2010, p. 279). Investigating this topic from a different perspective and using a different method, than which is described in prior literature, provides more in-depth and useful information.

The interviews consisted of open questions designed to answer the research question: ‘How do the categories of challenges associated with data analytics affect financial statement audits in terms of making these audits more or less efficient and effective?’ Appendix B shows when the interviews were conducted.

3.2.2 Type of roles

The author was the interviewer for all of the interviews. Ten interviews were conducted to collect a reasonable amount of information. The interviewees were contacted by email by the interviewer.

These interviewees were big data experts holding positions such as partner, director, manager, senior manager or senior associate who are in charge of different business units, including assurance, the national audit office and risk assurance. Relevant knowledge and involvement of the interviewees about using data analytics at Audit Firm X will be utilised to collect information.

It is crucial for the big data experts to be interviewed to be aware of the challenges presented by data analytics very well. They need to have an IT background and must specialise in big data, internal control or IT. The data consists of directly quotations with enough interpretable context.

(25)

3.2.3 Interview questions

The interviews included a series of questions addressing the challenges of the use of data analytics. The focus of the interviews lay on the experiences of the interviewee. Relevant interview questions that the big data experts were asked are presented in Appendix A. These questions were based on the research of Early (2015) into data analytics in auditing. The interviewees were asked to give examples of challenges associated with the use of data analytics in performing financial statement audits. The interviews were conducted in Dutch. English summaries of the interviews are included in Appendix B.

3.2.4 Interview setting

A room in which to take phone calls at Audit Firm X was selected as the location for conducting the interviews, since every interview will be conducted during work hours at this firm.

Audio recording was done using the author’s laptop. This method made recording more effective while conducting the interviews by telephone. Ultimately, the interviews were transcribed and analysed.

Each interview took approximately twenty minutes. The longest interview is approximately thirty minutes and the shortest interview is approximately fifteen minutes.

3.2.5 Theory

The theory used for this research is the grounded Theory. Grounded theory is inductive: you start with data and then build your theory around it. It also stresses qualitative rigor when conducting interviews. The research method is derived from the research question. Triangulation of researchers and interviewees is used. It was important to use a general interview guide approach. This approach allows interviewees to express themselves

according and let them share their own experiences (Gendron and Spira, 2010, p. 279). Gioia et al. (2013, p. 19) expressed shock at the willingness of interviewees to reveal private thoughts of the interviewees. This helps the interviewer to fully understand the interviewees’ experiences.

Grounded theory consists of four different phases; (1) the research design, (2) data collection, (3) data analysis, and (4) the grounded theory articulation (Gioia et al., 2013, p. 12).

In the first phase a specified research question is developed. By consulting existing literature, new insights are discovered.

(26)

In the second phase, the data is collected by, for example, conducting interviews. A great deal of flexibility is required, for instance, in backtracking to prior informants to ask questions that arise from subsequent interviews.

In the third phase, data is analysed by coding the data by first-order terms. The data will be coded (meaning labelling words or sentences) because the answers of the interviewees are important for providing an answer to the research question. In qualitative research, the researcher must prove that he or she has made the correct connections between the stories and answers. By coding the data, the researcher shows the process step-by-step. After coding this data by first-order terms, the data is organized by second-order themes. Categories or themes are created by bringing several codes together. The codes have to be constantly compared to decide if they are relevant. In this stage the data is conceptualized.

Finally, the second-order themes can be formulated into relationships in a data structure to construct a visual overview. New knowledge have been created. By comparing the results of the interviews you can decide if one category is more important than the other. The data structure is then transformed into a dynamic grounded theory model. The last step is explaining the connection with the literature. In the Findings section of this thesis, time will be taken to explain each evolving theme and focus on the new themes. New categories will be made and compared to existing categories of prior literature.

3.2.6 Result

Figure 1 shows, in steps, how the research question is answered. First, the categories of challenges associated with data analytics are made by using the grounded theory for conducting the interviews. The last step of the grounded theory is to make aggregate dimensions. The aggregate dimensions can be grouped into categories. After the categories have been created, the results from conducting the interviews are described as categories that have a more/less efficient or effective effect on the financial statement audits.

(27)

RQ: How do the categories of challenges associated with data analytics affect financial statement audits in terms of making these audits more or less efficient and effective?

Category 2

Category 1 Category 3

More/less efficiency More/less effectiveness

Categories of challenges associated with data analytics

Financial statement audits

N u m b er of c at egor ie s A d d ed val u e T yp e of au d it

(28)

4 Case analysis

This chapter presents the process of interviewing by applying grounded theory. As mentioned in Section 2.3.2, grounded theory consists of four different phases; (1) the research design, (2) data collection, (3) data analysis, and (4) the grounded theory articulation (Gioia et al., 2013, p. 12). This chapter ends with describing the relationship between the aggregate dimensions.

4.1 Grounded theory 1. The research design

The research question is: ‘How do the categories of challenges associated with data analytics affect financial statement audits in terms of making these audits more or less efficient and effective?’ A good in-depth research consist of open questions. To give an answer on every aspect of the research question, closed questions have also been asked. Probes were also used to get more clarity and details. The interview questions are provided in Appendix A and based on the challenges identified in prior literature, as mentioned in Section 2.1.5.

Before every interview started, the interviewee was informed about the purpose of the research, the interview questions, and the average time needed for the interviews. The

interviewees were also informed about the number of questions included in the interview. The data was processed by audio recording. The interviews were conducted by telephone and recorded on a laptop owned by Audit firm X and a telephone.

Next, the interview started with asking permission for recording the interviews. In addition, the research topic and research question were introduced.

2. Data collection

As mentioned in Section 3.2.1 data was collected by conducting interviews. Conducting the interviews consists of different stages. An interview is an intense experience. Credibility was established by asking questions that were relevant and meaningful for providing answers on this research topic. It is important to be alert to contradictions with what has been said earlier, decide how to phrase the next question, and pick up on hesitation, emotion and non-verbal signs. Additional tasks are keeping an eye on recording equipment and dealing with distractions that arise (Legard et al., 2003, p. 143). Notes have been taken on the tone of voice to provide insight into the meaning intended by the interviewee.

(29)

These notes were processed as part of the full transcriptions to improve the transparency of this research. The full transcriptions are not included in this thesis due to space limitations. See Appendix C for the basic transcription conventions used in the full recording

transcriptions to indicate the emotions of the interviewees, based on those used by Potter & Hepburn (2012). Appendix B presents English summaries of the interviews in short. The interviewees have been held anonymous to ensure their privacy.

The last question of the interview was announced at each interview. At the end of each interview the interviewees were asked if everything was clear and if they still had questions. Following the interviews, the interviewees were very interested in the progress of the research.

3. Data analysis

The next step consist of analysing data, and sorting and classifying the recorded data by coding the transcriptions. Coding involves moving from observation to interpretation. Data was analysed according to grounded theory, by creating categories and subcategories. As cited in Gioia (2012, p. 20) there are three stages of coding: (1) open coding (in first-order terms), (2) axial coding (in second-order themes) and (3) selective coding (in aggregate dimensions). Appendix D shows the open coding process step-by-step.

Open coding was done by line-by-line coding to stay focused on what has really happened. After finishing the open coding, the transcriptions are coded again to build deep connections across the data. In the first phase, first-order categories (sometimes a very large number of them) emerge. Appendix D shows the open coding process in first-order terms.

In the second phase, similarities and differences are sought among the many first-order categories. Similar categories are then given labels. The coder then considers whether the emerging themes suggest concepts that help to describe and explain what is being observed. Appendix D shows the axial coding process in second-order themes.

In the third phase, the coder investigates whether it is possible to distil the emergent second-order themes into further second-order aggregate dimensions. Now comes the moment to build a data structure. The data structure allows the coder to produce a visual display of the coding process. Appendix D presents the coding process for this study divided into first-order terms, second-order themes and aggregate dimensions. The colors stand for each aggregate dimension.

(30)

4. The grounded theory articulation

After coding the interviews, the research question has not yet been answered directly. Several aspects of the research question have been covered. The next step is to build a data structure. The data structure provides a visual display of the coding process. Appendix D includes the coding process divided into three stages: (1) first-order terms, (2) second-order themes and (3) aggregate dimensions.

4.1.1 Aggregate dimensions

The results consist of the aggregate dimensions as shown in Figure 2. These dimensions consist of (1) data quality, (2) exceptions, (3) technology, (4) analytical skills, (5) communication, (6) education, (7) guidelines, and (8) new challenges.

Data quality Analytical

skills Education

Exceptions Technology Communication Guidelines

A g g re g a te d im en si o n s New challenges

RQ: How do the categories of challenges associated with data analytics affect financial statement audits in terms of making these audits more or less efficient and effective?

Figure 2: Aggregate dimensions

As mentioned before, Figure 2 shows the third stage of coding. The research question is answered by combining these eight aggregate dimensions. These dimensions have an effect on the efficiency and effectiveness of the financial statement audits. The aggregate dimension ‘new challenges’ consist of the same topics as the other aggregate dimensions, except for the aggregate dimension ‘exceptions’.

(31)

First, data quality is specified by effectiveness, efficiency, relevancy and reliability. Effectiveness is about achieving a goal. Efficiency means obtaining a higher result with as little money or time as possible. Relevancy means that, for example, the data is essential to use. Reliable means, for example, that the data can be trusted.

Second, exceptions are specified with a better picture, exception identification and risk analysis. The definition of a better picture is obtaining a clear view. Identifying

exceptions is about determining an exception is a deviation. Risk analysis is about examining risks in parts for further study.

Third, technology is specified with data extraction, data sorting, program skills and tooling. Data extraction means retrieving relevant data from the database. After data

extraction, data is sorted. The data is layered in groups. Technology also includes the ability to work with different IT programs. Tooling is part of technology, because it includes the different tools and use of tools.

In addition, analytical skills are specified by being analytically strong, being able to make connections, using external sources and thinking in a certain way. Analytical skills means differentiating relevant aspects and parts of a problem. It is about gathering the information the auditor needs regarding the background and causes. The auditor makes connections between the data that collected and determines the relative importance of the elements. With analytical skills, the auditor can identify causes and think of adequate solutions. The auditor can use external sources to gather relevant information and to verify data.

Furthermore, communication consist of explaining data and interpreting data. Communication means responding to each other’s signals. Communication can be done by explaining data to the client and interpreting the data.

Next, education is divided into external and internal education. Education is about learning. Auditors can be educated internally and externally.

In addition, guidelines are specified with laws and regulations and type of guidelines. Guidelines are standards. Laws and regulations apply to every organisation. Type of

(32)

Finally, new challenges are specified in analytical skills, communication, education, data quality, guidelines and technology. Only the aggregate dimension of exceptions does not include new challenges. The second-order themes in these aggregate dimensions consist of existing themes, such as way of thinking, data extraction, effectiveness, reliability, education internal, laws and regulations and tooling. A new second-order theme has been made, namely job procedures, which is part of the aggregate dimension of technology. Job procedures is about the change in performing work procedures because of the innovation of technology. In addition, conditions is a new second-order theme, which is part of new challenges. Every client must have conditions, which can help the auditors to perform data extraction.

Part of the new challenges consists of important existing challenges. Important existing challenges are grouped under new challenges because the goal of this research is about discovering the new challenges and then deciding which challenges still exist and what are the effects of these challenges. These important existing challenges are discussed in the ‘Discussion and conclusions’ chapter after these new challenges are compared with

challenges of prior literature.

4.1.2 Categories

Figure 3 shows the categorisation between the aggregated dimensions. These aggregate dimensions are part of three self-made categories. Data quality, exceptions and technology are part of the data availability, innovation, and performance category. Analytical skills, communication and education are part of the training and expertise category. Guidelines are part of the category for guidelines of the financial statement users. As mentioned before, the new challenges, which resulted from the interviews, are part of the other six aggregate dimensions and are contained in all the three categories, as shown in Figure 3.

Finally, the aggregate dimensions will be explained as part of the categories. Appendix C explains the transcription conventions of the quoted text fragments.

(33)

Data quality Analytical

skills Education

Exceptions Technology Communication Guidelines

Training and expertise of auditors Data availability, innovation

and performance Guidelines of financial statement users A g g re g a te d im en si o n s C a te g o ri e s New challenges

RQ: How do the categories of challenges associated with data analytics affect financial statement audits in terms of making these audits more or less efficient and effective?

Figure 3: Categorising aggregated dimensions

1. Training and expertise of auditors

Analytical skills

Analytically strong

A skill that auditors need to work with data analytics consists of being analytically strong. For applying data analytics, Interviewee H brings people together who have experience with data analytics as well as the laws and regulations. That is a skill. When auditors apply data analysis to their control, it provides much more insights than they had before. Data analytics itself does have influence on the control, auditors have a sense of which direction they go in. The type of skills auditors need varies by type of organisation. Interviewee E examines retail organisations, e.g. supermarkets. He finds them more complicated to check, as he cannot review sales on the basis of management. Data analysis can help auditors in providing more insight to the clients.

(34)

‘I check retail organisations, supermarkets for example, yeah I find it a bit more complicated. I cannot check the sales on the basis of management. I think the type of organisation is very, very important. With Data analysis you can have many information together, but what does it

say? And can we also use it as control information?’ (Interviewee E)

Connections

Making connections between multiple objects is key. Interviewee I states, as an example, that ‘If you see a rise here, then you would expect to see a fall later on’. With all data available today, auditors can think of many more opportunities; they can therefore obtain more connections and can search for data for assurance regarding the transactions of the financial statements. Data analytics are used by auditors to give direction for gathering audit evidence. According to interviewee J, auditors can link many data sources and achieve ‘certainty on a whole different way. Interviewee G’s vision of how the control will ultimately look is that auditors will check all routine entries more through data analytics and will see links throughout the organisation.

‘With all data available today, you can think of many more opportunities, so you can get a lot more connection and can search for relationships for data in order to get assurance on the

transactions of the financial statements’ (Interviewee A)

External sources

If the auditor get information from the client and you have an external source that the auditor can connect, the auditor will be able see if it succeeds. Interviewee J finds that data extraction has to be outsourced to a specialized team. Interviewee F stated that these specialised data analysis teams are present. Data analyses can never stand on their own without external sources or other authentication information, but if the auditor has that, then data analysis can contribute to the quality of the control because the auditor can also focus specifically on areas of risk control. Data analyses can never stand on their own without performing verification with external sources or other information, but if the auditor has that, then data analysis can contribute to the quality of the control because auditors can focus specifically on areas of risk control.

(35)

‘Data analysis is a discussion that is still going on, which is that data analysis by itself does not equal control information, because you run data analysis on the basis of data of the

customer. And this data is the data that you should check. So data analysis can never stand on its own without checking it with external sources. When you checked it then data analysis can contribute to the quality of the control, because we also specifically on areas of can zoom

in risk control.’ (Interviewee B)

Way of thinking

The auditor should be able to ask questions regarding whether the data is complete and correct. He or she should be able to ask critical questions in data analysis. In addition, the auditor should have a proficient understanding of how the client has adjusted the processes and financial flows so that the auditor can start thinking of how to audit efficiently. The auditor must be able to think in the bigger picture; for example, he or she should be able to understand the results’ consequences for the rest of the practice. In particular, auditors are trained in the framework of thinking in processes and procedures and validating them. If the auditor examines big data in the future and the way that he or she is going to check in other ways and look for connections with data sources, the auditor might obtain different audit evidence. To work properly with big data, auditors must have a different mindset and the skills required to understand the data. If auditors need to understand the processes of an organisation, data analytics can help them to achieve a deeper understanding of the organisation, numbers and the data files.

'We are especially trained in thinking in processes and procedures, validating them and much less in the sense of data analysis and that sort of thing.'

(Interviewee D)

Conclusion

Possessing analytical skills means that auditors can think in a certain way to determine relations and provide insight. These auditors are continuously critical. By possessing these skills, they can perform their work more effectively, because they have a sense of which direction to go. Another way of making the audit more effective is by bringing people

together who have experience with data analytics on different levels, such as when the auditor observes something and uses an external source to connect it to see if it succeeds.

(36)

Being analytically strong, being able to make connections, thinking in a certain way and using external sources will not lead to more efficiency, because the auditors will obtain more insight by making connections between multiple objects. Auditors can link many data sources and achieve certainty on a whole different level.

Communication Explaining data

According to Interviewee A, it is important that auditors be able to explain ideas well to their clients. The auditor should be adept at continuously communicating with people, because not everyone has the same skills. In other words, communication skills are also relevant. It helps to explain what auditors do with the data and that the auditors then, on the basis of the new privacy legislation, the General Data Protection Regulation (GDPR) that came into force recently, also take the correct legal action. For example, auditors should process exit agreements and consult with the internal department within the audit firm regarding their tasks. Next, the auditor should explain to the client the importance of hard requirements in the control and how the client can follow up with data analytics.

Interpreting data

Clients struggle with interpreting data. They do not know the meaning of certain data. Auditors should consider whether data analysis really makes sense and whether they can run it.

‘And what I also find relevant, is that you have to be analytical, but you also must be able to explain well to your customer. So what you see is that customers face difficulty, because they

do not know what the data mean.’ (Interviewee D)

Conclusion

Explaining data to the clients will lead to more effectiveness, because an auditor constantly must communicate with departments to take the next best actions; otherwise, goals cannot be reached. This holds especially for the new privacy regulation, the GDPR. Interpreting data leads to more effectiveness. Clients cannot interpret data well, so auditors need to help them with it.

Data analytics : challenges in financial statement audits