An empirical analysis of the differences in audit activities associated with Aura implementation

Is Aura Equal?

An empirical analysis of the differences in audit activities associated with Aura implementation

F.Winius

Master Thesis Technology Management

University of Groningen

Faculty of Economics and Business

1 

Is Aura Equal?

An empirical analysis of the differences in audit activities associated with Aura implementation

Master Thesis Technology Management

PricewaterhouseCoopers Accountants N.V.

Address Zuiderzeelaan 53

P/o box 513 8017JV Zwolle Company supervisor ir. H.R. Selhorst RC

Email rudi.selhorst@nl.pwc.com

University of Groningen

Faculty Business & Economics

Address Netelbosje 2

9747AE Groningen Supervisor dr. ir. H. van de Water Second assessor drs. F.P. Bakker

Author Name F. Winius Address Wederik 69 8446AA Heerenveen Student # 1542028 Email frankwinius@hotmail.com Confidentiality

Please note that this report is confidential, and is not available to any other party than PwC and the University of Groningen. Accordingly, the University of Groningen may not provide copies of this report or make this report available for internal purposes and/or to any Third Party other than those involved in the assessment and/or quality inspection of the thesis. PwC will not accept any

2 

Summary

Enterprise Risk Management systems are applied to be able to identify, assess and react to risks facing the organization. These internal control systems are used by financial auditors as well to provide assurance over the extent to which financial information complies to rules and regulations. Recently PwC implemented Aura, as a replacement for the MyClient system to guide and structure the financial audit. PwC expects Aura to facilitate a more effective audit, while maintaining audit comprehensiveness. Although MyClient and Aura initially approach auditing from a different perspective, in the end they should yield equal results since the regulatory and professional requirements and regulations remained unchanged.

The assumption that the audit activities promoted by Aura are equal to those promoted by MyClient, is examined in this study, by comparing the activities present in the two systems to a generic audit model. The use of a benchmark for comparison contributes to the comprehensiveness of the study, as all elements deemed relevant by scientific literature are assessed. Two types of differences were identified; (1) General differences, which tend to be present throughout the audit and (2) Specific differences on elements of the audit.

In general, despite the presence of extensive guidance in Aura, the rationale behind, “why”, the use of specific activities and the resulting conclusions is scarcely documented. Interpretations of specific results, what these mean in the context of the audit, are at some points rarely given. These findings have a number of important implications;

Without proper consideration of appropriateness of actions and assessment of validity of results, less assurance over financial information is provided. Furthermore, audits are still susceptible to SALY mentality, the assumption that all aspects are Same As Last Year. This assumption may be valid in some cases, but for sufficient assurance it is essential to assess the underlying considerations of previous audits.

Considering the “why” is essential in understanding of the client’s risks and applied controls, this is required for solid judgment. Professional judgment and skepticism is commonly exercised in financial audits at clients, this study suggests it would be helpful to apply the same concepts to PwC’s own approaches.

3 

Abbreviations, Acronyms and Definitions

A&C Acceptance and Continuance (PwC procedure)

AFM Dutch Authority for Financial Markets

AICPA American Institute of Certified Public Accountants CobiT Control Objectives for Information Related Technology

COSO Committee of Sponsoring Organizations of the Treadway Commission

EGA Evidence Gathering Activity

ERM Enterprise Risk Management

GAAS Generally Accepted Auditing Standards

IAASB International Audit and Assurance Standards Board

IFAC International Federation of Accountants

ISA International Standard on Auditing

IT Information Technology

NIVRA Dutch Institute for Certified Public Accountants

4 

Table of contents

Summary ... 2

Abbreviations, Acronyms and Definitions ... 3

Table of contents ... 4

Table of figures & tables ... 6

1. Introduction ... 7

1.1 Background ... 7

1.2 Problem definition ... 8

1.2.1 Origin of the problem ... 8

1.2.2 Research Objective ... 9

1.2.3 Problem owner analysis ... 9

1.2.4 General conceptual model ... 9

1.2.6 Research questions ... 11

1.3 Product ... 12

1.4 Limiting Conditions ... 12

1.5 Research validity ... 12

1.6 Research overview ... 13

2. The Audit Process ... 14

2.1 Introduction ... 14

2.2 Professional Judgment & Skepticism ... 15

2.3 Core model of the audit approach ... 15

2.3.1 Phase I – Client acceptance and retention ... 15

2.3.2 Phase II – Knowledge acquisition of current condition ... 16

2.3.3 Phase III – Planning test of financial statements assertions ... 18

2.3.4 Phase IV – Test of financial statements assertions ... 19

5 

2.3.6 Phase VI – Audit reporting ... 19

2.4 Audit process overview ... 20

3 Audit activities at PwC ... 21

3.1 Introduction ... 21

3.2 Differences in MyClient and Aura audit activities ... 21

3.2.1 Phase I – Client acceptance and retention ... 22

3.2.2 Phase II – Knowledge acquisition of current conditions ... 22

3.2.3 Phase III – Planning test of financial statements assertions ... 25

3.2.4 Phase IV – Tests of financial statements assertions ... 25

3.2.5 Phase V – Completion of audit ... 26

3.2.6 Phase VI – Audit reporting ... 26

3.3 Main findings ... 27

3.3.1 General findings ... 27

3.3.2 Specific findings ... 27

4 Consequences of identified differences in audit activities ... 28

4.1 Introduction ... 28

4.2 Impact of general differences ... 28

4.3

Impact of specific differences ... 29

4.3.1 Omission assessment of enterprise risk appetite and risk tolerance ... 29

4.3.2 Omission assessment of event identification processes. ... 29

4.3.3 Omission of differentiation between risk and opportunity ... 29

4.3.4 Omission of consideration of risk responses other than reduction ... 29

4.3.5 No consideration of portfolio of risk response ... 29

4.3.6 No reconsideration of applicability of controls for specific risks ... 30

4.3.7 Omission assessment strategic management controls ... 30

Conclusion ... 31

Suggestion for further research ... 32

6 

Appendix ... 35

A1: COSO ERM Internal Control – Integrated Framework ... 35

A2: Risk ... 36

A3: Materiality ... 37

A4: Legal context and professional requirements ... 39

Dutch legal context ... 39

Professional requirements regarding the audit process ... 39

A5: Research elements ... 41

A6: Research Findings overview ... 44

Table of figures & tables

Fig 2: Conceptual model of the field of research ... 10

Fig 3: Research overview ... 13

Fig 4: Model of the audit process (Knechel et al., 2007) ... 15

Fig 5: Audit evidence supporting reasonable conclusions (Gray and Manson, 2008) ... 18

Fig 6: Overview of the audit process (Knechel et al., 2007) ... 20

Fig A 1: COSO ERM internal control - integrated framework (COSO, 2004) ... 35

Fig A 2: Process of applying materiality (Elder et al., 2010) ... 38

Table A 1: Engagement evaluation factors ... 41

Table A 2: COSO ERM Components ... 41

Table A 3: Steps for evaluating internal control ... 42

Table A 4: Preliminary analytical procedures ... 42

Table A 5: Quantitative factors in materiality ... 42

Table A 6: Qualitative factors in materiality... 43

Table A 7: Completion steps ... 43

7 

1. Introduction

1.1 Background

The lingering financial crisis raises the question to numerous stakeholders whether and how boards and senior executives are keeping track of their organizations’ most significant risk exposures. According to Beasly et al. (2010) most organizations lack sufficient infrastructures to identify, assess, manage and monitor emerging risks threatening stakeholder value. In many cases failures were attributed to overconfidence of management about ad hoc strategies to risk management. In reaction to these pressures towards more structured and sophisticated approaches to risk management, a holistic Enterprise Risk Management (ERM) – integrated framework was developed by the Committee of Sponsoring Organizations of the Treadway Commission (COSO), supported by the American Accounting Association, the American Institute of Certified Public Accountants, Financial Executives International, the Institute of Management Accountants and the Institute of Internal Auditors. They define ERM as:

“Enterprise risk management is a process, effected by an entity’s board of directors, management and other personnel, applied in strategy setting and across the enterprise, designed to identify potential events that may affect the entity, and manage risk to be within its risk appetite, to provide reasonable assurance regarding the achievement of entity objectives.” (COSO, 2004).

COSO recognizes that appropriate ERM systems are likely to vary between firms. Therefore they suggest a contingency perspective towards the design of ERM systems (COSO, 2004). This intuitive assumption that there is no universally ideal ERM system also has been commonly suggested

elsewhere in literature (The Financial Reporting Council, 2005; Beasley et al., 2005; Moeller, 2007). After the design and implementation of an internal control system, organizations are required to monitor the effectiveness of the system lest it become due to the dynamic changes in the environment and operations in the organization (Tsay, 2010).

Financial auditors are using the internal control systems as well to provide comfort during their audit activities. More and more internal control is part of the IT systems. Assurance over the reliability of information systems and processes is provided by PwC Systems and Process Assurance, through combined knowledge about IT systems and internal control.

Financial auditing at PwC is a COSO based auditing process which is guided and structured by the recently implemented IT tool “Aura” which replaced the “MyClient” system. Although both systems are COSO based, according to PwC they promote different ways of executing audits. The new system is expected to perform a more efficient and effective audit, but should at the same time guarantee audit comprehensiveness and reduce the occurrence of the SALY mentality. This is the tendency to assume that everything is Same As Last Year. An assumption that, according to Ramos (2009), leads to reduced audit quality.

8  The use of methodology in research is essential. Lack of structured methods will lead to ineffective research with low probability on reliable and relevant results (De Leeuw, 2005). In case of this research structure and guidance is provided by employing an appropriate business research

methodology by De Leeuw - Bedrijfskundige methodologie: management van onderzoek (2005). The use of this methodology has lead to the research approach elucidated in the following subsections.

1.2 Problem definition

1.2.1 Origin of the problem

This subsection aims to clarify the area subject to research. In Fig 1: Origin of the problem. The interdependencies between goals, systems and the following processes are visualized, which lead to the problem area.

Systems are applied to support the enterprise in the achievement of organizational goals. Although based on equal goals, these systems can be realized in different ways. The elucidation of the differences in processes that follow from these realizations (see Fig 1: Δ between process 1 and process 2), provide insights on differences in the types of activities employed to achieve

organizational goals. In case of the process facilitated by MyClient and Aura, the specific

organizational goal is the acquisition of the required level of comfort, i.e. certainty about the extent to which the financial information provided complies with the criteria established by authorities. The contents of these criteria are discussed in appendix A4: Legal context and professional requirements. Differences in types of activities may indicate changes in effectiveness of the achievement of

organizational goals.

Organizational Goals

Goal of the system

Model of the desired system

Realization Model 1

Realization Model 2

Process 1

Δ

9  . PwC strives to achieve the required level of comfort by a structured audit which is guided to a certain degree by an IT system. This system should guide the audit process towards performing all activities necessary to achieve the required comfort, in the most effective way, i.e. compliant to all applicable rules and regulations, without redundant activities or activities which contain unnecessary work. This research focuses on clarifying the differences in audit activities resulting from the systems, despite an unchanged framework of requirements.

1.2.2 Research Objective

The Aura system was implemented to provide guidance to conduct comprehensive PwC audits. The demand for replacement of the MyClient system initially ensued from technical matters. Next to coping with the technical demand, Aura was designed to increase effectiveness of the audit process by increasing audit guidance, thus supporting decision making. As both systems are based on the same philosophy, auditing principles, and both are bound by the same laws and professional regulations, PwC assumes that they should lead to the conduct of equal audit activities, since an equal level of comfort is required.

This assumption has however not been confirmed by research. PwC has intuitive indications that there might be significant differences in the type of activities performed. One of these indications is the increase in man-hours since introduction of Aura, despite the extensive training of staff

members. Although part of the increase can still be attributed to learning, the magnitude of the rise indicates the presence of other factors.

The primary objective of the study is therefore to provide empirical evidence on the accuracy of the

assumption that the activities resulting from use of the recently implemented Aura auditing system are equal to the activities encouraged by the MyClient auditing system.

1.2.3 Problem owner analysis

The notion of problem owner is often confused with that of a stakeholder. The problem owner in this research is defined as the individual who, based on personal assumptions and beliefs, perceives to

have a problem.

In contrast to the directly involved problem owner, a stakeholder, according to De Leeuw (2002), is anyone with an interest (of any kind) in the operation of the organization.

The problem owners in this research are the partners of PwC. Despite the fact that the audits are executed for a large part by their team, they are fully responsible and can be held personally accountable for the (deficiency in) comfort provided as well as the cost involved. Therefore their interest lies in gaining explicit knowledge of the differences in audit activities encouraged between the recently implemented Aura audit system and its predecessor. Even though most people involved with Aura intuitively believe it has been an improvement of some kind, a total overview is lacking. The Findings might (partially) confirm PwC’s intentions towards overall improvement of audits, but also may disclose for example the existence of excessive processes and/or activities. Therefore clear insight may yield opportunities to improve business performance.

1.2.4 General conceptual model

10  The model shows an overview of the financial audit environment. Legal and professional

requirements have lead to standardized auditing frameworks. The PwC audit approach is designed to comply with these regulations. For the purpose of conducting consistent and comprehensive audits in an efficient and effective manner, IT tools are used in the guidance of the audits. This tool used to be the MyClient system, but recently the Aura system has been implemented as a replacement with the same purpose.

As both systems are based on - and guided by the same theoretical frameworks, PwC assumes that similar audit cases should lead to equal audit activities, to achieve equal extent of comfort for PwC, i.e. degree of certainty of correct conclusions. This research will focus on the existence and content of the delta, in audit activities. If a delta is present, the related consequences will be interpreted. Audit practices are heavily regulated, the supervising authorities are included in the conceptual model to illustrate the importance of compliance to rules and regulations. If audit judgment is based on insufficient evidence (obtained through absence of specific activities), chances are great this will be noticed. As noted in 1.2.3., consequences may be immense for the specific partner responsible and the firm. The methods of working of the supervising authorities fall outside the scope of this research, for PwC intends to comply to all rules and regulations. Also those which may have a low risk of detection.

The research questions addressing the separate sections of the conceptual model are stated at the right side of the figure, these are elaborated further in

1.2.6 Research questions.

Recognized auditing frameworks Legal Requirements

Aura MyClient

Audit Process Outcome Aura NIVRA AFM Supervision PwC audit model Legislator PwC board Professional Requirements R e se a rc h S u b Q u e st io n 1 R e se a rc h S u b Q u e st io n s 2 & 3

Audit Process Outcome MyClient Δ

ISA NV COS

11  Standards on auditing are set by law and the regulating authorities to guarantee independency and quality of audits. The PwC audit methodology is designed to comply with all legal and professional requirements, as the International Standards on Auditing (ISA) and NV COS (PwC, 2010). The

compliance to these rules and regulations in the Netherlands is assessed by the AFM. Furthermore the PwC audit approach is closely aligned with the most widely recognized and accepted auditing frameworks (PwC, 2010).

1.2.6 Research questions

The implementation of Aura should increase the effectiveness of the audit, nevertheless all audits should still comply to all professional rules and regulations. In this respect Aura and MyClient should be equal. This leads to the primary research question:

Do MyClient and Aura promote equal audit activities?

This question is examined through discussion of sub-questions; first the basic principles and regulations which guide the internal auditing processes are described in a framework. The use of a framework with generic typologies will ensure a firm basis for independent and comprehensive comparison.

I. Which generic audit framework describes the PwC financial audit approach?

Sub question one yields a core model of the PwC audit approach. The model is used in sub question two to assess the similarities and deviations in process outcomes of the Aura end MyClient auditing systems.

II. What differences in in the type of audit activities performed can be identified between the MyClient and Aura systems?

Research sub question two is assessed through comparison of the activities promoted by MyClient and Aura in practice. The type of Audit activities guided by Aura respectively MyClient, executed in relatively stable companies are compared, based on the activities defined in the separate phases of PwC audit model which is developed at research sub question one. In the cases selected, the specific audits were executed for internal research as if the clients were new to PwC, i.e. without use of information which was gathered in preceding years. For this purpose, the audits were executed by new teams based at other offices, with no access to previous documentation.

. Sub question three assesses and interprets the consequences of the relevant differences identified at sub question two.

III. To what extent do the identified significant differences have an impact on the overall auditing process, with regard to the basic model found at I?

12 

1.3 Product

Research guided by this methodology should yield a reliable, relevant and comprehensive answer to the primary research question to achieve the research objective. In this specific case, the research provides empirical evidence on (1) a core model of the PwC audit approach, (2) clear insights in the key differences in audit activities performed between the Aura auditing system and MyClient auditing system and (3) judgment on to what extent these differences have an impact on the achievement of goals

1.4 Limiting Conditions

Research limitations constitute the boundaries of the research. These preconditions indicate the requirements the research has to comply to. De Leeuw (2005) defines two types of preconditions. (1) Research process preconditions, these define the timelines and other organizational boundaries of the research process, (2) research result preconditions define the boundaries of the solution, the research product. Furthermore, De Leeuw (2005) defines two types of limitations to the research; research feasibility limitations and research methodology limitations. Those applicable are noted below.

The study is confined to a research period of three months. It is required to be scientifically based, for academic purposes, as well as practically relevant, to be of use to PwC. Research results will be presented in a master’s thesis. The subject of research requires the thesis to be confidential.

The research is conducted in the Dutch context, results may therefore not be applicable to countries with regulatory and professional differences.

1.5 Research validity

A measurement instrument is considered valid when it measures what it is intended to measure. The concepts employed in the research inherently are a simplification of reality. It is therefore important to indicate to what degree the results of this research are valid and may be extended.

Audit activities resulting from the use of respectively Aura and MyClient are compared using cases from practice, based on the elements identified in the generic audit framework.

Comparison is however only partly dependent on the cases used. The type of activities promoted is mostly independent of the specific case. Comfort over financial statements needs to be acquired in generic risk areas, which are present in most organizations. Comfort in generic areas should be acquired by equal activities in Aura and MyClient.

13 

1.6 Research overview

An overview of the interrelations of the research sub-questions in the research structure is presented in Fig 3: Research overview. Research sub-question one, which yields a core model of the audit approach is discussed in chapter two. Research sub-question two clarifies the differences in process outcomes between MyClient and Aura in chapter three. An overview of these differences and their implications, research sub-question three, is given in chapter four.

Rsq I: Which generic audit framework describes

the PwC financial audit approach?

Rsq II: What differences in the type of audit

activities performed can be identified between the MyClient and Aura systems?

Rsq III: To what extent do the identified

significant differences have an impact on the overall auditing process, with regard to the

basic model found at I?

Research Approach

Core model of the PwC audit approach

Overview and implications of relevant differences Research results Core model – Aura Core model – MyClient

Differences in audit activities: MyClient vs Aura

Chapter 2

Chapter 3

Chapter 4

14 

2. The Audit Process

The aim of this chapter is to answer research sub-question one. For that purpose, first the context of the PwC audits is described, then an overview of the professional and regulatory requirements that guide audit processes is given and finally the audit process is described in a five phase model.

I. Which generic audit framework describes the PwC financial audit approach?

2.1 Introduction

Enterprises in today’s global business environment depend heavily on automated processing of information. Directors, management, employees, customers and all other stakeholders make critical decisions based on the information they receive. To be able to make informed decisions, the information used should be objective, relevant, reliable and understandable (Knechel et al., 2007). Information can be for example accidentally incorrect, incomplete or manipulated intentionally for the benefit of others. Therefore the need arises for assurance over the quality of information. Perfect information is not always necessary however. Information should be as reliable as it needs to

be for the purpose for which it is used (ISACA, 2009). For example a car’s speedometer does not need to be as highly accurate as life saving equipment used by medical professionals in hospitals.

According to ISACA (2009) this distinction is important for understanding that the importance and need for internal control are dependent on the intended use of information and the impact associated with the lack of integrity and loss of accuracy of the information.

The internal control system of an enterprise consists of a set policies and procedures, called controls, designed to provide the management with reasonable assurance that the company achieves its objectives and goals. The concept of reasonable assurance allows for only a remote likelihood that the internal control system will not prevent or detect relevant misstatements. All stakeholders depend for their decision making on the existence and functioning of these controls. Assurance over the correct functioning of these controls forms the foundation of the financial audits PwC performs. Financial audits should be conducted in such a way that reasonable assurance over the completeness and correctness of information is provided, in auditing this is called comfort (PwC, 2010). To achieve a satisfactory level of comfort, the complex audit process needs to be executed in a systematic

manner. Although a general methodology can be applied to execute audits, the fact that every client is unique will require the audit process to be tailored to fit enterprise specific circumstances (Knechel et al., 2007; Elder et al., 2010; PwC, 2010). Despite the resulting variation in audit processes due to tailoring, the quality of the audits is not allowed to vary. Poorly executed audits may result in serious information risks, which jeopardize the quality of decision making (Knechel et al., 2007; Elder et al., 2010). Therefore, given the public nature of the audit report and the substantial amount of

15 

2.2 Professional Judgment & Skepticism

Often the fact is that accounting information need more interpretation based on professional insights. In those cases professional judgment is used in the forming of conclusions about the validity of figures appearing in the financial statements.

In auditing various procedures may be employed for the same purposes. The selection of methods appropriate, given the specific circumstances, is considered a judgmental matter as stated in ISA 520:

Various methods may be used in performing audit procedures. These range from simple comparisons to complex analyses using advanced statistical techniques. Analytical procedures may be applied to consolidated financial statements, financial statements of components and individual elements of financial information. The auditor’s choice of audit procedures, methods and level of application is a matter of professional judgment.

The professional judgment in PwC audits is based on thorough understanding of the client, professional skepticism, knowledge and experience (PwC, 2010). Professional skepticism implies a questioning state of mind and critical assessment of audit evidence in all phases of the engagement. Hence, dishonesty should not be assumed, but at least considered in the process (Elder et al., 2010).

2.3 Core model of the audit approach

Knechel et al. (2007) devised a five phase model (Fig 4: Model of the audit process) to describe the financial audit process. In the following subsections subsequent phases and their core activities are outlined and used as a framework to analyze the PwC audit model in chapter 3.

Client Acceptance and Retention Knowledge Aquisition of Current Conditions Planning Test of Financial Statements Assertions

Test of Financial Statements Assertions Completion of Audit Audit Reporting I II III IV V VI

Fig 4: Model of the audit process (Knechel et al., 2007)

2.3.1 Phase I – Client acceptance and retention

16  These should provide reasonable assurance that PwC will not be associated with a client lacking integrity, professional services can be delivered by the audit firm, engagement risks are properly considered (see A2: Risk) and both parties fully understand the services to be provided by the Auditor.

The engagement acceptance procedures fall into four broad categories: (1) Acquire background information on client, (2) evaluate engagement risk factors, (3) decide on acceptability and (4) mutually agree on services to be provided.

According to Knechel et al. (2007), several factors need to be considered in the evaluation of an engagement, these can be found in appendix Table A 1: Engagement evaluation factors.

2.3.2 Phase II – Knowledge acquisition of current condition

Subsequent to accepting the engagement, the risks facing the organization are assessed and

evidence is acquired about the current state of the client. Audit evidence provides the auditors with to arguments to base their opinions on. The concept of evidence is defined in the International Standards on Auditing (ISA 500) as:

“Audit evidence” is all the information used by the auditor in arriving at the conclusions on which the audit opinion is based, and includes the information contained in the accounting records underlying the financial statements and other information. Auditors are not expected to address all information that may exist. Audit evidence, which is cumulative in nature, includes audit evidence obtained from audit procedures performed during the course of the audit and may include audit evidence obtained from other sources such as previous audits and a firm’s quality control procedures for client

acceptance and continuance.

Three categories of activities take place in phase II:

Understand Client Business Environment and Strategic Analysis

Evaluate Control Responses to Risk and Internal Control over Financial Reporting Preliminary Analytical Procedures

Based on the findings resulting from these activities the level of required detail of the audit is

determined. In auditing the concept used for this purpose is materiality. The concept of materiality is related to the significance of amounts in financial assertions, only material assertions are required to be stated in financial statements.

Understand Client Business Environment and Strategic Analysis

This step is an elaboration on the investigation conducted in the client acceptance phase. It promotes full understanding of the client, which is essential for virtually all auditing and financial reporting problems that are encountered during audits originate from the unique situation and operations of the enterprise (COSO, 2004; Knechel et al., 2007). In this section of phase II the specific risks facing the enterprise can be identified at the strategic level, the process level and those remaining after mitigating procedures, known as residual risks.

17  evaluated in an audit engagement (see appendix Table A 2: COSO ERM Components ). These can be found at all enterprise levels (COSO, 2004):

Evaluate Control Responses to Risk and Internal Control over Reporting

Internal controls over financial reporting are established to prevent and detect misstatements. They server three categories of objectives (Knechel et al, 2007): (1) improve effectiveness of management decision making and business processes, (2) increase reliability of information and (3) assure compliance with applicable rules, regulations and obligations. Evaluation takes place to determine the effectiveness of the controls and identify potential weaknesses. Prior to these evaluations it is therefore important that auditors understand the applied controls in an enterprise. Knechel et al., (2007) define six successive steps for evaluating internal control systems, these are summarized in appendix Table A 3: Steps for evaluating internal control.

Preliminary Analytical Procedures

Analytical procedures involve the evaluations of financial information by quantitative comparison of the recorded versus the expected amounts. These include the investigations of identified relevant fluctuations and other inconsistent relationships in the provided information (ISA 520; Gray and Manson, 2008). Knechel et al. (2007) distinguish six basic approaches for performing analytical procedures, which are outlined in appendix Table A 4: Preliminary analytical procedures. Materiality

Based on the considerations and findings resulting from the other activities in the knowledge acquisition phase, the level of materiality is determined. Unfortunately a universal definition of materiality has not been accepted (Messier et al., 2005). The definition of materiality pursued by the AFM in the Netherland is defined by the International Accounting Standards Board (IASB) in the International Standard on Auditing (ISA320) as:

Information is material if its omission or misstatement could influence the economic decisions of users taken on the basis of the financial statements. Materiality depends on the size of the item or error

judged in the particular circumstances of its omission or misstatement.

The smaller the level of misstatement that is considered to be important (i.e. the higher the

estimated risk), the more precise the auditor needs to check. The level of materiality has therefore a direct impact on the amount of effort required in the entire audit process.

Within the concept of materiality next to quantitative factors also qualitative factors are examined. The specific context is considered highly important in the judgment of misstatements in financial information (Gray and Manson, 2008; Knechel et al., 2007).

Appendix A3 provides further elaboration on the concept and process of determination of

materiality. The elements of materiality used for comparison in this research are outlined in appendix Table A 5: Quantitative factors in materiality and in appendix Table A 6: Qualitative factors in

18 

2.3.3 Phase III – Planning test of financial statements assertions

Based on the risk analysis in phase II, the auditor decides in phase III which evidence is required to obtain sufficient support of opinions about the fairness of financial statements in accordance with professional and regulatory standards.

To be able to come to conclusions there needs to be sufficient audit evidence of appropriate quality. The quality of the evidence depends on the relevance and the reliability of the information (see Fig 5: Audit evidence supporting reasonable conclusions (Gray and Manson, 2008)).

Resonable conclusions based on evidence Sufficient evidence (quantity) Appropriate evidence (quality) Relevant evidence (pertinent to the assertion to

be proven)

Reliable evidence (Trustworthy and

persuasive)

Fig 5: Audit evidence supporting reasonable conclusions (Gray and Manson, 2008)

A typical audit involves different types of tests, but depending on the circumstances, certain types may be emphasized for specific risks. The auditor’s choice of evidence mix is based on several factors including; availability of specific information, relative cost of the type of test, effectiveness of the test in the specific context and inherent risks of the tests (Elder et al., 2010).

Controls Testing is performed to gather evidence when the auditor’s assessment of the risk of material misstatement includes the statement that the internal controls were operating effectively at relevant times during the period under audit (ISA 330; Gray and Manson, 2008)

Substantive Testing is performed in order to be responsive to the risk of material misstatement,

because the auditor’s assessment of risk is subject to professional judgment and may not be

sufficiently precise to identify all risks of material misstatement. The tests include details of classes of transactions, account balances and disclosures (Gray and Manson, 2008).

Test of details focus on the statements on the income and balance sheet, these include confirmation of all relevant and significant amounts.

19 

2.3.4 Phase IV – Test of financial statements assertions

The planning of activities to obtain the required amount of comfort in phase III is executed in this phase. This involves the gathering and interpretation of evidence in all areas deemed important. The compilation of specific evidence is specified in audit policies and procedures (audit guidance), to ensure evidence reliability and compliance to rules and regulations.

2.3.5 Phase V – Completion of audit

In this phase all collected evidence is compared to the expected evidence and related disclosures as defined in 2.5.3. An overall opinion on the appropriateness of accounting policies and quality of information is formed. At this stage the auditor should have gained sufficient understanding of the enterprise to be able to explain all significant results and relationships reported. The steps in this phase are summarized in appendix Table A 7: Completion steps.

2.3.6 Phase VI – Audit reporting

20 

2.4 Audit process overview

This chapter provided the basis for comparison of the two financial audit systems by outlining an audit framework which describes the PwC audit approach. In the following chapter the application of the elements comprising this framework are compared for the MyClient and Aura system.

Client Acceptance and Retention

Understand Client Business Environment and Strategic Analysis

Evaluate Control Responses to Risk

And Internal Control over

Financial Reporting

Preliminary Analytical Procedures

Planning Test of Financial Statements Assertions

Sales and customer service

Supply chain and

production Resource management

Knowledge Aquisition of Current Conditions

Tests of Financial Statements Assertions

Completion of Audit Audit Reporting

I

II

III

IV

V

VI

21 

3 Audit activities at PwC

3.1 Introduction

The differences in audit activities between MyClient and Aura are identified in this chapter by separate assessment of the six audit phases that jointly constitute the generic financial audit model found in the previous chapter. These phases comprise of different elements, which are outlined and explained in appendix A5: Research elements. Two cases were selected for comparison. These concern the audit at the same company in consecutive years. Besides the fact that these were documented in respectively MyClient and Aura, for other internal research purposes these audits were executed by different audit teams. Furthermore the new team had for the same reason no (access to) prior knowledge. The company itself proved to be extremely stable during these two years, this was also a requirement for the other research.

Differences found in this chapter indicate that the audit systems do not operate unambiguously at these specific areas and that these differences may occur in other audits as well. Findings may give rise to the need for further research, or changes in the system.

To this end, the consequences of identified relevant differences are discussed in chapter 4.

Rsq II: What differences in the type of audit activities performed can be identified between the MyClient and Aura systems?

3.2 Differences in MyClient and Aura audit activities

In this section the significant differences between MyClient and Aura are outlined for each phase of the audit process, additional remarks are placed when the element under consideration was not discussed in either of the systems. The complete overview of all considered elements of each phase can be found in appendix A5: Research elements, the accompanying results in appendix A6: Research findings overview.

The initial approach to auditing differs fundamentally between MyClient and Aura. They follow respectively a process approach and a risk oriented approach. In MyClient key business processes are identified, and assessed on risk and mitigating controls separately. Assessment of the financial statements is based on an industry specific standardized framework.

In Aura, based on the understanding gained in phase II, risks are identified for all financial statement line items (FSLI’s). Transactions of the same type are grouped together on these FSLI’s. These are affected by the same main risks. As costs of the same type are grouped, the classification in FSLI’s often corresponds to the classification in processes in MyClient.

All identified risks need to be assessed. For this purpose evidence needs to be gathered by specific activities (EGA’s: Evidence Gathering Activities). Even though the terminology differs between MyClient and Aura, from this point on the two systems follow the same steps, based on PwC’s audit methodology which, as stated earlier, has not changed.

22  orders with the related invoice and receiving report on quantities, price per unit, etc. Operational effectiveness of these controls are tested in EGA’s; if they do not sufficiently cover related risks, additional comfort is gained by substantive analytics and tests of details. Whether the comfort gained is sufficient or not is decided through professional judgment (described in section 2.2).

Both MyClient and Aura offer guidelines for the appropriate execution of specific activities to guarantee quality, comprehensiveness and reliability of the actions performed. Remarkably, despite the fact that in Aura the guidance is more extensive, the reporting on the executed actions in all phases tends to be less elaborate. The reasons why activities are performed and the interpretation of the results (what does it mean for the audit and the enterprise?) are only occasionally and briefly documented by the responsible auditors. “Telling the story of the audit”, why actions were

performed, is however a very important element in the assessment by regulating authorities. In the following subsections the specific findings for every phase are outlined.

3.2.1 Phase I – Client acceptance and retention

In the first phase, based on considerations on engagement risk and opportunities, the decision is reached whether or not to accept or retain a client. Overall the two systems capture the considered elements equally. Remarks should be made however for two audit elements: (1) Assessment of the reputation of the client and (2) Assessment of the profitability of the engagement.

Reputation

In the consideration of client reputation, both systems refer to the completed Acceptance and Continuance (A&C) procedure for that specific client. Although the A&C process assesses among others the element of client reputation to determine the engagement risk, specific findings and considerations are not documented in either of the files. Members on this audit team, whom were not involved in the A&C process will therefore not have direct access to potential findings. Although obviously relevant findings will be shared, this depends on the evaluation of the assigned team member. Other team members may find certain findings more relevant for their activities than initially was determined by the assigned team member.

Profitability

An audit budget is prepared in MyClient and Aura, in both cases as an attached Excel file. In Aura only a budget for activities concerning testing of controls has been documented, which does not provide sufficient insight in the overall profitability of the engagement. The MyClient file does cover all engagement activities.

3.2.2 Phase II – Knowledge acquisition of current conditions

23  Understand Client Business Environment & Strategic Analysis

Although many activities aimed at understanding the client are conducted in either one or both MyClient and Aura, it is worth mentioning that several elements, which are considered relevant by scientific literature, are not present in either of the systems. These remarks are also explained in the upcoming subsections.

Internal environment

All elements required in this section are present in the extensive list of points for consideration located in the guidance section of Aura. Despite this list, only the standard required risk “Management attitude towards override of controls” is assessed in this engagement. Although guidance is not as explicit and extensive in MyClient, this section was covered more extensive. Other relevant considerations related to business ethics are part of the A&C procedure and analogous to the remark in the previous phase not present in both systems.

Objective setting

Both the MyClient and Aura systems apply the Business Analysis Framework (BAF) for clarification of the strategy of the enterprise. The BAF applied in the Aura audit was transferred directly from the MyClient audit, therefore there are no differences. Moreover, the specific template applied was intended for analysis of government institutions. Use of inappropriate templates may lead to evaluation of wrong and or redundant information, also in this case.

The applied framework, but also the general framework does an adequate job in outlining strategy. Both however fail to address risk appetite, which reflects the amount of risk an enterprise is willing to accept. Addressing the risk appetite of the organization is strongly recommended in scientific literature and therefore also included in the generic audit model.

Event identification and Risk assessment

Both MyClient and Aura do not assess the event identification process at the strategic level. This process enables management to identify possible incidents and occurrences which might affect the enterprise. At the strategic level, both systems do however mention some potential events implicitly in the Business Analysis Framework in Market Overview.

MyClient as well as Aura offers specific guidance on evaluation of the potential events and risks emanating from effects of the recent credit crunch on the entity. Although the focus areas have probably been considered during the audit, exclusion of some of those areas, even those which could be relevant, was not substantiated. Identified problem areas however were assessed adequately.

24  Risk response

Management response to risks other than reduce (avoid, accept or share) by controls, is not explicitly considered in either of the two systems. The response, thus the amount, location and type of

controls, chosen by management should match the risk appetite of the enterprise and result from cost-benefit analysis.

Whether or not the enterprise has considered other responses to the perceived risks, preceding implementation of the existing controls (implicit choice to reduce risk), is also not analyzed. In line with these findings there is no enterprise wide portfolio assessment of risk management, i.e. assessment of combination of risk responses compared to overall risk appetite.

Risks that remain after application of mitigating procedures – controls -, the residual risks are discussed in both systems after assessment of the specific control.

Information & Communication

Reliability of information is assessed in both systems with regard to the information used by controls and agreement and reconciliation of financial statements with underlying accounting records. Thus assurance is gained over the information used to provide comfort.

The information used throughout the organization in order to realize business goals by identifying, assessing and responding to risk, which poses a risk in itself, is however not audited consistently in either of the systems.

Monitoring

Both systems lead to confirmation of the existence of ongoing monitoring systems, which should enable the enterprise to react timely to changing conditions. Evaluation and interpretation of these systems is in contrast to MyClient, not present in the Aura audit. This is also the case for the

consideration of reliability of information used in the monitoring process and the controls on corrective action upon identified deficiencies.

Evaluate control responses to risks and internal control over financial reporting The controls identified during the preceding steps are evaluated at this step of the audit to

understand their operation and whether or not they sufficiently mitigate the organizational risks they are dedicated to.

Understanding and assessment of internal control over financial reporting

In line with the findings in the preceding subsections, both systems seldom address (the omission of) strategic management controls, but do gain thorough understanding of the functioning of the applied controls at the process level and their specific strengths and weaknesses. Correct functioning of the controls during the period under audit, is however not verified by hard evidence. In both cases this is assumed based on professional judgment considerations.

Furthermore both audit systems almost exclusively focus on controls addressing reliability of information (for assurance over financial reporting) and controls over compliance objectives.

25  Preliminary analytical procedures

Guidance in both MyClient and Aura encourage comparable analytical procedures in this phase, which are aimed at identification of unusual or unexpected relationships that may indicate

misstatements in financial information. In the specific document in MyClient this guidance was not followed. Some measures however did appear in other documents as the BAF. Only a selection of analytics was performed in Aura. Although the selection was probably based on professional judgment decisions, the rationale behind the choice for specific performed activities was not documented.

Following the previous subsection, and despite explicit guidance in both MyClient and Aura, analysis of disproportionate fluctuations, which could indicate misstatements, was not documented in either of the systems.

Materiality

Elaboration on the rationale behind the considerations on the calculation of materiality is strongly enforced in Aura. Although this enforcement is not present in MyClient, in both systems equal activities regarding the determination of materiality are documented.

3.2.3 Phase III – Planning test of financial statements assertions

Based on the understanding of the client gained in the previous two phases, preliminary conclusions are reached about internal control over financial reporting and the financial statements assertions made available. In this phase of the audit, a plan is prepared to gather the evidence required to support and confirm these conclusions.

In MyClient risks and related controls are assessed on individual basis. Controls mitigating multiple risks are often assessed all over again for every separate risk in MyClient. In Aura this approach was deliberately changed to improve efficiency. Identified controls are tested separately on design and existence (and essential controls also on functioning), and subsequently assigned to (multiple) related risks. Gathering of additional evidence is planned when the controls do not provide sufficient comfort. This process is displayed visually in Aura, which makes planning easier to link risks to controls. This clarifies which risks are mitigated and/or assessed. The centralized approach in Aura however implies that controls are not assessed on appropriateness for every separate risk. In

subsequent audits, it is essential for the auditor to keep in mind that the risks facing the organization may have changed which requires reconsideration of choice of controls and required additional evidence, to achieve a sufficient level of comfort.

3.2.4 Phase IV – Tests of financial statements assertions

Following the audit plan, in this phase the required evidence to support opinions about internal control over financial reporting and the financial statements assertions is gathered and evaluated. In both MyClient and Aura guidance is provided to assist in the conduct of the audit. These

procedures are not only more extensive in Aura, but also followed more closely in this system. In MyClient, the audit activities tend to be less methodologically structured and less standardized. This however does not imply that MyClient provides weaker arguments. Undefined borders often lead to more elaborate interpretations of findings. Surprisingly, increased guidance seems to have a negative effect on the comprehensiveness of the interpretation of results and the consideration of the

26  Both MyClient and Aura suffer from SALY due to reuse of templates. For certain audit activities predefined templates are used for structuring and facilitating the audit. Auditors may assume an unchanged situation and reuse last year’s results, also when this is not appropriate.

3.2.5 Phase V – Completion of audit

The evidence collected in phase four is compared to the expected evidence and the related disclosures. The findings in this phase reflect all judgments with regard to effectiveness of internal controls and the risks of material misstatement.

All significant completion activities are performed in both MyClient and Aura. The main difference between the two systems in this phase of the audit is that MyClient is far less readable, because of a fragmented multi level document structure (i.e. documents within documents). Instead of attaching work papers directly, these are stored in the central document library and can be referred to when applicable.

In the final analytical review the status and performance of the enterprise are measured based on all evidence gathered. Financial and non-financial performance measures should confirm consistency between the financial statements and the understanding of the enterprise. In both My MyClient and Aura the completion of analytical procedures is confirmed, however only high level conclusions are documented. The underlying considerations, which lead to the conclusions are not stated.

3.2.6 Phase VI – Audit reporting

27 

3.3 Main findings

In this chapter the generic financial audit model found in chapter two was applied to identify differences in the process of auditing between MyClient and Aura. Sometimes these differences are subtle, in other cases obvious, but viewed in coherence the main differences are summarized below from the Aura perspective. A distinction is made between (1) general findings; these tend to be present throughout the entire audit and (2) specific findings; which are remarks on explicit audit elements. The interpretation, what each of these findings means for the audit process, is given in chapter 4.

Rsq II: What differences in the type of audit activities performed can be identified between the MyClient and Aura systems?

3.3.1 General findings

Despite more extensive guidance in Aura, documentation of activities and findings tends to be less elaborate. This holds specifically for the remarks below.

1. Considerations which lead to conclusions are not always documented (“Why this conclusion?”)

2. Considerations on choice of methods/methodology/frameworks not documented. (“Why specifically this one? Why not this one?”)

3. Considerations on purpose of specific methods/models/frameworks not documented consistently. (“Why used in this way?”)

For the sake of clarity it is important to notice that Aura does support explanations, these are however not enforced by the system at those specific areas.

3.3.2 Specific findings

1. Omission assessment of enterprise risk appetite and risk tolerance 2. Omission assessment of strategic event identification processes. 3. Omission of differentiation between risk and opportunity 4. No consideration of risk responses other than reduction 5. No consideration of portfolio of risk response

6. No reconsideration of applicability of controls for specific risks 7. Omission assessment strategic management controls

28 

4 Consequences of identified differences in audit activities

4.1 Introduction

The importance and impact of the differences identified in the previous chapter are described in the upcoming sections. For the interpretation it is important to be aware of the fact that PwC has stated that they need to offer clients more than high quality financial audits and assurance over the past. Added value needs to be created by assisting them to get in control and providing them with the insights they need to improve their business performance in the future.

Rsq III:

process, with regard to the basic model found at I?

4.2 Impact of general differences

Despite more extensive guidance in Aura, documentation of activities and findings

is less elaborate.

The question “Why do we do what we do?” is an important starting point in auditing. The rationale underlying actions determines the value of the results. To be able to identify and acquire

comprehensive evidence on underlying problems affecting enterprises, it is essential to understand the rationale behind actions (Gray and Manson, 2008). They list several recent cases which dramatically confirm this necessity; in these cases critical signs were overlooked. The considerable damage to client and audit firm was attributed to the auditors whom had failed to gather comprehensive evidence.

Thorough documentation of actions establishes a solid basis for quality and comprehensiveness. It facilitates assessment of this year’s audit, but the audits in successive years as well. The importance is clarified further below for the remarks made in 3.3.1.

It is in Aura not always clear how certain conclusions were reached. Besides the fact that the AFM disapproves, it is also complicates internal quality checks. Furthermore absence of considerations hampers the use of previous activities in upcoming audits. To achieve efficiency benefits, it is more helpful to understand the reasons why decisions have been made, than just the conclusions in the end.

In cases where the origin of conclusions lies in evidence gathered by use of specific methods and frameworks, considerations on the appropriateness, validity and downsides of the applied

methodology are hardly documented. Assessment might however provide indications that the model or framework is not sufficient (anymore) for the current conditions due to changed circumstances. In line with the findings noted in the previous subsection, it is without documentation harder to identify and assess inappropriate application.

The cases researched show examples of reused models, applied in exactly the same way as previous audits. In some situations this leads to inefficiency due to superfluous activities. But more

29 

4.3 Impact of specific differences

4.3.1 Omission assessment of enterprise risk appetite and risk tolerance

Risk appetite is the amount of risk an enterprise is willing to take in return for growth, returns and increased stakeholder value. This should thus be in line with the enterprise’s overall strategy. Based on risk appetite considerations, risk tolerances are determined and mechanisms developed to manage the risks corresponding to the enacted risk appetite and strategy. Undefined risk appetite therefore leads to difficulties in risk identification and assessment (when is a risk significant or acceptable?) and evaluation of risk response (when is a control appropriate and adequate?).

4.3.2 Omission assessment of event identification processes.

It is uncertain whether or when events occur, and when they do how large their impact exactly will be. But merely expecting the unexpected, and deploying activities to identify emanating risks will enable the enterprise to respond both to opportunities and treats.

This processes used by the enterprise to identify, assess and respond to risk is should be assessed on design and operational effectiveness. Furthermore the information on which these systems are dependant should be assessed on reliability according to COSO (2004).

4.3.3 Omission of differentiation between risk and opportunity

Events, if occurring, may affect the enterprise in a positive, a negative or in both ways. Events that have a negative effect on the enterprise, risks, should be assessed and responded to when necessary (determined by the risk appetite). Events positively affecting the enterprise represent opportunities or compensation for risks. Whereas identified opportunities should be considered in strategy and objective setting, COSO (2004) suggests those events compensating risks should be considered in the assessment of risks and responses.

4.3.4 Omission of consideration of risk responses other than reduction

Often the only risk response considered is reduction through the use of controls (mitigation). Other risk responses as avoiding, accepting or sharing could be equally suitable or better at the same or lower operational cost. Furthermore COSO (2004) suggests that innovative responses may result in competitive advantage for the enterprise and should therefore be considered. As an example they mention the investments made by an insurance firm in enhancing road safety, which reduced incident claims.

4.3.5 No consideration of portfolio of risk response

30 

4.3.6 No reconsideration of applicability of controls for specific risks

Every identified specific control may be used for mitigating multiple risks in Aura. Previous audits at the same client often are used as basis for subsequent audits. Assessment whether or not controls are still appropriate, and risks have remained unchanged is not documented. A wrongful SALY assumption might lead to increased residual risk, because the assigned control(s) mitigate the specific risk to a lesser degree. Risk also may have reduced. If unnoticed, this may lead to redundant controls testing, hence a surplus of comfort.

4.3.7 Omission assessment strategic management controls

In both MyClient and Aura, much attention is paid to the design and operational effectiveness of process controls. Strategic level management controls are assessed to a far lesser degree. These include for example controls ensuring effectiveness of decision making. Knechel et al. (2007) emphasize the significance of management controls by stressing that process controls might not operate effective when management controls are not in place. Put the other way around, an extensive audit of management controls might render the audit of certain process controls redundant.

Furthermore, according to Knechel et al. (2007) proper management controls assist the enterprise to stay in control, by:

Mitigating strategic level risk

Promoting decision making effectiveness, thus enhancing efficiency of business activities Setting strategic boundaries, consequently determining risk appetite

Establishing lines of accountability

Implementing and executing enterprise risk management frameworks

31 

Conclusion

By replacing MyClient with Aura for the guidance and structuring of financial audits, PwC expects to be able to perform comprehensive as well as more efficient and effective audits. Aura should also reduce the occurrence of the SALY mentality, which leads to reduced audit quality. Although both systems are COSO based and need to comply with the same rules and regulations, they promote a different approach to financial audits. To determine whether both systems ultimately yield the same activities, as assumed by PwC, both are compared to a generic audit model which describes the PwC financial audit approach.

The overview of the audit process is very conveniently organized in Aura. Especially compared to MyClient, the links between financial statement line items, related risks and mitigating controls are obvious. Furthermore, the straightforward user interface has proved to enable new users to get familiar with the PwC audit process very easy. Although user friendliness is important, this study is confined to the differences in audit processes that ensue from them.

It has been shown that in Aura the rationale behind the activities conducted is not always

documented, despite more elaborate audit guidance. The reasons why conclusions are reached, and why specific methods and frameworks are used is often not clear. For the audit process this has a number of important implications;

First of all, audit comfort weakens when actions are not properly considered on appropriateness and validity. Furthermore, audits are still susceptible to the Same As Last Year mentality. This assumption may be valid in some cases, but to achieve a sufficient level of comfort, it is essential to evaluate considerations of previous audits. Furthermore, considering the “why” improves understanding of the client and its threats and opportunities. This will enable PwC’s auditors to deliver significant added value, by assisting the client to stay in control and improve client business performance. There is no universally ideal enterprise risk management system, enterprises should therefore take a contingency perspective towards the design of appropriate ERM systems (COSO, 2004; Beasly et al., 2005; Financial Reporting Council, 2005). As this research has shown, auditing these systems will therefore require a contingency perspective as well. Aura is equipped to handle such an approach, however not by providing elaborate guidance on every action, but rather by facilitating the process of applying the concept of professional skepticism and common sense.

32 

Suggestion for further research

33 

References

Autoriteit Financiële Markten (AFM), 2010. Aandachtspunten financiële verslaggeving 2010. Available online at:

<http://www.afm.nl/layouts/afm/default.aspx~/media/files/rapport/2010/rapport-aandachtspunten-financiele-verslaggeving-2010.ashx>

American Institute of Certified Public Accountants (AICPA), 2005. Letter of comment on the IAASB’s

Exposure Draft, Proposed International Statement on Auditing (ISA) 320: Materiality in the

Identification and Evaluation of misstatements.

Anderson, E.W., Fornell, C., Rust, R.T., 1997. Customer Satisfaction, Productivity, and Profitability:

Differences between Goods and Services. Marketing Science 16 (2).129-145.

Beasley, M.S., Clune, R., Hermanson, D.R., 2005. Enterprise risk management: an empirical analysis

of factors associated with the extent of implementation. Journal of Accounting and Public

Policy 24 (6), 521–531.

Beasley, M.S., Pagach, D., Warr, R., 2008. Information conveyed in hiring announcements of senior

executives overseeing enterprise-wide risk management processes. Journal of Accounting,

Auditing and Finance 23 (3), 311–332.

Bharadwaj, N., Walker Naylor, R., Hofstede ter, F., 2009. Consumer response to and choice of

customized versus standardized systems. International Journal of Research in Marketing

26(3). 216-227.

Committee of Sponsoring Organizations of the Treadway Commission (COSO), 2004. Enterprise Risk

Management-Integrated Framework. Published online at:

<http://www.coso.org/Publications>.

Elder, R.J., Beasly, M.S., Arens, A.A., 2010. Auditing and Assurance Services – an integrated approach, 13th . Person Education, Inc. New Jersey.

Financial Reporting Council (FRC), 2005. Internal Control: Guidance for Directors on the Combined

Code. Available online at: <http://www.frc.org.uk/corporate/internalcontrol.cfm>.

Gray, I, Manson, S., 2008. The Audit Process, principles, practice and cases, 4th edition. Thomson Learning, UK.

ISACA, 2009. CobiT and application controls – A management guide. ISACA, Rolling Meadows, IL. IT Governance Institute, 2005. Control objectives, management guidelines, maturity models. In:

CobiT 4.0. Rolling Meadows, IL

Knechel, R.W., Salterio, S.E., Ballou, B., 2007. Auditing: Assurance and Risk 3rd . Thompson South -Western, Mason, OH.

Leeuw de, A.C.J., 2993. Besturen van veranderingsprocessen, fundamenteel en praktijkgericht

management van organisatieveranderingen, 1e druk. Kon. Van Gorcum, Assen.

Leeuw de, A.C.J., 2002. Bedrijfskundig management, primair proces, strategie en organisatie, 2e druk . Kon. Van Gorcum, Assen.

Leeuw de, A.C.J., 2005. Bedrijfskundige methodologie: management van onderzoek, 6e druk. Kon. Van Gorcum, Assen.

Lightle, S.S., Castellano, J.F., Cutting, B.T., 2007. Assessing the control environment. Internal Auditor 64. 51-56.

Messier, W.F., Martinov-Bennie, N., Eilifsen, A., 2005. A review and integration of empirical research

on materiality: two decades later. Auditing: A Journal of Practice and Theory 24 (2). 153–188.

Moeller, R.R., 2007. COSO Enterprise Risk Management: Understanding the New Integrated ERM

Framework. John Wiley & Sons, Inc., Hoboken, New Jersey.