Capturing the power of data: using business analytics to manage organisational performance

(1)

Capturing the power of data: using

business analytics to manage

organisational performance

By

Bernd Jan Helms

s2591421

In partial fulfilment of the requirements for the degree of

MSc Business Administration: Organisational & Management Control

University of Groningen, Department of Economics and Business

June 24

th

_{, 2019}

Supervisor: Dr. A. Bellisario

(2)

1

Abstract

Since an understanding of how business analytics (BA) is used to manage performance is currently insufficient, this study examined how an organisation uses BA to manage its performance. Companies are struggling to create value from BA, making this a practical issue as well. Three theoretical propositions were tested by means of conducting a positivist case study, in which participants engaging in different levels of performance management were selected for interviews and observations. In total, 17 interviews and 3 group meeting observations have been conducted. The results indicate that data collection is not always directly related to the organisation’s strategy and can also be linked to short-term goals and smaller-sized KPIs. Analysts must bridge IT with decision-making and communication of insights from BA must be performed in an understandable way, either by tailoring communication to the receiving audience or by using a standardized method of communication. Trustworthiness of BA is considered important and several suggestions are developed on how to ensure high levels of trust. Future research is advised to perform longitudinal case studies in different industries, considering the use of business analytics in various departments.

(3)

2

1. Introduction

The environment in which organisations operate is becoming highly competitive, increasing the difficulty of sustaining competitive advantages (Schläfke, Silvi, & Möller, 2013). The amount of data that is generated by organisations nowadays is exploding, with companies capturing trillions of bytes of information about their operations, suppliers and customers (Manyika et al., 2011). Such amounts of data are now part of every sector of the global economy and much of modern economic activity, growth and innovation would not be possible without it. Leveraging data with the use of mathematics, statistics and econometrics supports fast and sophisticated decision-making (Schläfke et al., 2013), which allows enterprises to deal with competitive pressures by realizing higher productivity and returns (Brown, Chui, & Manyika, 2011). However, an understanding of how business analytics (BA) is used to manage performance is lacking (Visani, 2017) and a survey among more than 3,000 executives shows that the biggest challenge, when creating value from analytics, is a lack of knowledge about how to use analytics to improve the business (LaValle, Lesser, Shockley, Hopkins, & Kruschwitz, 2011). Therefore, studying the use of business analytics in the field of performance management (PM) is highly relevant and is the topic of this research.

Extant research about bussiness analytics in the field of performance management has identified the conditions to be met when aiming to realize advantages from BA. Decision-makers must know how to create value from data and have enough talent to derive insights from it (Manyika et al., 2011; McAfee & Brynjolfsson, 2012). Furthermore, organisations need their analysts to bridge BA with decision-making by being able to ‘speak the language of the business’ (Fosso Wamba, Akter, Edwards, Chopin, & Gnanzou, 2015; Raffoni, Visani, Bartolini, & Silvi, 2018). There must also be a direct link between BA and the organisation’s strategy, as BA should aim to answer strategically important questions (Raffoni et al., 2018) and measure causal relationships between strategic variables (Klatt, Schläfke, & Möller, 2011). High-quality data is paramount to enable comparisons between data sets and to gain economically valuable insights (Bose, 2009; Fosso Wamba et al., 2015). To facilitate control and decision-making processes, feedback loops must be present from various levels of the organisation, in which the relationships between performance drivers are revised regularly (Bititci, Carrie, & McDevitt, 1997; Schläfke et al., 2013).

(5)

4

Considering this lack of knowledge, the following research question was formulated: How is

business analytics used to manage organisational performance?

(6)

5

2. Theoretical background

This section first elaborates on the concepts of interest to this study and their definitions. This is followed by an explanation of how performance can improve when BA is used effectively. Finally, the literature on how to use BA for PM is discussed, illustrating the conditions to be met.

Arguably the most popular description of business analytics is provided by Chen, Chiang, and Storey (2012), who describe BA as the technologies, techniques, practices, systems and applications that analyse important business data to assist organisations to better understand their business and make timely decisions.

The process of performance management includes performance measurement, which is defined as

“the process of quantifying the efficiency and effectiveness of action” (Neely, Andy, Gregory, & Platts,

1995, p. 80). Neely (1998) describes that performance measurement provides information for decision-making and actions to be taken, since it quantifies the effectiveness of past actions through acquisition, sorting, analysing, interpreting and disseminating appropriate data. Performance management involves comparing such measures of outputs with their target values and taking corrective action when differences are found (Radnor & Barnes, 2007). It encompasses the process of assessing the differences between desired and actual outcomes, identifying the differences that are critical, understanding why the deficiencies have taken place and introducing corrective actions (Melnyk, Bititci, Platts, Tobias, & Andersen, 2014). The rationale for selecting this definition of PM is that this study is interested in how BA is used to manage overall organisational performance. Unlike other definitions of PM, such as those that describe managing the performance of individuals and teams (Aguinis, 2009), this definition encompasses the complete range of business performance aspects.

To move towards the discussion of how to use information from BA in the process of PM, it is useful have a picture of the how performance can be enhanced if BA is leveraged effectively. This emphasizes the relevance of the phenomenon and supports the understanding of how to determine whether companies are effectively using BA to support their PM.

2.1 Business analytics and effective performance management

(7)

6

Besides BA’s ability to enhance decision-making effectiveness, it can also improve strategy formulation and implementation (Mello, Leite, & Martins, 2014; Warren, Moffitt, & Byrnes, 2015) and test the impact of strategies (Schläfke et al., 2013). According to Visani (2017), new strategic patterns can be discovered by understanding relationships among performance variables. Simons (1995) provides a comprehensive explanation of how BA can support the exchanges of ideas and information between different departments as well as hierarchical levels, which assists the communication of strategies (Ittner & Larcker, 2005).

Furthermore, research identifies that BA can help to distinguish causation from correlation (Brown et al., 2011; George, Haas, & Pentland, 2014; Klatt et al., 2011). Schläfke et al. (2013) describe that BA can identify key success factors by revealing causal relationships, recognizing true profit drivers. According to Visani (2017) this can enhance firm performance by highlighting the relationships between strategic performance variables, measuring the impact of each variable and forecasting risks associated with specific actions. Strategic performance variables are key success factors, which must successfully be implemented or achieved to realize strategic goals (Simons, 1995). These can be features, elements or competencies (Ferreira & Otley, 2009) and examples are efficiency, market share, time to market, output quality and customer satisfaction.

From these studies it becomes clear that BA can have several positive effects on the quality and effectiveness of PM. The next step is to map out what is described in the literature about how BA is used to manage performance.

2.2 How to use business analytics for performance management

The discussion of how companies create value from BA starts with providing an overview of what is known about how successful implementers of BA differ from organisations that have difficulty with it. Firstly, decision-makers must understand how to leverage data (McAfee & Brynjolfsson, 2012). Manyika et al. (2011) suggest that leaders must understand BA’s value and have enough talent to derive useful insights from it. However, having a senior management team that knows how to make data-driven decisions is not sufficient. According to Raffoni et al. (2018), organisations need people that bridge IT with decision-making, referred to as ‘translators’, that communicate the outcomes of analyses to decision-makers. These analysts must be able to ‘speak the language of the business’ (Fosso Wamba et al., 2015), as insights emerge through engagement between business managers, analysts and employees from different departments (Sharma, Mithas, & Kankanhalli, 2014). The data has to be communicated in a way that allows decision-makers to understand its meaning and encourages refinement and exploration of the data (Evans, 2012; Miller & Mork, 2013).

(8)

7

(2005) highlight that organisations should invest in selecting measures that are leading indicators of performance and Nudurupati, Tebboune, and Hardman (2016) add that appropriate performance targets must be set. Schläfke et al. (2013) highlight the importance of feedback loops from various levels of the organisation, in which the relevance of performance drivers and their interrelationships are revised regularly. These feedback loops facilitate the decision-making and control processes (Bititci et al., 1997). Organisational culture also plays an essential role, as BA must be understandable and trustworthy to all employees (Fosso Wamba et al., 2015; Mello et al., 2014). A final element to mention is a need for high-quality data. If the data is inappropriate or of low-quality, decision-making will not be supported even when the most sophisticated analytical systems are used (Bose, 2009). High-quality data it is often described as data that is useful and generates economically worthy insights about costs and benefits (Fosso Wamba et al., 2015; Roden et al., 2017). Data should not be duplicated, conflicted, inaccurate or redundant (Beath, Becerra-Fernandez, Ross, & Short, 2012; Fosso Wamba et al., 2015).

These studies provide a first indication of the conditions that organizations must meet if they want to improve their PM practices by using BA. Nevertheless, empirical evidence is missing on how BA can be used to manage organisational performance (Raffoni et al., 2018) and research on how to make effective use of BA is underdeveloped (Nudurupati et al., 2011). Papers that concern the process of how organisations use BA to support their PM practices are remarkably limited. A few studies provide initial guidance on this, but profound investigations of how BA can be used to support PM purposes are still lacking (Visani, 2017).

(9)

8

cycle. From a diagnostic control perspective, BPA can make additional critical performance visible, which can be used to report on performance. From an interactive control perspective, information from BPA is debated in regular face-to-face meetings and used to make forecasts to periodically verify underlying assumptions. Managers must agree with decisions made in each step, in order to warrant trust in the process and outcomes. Sharma et al. (2014) emphasize the importance of making ‘good’ decisions, which means that the decisions can achieve their objectives and are acceptable for those responsible for its implementation. This framework provides a first step towards an understanding of how BA is used to manage organisational performance.

Figure 1: BPA framework, reprinted from “Business performance analytics: exploring the potential for

performance management systems”, by Raffoni et al., 2018, Production Planning & Control, 29(1), p. 62.

(10)

9

the CGMA (2014) illustrates that 86% of organisations struggle to get valuable insights from data. This problem can be assigned to the fact that many companies do not sufficiently know how to effectively use BA. The literature is missing an understanding of how business analytics is used to manage performance (Mello et al., 2014; Schläfke et al., 2013; Sheng, Amankwah-Amoah, & Wang, 2017), which is of both theoretical and practical concern.

These inferences resulted in the formulation of the following research question: How is business

analytics used to manage organisational performance?

This issue calls for further empirical investigation, since advances in the theoretical understanding of this phenomenon have not kept up with its increase in relevance and impact.

2.3 Research framework

To summarize what is known about how to use BA for PM, insights are combined from the articles that have been discussed. Table 1 presents key concepts and their associated sources.

Category Concept Source

Process of using BA for PM

Business strategy and performance model assessment Raffoni et al. (2018) Determination of the questions that BA should answer

Data collection Data analysis

Communication of insights from data Performance management

Conditions for using BA for PM

Ensure that the data is of high quality Bose (2009) and Fosso Wamba et al. (2015) Let data collection be guided by strategically important

questions

Raffoni et al. (2018) Search for causal relationships between strategic variables Klatt et al. (2011) and

Schläfke et al. (2013) Communicate the data in a way that supports exploration and

refinement and allows decision-makers to grasp its meaning

Evans (2012) and Miller & Mork (2013)

Ensure that ‘translators’ bridge IT with decision-making, by ‘speaking the language of the business’

Fosso Wamba et al. (2015) and Raffoni et al. (2018) Ensure that suggested actions can reach their objectives and

acceptable to those that are responsible for their implementation

Sharma et al. (2014) Schedule regular face-to-face feedback meetings to debate,

challenge, revise and learn from information that is generated by BA, to promote organisational learning

Raffoni et al. (2018) and Schläfke et al. (2013) Contextual

conditions

Create a culture in which BA is understandable and trustworthy, to promote its acceptance and use

Fosso Wamba et al. (2015) and Mello et al. (2014) Ensure that all actors involved understand BA’s value and know

how to derive insights from it

Manyika et al. (2011) Feedback

loops

Schedule regular feedback loops between the different steps of the process of using BA for PM, to revise relationships between performance drivers

Bititci et al. (1997) and Schläfke et al. (2013)

(11)

10

(12)

11

Figure 2. Preliminary research framework: how to use business analytics to manage performance

Senior management Senior management Data analysts Data analysts Data analysts Managers Managers Strategy assessment Determination of questions that BA should answer

•Communicate the data in a way that supports exploration and refinement and allows decision-makers to grasp its meaning

•Ensure that 'translators' bridge IT with decision-making, by ‘speaking the language of the business’

Data collection

•Ensure that the data is of high quality •Let the data collection be guided by the

determined questions

Data analysis

•Search for causal relationships between strategic variables Communication of insights from BA Performance management

•Schedule regular face-to-face feedback meetings to debate, challenge, revise and learn from information that is generated by BA, to promote organisational learning

•Ensure that suggested actions are capable of reaching their objectives and acceptable to those that are responsible for their implementation

Contextual conditions:

• Create a culture in which BA is understandable and trustworthy, to promote its acceptance and use

• Ensure that all actors involved understand BA's value and know how to derive insights from it

Actors

Process

As

(13)

12

This framework summarizes key insights from the literature on the use of BA for PM. From this framework, three propositions have been derived.

Proposition 1: Business analytics is used to manage organisational performance by collecting data

on variables that are of strategic importance, analysing their causal interdependencies and verifying strategic assumptions. This allows business analytics to assist in reaching strategic goals.

Proposition 2: Business analytics is used to manage organisational performance by making sure that

the data is of high quality and by effectively communicating insights to make the data understandable to decision-makers. This supports the exchange of information, ideas and views between different actors, ensuring the usefulness and value of the data.

Proposition 3: Business analytics is used to manage organisational performance by involving

employees from multiple parts of the organisation, by scheduling regular feedback meetings and fostering trust. This realizes effective collaboration between employees from different departments and hierarchical levels and make business analytics trustworthy for those that are responsible for implementing the resulting decisions.

(14)

13

3. Methodology

This study has explored how business analytics is used to manage organisational performance. In order improve understanding of this phenomenon, the propositions that are derived from the literature are put to the test of empirical data. Studying how business analytics is used to manage performance can either strengthen the validity of the theoretical propositions or offer an extension or refinement by augmenting knowledge about this topic.

3.1 Study design

At the heart of the research question is the use of business analytics, making organisational actors and their interactions central elements to this study. Hence, studying this phenomenon requires observation of organisational events and how these are interpreted by its members. The objective was to develop knowledge about how organisational phenomena occur and what they mean, for which qualitative research is particularly suitable (Golden-Biddle & Locke, 2007). Qualitative research is useful for studying phenomena in their natural setting, using social actors’ meanings to make sense of them (Denzin & Lincoln, 2008). This type of research is particularly useful for management studies, as it can provide detailed descriptions of actions in real-life contexts to enhance understanding of social processes and human interactions that underlie management (Rynes & Gephart, 2004).

A case study is the most appropriate research method according to Yin’s (2017) conditions, considering that this study contains a focus on contemporary events, has no need to control these events and studies a question of how something unfolds. Since theories about how BA is used in the context of PM have already been developed to some extent, the aim of this study was to test theory rather than to build theory (Voss, Tsikriktsis, & Frohlich, 2002). To discover whether the results from the study are in congruence with the propositions, this research used a positivist case study methodology with a descriptive nature (Guba & Lincoln, 2005), which fits with the aim of exploring how events unfold in a specific organisational context (Yin, 2017). Other rationales for pursuing a positivist case study are that the study aims to observe a real-life situation with the aim to discover the truth about how organisational practices take place, that the research question does not suggest that the researcher will influence or be influenced by the object of study and that the aim is to empirically test theoretical propositions (Denzin & Lincoln, 2008; Gephart, 2004). This careful selection of the methodology ensured that it is appropriate for answering the research question, strengthening the internal validity of the study (Crowe et al., 2011).

3.2 Case description

(15)

14

making necessary documents available. Avebe has around 1,300 employees and 2,500 affiliated Dutch and German farmers. Their main line of business consists of extracting starch and proteins from potatoes and supplying products to food, animal feed, paper, construction, textiles and adhesives industries. Avebe currently is worldwide market leader in potato starch extraction and sales. The organisation’s main goal is to optimize their growers’ returns, while achieving harmony between profit, environment and people. They are doing so by continuously offering new solutions that meet the needs of their clients and the society.

The decision was made to focus on the operations department of Avebe, as vast amounts of real-time data are collected and analysed there. This type of data fits perfectly with the subject of this study, as the data is collected to realize operational excellence. It provides insights about how different parts of the production process are related to performance indicators such as efficiency, costs and quality. Therefore, the population from which the sample is drawn consists of employees from operations departments of manufacturing companies. Data was collected at each of Avebe’s three Dutch production sites. Performance management in the operations department has various goals, such as improving operational efficiency, reducing costs, reducing harm to the environment and optimizing output quality. That is why from here on, the term ‘PM’ refers to performance management that aims to improve such types of performance. After having discussions with employees, to discover which specific group of employees would be best to select as the sample (i.e. those that can provide rich information about the topic of interest), it became evident that employees involved in World Class Operations Management (WCOM) initiatives were the most appropriate candidates. The aim of WCOM is to realize continuous improvement (CI) of performance, both in the short- and long-term. When observed that values of Key Performance Indicators (KPIs) are below their targets, possible immediate improvements are made in ‘daily control loops’. If such prompt improvements are impossible, a plan for improvement is developed and a suitable team is assembled. This team executes the plan and secures results to retain profits and guarantee long-term success. Business analytics plays a significant role here, since data-driven decision-making is a key component of WCOM. A graphical overview of this CI methodology applied by Avebe can be found in Appendix A. Because the Manufacturing IT (MIT) department of Avebe was involved in the implementation of WCOM since the early stages, the researcher also consulted members of this department on a regular basis.

(16)

15

Nr. Type Job Position / context Level of performance management

Site Interview date

Dura tion

1. Interview Team leader production, member of triangle, PCS pillar leader

Daily and weekly level 1 23-04-19 00:30 2. Interview Improve manager, member of FI

pillar

CI level 1 23-04-19 00:30

3. Interview Process engineer, member of Data Team, member of FI pillar

CI level 1 23-04-19 00:45

4. Interview WCOM consultant CI level 1 23-04-19 00:45 5. Interview Site controller, manager FI pillar Monthly and CI level 1 24-04-19 01:00 6. Interview Maintenance engineer, member of

triangle, member of PCS pillar

Daily and weekly level 1 24-04-19 00:45

7. Interview Site controller Monthly level 2 25-04-19 00:45 8. Interview Improve manager, FI pillar

member, PCS pillar member

Daily, weekly and CI level

2 25-04-19 00:45 9. Interview Process engineer, member of FI

pillar

2 25-04-19 00:45 10. Interview Operations manager, chairman of

morning consultation team, PCS pillar leader

11. Interview Process engineer, member of triangle, member of PCS pillar

12. Interview Site director, FI pillar Monthly and CI level 2 09-05-19 00:45 13. Interview Site controller Monthly and CI level 3 23-05-19 01:00 14. Interview Improve manager CI level 3 24-05-19 01:00 15. Interview Process engineer, FI pillar

member

3 04-06-19 01:00 16. Interview Project leader continuous

improvement

CI level 3 04-06-19 01:15

17. Interview Business data specialist CI level 3 04-06-19 00:45 1. Meeting Morning consultation (PCS) Daily level 1 04-04-19 01:00 2. Meeting FI pillar meeting Weekly, monthly and

CI level

1 02-05-19 02:00 3. Meeting Evaluating performance and

looking ahead to next year

Monthly and CI level 2 21-05-19 04:30

Table 2: sample description

(17)

16

researcher towards information-rich cases. Finally, theoretical sampling was used to identify cases that manifest the theoretical constructs of interest. The selected case is a critical case, which makes it suitable for proposition testing (Yin, 2017).

3.3 Data collection

Three different sources were used for data collection, namely interviews, observations and archival records. This data triangulation (Yin, 2017) makes use of multiple measures of the same phenomenon, strengthening the construct and internal validity of the research by providing stronger substantiation of propositions (Eisenhardt, 1989; Yin, 2017). During each interview, the researcher noted basic descriptive information such as the participant characteristics, field notes and time frame, which is known as attribute coding (Saldaña, 2015) and helps to manage data effectively (Miles, Huberman, & Saldaña, 1994). The participants were interviewed using semi-structured interviews and the participants signed a document to give their informed consent. In this way, participants’ privacy was protected, consent was given to record the interviews and participants were informed about the use of their information for the study. The construction of the interview protocol was guided by the information from the literature and specifically the propositions. After general questions were asked to get background information about the participants and their experiences with using BA, questions were asked to collect specific information about the phenomenon of interest. The questions contain as few terms from the propositions as possible, to avoid steering the conversation towards theoretically important concepts. Finally, additional questions were asked to delve deeper into a topic that the researcher felt the respondent had not elaborated on sufficiently. The complete protocol is presented in Appendix C. Semi-structured interviews were used to let the interviewee speak freely, without imposing predetermined views on the use of BA for PM. Besides, this offered the opportunity for interviewees to mention things that are not directly related to a predetermined question. Some structure is desirable however, to ensure that the main topics are covered and to enable comparisons between answers of different respondents. During the interview the researcher took notes and recorded the conversation on his phone. The first 12 interviews were transcribed immediately after the interviews. The final 5 interviews were not transcribed, since the majority of the content did not contain any new information. These recordings were exclusively used for the discovery of new and interesting insights.

(18)

17

to the codebook and no relevant information is forthcoming during interviews or observations (Guest, Bunce, & Johnson, 2006).

All collected data was safely stored on a USB stick for frequent reference. These files can be accessed and viewed upon request. Furthermore, the researcher kept a journal of all the research steps that are performed, to reflect on and keep track of the research process. Repetition of this study’s operations is facilitated by retaining the data and being transparent about the research process, further strengthening the study’s reliability and validity (Cepeda & Martin, 2005).

3.4 Data analysis

(19)

18

4. Evidence from the field

The aim of this research was to test the theoretical propositions against empirical data. Potential extensions or refinements are likely, as the theory on the use of BA for PM is still underdeveloped. This section describes the main findings from the case study at Avebe, resulting from the thematic content analysis. The next section discusses these results and compares them to the literature.

To provide a graphical representation of how the main themes emerged from the raw data, a Data Structure is provided in Figure 3. This helps to think theoretically about the data, stepping-up the level of abstractness (Gioia, Corley, & Hamilton, 2013). It enabled the researcher to balance the highly specific informants’ views with a helicopter view of the phenomenon of interest. From the list of 1st

order terms that were used for coding, 2nd_{order and aggregate themes emerged. All 1}st_{order concepts}

included in the Data Structure are supported by at least four different participants. The order of themes is inspired by the order of the steps from the preliminary research framework (Figure 2).

Driving force behind data

collection

Strategy

- Adjust KPIs to different levels of the organisation

- Constitute a plan/roadmap for BA, using the current state as a take-off point

Deciding which data to collect

- Collect data on variables/questions that are strategically important - Collect as much data as possible

- Higher management decides what data to collect - The experts on site determine what data to collect

Data quality

Definition of data quality

- Data quality is high when one source is used to store the data - Data quality is high when the data can be turned into information

Assessing and ensuring data quality

- Professionals assess the quality of the data (using their experience) - Data quality is ensured through standardization of data collection - Data quality is ensured through automatization of data collection - Data quality is high when human data collection is limited

Effective data analysis

Goal of data analysis

- To find root causes of issues - To provide real-time insights - To set up control mechanisms

- To measure the effects of actions that are aimed at optimizing processes - To improve organisational performance and reach KPIs

How to analyse data

- By combining different data sets

- Through continued inclusion of human activities - Through canalisation of efforts

Conditions for effective data analysis

- Actors must have know-how to derive insights/knowledge from data - Tools need to be easily accessible

- Analysts and users must know where the data comes from - Advice that results from BA must be checked with colleagues

Communicating insights from BA

How to communicate insights from BA

- Simplify the data: highlight important parts and leave out confusing parts - Use a standard visualisation tool

- Tailor communication method to the audience receiving the information - Substantiate facts: show where the data comes from and the underlying vision - Provide advice for concrete actions

Conditions for effective communication of

insights from BA

- Collaboration between actors - Involve the users: listen to their needs - Bride IT with decision-making

- Data analysts must have know-how of the business

Feedback loops and trust

Organising feedback loops

- Including employees from multiple parts of the organisation - Use feedback meetings to learn from each other and challenge the data - Plan feedback meetings regularly

Creating trust in BA

- Show how BA benefits the organisation

- Use BA to show connections between actions and outcomes - Foster a culture in which people are not afraid to ask questions - Provide training and education for using BA

1

st

_Order

₂

nd

_order

_Aggregate

Concepts

Themes

Dimensions

Figure 3: Data Structure

(20)

19

frequently by participants or are found to be interesting by the researcher. Most findings are common themes however, and the reader will be informed otherwise.

4.1 Driving force behind data collection

The first aggregate dimension is the driving force behind collecting data for PM. Virtually all research participants stated that one of the main driving forces behind BA is to gain insights to make fact-based decisions. Furthermore, some participants mentioned a link between data collection and Avebe’s strategy. For example, two respondents mentioned that business analytics should be an important item in the strategy formulation. The company’s vision must be translated into a roadmap, in which business analytics is mentioned explicitly.

‘It starts with a vision, what should the factory look like in 10 years? And then you have to set up a road map […], which should be a step-by-step plan towards that end point. This warrants that the company does not invest in something that gets in the way of this.’

This will speed up the progress of initiatives related to BA. Furthermore, explicitly stating the importance of BA in the strategy will signal its importance to the rest of the organisation, drawing attention to it. Moreover, this will reduce the risk of reactive data collection and will promote planned data collection instead. However, there were also a few participants who stated that data collection should only be linked to strategic goals that aim to be reached in the short-term. They proposed that data collection should be matched with the current state of development. The organisation must not be too far ahead with plans and current problems must be solved before going to the next step. Participants agreed that data collection should be linked to the organisation’s strategy, but opinions on whether this should mostly be linked to short- or long-term goals were mixed.

Additionally, many respondents mentioned that strategic goals must be translated into KPIs for different levels of the organisation. For example, operators on the shop floor should not be judged on the annual financial performance of Avebe as a whole. The main KPIs must be sliced into smaller-sized KPIs, which affect the main KPIs. From these small KPIs it must be defined which data is needed. Different norms must be defined for different parts of the organisation, to allow control mechanisms to be adjusted for different departments.

'You also want to make the translation there, 2 layers below. OEE is a KPI, below that hang the KPIs to achieve a good OEE, and therefore […] you want to show that to the users of the factory, the operators and team leaders. They […] will assess these KPIs and adjust accordingly, so that they are also able to influence that OEE.’

(21)

20

In terms of the actual driving force data collection, the points of view of respondents were divergent, with three points of view being expressed. From the first point of view it followed that data collection results from organisational goals and KPIs. According to this view, the decision to collect data is made by higher management. According to the second perspective, data collection results from a desire to collect as much data as possible.

‘We actually said from the beginning: all tags that are in the system […] will be logged per definition. We can make choices and uncheck things, but we just start with trying to log everything.’

Many participants mentioned that data storage is inexpensive, making it unnecessary to impose a restriction on the amount of data. Collecting as much information as possible increases the likelihood of information already being available when new questions emerge. This desire to collect as much data as possible partly resulted from a desire to innovate and keep up with others.

‘You have to keep up with the progress… You have to analyse better data, you want less people in the factory, you want to automate, you want to go to a higher level. Data is very important for that.’

A third group beliefs that experts should determine which data is to be collected. Legitimation for assigning these experts with the task to plan data collection is that they know the factory and are the most appropriate candidates for deciding which data is valuable to support improvement initiatives.

‘Problems often arise because things are conceived by people who do not have the knowledge and recognize the problems of the shop floor. The systems must be functional, but the knowledge, skills and feeling must also be in it.’

These experts were also the link between employees on the shop floor and the management team.

The frequency of occurrence of these three points of view was approximately similar. Therefore, besides strategically important questions, the desire to collect as much data as possible and the needs of experts to gain insights were identified as important drivers for data collection as well.

4.2 Data quality

The next theme is data quality. First, participants’ views on the definition of high-quality data are illustrated. One answer that was mentioned frequently was that data is of high quality when only one source is used to collect and store data.

‘Data must be standardised, to compare data with other data. Certain production lines run on different operating systems. It must be integrated.’

(22)

21

‘The problem is that we have many different systems which contain data, and a lot of manual work in Excel. […] Because the information is stored in so many different places, it is harder to do certain analyses because it costs more time.’

Many respondents also mentioned that data is of high quality when it can be turned into information.

‘Well, in the end it is not so much about what that data is, it is about: how do you turn that data into information? And only then does quality come into play.’

Related to that, another participant mentioned that data is of high quality when it is suitable for answering the question for which an answer is sought:

‘The data can be quite reliable for that question, but if someone uses the same data for a different question, then it may suddenly no longer be reliable.’

Finally, it was mentioned in a presentation from the MIT department that data is of high quality when it keeps knowledge inside the organisation. Data can deal with the issue of knowledge flowing out of organisations when employees are leaving.

‘There is a very dangerous thing associated with people leaving the organisation, because these people have a lot of knowledge. We want to face that threat by bringing more automation into our company and by saving the knowledge they have into our systems.’

One of the interviewees mentioned something similar:

‘The know-how of the software and the decisions made during the process of building it need to stay in the organisation. […] Otherwise you come across many issues of which there is no knowledge here, and I find that very worrying.’

High-quality data was thus described as being collected in a standardized manner and stored in one location. Furthermore, data is of high quality when it is useful for answering important questions and can be turned into information. Data that retains knowledge in the organisation was identified as being especially valuable.

(23)

22

‘For what purpose were those tags created? Is information compressed? When can I compare things? [...] Transferring the data from one system to another makes the validation of the data complicated. The person viewing the data must know where it comes from, whether it has filters, whether it is compressed, etc. If one is not aware of this, then data cannot be compared.’

However, some data cannot yet be collected by a system and will continue to require human involvement. To make sure that such data collection procedures still produce high-quality data, human data collection options must be limited.

‘In fact, systems should be such that you have little room to change things yourself.’

For example, reason-trees were frequently mentioned as a way to standardize the human data collection process, reducing the freedom of options to choose from. Additionally, training must be given to human data collectors to improve their decision-making quality. This allows some human action and knowledge to be included in the data collection process while maintaining a certain degree of standardization.

Many participants mentioned that the quality of the data should be assessed by experts.

‘... and the expert, who then wants to do the research or use that data, he or she determines at that moment whether it is reliable.’

These experts can assess the quality by asking themselves whether the value is realistic and logical.

‘See, a process engineer can say based on a certain tag: that should be within that and that value. And if that differs, then you know that something special is going on.’

By analysing the quality critically, large deviations and mistakes will be recognized. A downside from this is that the quality is often only assessed when there is suspicion about it.

Even though all participants agreed that automation of data collection is good in terms of data quality, most respondents also mentioned that automatically collected data should never be trusted blindly.

‘You will first verify or check with the relevant department. So, if it is something production related then you go to a process engineer or the manager of operations. If it is a matter of rejection then you start talking to Quality Control, for example, to first verify whether your picture matches their picture.’

(24)

23

For the type of data that is relevant in the area on which this study has focussed, the mechanisms to ensure high quality appear to be to automate and standardize the data collection process, warrant transparency about the origin of the data, limit human data collection options when human input is inevitable and to never blindly trust the quality of the data. Experts which have the most knowledge of the process for which the data is collected should be able to assess its quality.

4.3 Effective data analysis

The subsequent theme is that of effective data analysis. During the interviews and observations, two goals of analysing data were most frequently observed. Firstly, business analytics is used to find the root cause of issues, identifying where structural problems are located. One of the respondents used an analogy to describe why this goal is so imperative:

‘What you see way too often is that tires are being patched by people driving through spike strips. And we keep patching tires, because the spike strips are not removed.’

Secondly, data analysis served the purpose of providing real-time insights into the production process. This allowed employees to quickly understand where problems occur, improving the speed of implementing suitable solutions.

‘Where we now ultimately want to go [...] is that we make real-time dashboards, which […] make it very easy for an operator to understand how his factory is running at that time. […] you can create events that 'pop up' when they take place, whereas an operator does not normally see that. So, in that sense you can also be ahead of things, [...] that usually stay a little bit under the radar.’

Furthermore, analytics was used to measure the effects of optimisation initiatives, to evaluate whether actions have produced desired results. Business analytics was also consistently used to identify where big losses are and to check whether problems have occurred before. Besides that, analytics was used to define appropriate heights for targets. On the basis of analyses, norms for goals can be set with an appropriate bandwidth.

‘Based on data we have determined a certain level, our standard, and we say: you should be able to produce that, and around it we have agreed on a bandwidth. We can use that to control the process.’

This improved management control practices, as adjustments can be made within such a bandwidth. Additionally, several participants mentioned that analytics should be used to make predictions. Anticipating what will happen makes early adjustments possible and was an important goal for the case company.

(25)

24

Perhaps even more interesting however, is understanding how to use analytics to manage performance. A number of participants mentioned that it is important to combine different data sets when performing analytics.

‘So, if you want to answer difficult questions [...] you have to combine several things that are still islands now.’

One approach to facilitate this, suggested repeatedly by respondents, is to use a tool that automatically combines data from different sources. Such a tool enables employees to work with the data and make comparisons more easily. Furthermore, several participants mentioned that it is essential to keep a connection to humans instead of having only systems perform the analytics.

‘If you play with numbers yourself, you will often see more connections than a system which simply coughs up an amount for you.’

One respondent mentioned that there is a need to involve humans in the process of using BA, to reduce the risk of doing nothing with the information from BA:

‘The performance has to be filled in on paper, but that is actually because you want to make people aware of the fact that they have to pay attention to it. If something is only generated automatically, then you press print and put it in a folder and then it's done, it doesn't get the attention it deserves.’

Another added benefit from keeping the connection to humans was that humans are able to effectively identify where to pay attention to. A final piece of advice in terms of how to use analytics was to canalise initiatives. Having employees who pursue their own local initiatives is risky and might induce problems.

‘If you are doing your own research, ‘in your own tent’, so not integrated, then you are just doing something wrong. [...] We must meet on a structural basis to catch up on what is going on with everyone and what problems we face. Priorities must be set, and projects must be aligned.’

These results have provided some initial ideas about how to use analytics to manage performance. Different data sets must be combined, humans should play an active role in the process of using BA and initiatives must be canalised.

(26)

25

‘I actually belief that if you require education or training for using the tool, it is not good.’

Participants acknowledged two more conditions for effectively making use of analytics, which are strongly related to conditions for high-quality data. Firstly, employees must know what they are reading when looking at outcomes from analytics, such as how the data was collected, where it comes from and what lies behind the data. This is strongly connected to the need for transparent data collection procedures. Secondly, insights from BA should be checked with colleagues to verify the quality of the data and the resulting decisions.

According to the participants, the most important conditions for effective data analysis are having organisational actors that know how to turn data into information, making data easily accessible, having employees that understand where the data comes from and making sure that employees verify the outcomes of BA with colleagues.

4.4 Communicating insights from BA

From the literature it followed that in order to really benefit from BA, results must be effectively communicated to those that have to act on it. Only then can organisational change and benefits be realized. Here the results are presented about how to effectively communicate insights from BA. Similar to the rest of the findings, there probably will not be a ‘one size fits all’ solution to communicating insights. However, findings that may be relevant to a variety of contexts are described here. First of all, the importance of communicating insights in a way that is understandable to the receiving party was confirmed to be crucial. A recurring theme, perhaps being somewhat obvious but nevertheless of paramount importance, is to simplify data. Highlighting parts or leaving out confusing elements will make data significantly more understandable. Some respondents also mentioned that communication methods should be tailored to the audience. This means that the optimal communication method depends on the population, the party receiving the information. Tailoring communication can best be realized using one’s own intuition and understanding of the population. Other respondents were proponents of using a standardized communication tool. This idea contrasts that of tailoring communication to the population. An advantage of using a standardized way of communicating is that it makes it easier to compare things.

‘I want to bring a standard that I can make so that the moment I show it to that person, he understands what it means. [...] we have a check-list for this, listing the minimal criteria that the standard must meet.’

(27)

26

The indications from the case company’s representatives about how to communicate insights from BA, being partly contradictory, are to simplify the data, tailor communication methods to the receiving audience, use a standard communication tool and substantiate facts.

Besides recommendations on how to communicate insights, several conditions for effective communication emerged from the study. The findings illustrated that analysts and users of data must collaborate frequently. The user must be involved in each step of the process of using BA to improve performance. This is best illustrated with an example that often recurred during interviews: a client instructs a data specialist to create something new for a group of users. Before starting to use BA to meet the client’s demands, the analyst must involve the user:

‘If there is a question that something needs to be made, then someone starts working on it and makes it. And then it is at least first discussed with each other: is this about what you want?’

Once a solution has been finalized, the users must be instructed on how to use it:

‘And then the users all receive training, or at least an explanation of: this is what you see, this is the intention. And in that way, it is integrated and launched.’

Finally, the effect of the advice must be evaluated:

‘You have to put the builders, the users and the client together: Is this really what we all expected of it? And if not, what else do you want? So that is really every time that loop back to the client and the user.’

These examples illustrate that analysts must consult with their users on a frequent basis and listen to their needs, providing them with some control over the type of service they will receive. The reason for this is that the clients know best what the most understandable communication method is for them.

Another important condition is that analysts must be able to bridge IT with decision-making.

‘I want the data specialists to start in factories, work in factories, gain insights, recognize and acknowledge the problems that exist, and then work on a solution. But then you don't come from the top but from the bottom.’

The analysts must have affinity with the business and the ability to connect data with the business. Besides affinity, analysts must also have knowledge of the business and be able to think like decision-makers. As a result, analysts will be better able to provide decision-makers with what they need.

(28)

27

4.5 Feedback loops and trust

The literature review presented that it is important to have regular feedback meetings with employees from different departments, to learn from and have debates about insights generated by analytics. Actions that are suggested during these meetings must be acceptable to those that have to implement them. During the interviews, participants were asked about the existence of feedback meetings in the organisation and their opinion about it. Many respondents agreed that employees from multiple parts of the organisation, ranging from the Management Team to the shop floor, should be involved in feedback meetings. Respondents frequently described that meetings in which PM is evaluated are held in different levels, as described in Appendix B. One respondent mentioned:

‘I would not burden the organization too much with plenary meetings that are not very relevant to everyone.’

Others described that you should only include employees that can contribute to making decisions. There was consensus, however, that such meetings should include a reflection on what is going on and problems must be shared with one another. The respondents agreed about the need to have regular meetings, be it that these meetings do not solely serve the purpose of evaluating the use of BA. Feedback meetings that purely evaluate the use of BA only occurred in so-called Data Teams, where data integrity was discussed alongside issues such as whether the data is suitable for its intended purpose and whether employees interpret the data correctly. Other groups of employees did not feel the need to participate in this. However, some participants mentioned that there should be more central discussions, in which decisions are made about what to implement in terms of BA and how to proceed with it. Central organisation will improve uniformity of work, speed of progression and internal knowledge sharing.

In short, participants agreed that regular meetings are needed in which data is discussed and problems are shared with one another, instead of only evaluating the use of BA. This should be done in different levels, not including too many employees from different departments at once. Only when decisions must be made about how to proceed with BA in the future, large-scale central discussions are welcomed.

The final element to consider is trust. The literature indicated that it is important to create a culture in which BA is trustworthy, to promote its acceptance and use. Many times, participants mentioned that in order to make BA a success, some sort of culture needs to be formed in which people are motivated to use BA and have trust in it. To create trust, participants stated that it is important for analysts to show decision-makers how BA can benefit and help them, as well as to show that the aim of BA is to improve performance:

(29)

28

The main goal is to convince the users of the necessity of using analytics, creating a natural driver to do so. A potential way to assist in this is to use success stories to create trust. Another means for establishing trust is training and education. Broadening the knowledge of those with low levels of trust will result in BA becoming more trustworthy. Furthermore, connections between human actions and the data should be demonstrated:

‘At least the connection between their actions and the data, if you can make that visible, then the training is done for 80%.’

Also, indicating that there is a 1-to-1 relationship between reality and analytics can create trust:

‘That when they turn a knob, they see something changing in a trend. And when they see that, they gain trust in how it is structured.’

One respondent mentioned that to create trust, you need leadership. This means having charisma and a natural authority. Not looking down on people but signalling that the goal is to help them. Furthermore, a culture must be created in which people can raise the alarm and ask questions. Decision-makers should feel no barrier when it comes to asking questions to analysts, and vice versa.

To create trust among a group of employees that is sceptical about BA, it is advisable to first approach employees who are enthusiastic about digitalisation. After these employees have worked in projects in which data-driven decision are made and have been provided with guidance and feedback, their trust in BA will have improved and their enthusiasm will spread automatically.

One participant also mentioned something interesting, stating that trust should be created in an early stage of digitalisation, when the data is not yet too complex:

‘And you have to learn to trust it now, because the data is still clear and trustworthy.’

A final advice for gaining trust is linked to the advice for communicating insights and is to indicate where the data comes from and to explain what it means.

(30)

29

5. Discussion and conclusion

The aim of this research was to gain new insights into how organisations use BA to manage performance. The objective was to test three theoretical propositions against a real-life situation and to create new knowledge about the phenomenon of interest. A case study was performed at a company that has currently improved its efforts to move to more fact-based decision-making. The qualitative nature of this study enabled the investigation of many different organisational elements that are important in the process of using BA for PM. This wide orientation resulted in a holistic picture of how this process unfolds. The purposeful sampling of cases facilitated the formation of an understanding of how BA is used for PM, since a wide range of participants from different departments and backgrounds were involved. The findings of this study add to the body of literature concerning how organisations can leverage data and realize the potential of business analytics. Furthermore, this study provides insights about how to deal with obstacles that are faced when trying to move to fact-based decision-making.

We now turn to the initial propositions. The next three paragraphs will each compare one proposition to the findings, to draw conclusions about which theoretical elements are confirmed and which are contrasted. In the subsequent paragraph the main contributions are summarized, followed by managerial implications. Finally, the limitations of this study are described, along with implications for future research.

5.1 How to determine which data to collect and how to analyse it

(31)

30

collect as much data as possible. As a result, information is available when new questions develop, and the process of innovation is supported. Those with the third point of view assert that data collection is driven by experts and their needs for information. Adding these to the set of driving forces behind data collection provides a more complete picture of how companies decide which data to collect.

As stated in the proposition, the goal of analytics is to analyse causal interdependencies between variables (Klatt et al., 2011; Schläfke et al., 2013). Moreover, Warren et al. (2015) claim that analytics should explore which actions lead to outcomes and Sharma et al. (2014) state that analytics should investigate whether actions can reach their objectives and ensure that they are acceptable to those responsible for their implementation. Many different goals for analysing data emerged from the empirical findings. Investigating causal relationships was never mentioned explicitly, which means that the results do not confirm the proposition of Klatt et al. (2011) and Schläfke et al. (2013). The goals as described by Sharma et al. (2014) and Warren et al. (2015) are confirmed, even though the goals from the case study are more specific.

A condition for effective data analysis that emerged from the literature was that decision-makers must understand how to take advantage of data and have the talent to derive insights from it (Manyika et al., 2011; McAfee & Brynjolfsson, 2012). Strong support is found in the empirical findings for this proposition. Organisational actors must have the know-how and intelligence to convert data into information and must have sufficient time to do so. In addition, transparency about data collection procedures is essential to ensure that employees understand where the data comes from.

In light of these comparisons, it can be concluded that proposition 1 is predominantly disconfirmed by this study’s findings. Data collection in the case company is not mainly driven by questions that are of strategic importance, as hypothesized by Raffoni et al. (2018). Driving forces behind data collection were found to vary from the need to reach strategic goals to the desire to collect as much data as possible. A direct link to organisational strategy is almost non-existent in lower levels of the organisation, where a link with smaller-sized KPIs is more likely. Two additional driving forces behind data collection have been identified that may also be relevant in other organisations. The proposition from Klatt et al. (2011) and Schläfke et al. (2013) was also disconfirmed and the goals for analytics that resulted from this study can serve as input for improving the understanding of why companies choose to invest in BA.

5.2 Data quality and communicating insights from BA

(32)

31

from respondents about what high-quality data means, are not in line with those of Fosso Wamba et al. (2015) and Roden et al. (2017). The findings indicate that data can be of high quality for answering a certain set of questions, but of low quality for answering others. Data that generates economically valuable insights in a certain context can definitely be regarded as being of low quality in another context. Additional suggestions about what high-quality data means are that data is collected in a standardized way, can be turned into information and keeps knowledge inside the organisation. Furthermore, the literature has not provided information about how to ensure data quality. The empirical findings show that options to ensure data quality are to automate and standardize the data collection process and, if human data collection is inevitable, limit human data collection options and provide training on how to properly collect data. However, guaranteeing excellent data quality can only be achieved with sophisticated automation of data collection procedures.

The second proposition also states that insights from BA should be communicated using effective communication techniques, to make the data understandable to decision-makers and to support exploration and refinement (Evans, 2012; Miller & Mork, 2013). Fosso Wamba et al. (2015) and Raffoni et al. (2018) add that organisations need analysts that are able to bridge IT with decision-making. The empirical findings confirm each of these claims, illustrating that analysts should have affinity with the business, as well as the ability to bridge BA with making decisions for the business. As a result, the analysts will better be able to provide decision-makers with what they need. Furthermore, several practical implications on how to effectively communicate insights from BA have been derived.

Overall, it can be concluded that proposition 2 is predominantly confirmed, as high-quality data seems to be important, communication must be performed in an understandable way and analysts must be able to bridge IT with decision-making. Several potential extensions have emerged. Firstly, the findings add to the understanding of the meaning of high-quality data, as was suggested by Fosso Wamba et al. (2015) and Roden et al. (2017). Within this study’s case, data is considered to be of high quality when it is collected in a standardized way, is able to answer the question for which it is collected, can be turned into information and keeps knowledge from leaving the organisation. In addition, the results provide implications about how organisations can ensure that data quality is high. Automation and standardization are found to be of significant importance when attempting to collect high-quality data, alongside limiting human data collection options.

5.3 Feedback loops and trust

(33)

32

feedback meetings aim to discuss problems and to see whether the receivers of insights agree with and understand the data. The findings do not support the proposition from Schläfke et al. (2013), as the revision of the relevance of performance drivers was not mentioned by any of the participants. It follows that during these feedback meetings there must be a reflection on the use of BA and problems must be shared with one another. The participants did not fully agree with the hypotheses of Raffoni et al. (2018) and Sharma et al. (2014). The results indicate that feedback meetings should be held at different levels and only include employees who can contribute to making decisions.

The third proposition also articulates that fostering trust in BA is of significant importance when trying to turn the use of BA into a success. Those that have to implement advise that results from BA must have trust in its reliability (Sharma et al., 2014). Organisations must create a culture in which employees believe in the added value of BA, to promote its acceptance and use (Fosso Wamba et al., 2015; Mello et al., 2014). Participants fully agree that it is important to create a high level of trust in BA, confirming this proposition from the literature. However, this proposition is of no use as it is now and is in distinct need of expansion, since it is superficial, and no knowledge is provided on how to realize high levels of trust. Several implications on how organisations can create trust in management accounting information resulting from BA are provided in section 5.5.

It can be concluded that the findings are predominantly in contrast with the third proposition of this study. Revising the relevance of performance drivers, proposed by Schläfke et al. (2013) as the goal of feedback meetings, is not a goal in the case company. The propositions about the frequency and participants of feedback meetings are also contrasted, as the findings indicate that feedback meetings should only include a particular set of employees and should focus on a specific level of PM. The need for trust, as mentioned by Sharma et al. (2014), is clearly evident in the case company and several suggestions about how to ensure high levels of trust have been presented.

5.4 Main contributions

Capturing the power of data: using business analytics to manage organisational performance