DIGITAL BALANCED SCORECARDS

DIGITAL BALANCED SCORECARDS

How do managers use Artificial Intelligence to measure performance?

Master’s thesis

Laura Eleonora Farci · S4201221

MSc Business Administration · Management Accounting and Control

Faculty of Economics and Business

University of Groningen

Supervisor:

Dr. Andrea Bellisario

January 18, 2021

Word count: 8

456

In an always more digital world, global businesses are massively investing in Artificial Intelligence (AI) applied to Performance Measurement (PM). A growing number of companies worldwide are implementing digital Balanced Scorecards (BSCs) with performance metrics embedded in AI-enabled Business Intelligence & Analytics. Yet, empirical research on the use of such digital scorecards to measure performance is in its infancy and lively demanded by a rising number of academics. To shed light on how managers use digital BSCs at the operational level, an investigation encompassing purposeful sampling with maximum variation was conducted. Twelve in-depth semi-structured interviews were conducted in November 2020 with managers from ten companies in three domains (manufacturing, services, and university) and located in five countries (US, UK, Germany, Egypt, and Japan). The interviews proceeded in parallel with a database-led textual analysis. The results show three categories of digital BSC use and demonstrate that digital BSCs foster confidence and empowerment, facilitate the passive and proactive management of metrics, and promote resource-saving and synergies. These findings open up new areas of research in PM, namely towards (over)confidence, flexibility, visualization aspects, and future features.

TABLE OF CONTENTS

1. Introduction ... 1

2. Theoretical Framework ... 2

2.1 Performance Measurement and Balanced Scorecard ... 2

2.2 The digital side of the Balanced Scorecard ... 4

3. Methodology ... 7

3.1 Case setting ... 7

3.2 Data collection and analysis ... 8

4. Results ... 9

4.1 Digital BSCs used to foster confidence and empowerment ... 9

4.2 Digital BSCs used in the passive and proactive management of metrics ... 10

4.3 Digital BSCs used to promote resource-saving and synergies ... 12

1. INTRODUCTION

Let's face it: we live in a digital era. In this algorithmic world, organizations are undergoing digital transformations to take real-time action from massive amounts of data and enhance managerial and

organizational performance (Rikhardsson & Yigitbasioglub, 2018; Reinking et al., 2020; Hou, 2016;

Wamba-Taguimdje et al., 2020). To measure performance, companies have employed for decades traditional Balanced Scorecards (BSCs); however, the escalating amplitude and complexity of data are now driving the adoption of artificial intelligence (AI) to determine key performance indicators (KPIs). Currently, 73% of global businesses in the US, Europe, and China are exploring or implementing AI solutions, which are estimated to

contribute about $16 trillion to GDP by 2030 (IBM, 2020).

Under this trend, prior studies have investigated the positive effect of BSCs in association with digital

solutions (Hou, 2016; Shen et al., 2016; Alsharari et al., 2019; Fan, 2020; Shin et al., 2018; Frederico et al.,

2020; Kamble et al., 2020). However, a rising number of researchers (Schläfke et al., 2013; Reinking et al., 2020; Raffoni et al., 2018; Rikhardsson & Yigitbasioglub, 2018; Ransbotham et al., 2017; Davenport, 2018;

Robert et al., 2020) are showing concerns about blindingly relying on Business Intelligence & Analytics (BI&A) applied to scorecards. The effective adoption of digital BSCs is likely to be jeopardized when managers lack

the necessary skills and are unable to overcome the hurdles of using the framework (Van Camp & Braet,

2016; Robert et al., 2020; Wamba-Taguimdje et al., 2020; Raffoni et al., 2018; Rikhardsson & Yigitbasioglub, 2018; Amankwah-Amoah & Adomako, 2019; IBM, 2020; Neely et al., 2002, p. 344; Bititci et al., 2002). Managers’ inability to work with the measures may offset the potential benefits of BI&A and head them towards biased intuitions when supplementing the information retrieved by the system. Also, disruptive

innovations associated with AI are likely to suffer from cultural resistance to change (Walsh et al., 2020).For

this reason, a rising number of researchers are calling for empirical evidence on how managers use the BSC

with KPIs embedded in BI&A (Reinking et al., 2020; Rikhardsson & Yigitbasioglub, 2018; Raffoni et al., 2018),

especially in the combination of AI, BSCs, and BI&A (Schläfke et al., 2013). Due to the magnitude of current

AI investments by worldwide businesses, it is critical to explore how managers use digital BSCs at the operational level. In this regard, understanding the undermining factors of successful performance

measurement implementation is instrumental in minimizing the risk of PM or business failure (Lucianetti et

al., 2019; Amankwah-Amoah & Adomako, 2019; Van Camp & Braet, 2016; Parmenter, 2015, p. 34-38; Neely et al., 2002, p. 344; Bititci et al., 2002; Murray & Richardson, 2002).

2. THEORETICAL FRAMEWORK

2.1 Performance Measurement and Balanced Scorecard

Performance measurement (PM) is a knowledge-based process in which organizational processes and activities are accurately, objectively, and precisely quantified through a proportionate number of indicators

and then interpreted in the light of the organizational context and goals (Micheli & Mari, 2014). It consists of

measuring whether the present actions of managers will lead to a future measured value for their organization. At the strategic level, these indicators help management implement and challenge strategic

initiatives (Bourne et al., 2018). At the operational level, they facilitate the focus of employees on the most

critical factors for the success of the organization (Parmenter, 2015, p. 87). Besides, it exists a particular type

of measure, named Key Performance Indicators (KPIs), which have the dual role of linking the company activities to its operational critical success factors and planned strategic result.

Performance measures such as KPIs traditionally have a lifecycle that unfolds through design,

implementation, use, and refresh (Bourne et al., 2018). PM design is the process of identifying measures

representing all areas of the organization and defining how they relate to the organizational strategy and

objectives (Lucianetti et al., 2019). In PM design, measures can be either conventionally cascaded down with

a cause-and-effect strategy map (Kaplan & Norton, 1996) or cascaded up with modern approaches such as a

relationship-oriented result map (Parmenter, 2015, p. 187). In the conventional method, such metrics are

veritable KPIs in that they link the strategic priorities with the critical factors and the activities at the operational level. In the modern method, the indicators derive directly from such critical factors with no account for strategy. The two approaches differ tremendously in their implementation. With a cause-and-effect strategy map, an automotive company with the strategic priority of manufacturing high-quality components to increase customer satisfaction may tolerate higher waste to meet irreproachable quality and not see it as an operational critical success factor justifying a KPI. In turn, with a relationship-oriented result map, such an organization adopts a zero-waste policy in which "eliminate all waste" becomes one of the core

metrics regardless of the strategy (Petrillo et al., 2018). This second approach is not explored further in this

study, as it explicitly contradicts the definition of PM refresh. In this regard, PM use governs how metrics are used, whereas PM refresh directs how metrics are continuously updated to effectively test the strategy

implementation and challenge the assumption behind it (Kaplan & Norton, 1996; Bourne et al., 2018).

Hitherto, research has widely explored the phases of design and implementation, while leaving a compelling

need to develop theory around the continuing use and development of PM, which is the aim of this study

(Bourne et al., 2018).

PM use should center on a limited set of KPIs. However, environmental challenges and emerging

technologies provide an overload of strategic and operational performance measures (Kamble et al., 2020).

organizations often find themselves floundering with a myriad of conflicting, long to implement performance

indicators, and an overall inability to execute the strategic plan (Wiraeus & Creelman, 2019, p. 89; Kamble et

al., 2020; Murray & Richardson, 2002; Neely et al., 2002, p. 47, p. 344). These issues jeopardize the PM scope

of assisting decision-making when steering the organization in strategy implementation (Van Camp & Braet,

2016; Neely et al., 2002, p. 77). Some additional PM uses include achieving alignment with corporate goals, maximizing the improvements, monitoring and controlling, and – to a lesser extent – rewarding and

disciplining employees (Bititci et al., 2002).

If KPIs are to remain aligned with the strategy, they must be regularly refreshed (Bourne et al., 2018;

Micheli & Mari, 2014). In this review process, firms traditionally reported some heavy barriers, including lack of time, excessive availability of data, little understanding of the causal relationships among the measures, the cost of implementing a holistic system, the frequency of organizational change, and the lack of flexible IT

infrastructure to effectively and efficiently manage the dynamics of their metrics (Neely et al., 2002, p. 344;

Bititci et al., 2002). Nowadays, companies attempt to confront these barriers with powerful data-driven techniques that are more agile and adaptable in complex situations and offer proactive performance

measurement and real-time updates (Kamble et al., 2020; Wiraeus & Creelman, 2019, p. 14; Robert et al.,

2020). Nevertheless, these techniques are not always effective and organizational participants still cope with

many of these conventional problems. Accordingly, Van Camp & Braet (2016) analyzed over 250 articles and

shortlisted 36 causes of PM failure by grouping them in three levels: (i) difficulties with the dynamics of the metrics, such as confusion with the measurements, parameters, and KPIs; (ii) problems with the framework, such as lack of understanding and excessive complexity; (iii) wrong implementation of the PM system, such

as lack of alignment with the strategy. As noted by Lucianetti et al. (2019), understanding the factors that

may undermine successful PM is instrumental in minimizing the risk of its failure.

Over the past three decades, academics have expressed their dissatisfaction with traditional PM. This discontent prompted the spread of several multi-dimensional frameworks designed to provide a balanced

view among indicators of internal, external, and predicted future performance (Frederico et al., 2020; Shin

et al., 2018). Amongst them, the balanced scorecard (BSC) (Kaplan & Norton, 1992) is undeniably one of the most adopted PM frameworks. A generally-accepted explanation is that managers justify a PM system adoption with its cause-and-effect relationships – on which the BSC is devised – and with the perception of

its legitimacy and effectiveness (Frederico et al., 2020).

difference by Wiraeus & Creelman (2019, p. 146) explains that strategic BSCs do not exhibit detailed day-to-day activities but future-oriented goals and strategy-maps for new directions. Conversely, operational BSCs concentrate on real-time checking and operational monitoring, facilitate decision-making for a specific set of users, and effectively drive performance improvement against strategic objectives. Exploring the BSC operational use is of the greatest importance since the concerns on how organizational participants cope with many of the traditional problems of PM use are amplified now that they are exposed abruptly to

environmental and technological changes (Van Camp & Braet, 2016; Frederico et al., 2020).

In light of this, numerous academics tested whether using the BSC at an operational level may solve any of these concerns and determined that the framework yields higher responsiveness of the decision-making process and more intense and closer cooperation among units typically managed independently (Grando & Belvedere, 2008; Lucianetti et al., 2019). Yet, it seems inadequate under complex environmental

challenges and emerging disruptive technologies unless adapted to the industry-specifics (Frederico et al.,

2020). Despite its widespread use, several experts have identified shortcomings of the BSC use within an

organization, including the absence of additional perspectives (Lucianetti et al., 2019; Frederico et al., 2020;

Van Camp & Braet, 2016; Parmenter, 2015, p. 33; Neely et al., 2002, p. 148), the lack of understanding and

learning from the framework (Van Camp & Braet, 2016; Parmenter, 2015, p. 34), the difficulty of timely

implementation with the pace of change in the organization (Shin et al., 2018; Neely et al., 2002, p. 47), and

the overload, conflict, and confusion of metrics (Murray & Richardson, 2002; Van Camp & Braet, 2016;

Parmenter, 2015, p. 37-38). Due to the complexity of innovations, the employees frequently lack the necessary training and knowledge to understand how to use the framework and its metrics in their

correlation, causality, weights, and accuracy (Van Camp & Braet, 2016).

The fast-changing, complex environment may seriously compromise the effectiveness of the BSC. Changes in business conditions are emerging at a surprising speed. In this vein, several authors offer

suggestions on how the BSC needs to evolve to be effective in these disruptive times (Shin et al., 2018;

Frederico et al., 2020; Kamble et al., 2020). The next paragraph is devoted to addressing the concerns that recent literature has raised in this regard.

2.2 The digital side of the Balanced Scorecard

Compared to the uses of BSCs done over the past two decades, sophisticated technologies such as IT

infrastructures, process automation, advanced business analytics (Kaplan & Norton, 2004), and dynamic

scorecards (Wiraeus & Creelman, 2019, p. 19-20) allow nowadays managers to take real-time actions, as

these actions can be informed by massive amounts of data continuously elaborated. The result is that

organizational performance can be dramatically enhanced (Rikhardsson & Yigitbasioglub, 2018; Reinking et

al., 2020; Robert et al., 2020), especially compared to more classic approaches to PM (Schläfke et al., 2013;

with digital solutions such as business intelligence systems (Hou, 2016), digital enterprise resource planning (Shen et al., 2016), automated BSCs (Alsharari et al., 2019), complex big-data (Fan, 2020), and process

automation (Shin et al., 2018; Frederico et al., 2020). These authors all agree on the positive effect of digital

BSCs, which can also yield increased financial performance (Hou, 2016) and easiness in data management

(Fan, 2020).

These disruptive times, characterized by the escalating amplitude and complexity of business data, are driving the explosion of Artificial Intelligence (AI) solutions in Business Intelligence and Analytics (BI&A). AI-enabled BI&A are said to be the next natural step in the evolution of BI&A with a significantly larger and

more disruptive impact than in previous technology transitions (Davenport, 2018). In this algorithmic world,

AI plays a critical role in determining what KPIs are measured, how they are measured, and how best to

optimize them (Kiron & Schrage, 2019). Contrary to traditional BI&A, AI-enabled ones are likely to resolve

many of the BSC shortcomings as they clarify the relationships among metrics. Also, they contribute positively to improved synergies between people and systems, agile high-quality decisions in internal business processes, more thorough insights into customer behavior, and accurate prediction of financial trends (Davenport, 2018; Robert et al., 2020). Schläfke et al. (2013) state that effective PM occurs in the conjunction of Artificial Intelligence, Balanced Scorecard, and Analytical Methods and that further empirical research should investigate this connection. BI&A provide greater flexibility to managers, who can dynamically control the way measures are visualized and intercorrelated, discover hidden patterns, and be warned of critical factors when performing multiple operational activities such as reporting, measuring performance, making

decisions, and forecasting. Yet, a rising number of researchers (Schläfke et al., 2013; Reinking et al., 2020;

Raffoni et al., 2018; Rikhardsson & Yigitbasioglub, 2018; Ransbotham et al., 2017; Davenport, 2018; Robert et al., 2020) are showing concerns about blindingly relying on BI&A applied to scorecards.

At the operational level, BI&A increase the overall efficiency and effectiveness (Schläfke et al., 2013)

due to their particular focus on specifically tailored KPIs, which result in improved managerial and

organizational performance (Reinking et al., 2020). Nevertheless, when managers have little expertise with

BI&A, they are unable to extract valuable insights and are left with partial, useless, or even misleading

information (Raffoni et al., 2018) that may aggravate bias in decision-making (Rikhardsson & Yigitbasioglub,

2018). That is, managers’ inability to work with the measures may offset the potential benefits of BI&A and

Besides, disruptive innovations associated with AI are likely to suffer from manager cultural

resistance to change (Walsh et al., 2020). Distrust, high-costs offsetting the benefits, scalability issues, and

individual unwillingness to alter the ‘status quo’ are only a few causes fostering cultural resistance that may prevent the effective adoption of digital BSCs. Consequently, organizations need to develop new ways to combine user skills and powerful IT infrastructures to take advantage of the synergies between people and

systems (Rikhardsson & Yigitbasioglub, 2018; Ransbotham et al., 2017; Davenport, 2018; Kaplan & Norton,

2004; Robert et al., 2020). In this regard, training seems vital to acquire the necessary technical expertise and

skills (Van Camp & Braet, 2016; Robert et al., 2020; Wamba-Taguimdje et al., 2020) and overcome cultural

resistance (Walsh et al., 2020).

In this vein, scholars are explicitly asking for more empirical evidence on how managers use the BSC

with KPIs embedded in BI&A (Reinking et al., 2020; Rikhardsson & Yigitbasioglub, 2018; Raffoni et al., 2018;

Schläfke et al., 2013). This lack of understanding seems to raise urgent also among practitioners. For instance,

in a recent survey run at IBM (2020), lack of trust, poor technical skills, and limited knowledge surface as a

hindrance to the successful adoption of digital BSCs for day-by-day tasks. Scarce technical expertise is proven

to result in data and decision-making confusion, also creating conditions for business failure (

Amankwah-Amoah & Adomako, 2019). In turn, sound AI personnel expertise, AI management capabilities, and AI

infrastructure flexibility yield increased financial, marketing, and administrative performance (

Wamba-Taguimdje et al., 2020). Therefore, the mere possession of assets such as AI, BSCs, and BI&A do not automatically produce effective PM; rather, it is organizational participants’ ability to analyze data and extract value for decision-making that produce such result. Yet, research in this regard is still in its infancy and little is known on how managers interact with digital BSCs.

3. METHODOLOGY

3.1 Case setting

This study provides empirical evidence and expands the scientific understanding of how managers use digital BSCs at the operational level with KPIs embedded in AI-enabled BI&A. Due to the nascent state of the phenomenon under examination, qualitative field research encompassing semi-structured interviews and

textual analysis of relevant material from field sites was particularly indicated (Edmondson & McManus,

2007). Respondents were selected through purposeful sampling so that information-rich cases illuminate the

inquiry until theoretical saturation is reached (Patton, 2014, p. 264). Specifically, participants are managers

at the operational level who daily employ AI-enabled BI&A to measure performance. For three years in a row, IBM has been recognized as the leading company in providing AI solutions to companies, including AI-enabled

BI&A (IDC, 2018). Their clients are listed in a database (i.e., “Client stories”) according to the product they

bought, often with videos in which they describe their user experience and their company background. In this regard, IBM clients are a suitable research sample that relies on the maximum variation possible in three domains: manufacturing, services, and university. The Client stories database was valuable to skim the companies to the ones fitting the research criteria, directly identify the managers to interview, and retrieve

useful data on which performing textual analysis. APPENDIX A comprises a list of all the research participants.

Fig. 1: Digital BSC integrated into IBM® Planning Analytics (Readapted from Belden, 2020).

3.2 Data collection and analysis

Data collection unfolded through twelve in-depth semi-structured interviews with individuals from ten

companies in five countries over November 2020. The interviews had an average length of approximately 40 minutes and were conducted remotely. All of the respondents are made anonymous to ensure fair treatment.

An interview protocol (APPENDIX B) encompassing open-ended semi-structured questions was rigorously

established to ensure comparability across practices, prevent an overload of superfluous data, and guarantee

valid and reliable data (Miles & Huberman, 1994). Interview questions are of three types: (i) behavioral

questions on specific observable actions, (ii) opinion questions on the perception of such actions, (iii) feeling

questions on the emotional experience (King et al., 2018, p. 65; Patton, 2014). These types of questions allow

to obtain different kinds of responses (King et al., 2018, p. 65) and cover central issues both broadly and

in-depth. They identify which topics are relevant to the participants and what deep meaning do details have (Legard et al., 2003, p. 148). Additionally, probes such as different forms of follow-up questions are adopted

to explore the depth of responses (King et al., 2018, p. 69; Legard et al., 2003, p. 150-153).

Data analysis was performed iteratively during data collection to flexibly follow up promising leads

and abandon fruitless lines of inquiries (Edmondson & McManus, 2007). Transcripts were processed in open,

axial, and selective coding (Wolfswinkel et al., 2013) through thematic analysis software. In line with this

particularly-suitable method for nascent research (Edmondson & McManus, 2007), the Client Stories and the

interview transcripts were jointly broken-up into nineteen first-order concepts. Henceforth, the codes

representing similar ideas were grouped into seven second-order concepts following Gioia et al. (2012). Last,

the second-order concepts originated three aggregate dimensions, which shape the main findings of this

study. A table portraying the analysis of codes is offered in APPENDIX C along with an overview of the most

4. RESULTS

4.1 Digital BSCs used to foster confidence and empowerment

Companies undertake great work to challenge resistance and convince organizational participants of the potential of the new system. At the beginning of the digital BSC adoption, managers find it difficult to trust the system and risk to oppose cultural resistance by preferring old methods. In this regard, the understanding of cause-and-effect relationships among data and KPIs passes through a process of validation. Managers validate their intuitions against the system suggestions and start receiving confirmations that AI-enabled BI&A are trustworthy. For instance, P10 is confident that by leveraging BI&A data they will reduce their carbon footprint and eliminate coal from the company energy portfolio by 2050.

“Engineering is a very precise discipline, and engineers are accustomed to working with very high-quality data, often captured in laboratory conditions. But big data isn’t like that. It is messy, and often the valuable insights are hidden in a lot of irrelevant data. So, one of the biggest challenges was a cultural one: we needed to show our engineers how valuable big data can be when you have the right tools to deal with it.” (P5, C5)

“Another great example was when we were building a report for our human resources department, which showed that there was a sudden drop in workplace hand injuries in February and March. When we presented the report to the human resources team, they immediately recognized that the reason for the decrease was a health and safety education campaign they had run, which helped them to validate that their approach was working.” (P4, C4)

As managers trust AI-enabled BI&A, they can leverage data to implement strategy and make decisions. Indeed, data information is abridged in a limited set of KPIs, offering a more comprehensive view of the organizational dynamics. In strategy implementation, analysts use BI&A in combination with predictive modeling to determine how the decisions they make affect that data and bring them closer to the strategic objectives and KPIs. Predictive modeling is a novelty of digital BSCs, which simplifies the decision-making by comparing projected data against a reduced number of KPIs in a simulation environment. Managers can see the results of their actions before taking them and learn how KPIs are imperceptibly linked one to another. Hence, their decisions are more informed and not based on intuition or common-sense assumptions determined by their job experience. As organizational participants discover these imperceptible cause-and-effect relationships among data and KPIs, hard to find for a human eye in a reasonable time, they feel surprised, delighted, and comfortable because their decisions at the operational level are more educated and accurate. Empowered by this improved decisional effectiveness, they start trusting the digital BSC data accuracy and reliability to the extent that they sacrifice their intuitions to the stake of what is proposed by the system.

of the potential vendors had presented their new proposals, we uploaded our historical telephony usage data and the new terms and conditions. Within a matter of hours, [the system] helped our procurement team to determine which of the new proposals offered the best value. (…) On the other hand, it has also shown us that

common-sense assumptions can sometimes be incorrect.” (P4, C4)

Having faith in the abilities and recommendations of AI-enabled BI&A, managers also start gaining confidence in their decision-making capability. Thanks to data personalization, they are empowered to pull their tables and create their data modules to visualize and extrapolate the most pertinent information to meet their KPIs. For instance – to meet the KPI of “incrementing the four-year graduation rate” for the University in which they work – P1 and her colleagues regularly pull a table that lists the students who may not take the passing mark, depending on career history and investment, and offer supporting courses. A flexible personalization is vital in inhibiting resistance to change. Not only since some users may prefer tables and some other charts, but because each level of skills corresponds to a different method and depth of analysis. When organizational participants are trained to select the most meaningful data to display, they gain a deep understanding of the metrics in their correlation, causality, weights, and accuracy so they can better design, use, and review them. In C5, learning how customers drive in real-world conditions suggests how to manufacture cars so to improve R&D skills, internal processes efficiency, customer satisfaction, and eventually financial results. By retrieving and sorting the sensor data according to her preferences, P5 and her colleagues were able to identify a new project yielding new KPIs, streamline internal processes to increase customer satisfaction, and review the KPIs derived from obsolete assumptions on drivers’ behavior.

“Another important advantage of [our] approach is that it has shown the company’s engineers the value of big data analytics in understanding how cars and drivers behave in real-world conditions and identifying new engineering projects. For example, the company is using vehicle sensor data to monitor drivers steering maneuvers and gain a better understanding of customers' preferences.”(P5, C5)

Considered the importance of developing skills, three companies have severely invested in different types of continuous training – external for C7 and C10, and internal for C5 – resulting in enhanced performance. Continuous learning enables managers to steadily develop their know-how and adapt their functions and skills to the fast-changing work environment.

4.2 Digital BSCs used in the passive and proactive management of metrics

Organizational participants manage the digital BSC metrics in two ways: passively and proactively. Passive management consists of compulsorily adopting new work practices to overcome earlier criticalities for business needs. Proactive management resides in the managers' willingness to engage in non-compulsory, new activities that optimize their overall work.

Managers use digital BSCs to respond passively to previous issues. Participants repeatedly use terms such as needed, struggled, and difficult in association with past activities that were becoming too complex

and time-consuming to face trends expected to continue. In particular, P3 describes the exigency of enacting

a change before the whole system crashed on them and they lost all of their data. In this regard, three companies adopted the digital BSC as a response to multiple or obsolete PM systems. Two others implemented the framework to meet customer demand and cope with the increasing business scale, as they needed a more comprehensive PM system reflecting information from each part of the business. In both cases, external circumstances and business needs forced organizational participants to start using a new PM system abandoning old practices such as spreadsheet-based analytics. Whereas proactive use is characterized by a degree of flexibility to choose the analyses to perform and the method, passive use does not offer this option. Hence, firms were concerned that this would have engendered aversion to change – i.e., cultural resistance – and undertook great work to inhibit it, including training and informative meetings. As a result, managers clearly understood why the change was necessary and how they would have benefited from it, thus welcoming the new system. Although enduring change, they believe to have better data and tools compared to the past, as well as a more holistic view of the cause-and-effect relationships of metrics.

“People are generally averse to change. It’s hard for them to go from ‘I know whom to ask for this, and it will show up on my desk’ to ‘I can do this on my own’. When they’re faced with a whole new buffet of products, they can get overwhelmed.” (P1, C1)

“We grew from about 350 locations to about 520 locations. New opportunities, new markets are emerging. We knew we needed to grow headcount. We knew we needed to have better data. And, of course, we knew we needed a financial system that would scale to that growth. The IT department was looking for a BI solution and the FP&A team was looking for a reporting and planning solution. And [the new system] really fits the bill.” (P8, C7)

potential problems and enable drilling down the relationships among the metrics. An interesting aspect is that such non-compulsory analyses are spontaneously initiated to putting measures in place, meeting targets more effectively, and developing more efficient ways of working. Research participants repeatedly use terms such as optimize, enhance, and identify in association with future activities suggested or proposed by the system and possible risks that might arise. This creative search for optimizing their work and better meeting KPIs is described thoroughly by P4 below. By proactively analyzing with AI several factors to determine cost drivers, he unexpectedly discovered that wrong diagnoses were the leading cause. Thus, he successfully increased performance by providing additional training to the particular sub-offices whose accuracy was below average yet still meeting the KPIs. As KPIs were still met, this analysis was not mandatory but derived from P4’s confidence in his decision-making capability and the system recommendations. This spontaneous virtuous behavior allowed to improve performance rapidly.

“We wanted to find out what the main drivers for social housing repair costs were, and we had spent three weeks trying to analyze the data on spreadsheets. We thought the age of the property would be the most significant factor, but we were having a hard time testing this hypothesis. Then we fed our data into [the

system] and discovered the real answer in less than a day. In fact, the cost driver was how accurately our

clients were initially diagnosing faults in their properties and the work required. The result was that our teams were being sent on-site with incorrect equipment, and required more visits than usual to complete the work.”

(P4, C4)

In conclusion, managers use digital BSCs passively when they endure a change in their work practices and do not have the flexibility to choose which data, features, and type of analyses employ in their tasks. When companies undertake great work to make participants accept the change, managers welcome the new system and have a better understanding of metrics in their correlation, causality, weights, and accuracy. This better understanding impacts behavior as it inspires managers to proactively initiate real-time, complex analyses to optimize their work, eventually leading to improved performance.

4.3 Digital BSCs used to promote resource-saving

and synergies

The last way managers use digital BSCs is to promote resource-saving and synergies among units and employees. Digital BSCs efficiently reduce costs, time, and cognitive effort dedicated to executing tasks. In turn, the resources saved can be devoted to developing team-working and internal training.

positively affects the digital BSC in its financial and internal processes dimensions and modifies how teams are structured and collaborate in performing tasks. For instance, digital BSCs allow reducing the time and cognitive effort dedicated to executing an assignment thanks to the automation of KPIs, alerts, and reports. As described by research participants, meaningful data is extracted in real-time by the BI&A and automatically converted into accurate KPIs displayed on the digital BSC in an interactive and customizable fashion. From that point, each time a manager is not going to meet a KPI or a goal, they receive an automated alert to correct the trend and deliver high-quality results. Last, a proposed solution complemented by other data is extrapolated by a database and synthesized in automated reports.

“The business intelligence platform starts by the extraction process, the ETL, and that's the tool that actually connects with all of these data sources on a nightly batch or, depending on how you want to schedule it, pulls the data that is of value for us, places it in a staging area, then transforms it into KPIs that is hosted in the data warehouse, and then visualized by the [system].” (P2, C2)

The information condensed in these reports and KPIs is perceived as a launch-pad for discussion among the teams, and a set of valuable suggestions that could have been missed with the old system. When participants flexibly have ‘the last word’ on which data, features, and type of analyses employ, they engage in a mutual, beneficial relationship with the system. As they gradually choose how to modify their way of executing tasks, their cultural resistance is inhibited. Even when it is their workload to be automated, this is not perceived as a constraint. Workload automation is proven to enhance the synergies among units and employees, as managers' tasks are re-ordered and combined with the priorities of other teams or individuals.

Another way by which organizational participants promote resource-saving and synergies is by retrieving useful data in real-time. Faster data-fetching reduces the time of fruitless collaboration, as participants can independently retrieve data for which they would have had to go to different locations or people. Nonetheless, it improves the quality of the collaboration itself and promotes non-formal, internal training.

“I call it the Big Data Friends Club. Bringing our engineers together has been a benefit in itself because they are more likely to share their data and learn from each other. For example, our chassis team was working on a project to study deterioration in exhaust pipes, and they needed some data about gasoline. Through the Big Data Friends Club, they discovered that our fuel economy team already had the data they needed, so it saved a lot of effort and expense. (…) Another valuable feature is the ability to monitor users and see how they are interacting with the tool.” (P5, C5)

5. DISCUSSION

At the operational level, managers use digital BSCs with KPIs embedded in AI-enabled BI&A as an instrument to (i) foster confidence and empowerment, (ii) passively and proactively manage metrics, and (iii) promote resource-saving and synergies. Organizations undertake great work to make participants accept the change. Even when managers passively endure a transformation in their work practices and do not have the flexibility to choose which data, features, and type of analyses employ in their tasks, they welcome the new system. Once the initial cultural resistance is inhibited, they start leveraging data to better implement the strategic initiatives and make more informed decisions. In turn, they gain confidence in their decision-making capabilities and are empowered by the visualization features and flexible use of the AI-enabled BI&A. This, in association with continuous training, results in a better understanding of metrics in their correlation, causality, weights, and accuracy. This deeper understanding inspires managers to proactively initiate complex analyses to optimize their work, eventually stimulating an overall positive effect on the BSC’s dimensions and an increase in performance. Next to this, digital BSCs modify how teams are structured and collaborate by reducing unproductive costs, the time of fruitless collaborations, and the cognitive effort to executing tasks. Besides such savings in resources, they foster synergies by increasing the quality of the collaboration among team members and proposing valuable starting points for discussion.

The study provides empirical evidence on the BSC use as a PM framework by examining its conjunction with AI and BI&A at the operational level. It proposes new material for research, as lively asked

by scholars (Bourne et al., 2018; Reinking et al., 2020; Rikhardsson & Yigitbasioglub, 2018; Raffoni et al., 2018;

Schläfke et al., 2013). These results offer a meaningful contribution to the PM and BI&A literature related to behavior and decision-making. Primarily, they corroborate the idea that PM systems directly influence

behavior (Micheli & Mari, 2014). The digital BSC provokes a change in how managers perform their tasks

from the very beginning of the adoption. Compared to traditional BSCs and BI&A, its visualization features empower managers to drill down measures and dynamically control the presentation format depending on the type of tasks. AI is then used to test the expected outcome of a decision on a set of KPIs before making it. The projected result derives from data, regardless of the visualization chosen, thus preventing suboptimal decisions. This mechanism, which is peculiar to digital BSCs with KPIs embedded in AI-enabled BI&A, contrasts the idea of drill-down paths leading to suboptimal decisions in the absence of fit between presentation

format and type of task (Rikhardsson & Yigitbasioglub, 2018). Rather, this study confirms Reinking et al.

(2020) by suggesting that, in data-driven decision-making, the presentation format does not affect the

decision quality and that drill-down visualization features and projections aid decision-making. The findings directly respond to the need for research on how performance dashboards and similar visualization tools affect managerial performance and behavior and how employees can ensure, through the use of BI&A, that

data-to-insight is turned effectively into decision-to-value (Rikhardsson & Yigitbasioglub, 2018). Also,

consequence is particularly positive when adopting the digital BSC because it helps inhibit cultural resistance. In the very beginning, participants validate their intuitions against the system suggestions and receive confirmations on the reliability of AI-enabled BI&A. After a while, they start sacrificing these intuitions at the stake of what is proposed by the system. However, the results do not support nor exclude the possibility that, in the long term, confidence may degenerate into overconfidence and encourage risky behaviors (Rikhardsson & Yigitbasioglub, 2018). Due to the nascent state of digital BSCs, it is still premature to look empirically for evidence of overconfidence.

This study also contributes to the change and innovation literature associated with how people receive technological innovations and structure their relationship with machines. Literature displays a clear

lack of consensus on the inhibiting factors of cultural resistance to AI-based disruptive innovations (Robert

et al., 2020; Reinking et al., 2020; Walsh et al., 2020). Robert et al. (2020) identify as determinants the managerial involvement in the PM system implementation, the active participation in the decision-making,

and the autonomy in using transparent indicators. To Reinking et al. (2020), the foremost factor is the extent

to which operational managers perceive an alignment between strategic objectives and performance

measures. Last, Walsh et al. (2020) recognize that the simplicity of the infrastructure and the environmental

support intensify the innovation’s perceived value, thus decreasing managers’ cultural resistance. The present investigation adds to such studies and reveals that cultural resistance is overcome thanks to strong organizational support that includes training and informative meetings. Here, the chief inhibiting factors are the perceived need for a new infrastructure, the expected benefits of better data management and PM, and the flexibility to choose which data, features, and type of analyses employ. Flexibility is the core factor governing how employees effectively structure their relationship with machines, yet still scarcely explored in literature. It contributes to a better understanding of metrics, thus incrementing the perceived value of the innovation.

Additionally, the study contributes to the BSC literature. The findings suggest that flexibility, a better understanding of metrics, and the savings in resources offered by the automation are accountable for an overall positive effect on the BSC’s dimensions and a managerial, financial, and administrative performance increase. These results provide a new understanding of the positive relationship between the proactive use of AI-enabled BI&A and increased financial performance by validating the cause-and-effect relationship of

the BSC’s dimensions. Recent studies (Reinking et al., 2020; Hou, 2016) demonstrated that advanced BI&A

itself. An attempt to investigate these internal mechanisms comes from Wamba-Taguimdje et al. (2020), who discovered that the combination of management capabilities, personnel expertise, and infrastructure

flexibility enhances organizational performance. It is important to note that Wamba-Taguimdje et al. (2020)

intend flexibility as the customization with multiple software modules and the interoperability among different systems (i.e., how AI or analytics can be used for diverse functions). On the contrary, this study proposes flexibility as the users' possibility to determine their extent of use (i.e., which data, features, and type of analyses employ at their own pace and will). Managers can flexibly decide to engage in non-compulsory, virtuous behaviors to optimize their work. This optimization – in association with a better understanding of metrics and the savings in resources given by the automation – is predicted to have a positive effect on financial performance.

Last, this research contributes to the organizational literature linked to team structure and

collaboration. Whereas previous research (Grando & Belvedere, 2008; Lucianetti et al., 2019) has shown that

traditional BSCs offer more intense and closer cooperation among units and employees, these results unveil that digital BSCs reduce the frequency of team collaboration but increase its quality. That is, managers refer less to other team-members for retrieving information that the system can automatically generate. However, when they do, they dedicate the time and cognitive effort saved throughout the process to share their data and learn from each other. This faster, high-quality knowledge-sharing process strengthens the BSC’s dimension of learning and growth. As such, digital BSCs are predicted to offer a stronger positive effect on the customer dimension than traditional ones. These results directly address how cross-functional

collaboration changes when implementing AI-enabled technologies (Frederico et al., 2020) and provide a

6. CONCLUSION

This study has been conducted to shed light on the implications of digital BSCs and provide empirical evidence on how managers use them at the operational level. The initial motive was investigating the peculiarities of effective digital BSC adoption, primarily because of the magnitude of current AI investments by worldwide businesses. The investigation embraced purposeful sampling with maximum variation. Twelve in-depth semi-structured interviews were conducted in November 2020 with managers from ten companies in three domains (manufacturing, services, and university) and located in five countries (US, UK, Germany, Egypt, and Japan). The interviews proceeded in parallel with a database-led textual analysis. The results of this study demonstrate that digital BSCs have an impact on human behavior, on the relationship between people and machines, and on team structure and collaboration. It was found that, at the operational level, managers use digital BSCs with KPIs embedded in AI-enabled BI&A as an instrument to foster confidence and empowerment, passively and proactively manage metrics, and promote resource-saving and synergies.

From an executive standpoint, this study describes the experience of several companies in such a way that firms can better weigh the advantages and disadvantages of adopting AI-enabled digital BSCs. Besides, the results propose several recommendations. First, organizations are invited not to underestimate the impact of strong organizational support on cultural resistance. Informative meetings, training, and the direct involvement of operational managers in the BSC implementation are only a few of the activities to put at the forefront. Second, managers shall be empowered to flexibly decide – time after time – which data, features, and type of analyses employ in their tasks. Such flexibility will contribute to a more personalized user experience, a better understanding of metrics, an improvement in managers’ confidence about their decision-making, and expected proactive virtuous behaviors. The combination of these elements is predicted to increase financial performance. Last, organizations shall govern and take advantage of the resource-saving given by automation. As managers save money, time, and cognitive effort while accomplishing a task, organizations should strategically redirect such resources to the most critical factors for their success and offer specific tracks for reinforcing skills in cross-functional teams.

Nonetheless, these recommendations ought to be appraised in light of the limitations of this study. Although interviews are a powerful technique for describing, interpreting, and contextualizing the insights of new phenomena from those directly experiencing them, they cannot be generalized beyond the sample

group (King et al., 2018, p. 11; Legard et al., 2003; Patton, 2014, p. 246). In particular, many of the participants

could present more valid data with a longitudinal approach. For instance, there is a high likelihood that a longitudinal study will show that the digital BSC’s power of stimulating confidence is valid only in the first phases of the adoption. If it is demonstrated that confidence degenerates over time into overconfidence, firms will prevent this issue by adopting management control fashions that could entirely overturn the investigation results.

REFERENCES

Amankwah-Amoah, J., & Adomako, S. (2019). Big data analytics and business failures in data-Rich environments: An organizing framework. Computers in Industry, 105, 204–212.

Alsharari, N.M., Eid, R., & Assiri, A. (2019). Institutional contradiction and BSC implementation: comparative organizational analysis.

International Journal of Organizational Analysis, Vol. 27 No. 3, pp. 414-440.

Belden, C. (2020, July 7). Metrics and Scorecards in IBM Planning Analytics. ECapital Advisors.

https://ecapitaladvisors.com/blog/metrics-and-scorecards-in-ibm-planning-analytics/

Bititci, U., Carrie, A., & Turner, T. (2002). Integrated performance measurement systems: Structure and dynamics. In A. Neely (Ed.), Business Performance Measurement: Theory and Practice (pp. 174-197). Cambridge: Cambridge University Press.

Bourne, M., Melnyk, S., & Bititci, U. S. (2018). Performance measurement and management: theory and practice. International

Journal of Operations & Production Management, 38(11), 2010–2021.

Davenport, T. H. (2018). From analytics to artificial intelligence. Journal of Business Analytics, 1(2), 73–80.

Edmondson, A. & McManus, S. (2007). Methodological fit in management field research. Academy of Management Review, 32(4), 1155–1179.

Fan, H. (2020). Data Monitoring and Anomaly Analysis for Information Systems based on Balanced Scorecard and Fuzzy Neural Network. 2020 International Conference on Inventive Computation Technologies (ICICT).

Frederico, G., Garza-Reyes, J.A., Kumar, A., Kumar, V. (2020). Performance Measurement for Supply Chains in the Industry 4.0 Era: A Balanced Scorecard Approach. International Journal of Productivity and Performance Management.

Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2012). Seeking Qualitative Rigor in Inductive Research. Organizational Research Methods, 16(1), 15–31.

Grando, A. & Belvedere, V. (2008) Exploiting the balanced scorecard in the Operations Department: the Ducati Motor Holding case.

Production Planning & Control: The Management of Operations, 19:5, 495-507.

Hou, C.-K. (2016). Using the balanced scorecard in assessing the impact of BI system usage on organizational performance.

Information Development, 32(5), 1545–1569.

IBM. (2020). Client Stories. https://www.ibm.com/analytics/client-stories IBM. (2020). From Roadblock to Scale: The Global Sprint Towards AI.

http://filecache.mediaroom.com/mr5mr_ibmnews/183710/Roadblock-to-Scale-exec-summary.pdf

IDC. (2018). IDC Market Share: Worldwide Artificial Intelligence Market Shares, 2018: Steady Growth — POCs Poised to Enter Full-Blown Production (Doc #US45334719). Available at https://www.ibm.com/downloads/cas/MK85Y8V3

Kamble, S.S., Gunasekaran, A., Ghadge, A., Raut, R. (2020) A performance measurement system for industry 4.0: enabled smart manufacturing system in SMMEs - A review and empirical investigation. International Journal of Production Economics.

Kaplan, R. S., & Norton, D.P. (1992). The Balanced Scorecard—Measures that drive performance. Harvard Business Review, 71-79 Kaplan, R. S., & Norton, D. P. (1996). Using the balanced scorecard as a strategic management system. Harvard Business Review,

74(1), 75–85.

King, N., Horrocks, C., & Brooks, J. (2018). Interviews in qualitative research. SAGE Publications Limited.

Kiron, D., & Schrage, M. (2019). Strategy for and with AI. MIT Sloan Management Review.

https://sloanreview.mit.edu/article/strategy-for-and-with-ai/

Legard, R., Keegan, J., & Ward, K. (2003). In-depth interviews. Qualitative research practice: A guide for social science students and researchers, 6(1), 138-169.

Miles, M.B. & Huberman, A.M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks: Sage.

Murray, E. & Richardson, P. (2002). The critical few: First among equals as parameters of strategic effectiveness. Business Performance

Measurement, 156–173.

Neely, A. et al. (2002). Business Performance Measurement: Theory and Practice. Cambridge University Press. Parmenter, D. (2015). Key Performance Indicators (3rd ed.) [E-book]. Wiley. ISBN 978-1-119-01984-8

Patton, M. Q. (2014). Qualitative research & evaluation methods: Integrating theory and practice. Sage publications.

Petrillo, A., De Felice, F., & Zomparelli, F. (2018). Performance measurement for world-class manufacturing: a model for the Italian automotive industry. Total Quality Management & Business Excellence, 1–28.

Raffoni, A., Visani, F., Bartolini, M., Silvi, R. (2018). Business Performance Analytics: exploring the potential for Performance Management Systems. Production Planning & Control, 29(1), 51–67.

Ransbotham, S., Kiron, D., Gerbert, P., Reeves, M. (2017). Reshaping business with artificial intelligence: Closing the gap between ambition and action. MIT Sloan Management Review. https://sloanreview.mit.edu/projects/reshaping-business-with-artificial-intelligence/

Reinking, J., Arnold, V., & Sutton, S. G. (2020). Synthesizing enterprise data through digital dashboards to strategically align performance: Why do operational managers use dashboards? International Journal of Accounting Information Systems, 37, 100452. Rikhardsson, P. & Yigitbasioglu, O. (2018). Business intelligence & analytics in management accounting research: Status and future

focus. International Journal of Accounting Information Systems, Elsevier, vol. 29(C), pages 37-58.

Robert, M., Giuliani, P., & Gurau, C. (2020). Implementing industry 4.0 real-time performance management systems: the case of Schneider Electric. Production Planning & Control, 1–17.

Schläfke, M., Silvi, R., & Möller, K. (2013). A framework for business analytics in performance management. International Journal of

Productivity and Performance Management, 62(1), 110–122.

Shen, Y.-C., Chen, P.-S., & Wang, C.-H. (2016). A study of enterprise resource planning (ERP) system performance measurement using the quantitative balanced scorecard approach. Computers in Industry, 75, 127–139.

Shin, W. S., Dahlgaard, J. J., Dahlgaard-Park, S. M., & Kim, M. G. (2018). A Quality Scorecard for the era of Industry 4.0. Total Quality

Management & Business Excellence, 29(9-10), 959–976.

Van Camp, J. & Braet, J. (2016), "Taxonomizing performance measurement systems’ failures". International Journal of Productivity

and Performance Management, Vol. 65 Iss 5 pp. 672 – 693.

Walsh, C., O’Reilly, P., Gleasure, R., McAvoy, J., O’Leary, K. (2020) Understanding manager resistance to blockchain systems. European

Management Journal

Wamba-Taguimdje SL., Wamba S.F., Kamdjoug J.R.K., Wanko C.E.T. (2020) Impact of Artificial Intelligence on Firm Performance: Exploring the Mediating Effect of Process-Oriented Dynamic Capabilities. In: Agrifoglio R., Lamboglia R., Mancini D., Ricciardi F. (eds) Digital Business Transformation. Lecture Notes in Information Systems and Organisation, vol 38. Springer, Cham.

Wiraeus, D., & Creelman, J. (2019). Agile Strategy Management in the Digital Age. Springer.

Wolfswinkel, J. F., Furtmueller, E., & Wilderom, C. P. M. (2013). Using grounded theory as a method for rigorously reviewing literature.

APPENDICES

A. PARTICIPANTS LIST

PAR. CO. PARTICIPANT ROLE UNIT COMPANY SECTOR CO.

P1 C1 IT Manager of Reporting and Analytics IT Department University US

P2 C2 Chief Knowledge and Strategy Officer Digital Transformation University EG

P3 C3 Manager of Vendor Incentives Sales Electronics Retailer US

P4 C4 IT and Business System Director IT Services Housing Maintenance UK

P5 C5 Chief Engineer, TAC Automobile R&D Center Manufacturer (Automotive) JP

P6 C6 Executive Director, SRM Audit, Risk, and Control Manufacturer (Automotive) US

P7 C6 Project Manager for the implementation Audit team Manufacturer (Automotive) US

P8 C7 Senior Manager Financial Planning &Analysis Industrial Equipment Rental US

P9 C8 Head of Data & Analytics Service Help Center Aviation Services DE

P10 C9 Chief Digital Officer Asset Information Service Marine Engineering Services UK

P11 C9 Executive Director High-Voltage Marine Engineering Services UK

P12 C10 Business Analyst Business Analysis Energy Services US

Back to Methodology

B. INTERVIEW PROTOCOL

I – Introduction

Introduce theme, timing, confidentiality, recording devices, small talks. 1. What is your role and experience in the company?

2. Would you describe the main tasks you daily carry out?

3. Can you give me an overview of the performance indicators you deal with in your daily job?

II – Main Questions

4. Why did your company decide to introduce AI in your Performance Measurement?

5. How was this change received by you and your colleagues? Which were your main concerns before and after?

6. What actions did your company put in place to prepare you for the new method of work or system? (e.g. training, informative meetings, etc.). To what extent was this helpful?

7. To your knowledge, how is the new system implicated in managing key performance indicators? 8. How do you use the recommendations provided by the system to take decisions in your daily job?

a. If an indicator is forecasting bad performance results, do you try to correct it with the help of the AI or not? b. Can you choose which problems should be solved with the aid of AI or not? How do you choose?

9. How are key performance indicators reviewed and updated? Did something change from before?

10. What has significantly changed in the department activities now that you are employing AI? And what about the KPIs related to those activities?

11. How would you use to make decisions before the new system? a. Do you think the new system improves your job efficiency? Why? 12. Do you perceive to have more control or less control on the metrics? Why? 13. What are the main ease and struggle when using the new system?

III – Conclusion

C. DATA ANALYSIS

Passive change of work practices

Automation to save time and cognitive efforts - Perform real-time complex analyses (C4, C5)

- Analyze the most spending (C3, C10) - Identify trends in contracting, make predictions, and find counter-measures (C1) - Identify product issues (C5, C6)

- Identify possible solutions (C4, C8) - More quality (C5) and efficiency (C1, C6)

- Automated KPIs (C2, C3) - Automated alerts (C10) - Automated reporting (C6, C7)

- Previous expansions (C4, C7) with difficulties in meeting objectives (C4, C1)

- Issues with multiple/old systems (C3, C4, C6) - Meeting strategic objectives (C4, C6, C10) - Better decisions (C1, C2, C3, C4, C5, C7, C8)

- Prioritize employee workload (C4, C10) - Faster data retrieval (C1, C3, C4, C6, C7) resulting in more teamwork (C5)

FIRST-ORDER CODES SECOND-ORDER CODES AGGREGATE THEMES

Data-driven strategy implementation and decisions

FOSTERING CONFIDENCE AND EMPOWERMENT

PROMOTING RESOURCE-SAVING AND SYNERGIES

Cooperation among units and employees

Proactive work optimization

PASSIVE AND PROACTIVE MANAGEMENT OF METRICS Deeper understanding of metrics

- Cutting costs (C6, C8, C10) Reducing operational costs

- Personalization (C1, C10)

- Make sense of data (C4, C5, C6, C7, C9) - Training (C5, C7, C10)

D. USES OF DIGITAL BSCs

“With [the new system] providing a more comprehensive view of risks, controls, and issues across the company, we are much better positioned to meet our strategic objectives.” (P6, C6)

"Our work is helping us to transform into a social, data-driven organization, and we are

delighted with what we have achieved so far.” (P4, C4)

Better decisions “Recently, we were discussing where to locate an engine start button in our vehicles. From analyzing the NHTSA records, we found that several drivers in the US had complained that they had accidentally pressed the engine start button with their knees. This was a big surprise for us because it is only possible if you put the car into cruise control, take your feet off the pedals, and fold your legs up on the seat. We had never imagined that people would actually choose to drive like this so it was a valuable insight for our product design teams.” (P5, C5) “This has changed the way our end users access financial data and reports on the web and the iPad. And enables them to make educated business decisions.” (P8, C7)

“It gives actually better answers, so [the call center agents] feel more comfortable on the phone”. (P9, C8)

“We took the decision, a very conscious decision, to implement a business intelligence platform to enable and support all decision-makers, all stakeholders, to make better decisions

based on information.” (P2, C2)

Deeper

understanding of metrics

Personalization “I’m most impressed with things that we do from the bottom up. We can connect [the internal users in Asset Management and Operations] directly to the data that matters to them, so they can pull their own tables and create data modules. It’s an exciting way to use the analytics tools, which have been super successful.” (P12, C10)

“Every little new feature is like another brick falling from people’s walls against change. They realize: Oh, I can use this type of visualization now? I can bring in my own custom visualizations or my high charts or any of those things? Every release further empowers our end-users.” (P1, C1)

Make sense of data

“It is all about trying to aggregate some of the data we have across industries and make sense out of it. We've used [the system] to develop toolkits for offshore wind, to better understand how to maintain cables in a predictive way.” (P10, C9)

“When I walked into the room and saw the dashboard for the first time, (…) I thought: Yes, eventually we've got something that gives me an indication of the condition of those cables.” (P11, C9)

“Previously we stored information in a collection of documents and spreadsheets, which made it difficult to conduct meaningful analytics.” (P7, C6)

“It's no longer questioning: is the data or the report right? This is, this is the result. Go ahead and take action and improve your business.” (P8, C7)

Passive change of work practices Previous expansions with difficulties in meeting objectives

“As our business has grown, the demand for analytics has also increased and we see that this trend will only continue in the future. (…) To [meet growing demand from the business], we looked for a way to provide our decision-makers with faster access to the information they need.” (P4, C4)

Issues with

multiple or old systems

“We decided that we needed to upgrade to something before the whole system just crashed on us and we lost all of our data.” (P3, C3)

“In the past, we relied on manual, spreadsheet-based approaches to analytics, which were

becoming too complex and time-consuming.” (P4, C4)

“We were using four different systems to document and monitor risk, control, and audit information. As a result, we struggled to get a holistic view of the potential risks [the company] faced and the inventory of controls in place. It was also difficult to see how a risk in one area of the business might impact another division." (P6, C6)

Proactive work

optimization

Perform

real-time complex analyses

“[The system] is very good for organizing raw data into usable data-sets, so that it can be analyzed easily. It is also very easy to use for complex analyses. (…) We logged into [the system], analyzed over a million records in the NHTSA data-set, and within 10 minutes we had found three or four examples of relevant feedback from customers. This is the kind of analysis that would be almost impossible to perform manually. (…) Instead of simply analyzing the parameters that they think are important, they can use data mining techniques to uncover patterns and clues that they might never have thought about.” (P5, C5)

Analyze the

most spending

“With this implementation, the executive team was able to analyze travel by category, which helped them understand where they were spending the most money. From there, they were able to put policies in place that allowed them to optimize the travel budget.” (P12, C10) “Analytics insights help us to identify the cost drivers behind our KPIs and optimize our costs, which enhances our ability to win and retain valuable maintenance contracts. (…) We simply upload a set of unstructured data into the cloud, and the solution automatically detects hidden trends and suggests additional types of analysis we might want to perform.” (P4, C4) Identify trends in contracting, make predictions, and identify counter-measures

“Our HR and finance departments can use [the system] to discover trends among the agencies and organizations awarding the contracts. (…) The four-year graduation rate is where we need to focus. So, we’re doing predictive modeling. Some of the metrics or indicators may show us there are students who might not make that mark. So, we can take that information and offer courses to increase that four-year graduation rate.” (P1, C1) “There is also a lot of excitement around predictive modeling in every function of our business. The solution that we've implemented puts the actionable items at the forefront.” (P8, C7)

Identify product issues

“Analytic insights can help identify quality or manufacturing asset issues in a sandbox environment.” (P5, C5)

“As well as providing a macro view of all risks that we currently face, the solution also enables us to drill down and assess the specific risks in one manufacturing process or link in our supply chain.” (P6, C6)

Identify possible solutions

“And what it does actually, it looks into the, our ticket database. Was there a similar solution in the past? How was that solved? And was it solved well? And then it proposes a solution to the call center agent.” (P9, C8)

More quality

and efficiency