1
Digitalization of Performance Management
A qualitative study in a service company
Herbert-Jan Hattem
S3227715
Master Thesis
MSc Business Administration: Management Accounting and Control
University of Groningen, Faculty of Economics and Business
Supervisor
Dr. A. Bellisario
Date
January 18, 2021
Word count
(excluding references and appendix)
2
Abstract
The aim of this paper is exploring changes in the process of controlling after digitalizing performance management (PM) within service organizations. Data-collection took place by semi-structured interviews and observations during an in-depth single-case study within a small Dutch IT company. The main results show that digitalization of PM causes decision-making to be more data-driven. However, there are several aspects that explain why the situation within service organizations is more nuanced. More specifically, these organizations need to find a balance between making decisions based on intuition and based on data. This is mainly because services have a high variability, which should be considered, and services are less suitable to be quantified due to their uniqueness. Data validity and reliability seems to be challenging and important aspects when digitalizing PM within a service organization. These findings are a first step into the search towards the change in the controlling process of service organization after digitalizing PM. Future researchers are advised to conduct a multiple-case study with research organizations in different industries over a longer period. That will generate more generalizable results about the different changes during the several stages of implementing digitalized PM.
Key words: Performance measurement, Digitalization, Business Analytics, controlling, Service
3
Table of Contents
Abstract
2
1.
Introduction
4
2.
Literature review
7
2.1. Controlling Service Organizations 7 2.2. Performance Measurement and Business Analytics 9
3.
Methodology
12
3.1. Data Collection 12
3.2. Data Analysis 13
4.
Results
15
4.1. Balance Between Intuition-driven and Data-driven PM 16 4.2. Revision of Performance Measurement 21 4.3. Data Validity and Reliability within Service Organizations 23
5.
Discussion and conclusion
27
5.1. Contribution to Existing Literature 27
5.2. Managerial Contribution 28
5.3. Limitations and Future Research 29
6.
References
31
7.
Appendices
37
4
1. Introduction
Today’s world is more and more digitalising, companies over the whole world collect and analyse data to support operational and strategic decisions, this process of collecting and analysing data is called Business Analytics (BA). Globally, the volume of data that is produced by organizations is expanding by 35-50% every year. And this trend is likely to accelerate over the next 10 years (Manyika, et al., 2011), which generates an enormous volume and variety of data (McAfee & Brynjolfsson, 2012) that enables firms to make data-driven decisions. This increasing amount of data enables managers to digitize the performance management systems (PMS) within their organization (Chapman & & Kihn, 2009) and refocus their daily activities since this digitalization is expected to influence the job of managers and controllers. This is mainly because part of a manager’s job can be digitized, which gives the opportunity to focus on other important parts of the job.
PMS is found to have positive effects of the overall firm performance (Raffoni, Visani, Bartolini, & Silvi, 2018; Wiengarten, Humphreys, Cao, & McHugh, 2013) by comparing performance levels to the target values (Radnor & Barnes, 2007) and informing the organization about whether corrective action is necessary (Margretta & Stone, 2002). PM enables firms to easily access the data and base their decision on the values that indicate firm performance (Davenport, 1997). PMS is a supportive tool for managers to control and manage organizations. Due to this, the role of controllers is expected to change from collecting historic data towards analysing automatically collected data and providing advice based on this data (Stransky, Reder, Huber, & Hauer, 2019).
BA is one of the ways to digitalize performance measurement and management in an organization and it brings big changes in the organizational usage of data (Dubey & Gunasekaran, 2015; Waller & Fawcett, 2013). It is defined as ‘the techniques, technologies, systems, practices, methodologies, and applications that analyse critical business data to help an enterprise better understand the business and market and make timely business decisions’ (Chen, Chiang, & Storey, 2012). This paper will use BA as an example of digitalized PMS, because it is believed to have high potential to be used for PM purposes (Bhimani & Willcocks, 2014). Research has shown that data analysis is of equal importance as allocating the organizational resources, understanding the strategy, and defining the PMS formal aspects (Ittner & Larcker, 2005). Additionally, current market conditions demand more sophisticated control that enables them to make decisions, plan tasks and keep control over the organization (Widener, 2007). BA has been studied a lot during the last years since it is an upcoming tool for organizations which are operating in a digitalizing world.
5
two main reasons, according to previous research. Firstly, the nature of operations is different from a production process due to intangibility (Doney, Barry, & Abratt, 2007) and heterogeneity of services (Hope & Muhlemann, 1998; Fitzgerald, Johnston, Brignall, Silvestro, & Voss, 1991). Secondly, due to the uncontrollable influence of outside the organization, it is more challenging to compare performance of service organizations to targets or industry average (Grönroos, 2000; Heymann, 2018).
The goal of this study is to explore the changes in the process of controlling within service organizations after digitalizing PM. Whereas previous research has studied the impact of digitalizing PM and also the process of controlling service operations, this study aims at closing the gap in the literature. The impact of digitalizing PM within service organizations is namely currently unknown. Additionally, this study aims at understanding the aspects that make digital PM within service organizations different compared to manufacturing settings. The objective of the case study is to gain insights into the role of humans in controlling a company when their current jobs are more digitised. This study questions how BA will help them in doing their job and how they should change their activities to stay valuable for the company.
This study can add to current literature by looking at the human aspect of digitalizing organizations. Whereas other studies more looked at the effect of implementing BA on organizational performance (Brynjolfsson, Hitt, & Kim, 2011; Chen, Chiang, & Storey, 2012), the design of PMS (Ferreira & Otley, 2009), or the future of PMS (Melnyk, Bititci, Platts, Tobias, & Andersen, 2014; Mello, Leite, & Martings, 2014), this study will research how the process of controlling a service organization will change due to digitalization. These presented changes also have their own managerial interest because managers should be aware of the implications of digitalizing their PMS.
As described before, the introduction of data-driven decision making has been researched, as well as the impact of BA on firm performance (Brynjolfsson, Hitt, & Kim, 2011; Chen, Chiang, & Storey, 2012). The role of controllers and managers in their goal to control an organization will change from a job that is preparing historical data, towards a job that is more aiming at analysing automatically prepared and real-time data (Goretzki, Strauss, & Weber, 2013). The gap in the literature, about the role of humans in controlling an organization, will be addressed in this study. More specifically, since there is also a lack of research towards controlling a service company, the focus of this study will be on digitalization of performance measurement in service companies. Therefore, the following research question is formulated:
Research question: ‘How is digitalization of performance management changing the way a
service company is controlled?’
6
7
2. Literature review
Organizational performance is difficult to measure and analyse. There is an ongoing debate on whether it is a one-dimensional or two-dimensional concept, whether quantitative or qualitative measures are better and whether the data to be used should be subjective or objective (Dess & Jr., 1984; March & Sutton, 1997). Measurement of performance also has different meanings to the various stakeholders in organizations. Owners and investors want information about performance to determine whether they have invested in the right organization, managers are more interested in whether the available resources are allocated right and how the available resources can be used even more efficiently (Duursema, 1999). Previous research found that the existence of a PMS has positive effects on the overall performance of a firm (Raffoni, Visani, Bartolini, & Silvi, 2018; Wiengarten, Humphreys, Cao, & McHugh, 2013) and it is therefore valuable to study how this important element of an organization influences other aspects. PMS can be interpreted as a system that provides and integrates relevant information for decision making (Bititci, Carrie, & McDevitt, 1997), which is essential for management (Pavlov & Bourne, 2011) when quantifying efficiency and effectiveness (Neely, Richards, Mills, Platts, & Bourne, 1997).
This literature review will firstly discuss challenging aspects of controlling a service organization, followed by an analysis on these challenging aspects in light of digitalizing PMS by introducing BA.
2.1.
Controlling Service Organizations
This first section will discuss the different aspects of controlling within service organizations. It will argue why service operations are generally less suitable to be monitored by a formal control system (Abernethy & Stoelwinder, 1991; Pierce & Sweeney, 2005; Lowry, 1993). In the past, few studies looked at development and innovations in PM, but this took mainly place in manufacturing settings (Bititci, Garengo, Dörfler, & Nudurupati, 2012). PM in service companies is currently thus far less researched than PM in manufacturing settings. Due to the unique characteristics of the service sector (Anthony & Govindarajan, 2014), it is not appropriate to apply theory developed at the manufacturing industry directly to the service industry. But, since the service industry is a fast-growing sector and is seen as an important driver for the economy growth, it is important to study this sector. Services are dominant in the industrialized countries, while in the economies which are less developed the process of a growing service sector is still under way (Breitenfellner & Hildebrandt, 2006).
8
controllability is therefore also lower. Furthermore, there is also the fact that there is often a trade-off between a change in productivity and the quality of a service (Anderson, Fornell, & Rust, 1997; Singh, 2000). This increases the challenge to compare performance of different services to each other. In addition to this, the process of measuring and managing service performance is much more complex due to this trade-off.
Secondly, the degree of routine within service operations is low (Mohd Amir, 2014), which means that services are also more heterogeneous, they are likely to differ over time, per person, or per customer, this human aspect in the process makes it more difficult to control and study the service industry (Hope & Muhlemann, 1998; Fitzgerald, Johnston, Brignall, Silvestro, & Voss, 1991). Additionally, the human aspect of employees providing services also decrease the degree of routine (Mohd Amir, 2014). Mass services are an exception for this, due to the little customer contact and little customization (Mohd Amir, 2014). But professional services, where customer interaction and customization are high, have a low degree of routine, indicating that there is a high degree of risk and uncertainty. Most provided services are driven by customer interaction, and these demands vary every day and even within the day. These demand patterns also affect productivity of an organization (Heymann, 2018). Therefore, comparing actual performance with the targeted performance or performance of comparable firms, is more difficult. However, the importance of comparing results to what they should be remains important. Service organizations need standardized key indicators that represent performance accurately (Heymann, 2018). So, due to the low degree of routine of operations (Mohd Amir, 2014), PM is relatively more challenging for service organizations compared to manufacturing organizations.
Thirdly, the operations that are performed by service organizations generally have to deal with more influence from outside the organization, which cannot be controlled by the organization itself. Customers are strongly participating in the process of a service (Grönroos, 2000; Heymann, 2018), which decreases the comparability of different services. Employees of service organizations interact with customers, which increases the variability and unpredictability of service outcomes, and complicates the controllability even more (Mohd Amir, 2014). Due to variability and unpredictability of this influence from outside, it is very complex to monitor and manage this when controlling an organization.
Lastly, the complexity of the service organization is positively correlated with the difficulty to measure performance (Heymann, 2018). As stated before, PM creates a context for performance measurement to measure, but also analyses the outcomes of this measurement (Radnor & Barnes, 2007; Melnyk, Bititci, Platts, Tobias, & Andersen, 2014). Therefore, if performance is more difficult to measure, it is more challenging to analyse these results and use them for further improvement of performance.
9
2.2.
Performance Measurement and Business Analytics
After describing the challenging aspects of controlling service organization, this section will take one step further and discuss how these aspects become compromised when digitalizing PMS, by implementing BA within a service organization. According to previous research, PMS should be aligned with the corporate strategic orientation to ensure that it is a tool that can be used to effectively manage the firm (Pnevmatikoudi & Stavrinoudis, 2016). PMS should contain both formal mechanisms, processes, systems and networks and the more informal mechanisms that are also quite important (Chenhall, 2003; Malmi & Brown, 2008).
Over the past decades, firms more and more have implemented IT in their organizations. This has dramatically transformed the ways in which firms control their operations and manage their products (Sohal, Moss, & Ng, 2001). Traditionally, organizations used IT to reduce their costs, but currently IT is more and more a way to achieve a competitive advantage. The introduction of systems that manage and monitor all processes have boosted the implementation of PM in firms, due to all new possibilities in the digitizing environment (Chapman & & Kihn, 2009). PM therefore has nowadays a strong link with technological developments (Dechow & Mouritsen, 2005), since it provides easy access to the large amounts of data that are generated by all systems withing the organization (Davenport, 1997). This again enables organizations to increase their efficiency and detect room for improvements in the organizational processes (Noudoostbeni, Ismail, Jenatabadi, & Yasin, 2010).
10
One of the main questions for this study is whether this increase in knowledge also holds for service organizations, and how digitalization of PMS in service organizations changes the way of controlling service organizations. There are several aspects that should be considered when BA is implemented within an organization. This digitalization of PM also must deal with the challenging aspects discussed before and will therefore change the way of controlling within service organizations.
Firstly, without using digitalized ways of performance measurement and management, managers base their decisions on experience that they have built up during the years they were active, they recognize patterns and use their intuition to state their opinion (Mauboussin, 2012). Important decisions are mostly made by people high in the organizational hierarchy, who use their personal experience and intuition (McAfee & Brynjolfsson, 2012). After the digitalization of PM, managers no longer base their decisions on personal judgement and intuition, but they use data to base decisions on organizational facts (McAfee & Brynjolfsson, 2012). The first change therefore is a shift from intuition-driven to data-driven decision-making, but current research does not explain how this is influenced by the intangibility and heterogeneity of operations within a service organization.
Secondly, to make data-driven decisions, it is essential that data is translated from data into useful information (Manyika, et al., 2011). Therefore, organizations need people that can bridge IT and data issues with the decisions to be made (Brown, Court, & McGuire, 2020) to ensure that the raw data is analysed properly and brings the right information to make decisions (Manyika, et al., 2011). A proper analysis in this case, is an analysis that gives insights in current and future expected performance, this can then be used to make decisions that increase firm performance. This means, that managerial experience and intuition will be of less importance (McAfee & Brynjolfsson, 2012), but that managers and controllers should be more skilled in analysing the data related to current en expected future performance. But the intangibility of operations (Doney, Barry, & Abratt, 2007) and low degree of routine (Mohd Amir, 2014), which are discussed earlier, are expected to make data availability and quality more challenging within service organizations. This is an important aspect, since these elements are crucial for the implementation of BA within organizations.
11
2000; Anderson, Fornell, & Rust, 1997), which should be considered when measuring and managing performance within a service organization. Additionally, service organizations are more complex than manufacturing organizations (Heymann, 2018), which also can influence the change after digitalization of PM.
12
3. Methodology
This research aims to analyse the digitalization of PMS within service organizations, and more specifically the changing role of managers and controllers due to this digitalization. The research question asks how a certain element will change certain behaviour. So, it is not addressing whether it will change, but how it will change. This asks for more a discovery of ideas, feelings, and insights from involved people, questions of why, what, and how should be answered to get a full understanding of the phenomenon (Voss, Tsikriktsis, & Frohlich, 2002). Therefore, the nature of this research is exploratory, since the current lack of literature makes this method more suitable (Edmondson & McManus, 2007). More specifically, a single-case study will be conducted, this can improve the knowledge about changes in controlling service organizations when PMS are digitalized.
3.1.
Data Collection
The organization to study is a relatively young and growing Dutch IT company, Bloemert IT. Over the past 15 years, it has grown from a sole proprietorship towards a company with more than 50 employees and yearly revenues of approximately 5 million euros. Nowadays, this company provides a large range of services related to IT, to ensure that their large number of customers all can benefit from excellent IT solutions. Recently, the company has been split up into three different divisions and the CEO is less involved in the practical issues in the organization, there are now three division managers that are responsible for this. However, due to this shift in responsibility, the CEO has lost control over the company, and he aims at using BA to have more control over the whole organisation. Additionally, managers would like to have real-time insight in company performance, so they can make data-driven decisions, instead of making decisions solely based on intuition and experience. Since this company realised the importance of data availability quite early, they have collected a lot of data over the past years that can be used to make data-driven decisions. This is firstly data about past performance of the company. Secondly, they gather data of the clients they serve. And thirdly, they collect data about possible business opportunities. So, there is data about the past, but also data about potential future activities. Research has shown that especially companies in a rapidly changing industry should monitor the external environment (Nudurupati, Garengo, Bititci, & Sardi, 2014), this can therefore also help this IT company. Due to the availability of a lot of data, combined with good insights into their current financial position, this organisation is appropriate to digitalize performance management by introducing BA.
13
of people that are available for interviews, and this will also provide different perspectives from the three separate departments.
The data that is needed for this study mainly comes from two rounds of interviews, more informal conversations and from observations during the internship period. The main purpose of the first round of interviews, which were more informal, took place in September 2020 at the beginning of the research project, and was meant to gather data about the current situation in the company and information about the incentives of managers to digitalize PM. The interviews during the second round took place in November and December 2020 and were semi-structured to have the conversation as broad as possible and allow the participant to bring additional insights related to the theme. These meetings aimed at getting an overview of the perspectives of the interviewees on the usage and introduction of BA within the organization and their personal ideas of the changing situation within Bloemert IT as a service organization. The interview protocol of both interview rounds can be found in appendix 1 since the same protocol was used for both rounds. The questions that could not be answered during the first round were asked again during the second round. This ensures that a clear overall view of all the different respondents was generated. Table 1 shows the dates, participants, and duration of the interviews.
Table 1
Interview round Date Function of the interviewee Duration
1 August 31, 2020 CEO 60 minutes
1 September 7, 2020 Manager Division Managed Services (MS) 45 minutes
1 September 8, 2020 Financial Controller 30 minutes
1 September 9, 2020 Manager Division LT 45 minutes
1 September 10, 2020 Manager Division IT Staffing (ITS) 75 minutes
2 November 12, 2020 Manager Division LT 45 minutes
2 December 2, 2020 CEO 45 minutes
2 December 7, 2020 Manager Division Managed Services 30 minutes
2 December 7, 2020 Manager Division IT Staffing 105 minutes
2 December 8, 2020 Financial Controller 60 minutes
3.2.
Data Analysis
14
15
4. Results
Currently, not much research has been done on digitalization of PM within service organizations. While studying the situation at the research organization, by observations and interviews, the goal was to look how controlling a service organization will change after digitalization of PM. In figure 1, which is presented below, the main findings of the interviews are presented. This structural representation of the data is crucial to get a proper overview and interpret the outcomes (Gioia, Corley, & Hamilton, 2013). The first order concepts result, after axial coding them into the second order concepts, into three different dimensions, which will be discussed below.
Figure 1 Balance between intuition-driven and data-driven PM Intuïtion-driven Consequences of PM by intuition
Manager should be more involved in daily operation Link between financial and operational indicators
Data-driven At higher levels of PM, intuition becomes less raliable
Degree of involvement in daily operations
Shift in indicator type Financial indicators Operational indicators Revision of performance measurment Quantifying variability of services Services are confronted with a lot of influence from
outside the organization
Participation of customers lowers controllability of services
Measuring services Every service is unique
Productivity is less suitable to quantify
Data validity and reliability within
service organizations Trusting the data
Industry specific activities A lot of data is generated by hand
Collecting more data Impossible to measure all provided services
Organization generates lot of data
Changing time-frames Taking actions before things go wrong
16
4.1.
Balance Between Intuition-driven and Data-driven PM
The first dimension that will be discussed in this section is the balance between intuition-driven and data-driven PM. During the research period there was a lot of discussion about the possibility to manage performance completely data-driven or not. The service organization struggled to catch all their activities within numbers on one hand, and on the other hand the division managers lost control when they were less involved in the daily operations. The organizational level of PM and the type of indicator also seemed important in seeking the balance between human intuition and data-driven information. The shift to data-driven decision-making brings several consequences. Managers and controllers have to change their way of working and thinking. The individuals who were responsible for performance management at the organization were on one hand enthusiastic to control their division more data-driven, which also happened more and more. However, on the other hand they valued their own intuition and experience as important and necessary in their daily tasks. Previously, almost all decisions were taken based on intuition. Due to the involvement of managers into the daily operations, this was possible until the organizations and divisions became too large. All the interviewees came up with the statement that intuition became less valuable and trustful when the department they are responsible for became larger.
‘Due to the daily involvement in the operations, I know how the divisions perform. But since the divisions becomes larger, intuition is less reliable, and data becomes more important to manage performance.’ - Manager division MS
17
‘With the analysis of the monthly financial figures, I could manage performance very well. Until my company grew, and I was less involved in the daily operations, at that point I lost control and saw the necessity of data.’ - CEO
Additionally, during the different interviews with division managers, evidence was found that they are struggling to manage services completely data-driven. The managers realise that data gives them a lot of insights, which they can use a lot. But at the other hand, they see the shortcomings of data-driven decision making. Data will provide sufficient information about most of the daily operations, but there are always some situations which are special and cannot be included in and analysed by the data. For these special circumstances, the data should be interpreted with help of intuition and experience that comes from daily involvement in the operations. In other words, one should not blindly trust data, but keep in mind that it could be that the data does not exactly represent the real situation. This is however more often the case at lower levels in the organization, at the overarching higher level, data is less influenced by distortions.
‘Keep in mind that the data will never generate a complete detailed overview of firm performance. There will always be issues where intuition or experience is important, and if I do not have a proper view on the situation, I will meet up with the team leaders to ask for their ideas.’ – Manager Division LT
The manager in this citation means that there can always occur special circumstances where presented numbers do not represent the actual situation correctly. This can for example happen when input of the data happened not correctly. Or it can be that some nuance is necessary when a manager knows more about historic or upcoming events that are not caught in the data. This nuance obviously cannot be represented into numbers and needs human intuition, therefore.
‘I am only interested in financial figures and will contact the division managers if I see anything noticeable, they should tell me what happened at the operational side.’ – Financial Controller
18
During the interviews it seemed that people at different hierarchical levels look different at digitization of PM and the use of intuition. Where the CEO is more interested in numbers that can help him to make strategic decisions, are the team leaders more looking at indicators that help them to manage the operations that take place today or this week. This also influences the balance between intuition-driven and data-intuition-driven decision making. On one hand, PM at the lower levels is mostly more influenced by intuition, and it also better possible for these people. This mainly comes because involvement in daily operations is higher at lower managerial levels. PM at the higher levels, however, is more forced to work data-driven, they are less involved in the daily operations and dependent on periodical figures. The manager of the largest division and the CEO both explained that they will use knowledge of their subordinates to interpret the figures then.
‘I will always try to link the data to intuition from myself or people at a lower hierarchical level, because I believe that it is crucial to interpret the data correctly. […] the team leaders are involved in the daily operations and therefore more focussed on the current performance.’ – Manager Division LT
So, the way of working from managers will change after digitalization, since they have more and real-time insights in the firm performance. However, they are still looking for a proper interpretation of the numbers, which can be done by themselves of help from subordinates. The same pattern came up during the interviews with the CEO. He realizes that not being involved in the daily operations, increases the necessity of data-driven decision making, but also added new challenges when interpreting the reports with data.
‘For me as a CEO it was not a big issue to miss the details in the data, since the indicators that I use, mostly financial figures, were already known. However, since I monitor and try to support the division managers, I was also looking for more detailed insights about firm performance’ – CEO
Additionally, people at different organizational levels normally are focussed on different time periods. The CEO is mainly looking at the period on year from now, and perhaps even several years from now. Division managers are on one hand busy to manage their division today, but also for the coming months. And team leaders are focussing on today’s performance. Therefore, all these people have different views on the organization and the performance. For the short term, this has not much influence. But for the long term, where the CEO is focussing on, data cannot provide one with proper data, since the future is not measurable. Therefore, whereas the CEO is not able to use intuition when looking at current performance, he is forced to use intuition and experience when making strategic decisions.
19
Thus, PM in this service organization needs to be supported with intuitive interpretation of people who are involved in the operations. In the short term, the larger distance between the daily operations and the CEO causes that he is only able to monitor at the top level, but the details are not monitorable for him. This is however better doable by people who are involved in the operations, they can provide their supervisors with useful insights and interpretations about the data. In the long term, the CEO is more dependent on intuition and experience, to make estimations about the future. This means that within service organizations, decisions can never be taken completely based on data. At the higher hierarchical level, details are less relevant and managing performance is possible. But if details and operational aspects become more important, experience or involvement in daily operations becomes an essential aspect of managing performance.
The interviewees talked about two main types of performance indicators. On one hand, there are the financial indicators. These are the main financial figures that were already known by the company, this data mainly provides the company about performance of the firm in the past, since they represent past actions. Also, before the introduction of digitalized PM, the organizational financial performance was known and monitorable. However, these figures were mostly discussed monthly, that means that managers and the CEO had no real-time insight in the financials. Digitalized PM can therefore also improve the use of financial performance indicators. During the study period, it became clear that the financials were not the main problem when talking about digitalizing PM.
‘The monthly financial results were already known within the company. These were compared monthly to the budgeted revenue and costs. With this analysis, we could monitor our organizational performance as long as I was involved in the daily operations’ – CEO
On the other hand, there are the operational indicators which represent the performance of the daily operations within the organization. Knowledge about these numbers mainly causes the change from intuition-driven decision-making to data-driven decision-making. Previously, managers were able to indicate performance on basis of their involvement in the daily operations. But as discussed before, due to growing divisions, intuitive insights on daily operations became impossible. Therefore, insight in these operational indicators is the most important part of digitalizing PM.
‘How effective are the employees working on intern projects?’ – Manager Division ITS
20
clear by operational indicators. The link between financial performance and operational performance should be clear.
‘The goal is to monitor the daily operations. Changes in financial figures are only visible after the bad performance already occurred. We want to take a step back and prevent bad performance before this happened. Therefore, we wanted insights into operational indicators.’ – Manager Division LT
The above representation of the insights gathered from observations and interviews makes clear that managers want to translate their financial results into operational indicators. Managers want to and have to explain performance of their division. Only presenting financial figures in PM will therefore not change much in the process of controlling. The main difference and impact will be made by the operational numbers. That will increase the possibility of managers to monitor their division even better. Additionally, operational indicators will present possible bad performance in an earlier stage than financial data will do. This changes the way of working of a controller or manager in such a way that they can react earlier, instead of intuitively reacting on assumed performance issues.
21
4.2.
Revision of Performance Measurement
The second important aspect from the interviews is the discussion about performance measurement. This seems to be a challenging aspect within service organizations, which becomes even more clear when PM is digitalized. The specific nature of operations within service organizations influences the process of measuring performance. Digitalization of PM has several consequences for measurement of performance, there are two main aspects that will be discussed in this results section. Quantification of the variability of services and measurement of provided services were the two main aspects that were mentioned by the interviewees.
Whereas manufacturing operations normally have a clear relation between input and output, service operations are more influenced by factors from outside the organization or other uncontrollable influences. Productivity is therefore also relatively abstract within service organizations. Due to the interaction with customers, almost all services are unique. The relation between input and output is not clear. In addition to this, targets are more subjective within our service organization than in manufacturing settings. Performance indicators about production are therefore more challenging to measure and monitor. This is also stated by the CEO.
‘Related to the fact that we mainly earn our money by providing services, we have a less specific goal than a manufacturing organization. Productivity is therefore more difficult to measure.’ – CEO
Performance measurement of services and comparing actual performance to targeted performance was a challenging aspect of the controlling process within the research organization. This is mainly due to the fact that the relationship between input and output of the organization is less clear. PM in a service organization therefore differs completely from PM within the manufacturing industry. In addition to this, employees have to enter the data often manually. They should register the completed tasks and more details about this, this is more accessible to biases within the data.
According to the interviewees, controlling the operational side is challenging due to the variability of activities within service organizations. The research organization is a company that offers a lot of different services to their customers. The divisions all have their own expertise and services or products they offer. During the interviews, the relation between a variety in operations and controllability becomes clearer. Variety in operations can have several reasons. The variety meant here is variety between similar services, after being influenced by uncontrollable factors.
‘Due to the interaction with the customer, every provided service is unique’ – Financial Controller
22
all customers are unique and have their own ideas or demands. This uniqueness means that it is difficult to set goals for providing a service. At the higher level, one can look at the averages, but if you look more into detail, comparing services to their target is almost undoable. Additionally, services often need some explanation from the responsible employee, to interpret a particular performance indicator about the provided service.
‘It can be challenging to set goals, we should take more aspects into account, since we have to deal with influence from outside, like customer satisfaction. The ultimate goal is to generate more revenue with less employees, but it is difficult to monitor the influence of this exactly.’ – CEO
‘Customer service for example is really hard to analyse. The variety of customers, which influence the process, is so large and therefore the services cannot be compared to something like a budget.’ – Financial Controller
The above quotes explain that influence from outside cannot be fully quantified. One can of course measure whether customer satisfaction increased or decreased, but if an employee works more hours on providing a service, customer satisfaction will normally increase. However, due to this extra provided service, costs also increased. Additionally, this employee was during the extra hours not able to help other customers and generate revenue. Therefore, there are so many aspects that should be considered when setting targets for services. The influence of customers in the process of providing a service, cannot be quantified, which means that it is hard to quantify performance of services that are provided.
‘It will always be important to analyse the numbers by looking at the situation as a whole, there are often situations where some extra explanation is necessary to create a right view on performance’ – Manager Division ITS
The consequence of the fact that all services are unique is that they are less suitable to be quantified. And even if they are quantified, they will often ask for explanation. This is the point where intuition and experience of managers and controllers comes again. People who are involved in the daily operations can add their experiences to the operational indicators. So, controlling service organizations in this case will change by trusting fully on intuition, to trusting fully on data combined with personal intuition or experience to explain the indicators.
23
‘The divisions that are fully focussed on providing services are hard to control, the process from input to output is so unclear. It is therefore hard to explain financial distortions.’ – Financial Controller
The quote explains this by comparing the different divisions to each other. The divisions all have their own specific processes and therefore their own activities to transform the inputs in as much output as possible. The divisions where the activities are less complex can be better monitored from a little distance, whereas the most complex division asks for a lot of involvement in the daily operation and thereby a good interpretation of the financial figures.
‘There is one division which is mostly specialized in providing customers will workplaces in their offices. The relation between the input and output is much clearer. This ensures that I am much more able to follow the activities within this division and the financial figures therefore need less explanation on basis of operational numbers.’ – Financial Controller
So, there are three challenging aspects for measurement of performance when digitalizing PM within service organizations. Evidence was found that the specific nature of operations influences the way of controlling when PM is digitalized. Firstly, measurement of services seems to be different from measuring manufacturing operations. The unclear relationship between input and output makes the process of digitalizing more challenging within these organizations. One of the changes for managers is then that they must set targets that take into account that every service is unique. Secondly, service organizations must quantify the variability of services. Services have a high variability and need therefore special attention when being quantified. This variety mainly comes from uncontrollable influence outside the organization. Therefore, digitalization of PM changes controlling in such a way that variable services must be quantified, whereas this was previously not necessary, when performance management mainly was based on financial figures. Quantification of the variability is important since this strongly influences performance of the provided services. Lastly, a large part of the data about performed services has to be entered manually, which can easily cause errors. This problem must be solved within service organizations when PM is digitalized. Organizations should take action to ensure that their data quality is high, despite this manual entrance by employees.
4.3.
Data Validity and Reliability within Service Organizations
The third and last dimension to be discussed in this findings section is related the data. To make data-driven decisions, one should beforehand ensure that the data is available within the organization. Additionally, the available data should also be valid and appropriate for proper analysis. The next sections will discuss these aspects and the change on controlling a service organization.
24
within the organization. The financial indicators normally do not cause that much discussion within the research organization. The operational indicators, however, do cause discussion. Firstly, it is always the question whether the data input of every employee is equal. Since everyone has to enter his or her own information about provided services into the project management system, it can always cause biases when not everyone uses the same criteria for this. Secondly, mangers sometimes doubt the results that come out of the datasets, they do not take for granted that the numbers are truth and discuss them.
‘The data sometimes needs extra explanation. For example, since we do secondment of IT personnel, I normally look for several sales opportunities if someone needs another project. If then one of the opportunities succeeds, the other opportunities obviously fail, but that does not mean that I performed badly. These failures are caused by another success’ – Manager Division ITS
The quote above shows one example of a situation where the manager not fully trusts the data. This can be the case when a particular division is active in a special industry, which requires special treatment when collecting and entering data. Thirdly, managers should realize that their intuition does not always represent the truth. During the research period, several times discussion came up about whether the presented numbers were valid or not. But this is something managers have to change within their mind.
Data availability is one of the most important aspects when digitalizing PM. Important during the interviews were question about whether the necessary data was available. The research organization collects a lot of data about the services they provide to their customers. Financial data is of course available since firms are obliged to deliver them yearly to the Chamber of Commerce.
‘We mainly monitor the drivers of our revenue. That means the quantity of subscriptions, the turnover made by implementing software at our customers, all worked hours, and important information per customer.’ – Manager Division LT
So, additionally to the financial figures, there is also a lot of operational data collected and stored in the datasets. This service organization therefore properly collects and stores data that can be used for BA. The above two aspects, availability, and validity of data, are necessary conditions for a company to digitalize their PM. When these are in place, organizations can implement a technique like BA, according to the interviewees. During the conversations, the main point of interest was the change in the process of controlling. This will be discussed in the following section, therefore.
25
due to the digitization of PM, the presentation of real-time figures about the performance, managers are better able to monitor their division and the CEO is better able to manage the situation. Data provides them with real-time insights in performance and business opportunities,
‘From the moment our organization became larger and I am less involved in the daily operations, managing performance based on intuition and experience is more challenging for me. The real performance was unknown within the organization.’ – CEO
As stated by the CEO above, data-driven decision-making, and performance managing was necessary to manage this growing organization. Only trusting on intuition was inappropriate, since the activities were more diverse, and the number of activities increased enormously.
Secondly, as discussed in previous sections, intuition and personal experience is less useful when one is not involved in the daily operations. The decisions that must be made will be based more on data and less on intuition. This is of course a well-known phenomenon, but it is important te realize in this discussion about changing controlling processes within service organizations.
‘By using BA, decision-making will be based less on my intuition on basis of daily involvement within the operations, but more on basis on real numbers’ – Manager Division MS
The quote above makes it clearer that the managers change the basis of their decision-making. The main question that arises, is whether they are willing to and able to trust completely on their data. As discussed earlier, due to the nature of operations and intangibility of activities, performance is difficult to quantify. This is especially for service organizations a major problem when digitalizing PM. The collected and presented data of business activities often asks for extra explanation. This is also recognized by the manager who is cited below.
‘Since providing a service is not equal to a normal manufacturing process, we think that it is important to stay connected to the daily operations. One cannot interpret the daily figures without being involved in the daily processes.’ – Manager Division LT
Thirdly, the timeframe of people who are managing performance is changing. Previously, when the managers only monitored their performance with help of monthly financial figures, they noticed bad performance at the moment when the bad performing activities already took place. However, with digitized PM, performance indicators are real-time available, and manager can intervene immediately if some elements threaten to perform below budget.
‘The main change is that I will be able to see changes in an earlier stage, and not at the moment in time when the financial figures represent bad performance, this is too late.’ – Manager Division LT
26
Entrepreneurship and management always asks for timely decisions, actions that are in line with the strategic orientation, and these can only be made if a detailed overview of current performance is available.
‘We are also better able to monitor whether we will reach our strategic targets. This can be monitored throughout the financial periods’ – Manager Division LT
Conceptually, managers do benefit from the higher amount of data within the organization. They can manage performance more accurately and have real-time insights into the daily business. This also brings several changes with it, which are more specific for service organizations. The first change in the process of controlling service organizations is that managers have to trust the data that is generated within the organization. Since this data is mainly about the provided services, a lot of data is generated by hand, service organizations therefore have to make sure that their data is generated accurately and according to developed standards. The second change has to do with data collection. Even though also service organization generate a lot of data, it is impossible to measure and count every provided service. Managers and controllers therefore must increase the amount of data that is being collected. The last change is that the process of controlling will focus on another timeframe than before. This means firstly that actions will be taken earlier, digitalized PM will earlier show aspects within the organization that are performing badly. Secondly, performance can be monitored more frequently, which also means that corrective actions can be taken earlier. As a third change, the timeframe of controlling will be longer, which means that the organization is better able to monitor whether long-term strategic goals will be achieved or not.
27
5. Discussion and conclusion
The aim of this research project is exploring changes in the process of controlling within service organizations after digitalizing PM. A Dutch IT company was used to conduct a single case study. Data was collected and analysed, and it seemed that the process of digitalizing PM has specific consequences within service organizations. There were three main outcomes out of this study that explained how and why the process of controlling changes after implementation of digitalized PM in service organizations. These elements are all answering the following research question: ‘How is digitalization of performance
management changing the way a service company is controlled?’ The previously discussed findings
show that digitalization of PMS within service organizations seriously changes the process of controlling. The next section will discuss the findings conceptually in the light of the currently existing knowledge from previous research. Afterwards, the managerial contribution of this study will be explained, followed by a look at the limitations and recommendations for future research.
5.1.
Contribution to Existing Literature
Firstly, existing literature argues that digitalization of PM causes a shift from intuitive to data-driven decision-making (McAfee & Brynjolfsson, 2012; Holsapple, Lee-Post, & Pakath, 2014). Without digitalized PM, managers base their decisions on experience and intuition, which help them to recognize patterns of performance (Mauboussin, 2012). This study confirms this statement, but also adds some nuance for service organizations by looking at how intuitive and data-driven decision-making are balancing within service organizations. The finding of this study that data-driven decision-making becomes essential when the organization becomes more complex matches with the findings of previous research (Davenport, 1997; Bhimani & Willcocks, 2014). This study contributes to current literature by arguing that involvement in daily operations will stay important to manage performance. Whereas previous studies present a shift from intuition-driven to data-driven, this study found evidence that this shift is more nuanced and less obvious for service organizations. Data from service operations should namely be interpreted by people who are involved in the operations and are therefore able to intuitively analyse the data. Additionally, this shift is more obvious for people at lower hierarchical levels since they are more involved in the operations and more focussing at the short term. For PM at the long term, at the level of a CEO, intuitive decision-making will stay important, because these individuals are less involved in the daily operations and because the future is not measurable in the service industry. So, the shift from intuitive to data-driven decision-making is more nuanced within service operations than it is within manufacturing settings.
28
1998; Fitzgerald, Johnston, Brignall, Silvestro, & Voss, 1991), which makes the process of controlling more challenging. This study confirms this variability of services, but also contributes to the current literature by the following points. Manual entrance of the data seems to be a potential aspect to cause errors, service operations have an unclear relationship between input and output, and the variability of services also seems to influence the measurability. These are three different aspects that complement previous studies about digitalization of PM within the service industry.
Lastly, previous studies mention that BA will bring understanding of the historical, current, and future expected situation (Raffoni, Visani, Bartolini, & Silvi, 2018) by transforming unstructured data into structured dashboards that are ready to be used in analysis of firm performance (Chen, Chiang, & Storey, 2012; Bughin, Livingston, & Marwaha, 2011). This perfect situation does not hold for service organizations, according to our research. Whereas previous studies barely discusses data validity and reliability within service organizations, this study found evidence that these are fundamental aspects when studying digitalization of PM. Managers should trust more on the provided data and they should be even more focussing on collecting the right and interesting data. BA within service organizations therefore also will bring more understanding by transforming the unstructured data into dashboards, but there are several challenges when talking about data generation, data collection, and the specificness and uniqueness of services within this industry. Thus, the introduction of digitalized PM in service organizations will bring advantages similar to the advantages in manufacturing organizations, but the process to do so is more abstract and challenging. Existing literature found that the role of controllers will change from preparing historical data, towards a job where the analysis of digitalized presented performance indicators is more important. This is in line with the conclusion that the timeframe of controlling will change. Managers will be able to detect possible bad performance earlier, which enables them to intervene at an earlier stage. So, this study also complements current literature by arguing that validity and reliability of data are significant within service organizations to successfully profit from all the benefits of digitalized PM.
5.2.
Managerial Contribution
The managerial interest of this study can be divided into several aspects. Firstly, the world is more and more digitalizing, and this has consequences for PM. Over time, managers will be more and more forced to use digitalized PM. Or they will decide by themselves to digitalize these processes. However, it is important to weigh the pros and cons and to consider the potential consequences. Therefore, this paper explains how PM in service organizations will change due to digitalization. This study found that managers within these organizations will always need their intuition, indicating that it is important to be partly involved in the daily operations, to be able to interpret the data.
29
and systems in such a way that they will provide enough data to be used for PM. Quantifying services is often more difficult than quantifying manufacturing activities. Managers should realize that their services are influenced by several factors that increase the difficulty to measure them accurately. So, it is important for managers to deal with the issues that can occur within the organization when implementing digitalized PM.
Lastly, data-driven PM asks for a change within an organization. Managers should make sure that data generation and performance measurement are well organized within their organization, so that they can trust the presented data. Employees should be encouraged to list all their provided services in a correct way. And the managers should adopt their timeframe. They should be encouraged to use the data in such a way that they can take their decisions at an earlier stage. Normally, decisions based on data will be more accurately than decisions based on intuition. In the first case, decisions are mostly made at an earlier stage, whereas decisions based on data are mostly reactions on bad performance which is currently happening. Therefore, it is important for managers to realize that digitalized PM will allow them to change their timeframe, to take decisions earlier, which is beneficial for performance of the organization.
5.3.
Limitations and Future Research
Like all other studies, this study has limitations that should be considered. However, these limitations also generate opportunities for future research. The two main limitations will be discussed, both accompanied with a suggestion for future research.
Firstly, even though the generalizability of this research was attempted to be high by selecting a proper research organization, the findings are still context specific. This is however always the case, especially within research project based on single case studies. The conclusions that come from this IT organization, cannot be copied one-to-one to other organizations in different situations or industries. Part of the results is only applicable if a service organization is active in the IT industry. However, most of the conclusions that were made are theoretically relevant, also for situations in other industries. To overcome this problem, future researchers are advised to conduct multiple-case study with research organization in different sectors or industries. This will result in a more generalizable view on the impact of digitalized PM on the way of controlling service organizations, which then will further complement current knowledge.
30
31
6. References
Abernethy, M., & Stoelwinder, J. (1991). Budget use, task uncertainty, system goal orientation and subunit performance: a test of the fit hypothesis in not-for-profit hospitals. Accounting,
Organizations and Society, Vol. 15 No. 4, pp. 105-171.
Anderson, E. W., Fornell, C., & Rust, R. T. (1997). Customer Satisfaction, Productivity, and Profitability: Differences between Goods and Services. Marketing Science, 16: 129-145. Anthony, R., & Govindarajan, V. (2014). Management Control Systems. Mcgraw-Hill Education -
Europe.
Berman, S. (2012). Digital transformation: opportunities to create new business models. . Strategy &
Leadership, 40(2): 16-24.
Bhimani, A., & Willcocks, L. (2014). Digitisation, ‘Big Data’ and the transformation of accounting
information.
Bititci, U., Carrie, A., & McDevitt, L. (1997). Integrated performance measurement systems: a development guide. International Journal of Operations & Production Managemetn, Vol. 17, No. 5: 522-34.
Bititci, U., Garengo, P., Dörfler, V., & Nudurupati, S. (2012). Performance measurement: challenges for tomorrow. International Journal of Management Reviews, Vol. 14, No. 3: 305-327.
Bose, R. (2009). Advanced Analytics: Opportunities and challenges. Industrial Management & Data
Systems, 109(2): 155-172.
Bourne, M. C., Neely, A. D., Mills, J. F., & Platts, K. W. (2003). Implementing performance measurement systems: a literature review. International Journal of Business Performance
Management, Vol. 5, No. 1, 1-24.
Breitenfellner, A., & Hildebrandt, A. (2006). High employment with low productivity? The service sector as a determinant of economic development. Monetary Policy and the Economy, Vol. Q1/06, pp. 110-135.
Brown, B., Court, D., & McGuire, T. (2020, 09 23). Views from the Front Lines of the Data-analytics
Revolution. Retrieved from McKinsey Digital:
https://www.mckinsey.com/business- functions/mckinsey-digital/our-insights/views-from-the-front-lines-of-the-data-analytics-revolution#
Brynjolfsson, E., Hitt, L. M., & Kim, H. H. (2011). Strength in numbers: How does data-driven decisionmaking affect firm performance? Available at SSRN 1819486.
Bughin, J., Livingston, J., & Marwaha, S. (2011). Seizing the potential of "Big Data". McKinsey
Quarterly, October: 103-109.
32
Chen, H., Chiang, R. H., & Storey, V. C. (2012). Business Intelligence and Analytics: From Big Data to Big Impact. Mis Quarterly 36(4), 1165–88.
Chenhall, R. (2003). Management control systems design within its organizational context: findings from contingency-based research and directions for the future. Accounting, Organizations and
Society, 28: 127-168.
Choi, T., Wallace, S., & Wang, Y. (2017). Big Data in Operations Management. Production and
Operations Management, 27(10): 1868-1883.
Choong, K. K. (2013). Understanding the features of performance measurement system: A literature review. Measuring Business Excellence, 17(4): 102-121.
Davenport, T. (1997). Information ecology: Mastering the information and knowledge environment. New York: Oxford University Press.
Dechow, N., & Mouritsen, J. (2005). Enterprise resource planning systems, management control and the quest for integration. Accounting, Organizations and Society, 691-733.
Dess, G. G., & Jr., R. B. (1984). Measuring organizational performance in the absence of objective measures: The case of the privately‐held firm and conglomerate business unit. Strategic
Management Journal, 5(3), 265-273.
Doney, P. M., Barry, J. M., & Abratt, R. (2007). Trust determinants and outcomes. European Journal
of Marketing, Vol. 41 No. 9/10.
Dubey, R., & Gunasekaran, A. (2015). Education and training for successful career in Big Data and Business Analytics. Industrial and Commercial Training, 47(4): 174-181.
Duursema, N. (1999). Balanced scorecard performance measurement for cost engineering. AACE
International Transactions, C81.
Edmondson, A. C., & McManus, S. E. (2007). Methodological fit in management field research.
Acedemy of management review, 32 (4): 1246-1264.
Ferreira, A., & Otley, D. (2009). The design and use of performance management systems: An extended framework for analysis. Management Accounting Research, 20: 263-282.
Fitzgerald, L., Johnston, R., Brignall, T. J., Silvestro, R., & Voss, C. (1991). Performance measurement in service businesses. CIMA Publishing, London.
Fosso Wamba, S., Akter, S., Edwards, A., Chopin, G., & Gnanzou, D. (2015). How ‘big data’ can make big impact: Findings from a systematic review and a longitudinal case study. . International
Journal of Production Economics, 165, 234-246.
Franco-Santos, M., Lucianetti, L., & Bourne, M. (2012). Contemporary performance measurement systems: A review of their consequences and a framework for research. Management
accounting research, 23(2), 79-119.
33
Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2013). Seeking Qualitative Rigor in Inductive Research: Notes on teh Gioia Methodology. Organizational Reserach Methods, 16(1): 15-31.
Goretzki, L., Strauss, E., & Weber, J. (2013). An institutional perspective on the changes in management accountants’ professional role. Management Accounting Research, 24(1), 41-63.
Griffin, P. A., & Wright, A. M. (2015). Commentaries on big data's importance for accounting and auditing. Accounting Horizon, 29(2), 377–379.
Grönroos, C. (2000). “Service Reflections: Service Marketing Comes of Age. Handboek of Service
Marketing and Management, 13-16.
Heshmati, A. (2003). Productivity Growth, Efficiency and Outsourcing in Manufacturing and Service Industries. Journal of Economic Surveys, 17: 79-112.
Heymann, M. (2018). How the service industry can corral big data. Global Business and Organizational
Excellence, 37(5): 39–46.
Holsapple, C., Lee-Post, A., & Pakath, R. (2014). A unified foundation for business analytics. Decision
Support Systems, 64: 130-141.
Hope, C., & Muhlemann, A. (1998). Services Operations Management-Strategy, Design and Delivery.
International Journal of Operations & Production Management, 18(5) 525-526.
Ittner, C. D., & Larcker, D. F. (2005). Moving from strategic measurement to strategic data analysis. In
Controlling strategy (p. 86).
Ittner, C., Larcker, D., & Randall, T. (2003). Performance implications of strategic performance measurement in financial service firms. Accounting, Organizations and Society, Vol. 28 Nos 7/8, pp. 715-741.
Janin, F. (2017). When being a partner means more: The external role of football club management accountants. Management Accounting Research, 5-19.
Johnston, R., & Jones, P. (2004). Service Productivity: Towards Understanding the Relationship Between Operational and Customer Productivity. International Journal of Productivity and
Performance Management, 53: 33-41.
Kennerly, M., & Neely, A. (2003). Measuring performance in a changing business environment.
International Journal of Operations and Production Management, 23(2): 213-229.
Klatt, T., Schläfke, M., & Möller., a. K. (2011). Integrating Business Analytics into Strategic Planning for Better Performance. Journal of Business Strategy, 30-39.
Kotler, P. (1991). Organization Effectiveness Measurement and Policy Research. Academy of
Management Review, 347-355.
Lebas, M. J. (1995). Performance measurement and performance management. International Journal of
Production Economics, 41(1-3): 23-35.
