A Performance Measurement Framework for
the Electricity Generation Sector
---
Master Thesis for Technology Management First Supervisor: Prof. dr. I.F.A. Vis Second Supervisor: Dr. ir. W. Klingenberg
Date: August 2012
Preface
This research is executed for the study Technology Management at the University of Groningen, in the form of a Master Thesis. Completing this thesis has been a big but interesting challenge. I gained a lot of new qualities, gathered new knowledge and learned a lot about myself.
I would like to thank everyone who believed in me and who supported me to achieve this result. I would like to thank my university supervisor, Prof. dr. Iris F.A. Vis for her guidance, useful comments and her time, the consulted experts for their time and their help and all the people who have helped me for their sincere interest, which gave me the strength and motivation to write this thesis. Thank you all.
Management Summary
During the last years the electricity generation sector has changed considerably, due to liberalizations and merging markets. This has led to a changing field of competition, which resulted in a bigger need for Performance Measurement to see how the organisation is performing.
A rich amount of Performance Measurement Frameworks has been created, which has resulted in an unmanageable diversity. This presents a significant problem since there appears to be no agreement as to which are the important themes and theories in the field, which also makes it almost impossible to build upon the work of others. In the search for Performance Measurement Frameworks, no literature could be found about a framework specific for the electricity generation sector. Therefore this research will fill this gap and design a Performance Measurement Framework (PMF), specifically for the electricity generation sector.
The focus of the research is on power plants, which are, or will be, in operation. Performance improvement of operational power plants is the most cost-effective way to increase the energy producing capabilities of a power plant while improving the overall energy efficiency of the industry and producing substantial environmental benefits. The outcome of this research will help managers in power plants to create a Performance Measurement System (PMS) for their organization.
The main research question “What should a PMF look like for the electricity generation sector?” will be answered by combining the theory on PMFs and the characteristics of the electricity generation sector.
The theory from the field of Performance Measurement, which is provided, gives a solid basis for creating a PMS. It indicates what characteristics a good performance measure should have and what details should be taken into account, so it becomes clear exactly what is done, why this is done, what action is required, by whom, etc.
From this theoretical basis a framework is created with the help of several experts, from the different fields of research. The experts have been able to react on each other’s views and via an iterative process; the final PMF is created on these foundations.
Figure 1 The Balanced Scorecard for the electricity generation sector.
Index
Table of content
Preface... 2 Management Summary ... 4 Index... 8 1 - Introduction... 122 - The electricity generation sector ... 22
3 - Performance Measurement Frameworks ... 28
4 - Rigor Cycle, The selection of a base PMF for the electricity generation sector... 44
5 - Relevance Cycle, The selection of the indicators for the electricity generation sector .. 52
6 - Conclusion ... 64
Appendix 1 - Explanation of the framework indicators ... 78
Appendix 2 - Availability indicators (Source: VGB PowerTech, 2011) ... 104
Appendix 3 - Reliability and dispatchability indicators (Source: VGB PowerTech, 2011) ... 106
List of tables
Table 1 PMS evolution (Source: De Toni and Tonchia, 2001)... 34
Table 2 Overview of the PMFs. ... 40
Table 3 Instructions with regard to the design of performance measures. (Source: Neely et. al. 1997)... 41
Table 4 The Performance Measure record sheet. (Source: Neely et. al. 1997)... 41
List of figures
Figure 1 The Balanced Scorecard for the electricity generation sector. ... 6Figure 2 Conceptual model (Source: Hevner, 2007)... 17
Figure 3 Outline of the research... 20
Figure 4 The electricity supply chain before and after liberalisation. (Source: Energy Delta Institute, Energy Business School. 2010)... 23
Figure 5 How is electricity generated? (Source: Zactruba, J. 2009. Bright Hub.com) ... 24
Figure 6 Optimization of a power plant. ... 26
Figure 7 The closed loop deployment and feedback system for the PM process (Bititci, et. al. 1997)... 29
Figure 8 Porter’s five forces model. (Source: Porter, 1990) ... 30
Figure 9 The organization’s purpose & vision framework. (Source: Casey et.al.2008)... 32
Figure 10 The Balanced Scorecard. (Source: Kaplan and Norton, 1996)... 35
Figure 11 The Performance Measurement Matrix. (Source: Keegan et. al. 1989). ... 36
Figure 12 The Results and Determinants Framework. (Source: Fritzgerald et. al. 1991). ... 36
Figure 13 The Performance Pyramid. (Source: Lynch and Cross, 1991). ... 37
Figure 14 The Business Excellence Framework. (Source: Neely et. al. 2000)... 38
Figure 15 The Performance Prism. (Source: Neely et. al. 2002). ... 39
Figure 16 Keeping the PMS up-to-date and successful. (Source: Neely et. al. 2000). ... 42
Figure 17 Performance Management Framework©. (Source: Chris Rodgers Consulting Ltd. 2011)... 47
Figure 18 The Balanced Scorecard for commercial companies (Source: Kaplan and Norton, 2004)... 53
Figure 19 Initial Balanced Scorecard for the electricity generation sector. ... 56
Figure 21 Availability from the view of power plant operators and dispatchers and in time- and energy availability. (Source: VGB PowerTech, 2011) ... 81 Figure 22 Types of unavailability. (Source: VGB PowerTech, 2008)... 82 Figure 23 The electricity merit order and the effect of renewable energy sources on the merit
order. (Source: UCPartners / Energy Delta Institute, 2010) ... 84 Figure 24 Example of energetical efficiency between a seperate heat boiler and a power plant
vs. a CHP installation. (Source: Essent 2011)... 87 Figure 25 Safety matrix... 89 Figure 26 Architecture of the Risk-Based Maintenance methodology. (Source: Krishnasamy
et. al. 2005)... 92 Figure 27 An example of a classification of power plant subsystems and components.
(Source: Krishnasamy et. al. 2005). ... 93 Figure 28 Power / Interest Matrix for stakeholder prioritization. (Source: Gardner et. al.
1 - Introduction
In this chapter an introduction will we given of this research in which the need for a sector specific Performance Measurement Framework (PMF) will be addressed, the scope of the research and the research question will be described as well as the methodology and the outline of the research.
The chapter consists of the following paragraphs: - Background and literature gaps
- Scope of the research - Research question - Methodology
- Roles of the consulted experts - Outline of the research
- Conclusion.
1.1) Background and literature gaps
Since the last decade the energy market has been moving a lot. The big changes started with the liberalisation of the energy market and the forced separation of the power generation and the distribution network. Before the separation of the production and distribution there were a few big companies, which had an oligopoly on the entire supply chain. Because of the separation a lot of smaller companies were created, opportunities for incumbents arose and a new perspective on competition was necessary.
In the last few years the energy market is becoming more and more international (NMA, 2010). There were a lot of takeovers and merges. Next too these takeovers and merges the regulations and possibilities for cross-border trade became more easy and cheaper.
differences and market imperfections have ensured that the Dark-spread and Spark-spread (margins for coal-fired and gas-fired plants) dropped significantly. It is expected that the margins for electricity are dropping even more in the coming years (Bökkerink and Schiffers, 2011).
It is clear that because of these decreasing margins the electric power generation sector is in rough weather and the prospects are bad. (Bökkerink and Schiffers, 2011) According to the marketing perspective, organisations achieve their goals by satisfying their customers with greater efficiency and effectiveness then their competitors (Kotler, 1984). For this reason it is becoming increasingly important to manage the production of electricity in an effective and an efficient way.
Effectiveness refers to the extent to which customer requirements are met, while efficiency is a measure of how economically the firm’s resources are utilized when providing a given level of customer satisfaction (Neely, et. al. 2005). Especially for organisations in rough weather it is important to improve the effectiveness and efficiency of the organisation to stay in competition (Besanko, et. al. 2003). Therefore the effectiveness and efficiency needs to be controlled and improved.
Harrington stated the importance of performance measurement by saying; "Measurement is the first step that leads to control and eventually to improvement. If you can't measure something, you can't understand it. If you can't understand it, you can't control it. If you can't control it, you can't improve it" (Harrington, 1991). For this reason it is important to have data about the performance of the power plants.
To make sure a company systematically measures the performance, a Performance Measurement System (PMS) is needed (Bourne, Et. Al. 2000; Fitzgerald and Moon, 1996; Harrington, 1991; Kaplan and Norton, 2000; Neely, 1999; De Toni and Tonchia, 2001). For the definition of a PMS this report will use the definition from Neely:
“A performance measurement system can be defined as the set of metrics used to quantify both the efficiency and effectiveness of actions” (Neely, 1994).
In order to create a PMS, a PMF is needed to provide guidelines for what a PMS should look like. The motivation for this research is based on the believe that a PMF for the electricity generation sector will help managers in the sector to create their own PMS. This sector specific PMF should save managers in the sector time and effort in searching the web for the information and it should decrease the chance that important items will be forgotten or the wrong metrics will be used.
In academic research the topic of performance measurement is researched by people with a wide variety of different functional backgrounds, like accounting, economics, human resource management, marketing, operations management, psychology, sociology. Many PMFs were created last years, which can be used as a basis to design a PMS. Some of these frameworks are for broad use in different industries, some specifically designed for one industry. Each of these frameworks has their own characteristics. Most researchers are doing research independently and discuss the topic mainly in their own field of business. Therefore a rich amount of PMFs are created, which turned out to be an unmanageable diversity (Neely, 2007). Neely (2007) states this gives a significant problem since there appears to be no agreement as to which are the important themes and theories in the field and this makes it almost impossible to build upon the work of others (Neely, 2007). Moreover, companies and organizations don’t want to create a PMS from scratch and they are interested in improving their current systems (Medori and Steeple, 2000).
Performance measurement however is an upcoming topic for the sector. Research platforms for the power generation sector are doing research on performance indicators for the electricity generation sector. “Definition of new performance indicators for the assessment of the operating performance of power plants on the liberalized power market”, is now a current topic within VGB PowerTech, a neutral research platform for the power generation sector (VGB PowerTech, 2011).
1.2) Scope of the research
Building a power plant is very capital intensive. A power plant will have a lifespan of about 30 or 40 years. Building a plant is an important part of a company’s strategy, since this should secure future cash flows. Therefore company’s need to take a look into the future and consider different aspects, like legislation, market regulation, current and future customer demands, future fuel prices, etc. This aspect of business development is very important for the positioning of the company in a future market. This positioning for a future market and the portfolio management within the trading department are fields of business of their own. If the choice of building a power plant is made and the plant is operational, it needs to be operated for its entire lifetime, to earn back the huge investments. Performance improvement of operational power plants is the most cost-effective way to increase the energy producing capabilities of a utility while improving the overall energy efficiency of the industry and producing substantial environmental benefits (Theis, 2004). Therefore this research will focus on the important performance aspects for power plants, which are, or will be in operation. This means that investments in new power plants or big investments in modifications for future benefits will be out of the scope of this research.
Since all plants have different characteristics, are a part of different portfolios, etc. the research will focus on creating a PMF for the electricity generation sector as a whole and not on creating a PMS and creating metrics for one company or power plant.
In the electric power generation sector a lot has been changed last years. Because of the liberalisation it became increasingly more important to work in an effective and efficient way, but it is unclear which measurements and what KPIs are needed to manage the organisation to improve performance.
This research intends to deliver a PMF for organisations in the electricity generation sector. This is done, to provide the organizations with the most important performance indicators to enable organizations to create their own PMS to manage the performance and to become effective and efficient operating organizations.
The research question is:
What should a PMF look like for the electricity generation sector? The sub questions are:
- What are the important aspects of the electricity generation sector?
- What types of PMFs are described in literature and what characteristics do they have?
- Which PMF can best be used as a basis for the design of a specific PMF for the electricity generation sector?
- What indicators are needed to make the PMF fit the electricity generation sector and how are the indicators interrelated?
1.4) Research methodology
people-, organizational-, technological aspects (Hevner, 2004). With the combination of these two a sector specific PMF can be designed.
Hevner (2007) created a Design Science Research methodology, which has a special focus on the design of Information Systems. Since this research is about the design of a framework for the creation of a PMS, which is an Information System, this methodology suits this research. The conceptual model of the methodology can be seen in figure 2.
Figure 2 Conceptual model (Source: Hevner, 2007)
According to Fens (2012) this methodology suits the topic of the research very well, since it gives sufficient attention to both the design/architecture of the framework and the specific sector characteristics. Therefore it is chosen to use this methodology in this research.
In the conceptual model (figure X) can be seen that there are 3 fields which are discussed, namely the Environmental field, the Knowledge Base field and the Design Science Research field in which the other fields come together. In each field one of sub questions of the research questions will be answered.
Environmental Section
on the first sub question of the research, ‘What are the important aspects of the electricity generation sector?’
Knowledge Base Section
In the Knowledge Base Section, research is done on performance management frameworks. The different approaches are discussed; the most important PMFs are evaluated on their characteristics and the design criteria. This is done by the means of a literature review. An answer will be given on the second sub question of the research, ‘What types of PMFs are described in literature and what characteristics do they have?’
Design Science Research Section
In the section of the Design Science Research, the Environmental Section and the Knowledge Base Section come together. This will be done by the means of two cycles, namely the Rigor Cycle and the Relevance Cycle. The final PMF for the sector will be designed via iterative processes of these two cycles. In the Rigor Cycle one of the frameworks, discussed in the Knowledge Base Section, is chosen as a basis for the sector specific framework and adjusted to fit the electricity generation sector. Via the Relevance Cycle the sector specific characteristics from the Environmental section will be implemented in the design. Via these iterative processes the aspects of both fields of knowledge will come together and influence each other. The framework, which is designed in this section, will be created with the help of experts from the field of the electricity generation sector as well as experts from a performance management and / or performance measurement background.
Based on the outcome of the literature study and the author’s experience in Performance Measurement in the electricity generation sector an initial design is created, which is submitted to the experts for feedback. Both the validity of the choice of the framework as well as the choice of the perspectives and indicators are thoroughly questioned and revised when necessary.
1.5) Experts
As mentioned before, several experts will be used in this research. Because of the different characteristics of the research fields, several expert roles need to be fulfilled to provide a solid expertise in the different fields of experience. The experts used for the Environmental part need to provide objective information about characteristics of the electricity sector and performance measurement models. To secure the objectivity of the experts in the creation of this framework the experts will need to match the following profiles:
- The chosen experts will have at least 10 years of experience in sr. functions in the electricity sector and a background in Performance Measurement in the electricity sector. - The chosen experts have at least 10 years of experience with creating and implementing
PMS in different industries and are familiar with the latest PMS literature.
The names and backgrounds of the experts can be found in the appendix together with a summary of the interviews.
1.6) Outline
The research will be held according to the structure of the methodology of the research. In de second chapter a literature study will be held on the Environmental Part. In the third chapter the research on the Knowledge Part will be held.
The elements from these two chapters will be brought together in chapter four and five, which will describe the rigor cycle and the relevance cycle.
In chapter six the conclusion will be held in which the result of the research from both the rigor and relevance cycle can be seen.
Figure 3 Outline of the research
1.7) Conclusion
In the electricity sector there is a request for additional literature about Performance Measurement. This research will create a PMF for the electricity sector to fulfil this demand. The framework should provide managers in power plants, which are, or will be, in operation with the information to build there own PMS. The research is done via the methodology shown in figure 2 (page 14). The combination of both a literature study and the use of experts from the different fields of research will provide with the research with a solid theoretical and methodological basis.
2 - The electricity generation sector
In this chapter an overview will be given of the electricity generation sector, which will cover the environmental part of the design, shown in the methodology paragraph (figure 2, page 14). The chapter will answer the first sub question of the research “What are the important aspects of the electricity generation sector?”
This chapter will cover the following subjects; - The energy sector characteristics - Changes because of the liberalization - Energy generation
- Relation between Trading and Production - Business models
- Important sector characteristics - The main aspects within Production - Costs and Revenue
At the end of the chapter a conclusion with a short overview will be sketched. 2.1) The energy sector characteristics
For a long time the energy supply was a government responsibility. Changes in the energy sector are relatively slow compared to other industries. Operations are ongoing, investments are expensive and planning time is long (Haapasalo, Et. Al. 2006). Innovation was slow, energy prizes were increasing and governments wanted to sell to get the sector more competitive. Therefore most governments decided to liberalize the sector. The main idea was to decrease the costs of energy and to get a more competitive industry. In the ‘90’s it was decided to liberalize the electricity sector in several European countries (Joskow, 1998). 2.2) Changes because of the liberalization
competition for customers is growing, and the strategic position in the markets can change quite fast (Haapasalo, et.al., 2006). Because electricity is a homogeneous product the competition is on costs. Therefore economies of scale are increasingly important in the electricity generation sector. Because of this big scale, investments in production plants are massive and mostly irreversible (Soto, 1999). Since it is almost impossible and extremely costly to store big amounts of electricity, the electricity market must rely on equating supply and demand in every instant. This, in turn, implies both that an important effort of coordination is required and that, as a system, the electricity sector would usually present excess capacity (Soto, 1999).
Because of the liberalization the electricity sector has changed dramatically last decade. In some countries the power generation and the distribution network were separated and in all the countries the power generation sector is fully liberalised. In figure 4 the pre-liberalization and post-liberalization supply chains are shown.
Figure 4 The electricity supply chain before and after liberalisation. (Source: Energy Delta Institute, Energy Business School. 2010)
Now the focus of the power generation / Production part changed more towards the focus on financial figures, in relation to other issues. Sector experts say it is believed that the performance measurements are outdated and should be looked at again (VGB PowerTech, 2011).
be entering the transmission network at the exact moment of generation, without anyone or anything in between. From the actor perspective the Trading department is an important party and the network. The network is in this aspect not an important factor for the power plants. This can be seen in figure X by means of the difference in colour in the post-liberalization supply chain.
2.3) Electricity generation
There are different ways to generate electricity. This could be done by using hydro- or wind power, converting solar energy, by nuclear reactions or by burning fuels, see figure 5.
Figure 5 How is electricity generated? (Source: Zactruba, J. 2009. Bright Hub.com)
2.4) Relation between Trading and Production
In figure 4 can be seen that the business units Production and Trading are next to each other in the supply chain. These business units are closely linked to each other. Trading needs to sell the electricity generated by the Production facilities and will decide which plant should run at what time. Both parties are each other’s customers. For this reason there is a complex supplier/client relation between these business units. The exact relation is mainly set by the company’s business model.
2.5) Business models
Within the companies operating in the market, there are different business models, which have their impact on the relation between the departments of Production and Trading. In some electricity companies the Production department is considered a cost-centre, in other companies Production department is a profit-centre. These differences in business models have their impact on which steering information is required and therefore about what is considered more important.
2.6) Important sector characteristics
2.7) The main aspects within Production
Within the department of Production, there are two main aspects, which are Operations and Maintenance (O&M). The optimization of these two aspects is called Asset Management.
Figure 6 Optimization of a power plant.
The income of a power plant is created by delivering electricity to the market. This is done by Operations. The plant operators need to make sure that the dispatchers from Trading can utilise the plant to its full capacity. For this reason the time that a power plant is ready for operation should be maximised. They need to make sure that the plant runs smoothly and keep a focus on a safe and efficient operation.
Maintenance consists of only costs and should make sure the operation runs smoothly. In case of bad or too little maintenance, the reliability of the plant is will decrease. Preventive maintenance can save a lot of costs and missed revenue if a plant breakdown, with associated expensive repairs, can be prevented. On the other hand, the hours a plant is in maintenance it is not ready for operations, which will be at the cost of missed revenue. In order to sustain high plant availability and at the same time meet the cost and regulatory requirements, the appropriate maintenance strategies need to be integrated with other management functions (Nakajima, 1989). To know which O&M practices are critical predictors of plant performance a deeper understanding of best predictors of plant performance is needed. This would assist managers to allocate limited resources to those areas, which have the most significant contribution to plant performance (Nakajima, 1989).
positive and significant predictors of plant performance (Shyong Wai Foon and Terziovski, 2008).
2.8) Costs and Revenue
The costs of a plant consist of the fuel costs, maintenance cost and fixed costs, like overhead. The revenue comes from several aspects. The majority of it will come via electricity sold on the wholesale market (APX). Next to that revenue can be made via the unbalance market. Via these markets the network provider regulates the production and demand to get a stable frequency in the net and to make sure the production companies will adjust their production towards the demand.
2.9) Conclusion
Because of the liberalization of the electricity generation sector, the focus changed towards being competitive and more towards a focus on continuous improvement. Electricity is a homogeneous good, which is sold via a wholesale market. Therefore the end-consumers of the electricity are irrelevant for a power plant and the Trading department can be seen as the customer. The Trading department deploys the power plant and therefore the aspects, which are have an impact on the deployment of the power plant, have a direct impact on the revenue. An optimum must be found between the costs and the revenue, to maximize shareholders value. In the electricity generation sector all activities should be done with a constant focus on safety. It has been found that both people-oriented “soft” as well as technically oriented “hard” practices were found to be positive and significant predictors of plant performance. The most important aspects of the electricity generation sector are the following:
3 - Performance Measurement Frameworks
In this chapter a review will be given on the latest theory of PMFs. First a short introduction will be given on Performance Management (PM), to create a clear picture of the role of Performance Measurement in a broader context. In the second part of the chapter the main aspects of Performance Measurement will be described, as well as an overview of the modern PMFs.
The paragraph on PM will treat the following aspects:
- An overview of the different levels of PM will be given.
- An overview will be given on the different philosophies of the highest levels of PM, which is the basis of the creation of a PMS.
- The place of the PMS in the field of PM will be given, to put it in a bigger perspective. The paragraph on Performance Measurement will treat the following aspects:
- An overview will be given of the different disciplinary approaches for the design of a PMS.
- The historical perspective on Traditional and modern Innovative PMS will be given. - An overview will be given of the modern PMFs and their characteristics.
- An overview will be given of the important characteristics, which a performance measure should meet when designing a PMS.
- An overview will be given of the action that should be taken when designing a PMS. - An overview of the process of maintaining a PMS will be given to make sure the PMS
won’t loose its value over time. 3.1) Performance Management
goals are as much about communication as they are about evaluation and targets. In order to achieve the best outcome, measurement must be applied systematically (Pulford, Et. Al. 1996).
Bititci et. al. created the closed loop deployment and feedback system for the performance management process, visible in figure 7 (Bititci, et. al. 1997). In this pyramid can be seen by what means the performance measures are created.
Figure 7 The closed loop deployment and feedback system for the PM process (Bititci, et. al. 1997).
The strategic goals should be deployed throughout the entire organization via the business processes, activities and the personnel. In every layer of the pyramid the goals or action will be worked out in further detail. The performance measures in each layer should support the goal or action in the layer above. In this pyramid, the performance measurement system provides the feedback. Therefore the performance measures should not only be at the bottom of the pyramid, but at every layer there should be performance measures, which support the level of detail, appropriate for the level.
“Every level needs to have its own performance measures. To control the process at every level, you need performance measures at all levels” (Rodgers, 2012).
objectives and the strategic goals need to be clear. Creating a PMS starts here (Bitici et. al 1997; Porter, 1990).
Vision and Business Objectives
According to Kaplan (2000) for a company to create its vision, it is important to look at the mission statement and their core values – why does the company exist and what does it believe in. With that inside, managers can develop a vision and what the company wants to become. This vision should create a clear picture of the company’s overall goal and the business objectives (Kaplan and Norton, 2000; Bititci, et. al. 1997).
Strategy
With the vision and business objectives as a starting point, a strategy and strategic goals can be created. There are differences of opinion about how to achieve this. Some people like Porter say that the external environment is of a big influence for the strategy of an organization and its success. Porter stated that it’s the main influence, which a company should build its strategy around, and created the 5-forces model (Porter, 1990) (see figure 8).
Figure 8 Porter’s five forces model. (Source: Porter, 1990)
assets, such as company culture, employee skills and knowledge, customer relationships, etc. since these are competitive advantages that are hard to copy and becoming increasingly more important (Kaplan and Norton, 2000).
There is a general agreement that there are three main strategies in the production industry to market leadership (Porter, 2000; Kaplan and Norton, 2000; Treacy and Wiersema, 1993). These main strategies are the following:
- Operational excellence - Product leadership - Customer intimacy
Once there is a thorough understanding of the company’s core competences, its competitive advantage and a clear view on the external influences on the business, the vision, business objectives and strategy can be formed. This is the basis for creating a PMS. The discussions are necessary for ‘sense-making’ and for aligning the views throughout the organization and building critical success factors, like skills, capabilities and help to build a motivated team (Rodgers, 2007).
The information system is at the heart of the performance management process and should integrate the relevant information and should enable the closed loop deployment and feedback system as shown in figure X. In this context integration means that the PMS should enable the correct deployment of the strategical and tactical objectives of the business as well as providing a structured framework to allow the relevant information feed back to the appropriate point to facilitate the decision and control processes (Bitici et al. 1997).
Figure 9 The organization’s purpose & vision framework. (Source: Casey et.al.2008)
It’s important to realize that KPIs are indicators and no more than that. In a changing environment it’s good to constantly think about what the indicator is telling you. “KPIs are meant to help achieving the goals. They shouldn’t become goals themselves” (Casey et. al. 2008). The internal discussion about what it is, how to improve the organization, how to plan and how to implement the changes is of utmost important.
3.2) Performance Measurement System
Performance Measurement is a topic, which is often discussed, but rarely defined. Literally it is the process of quantifying action, where measurement is the process of quantification and action leads to performance (Neely et. al. 2005).
In literature performance measures within an organization can be designed on the basis of several disciplinary approaches, such as:
- The engineering approach - The systems approach
- The statistical approach
- The consumer marketing approach
- The ‘conformance to specifications’ approach (Waggoner et. al. 1999)
In the past more traditional accounting performance measurement systems were used. The systems had several shortcomings and failed to convey strategies and priorities effectively within an organization (Maskell, 1991; Skinner, 1974), encouraging short-termism (Johnson and Kaplan, 1987), and inflexibility to change (Richardson and Gordon, 1980). These traditional systems had the weakness that they were uni-dimensional and only looking backwards. This led to the development of more balanced and innovative systems, which were able to generate multi-dimensional and more focussed on more leading measures, than the traditional financial measurement systems, which provided more lagging measures. This made it possible to take action earlier in the business process. The information was also available at a lower level in the organization, which made it possible to react with more speed and made it easier to control the organization.
According to De Toni and Tonchia (2001) Traditional PMS have the following characteristics:
- Based on cost/efficiency
- Trade-off between performances - Profit oriented
- Short-term orientation
- Prevalence of individual measures - Prevalence of functional measures - Comparison with standard
- Aim at evaluating
Innovative PMS on the other hand have the following characteristics: - Value-based
- Performance compatibility - Customer oriented
- Improvement monitoring
- Aim at evaluating and involving
These differences between Traditional PMS and Innovative PMS are shown in table 1 for an easy comparison between them:
Table 1 PMS evolution (Source: De Toni and Tonchia, 2001).
Examples of innovative PMFs are the Balanced Scorecard (BSC), the Performance
Measurement Matrix, the Results and Determinants framework, the Performance Pyramid, the Business Excellence Framework and the Performance Prism.
A short explanation will be given of the above-mentioned frameworks. The Balanced Scorecard (Kaplan and Norton, 1992)
The Balanced scorecard is the most famous PMS-framework and by far the most used framework (Neely, 2000). The BSC is based on four fundamental questions that need to be answered.
- How do we look to our shareholders (financial perspective)? - What must we excel at (internal business perspective)? - How do our customers see us (the customer perspective)?
Figure 10 The Balanced Scorecard. (Source: Kaplan and Norton, 1996)
The strength of the balanced scorecard lies in the way it seeks to integrate different classes of business performance ± financial and non-financial, internal and external. The balanced scorecard is well packaged and does makes explicit the links between the different dimensions of business performance, which is arguably one of the greatest strengths of Kaplan and Norton's balanced scorecard. (Neely et. al. 2000).
A few shortcomings have been identified. These include the absence of a competitiveness dimension (Neely et al. 1995), failure to consider the human resources perspective, employee satisfaction, supplier performance, product, service quality, environment and community perspectives (Lingle and Schiemann, 1996; Brown, 1996), whilst Neely and Bourne (2000) and Schneiderman (1999) argue that the four perspectives of the BSC are insufficient.
The Performance Measurement Matrix (Keegan et. al. 1989)
Figure 11 The Performance Measurement Matrix. (Source: Keegan et. al. 1989).
The matrix is sort of similar to the BSC of Kaplan and Norton, but the matrix, however, is not as well packaged as the BSC and the links between the categories are not described very explicit (Neely et. al. 2000).
The Results and Determinants framework (Fitzgerald et. al. 1991)
The results and determinants framework shown in Figure 12 does make a link between the different dimensions and the business performance. This framework is based on the premise that there are two basic types of performance measures in any organization, those that relate to results (competitiveness, financial performance), and those that focus on the determinants of the results (quality, flexibility, resource utilization and innovation). The appeal of this distinction is that it highlights the fact that the results obtained are a function of past business performance with regard to specific determinants ± i.e. results are lagging indicators, whereas determinants are leading indicators.
Similar to Kaplan and Norton’s BSC, Fitzgerald et al. (1991) developed another PM framework by considering leading and lagging performance measures. This PM framework specifically targets the PM in the service sector. The framework identifies six performance measures: two measure the results (lagging indicators) of competitive success (competitiveness and financial performance) while the other four measure the determinants (leading indicators) of competitive success (quality of service, flexibility, resource utilization and innovation).
Just like the BSC it got the critic that it does not focus enough on the stakeholders (Neely, 2002). The main focus is on the service sector; therefore it is questionable how well the leading indicators fit other sectors.
The Performance Pyramid (Lynch and Cross, 1991)
The strengths of this framework (figure 13) are that it ties together the hierarchical view of business performance measurement with the business process view. It also makes explicit the difference between measures that are of interest to external parties-customer satisfaction, quality and delivery, and measures that are primarily of interest within the business ± productivity, cycle time and waste.
One of the weaknesses of this framework, however, is that it is difficult to operationalise (Neely et. al. 2000). Another shortcoming is that it does not provide any mechanisms to identify key performance indicators, nor does it explicitly integrate the concept of continuous improvement (Ghalayini et al., 1997). It should be also noted that the system has not been empirically tested (Metawie and Gilman, 2005).
Business Excellence framework
The strength of this framework (figure 14) is that it combines the results and the enablers of this result. The weakness however is that the terms used in the framework are so open, that they can be interpreted in many ways, that any single organization could decide to capture any one of several dozen different measures of performance under each of the headings. This makes it hard to decide which performance measures the organization should rely on (Neely et. al. 2000).
Figure 14 The Business Excellence Framework. (Source: Neely et. al. 2000)
The Performance Prism (Neely, 2002)
The Performance Prism is a balanced framework with the following questions:
- Stakeholder Satisfaction: Who are our stakeholders and what do they want and need? - Stakeholder Contribution: What do we want and need from our stakeholders?
- Strategies: What strategies do we need to put in place to satisfy these sets of wants and needs?
- Processes: What processes do we need to put in place to satisfy these sets of wants and needs?
infrastructure – do we need to put in place to allow us to operate our processes more effectively and efficiently?
Together these five viewpoints provide a comprehensive and integrated framework for managing organizational performance and, by answering the related questions, organizations can build a structured business performance model.
Figure 15 The Performance Prism. (Source: Neely et. al. 2002).
In the Performance Prism the measures however, are not derived from the strategy, as stated as the no.1 important characteristic in the list of characteristics that a performance measure should have (table 3, page 39). The strategy is derived by looking at the stakeholders’ perspective instead of a translation from the vision and business objectives, as can be seen in the framework of Bititci et. al. (figure 7, page 27). Next to that the performance prism tends to neglect issues such as how the performance measures are going to be realized, hence, little concentration is given to the process of designing the system (Tangen, 2004).
Table 2 Overview of the PMFs. Indi ca tors de ri ve d f rom s tra te gy Bot h f ina nc ia l a nd non -f ina nc ia l m ea sure s i nc lude d Bot h i nt erna l a nd e xt erna l proc es se s i nc lude d F oc us on t he c om pe ti ti on i nc lude d S ta ke hol de r f oc us inc lude d F oc us on t he c li ent inc lude d M ent io ne d i n l it era ture a s w el l ba la nc ed A re t he l inks be tw ee n the e le m ent s m ent ione d in li te ra ture a s e xpl ic it ? F oc us on Cont inuous Im prove m ent M ec ha ni sm s f or f indi ng K P Is provi de d M ent ione d i n l it era ture a s a prove n c onc ept Total amou n t of e le me n ts in th e fr am ew or k s Balanced scorecard X X X 0 - X X X X X X 9 Performance Measurement Matrix X X X 0 0 - - - X 0 - 4 Results and Determinants
framework X X X X - 0 X X X - - 7 Performance Pyramid X X X X - X - - - 0 - 5 Business Excellence framework X X X - X X - 0 - 0 - 5 Performance Prism 0 X X X X X X - X - - 7
Table 3 Instructions with regard to the design of performance measures. (Source: Neely et. al. 1997)
Just having the right measures is not enough to improve performance. Therefore action is required. In table 4 the details of the performance measure can be worked out, so it becomes clear exactly what is done, why this is done, what action is required and by whom, etc.
Table 4 The Performance Measure record sheet. (Source: Neely et. al. 1997)
When the implementation is successful, over time, organizations often find themselves short of people who have the skills required to analyze and interpret the data. Therefore measurement systems could become less and less valuable. To prevent this from happening continues attention and improvement is needed. In figure 16 the process of PMS-improvement is shown, with the enablers shown in the middle.
Figure 16 Keeping the PMS up-to-date and successful. (Source: Neely et. al. 2000).
3.3) Conclusion
Performance Measurement can be seen as one of the important aspects of the broader field of Performance Management. Performance Management starts with a vision of the organization and the environment it is in and in which direction the organization wants to evolve. Based on this vision a strategy must be created. If this strategy is clear and it is clear what the strategic goals of the organization are, a PMS can be created.
When a PMF is used, it is important that the measures meet certain criteria (mentioned in table 2) and that the details of a performance measure are worked out, so it becomes clear exactly what is done, why this is done, what action is required and by whom, etc.
4 - Rigor Cycle, The selection of a base PMF for the electricity
generation sector
In this chapter a PMF will be chosen, which will be the basis of the sector specific framework. The selection of this framework is based on the characteristics of the different PMFs and the characteristics of the electricity generation sector, described in the previous chapters. This chapter will cover the Rigor Cycle, described in the methodology paragraph (figure 2). In this chapter an answer will be given on the sub question: Which PMF can best be used as a basis for the design of a specific PMF for the electricity generation sector?
The chapter will comprise of the following paragraphs: - Basis of design
- Experts opinions - Framework selection
- Description of the selected framework - Conclusion
4.1) Basis of design
The first choice in the selection process of a PMF is the choice of disciplinary approach. The disciplinary approaches mentioned in the previous chapter will be reviewed. Before choosing an approach, first the main characteristics of the sector need to be looked at.
In the chapter of the electricity generation sector (chapter 2) can be seen that the following subjects are of importance in the electricity generation sector.
On the basis of the above-mentioned characteristics and elements, each approach will be reviewed and judged.
- The engineering approach – This approach focuses on only the technically oriented measures. The people-oriented measures would be missing. Therefore this approach is not suitable for the sector.
- The management accounting approach – This approach focuses on the financial elements. Because of the mixed type of characteristics, which are important for the sector, with both financial and non-financial indicators, this approach is not suitable. - The statistical approach – Only the technically oriented measures could be made and
the people-oriented measures would be missing. Therefore this approach is not appropriate for the sector.
- The consumer marketing approach – Electricity has no quality. The consumers are unknown and irrelevant to a power plant. Therefore this approach is not suitable. - The ‘conformance to specifications’ approach – Electricity is a product that doesn’t
know any quality. It’s there, or it’s not. For this reason it is not a question if it is conform specifications. This approach is not suitable for the sector.
The last approach is the systems approach. In the systems approach objectives are set. The achievement of these objectives must be measured. In the chapter of the Electricity sector we have seen that different objectives are of importance and must be measured. For this reason the systems approach fits the electricity generation sector and a framework based on this approach will be chosen.
4.2) Experts opinions
These three frameworks are given to the experts for review, together with the following questions:
- Which of the three selected frameworks is best suitable for the electricity generation sector and why do you think so?
- Can you tell me if there is a framework, which is less suitable and if so, why do you think this is?
- Are there other frameworks, which are not mentioned in the review, which you think could be very suitable and even better then the three frameworks, which are presented to you? If you think so could you explain to me why you think this is?
Their reactions were as follows: Chris Rodgers:
Figure 17 Performance Management Framework©. (Source: Chris Rodgers Consulting Ltd. 2011)
The Performance Management Framework is intended to encourage and facilitate ongoing sense- making conversations amongst managers and staff, about the strategic and operational challenges that they face. Within a particular local context, it directs attention towards those actions required to meet current commitments and, at the same time, to address emerging demands. Equally importantly, it raises questions about the conditions that need to be established to enable this agenda to be delivered on a continuing basis.
“This framework provides a good mix between the focus on the results of the current operation and the work that needs to be done to achieve the long-term goals of the organisation. The framework looks at the goals of the organisation and what the enablers are to get these results” (Rodgers, 2012).
Theo Fens:
“The Balanced Scorecard is a very suitable framework which has proved to be a successful framework in many sectors. In the electricity sector this type of framework can also be used perfectly fine”.
framework and not as a PMF. Therefore it is too generic and not explicit enough. The BSC has its focus on Performance Measurement and therefore it suits better for this type of research”.
Peter Stoppelenburg:
“The Balanced Scorecard is a very suitable PMF. These PMS are already in use in different companies or parts of companies in the sector. I think this framework can be very useful. To add value the framework it should however be made very explicit. Especially the elements in the Internal Processes Perspective and the Learning and Growth Perspective should be very explicit. Otherwise it keeps the high level focus and it won’t be of any practical use to managers in lower levels of the organisation. Maybe it is helpful if an elaborated example is added”.
4.3) Framework selection
Before selecting a framework first the strategy needs to be clear, to be able to make a sound decision on the importance of several aspects and therefore to make a decision on the focus of the framework.
Electricity is a product that does not have quality difference and is sold via a wholesale market without any contact with the buyer. Product differentiation or product leadership is therefore not possible in the electricity generation sector. Operational excellence should therefore be the leading strategy in this sector. It is better to focus on the internal competences and to invest in intangible assets, such as company culture, employee skills and knowledge, customer relationships, etc. than on the competition.
The consulted experts were slightly in favour of the BSC. Despite that this research will compare the pro’s and con’s of the different frameworks with the sector characteristics to come to a sound conclusion.
The main shortcoming of the BSC is that the factor ‘competition’ is not taken into account. Competition is taken into account in both the Results and Determinants framework and the Performance Prism. For the electricity generation sector however, this is not such an important aspect. The lifespan of a power plant is something like 30 or 40 years. When the decision to build a plant is taken, it needs be operated for its entire lifetime to earn back the investment. Since it’s not possible to act upon the competition, because of the technical restrictions, it is not necessary to focus upon the competition. Therefore this is not considered a disadvantage of the BSC.
Another issue described as a shortcoming of both the BSC and the Results and Determinants framework, compared to the Performance Prism, is the lack of focus on the stakeholders’ perspectives. There could however be argued that this stakeholder perspective is an integral part of the internal processes- and learning and growth perspective.
The Performance Prism is on the other hand not as strong in the supporting mechanisms for finding the KPIs. Next to that, the framework is not based on the strategy, but strategy is based upon the framework. It can therefore be a nice tool to reform the strategy and be used as a framework in the field of Performance Management, but it is considered less suitable as a PMF.
The BSC does have some other advantages over the Results and Determinants framework and the Performance Prism. The BSC is the best-known framework and is already in use in some organisations or parts of organisations in the electricity generation sector. It is by far the most used framework, because of its operational success and over time it is already improved by extensive research from both the academic as the business world. It has a proven effectiveness and the economic benefits the BSC yields are proven many times. Next to that the consulted experts all stated that the BSC was a suitable framework for the sector.
4.4) Description of the selected framework The Balanced Scorecard
The BSC incorporates four perspectives: financial, internal business, innovation and learning, and the customer. These perspectives tend to answer the following four questions to ensure that the organisation is doing well in the competitive market.
- How do we look to our shareholders? (Financial perspective) - What must we excel at? (Internal business perspective) - How do our customers see us? (Customer perspective)
- How can we continue to improve and create value? (Innovation and learning perspective)
The four perspectives of the BSC minimises the overloading of information but focuses on the most critical success factors of the organisation (Kaplan and Norton, 1992). Furthermore, the BSC can be used to translate the organisation’s mission and strategic objectives to a set of performance measures and help communicate and implement the organisation’s strategy throughout the organisation, consequently enabling the employees to identify the drivers of current and future success (Kaplan and Norton, 1996). A major strength of the BSC is that it links the PM with organisational strategy. The BSC differs from the traditional approach of performance measurement in combining both “lagging” and “leading” measures.
Furthermore, the measures in the BSC are balanced not only between external measures (shareholders and customers) and internal measures (critical business process, innovation, and learning and growth), but also between the result measures (outcomes) and driver measures (measures for future improvement) (Kulatunga et. al. 2006).
4.5) Conclusion
5 - Relevance Cycle, The selection of the indicators for the
electricity generation sector
In the previous chapter can be seen that the Balanced Scorecard is suitable as a framework for the electricity generation sector. This chapter will focus on finding the right indicators to fit the reporting mechanisms in the electricity generation sector. The selection of the indicators will be based upon the characteristics of the Balanced Scorecard and the characteristics of the electricity generation sector, as described in the previous chapters. This chapter will cover the Relevance Cycle, described in the methodology paragraph (figure 2). The final indicators will be reached by means of expert consultation. This will be done via several feedback loops with the consulted experts. In this chapter an answer will be given on the sub question: What indicators are needed to make the PMF fit the electricity generation sector and how are the indicators interrelated?
The chapter will comprise of the following paragraphs: - The BSC for commercial companies
- Initial changes to make it fit the electricity generation sector - Experts’ feedback
- Final changes
5.1) The BSC for commercial companies
Figure 18 The Balanced Scorecard for commercial companies (Source: Kaplan and Norton, 2004).
5.2) Changes for the sector
Based upon the characteristics described in the electricity generation chapter, several discussions on Performance Indicators in the electricity generation sector with employees from the sector and the authors experience in Performance Measurement in an electricity generation company, the following changes are made in comparison with the BSC for commercial companies from Kaplan and Norton.
The initial changes to BSC for commercial companies are done by the author, based upon the characteristics found in the previous chapter of the electricity generation sector. There are also indicators in the Financial Perspective and the Internal Processes Perspective, which has been changed by the author, based upon his experience in Performance Measurement in the
rather present the experts a first draft on which they could react; to be able to get into a more in-depth discussion on the indicators instead of spending there limited time on the creation of a first draft.
These initial changes will be described per perspective. 5.2.1) Financial Perspective
In the Financial Perspective it is decided to change the term Productivity for the term Costs and the term Revenue Growth into Revenue. The goal is to create as much value as possible. The objectives of increasing the revenue, decreasing the costs, or a combination of these two measures, can do this.
5.2.2) Customer Perspective
In the Customer Perspective the view of the dispatcher is chosen to find the right indicators. The elements that are essential for the dispatcher are:
- Availability - Reliability - Flexibility - Efficiency
These measures are chosen to put in the framework. 5.2.3) Internal Processes Perspective
In the Internal Process Perspective the following fields are mentioned in the BSC for commercial companies:
- The field Manage Operations - The field Manage Customers - The field Manage Innovation
One of the most important aspects in the electricity generation sector is Safety, Health and Environment (HSE). This field was not mentioned in the BSC for commercial companies. This field needs to be added for the sector.
For each of the other fields is investigated if these are suitable for the sector, or that some change is needed to make it fit the sector. Each field is separately treated.
The field Manage Operations
For this field it is decided to change the term Manage Operations into Manage Operations and Maintenance. This is decided because Operations and Maintenance are the two main
processes in a power plant. The field Manage Customers
This is changed into Manage Stakeholder Processes, since there are more stakeholder processes that need to be managed, than just the processes that are related to the customer. Other stakeholder processes, which need to be managed, are supplier management and processes related to the employees.
The field Manage Innovation
The field Manage Innovation is kept unchanged. Innovation is important in the electricity generation sector, just as in other sectors, to make sure that the company doesn’t fall behind with the competition.
The field Manage Regulatory and Social Processes
For the term Manage Regulatory and Social Processes is decided to change it into Manage Regulatory and Contractual Processes. In the energy sector it is very important to manage the regulatory aspects, because there is the risk of loosing the operating permit, in case of
5.2.4) Learning and Growth Perspective
The fields in this perspective are not changed in respect to the Balanced Scorecard for commercial companies, since these elements are important in the electricity generation as well.
These changes in the different fields, led to the BSC for organisations in the electricity generation sector, visible in figure 19, which has been presented to the experts for consultation.
5.3) Experts’ feedback
The special for the electricity generation sector adjusted Balanced Scorecard (see figure 19) has been send to the experts for consultation. With the framework they have been presented with the following questions:
- What do you think of the chosen indicators for the sector? - Are there any indicators that are missing? What would they be?
- Are there any indicators that are not that important to put in the framework? Which indicators would that be?
- Are there any changes that you would suggest? Their reactions were as follows:
Theo Fens:
“I think the selected indicators give a good indication of the important indicators in the electricity generation sector. Maybe the customer perspective could be renamed into Market Perspective, since this term is more common in the electricity sector. This also prevents misunderstanding from happening about who’s the customer. I think there is one missing indicator, which is sustainability. This becomes increasingly important in the sector. This indicator could be added in the Customer / Market Perspective, since this can lead to direct deployment decisions, since sustainable electricity production can yield extra revenue if green certificates can be obtained, or costs can be saved because of less CO2-emissions. I would change the indicator ‘Manage Regulatory and Contractual Processes’ into ‘Manage
regulatory processes’. I would do this because the regulatory processes are of a much greater importance than the contractual processes, since violation of the regulations can endanger the operating permit. Next to that, I think the contractual processes are already covered in the ‘Manage Stakeholder processes’ indicator.”
Chris Rodgers:
I would suggest changing the term Customer Perspective into Market Perspective. Managers in the generation business tend to talk in terms of the market, rather than the customer. This tends to be because generators supply power in bulk to the market and because the term “customer” tends to feature widely in the retail/commercial side of the business.
Next to that I would change the name of the Internal Processes Perspective into the Core Processes Perspective, since this perspective relates to the processes that are core to effective management of the generation business. In this perspective I would add the “Cost/Value for money management” and “Budgetary discipline” indicators. I see these as core processes in the context of the commercial management of the business.
I would also suggest splitting the Health, Safety and Environmental indicator into a Health and Safety indicator and an Environmental indicator. I would do this because achieving the best possible health and safety performance is part of the fundamental ‘ethos’ of the power generation business and deserves, I believe, separate mention. The last one could be implemented into the Regulatory processes indicator, since most of the environmental indicators are already a part of the measures obliged by regulatory institution to keep an operating permit and therefore they are already measured as a part of these processes in the daily operation”.
Peter Stoppelenburg:
“I think these indicators are indeed the most important Performance Measurement indicators for the electricity generation sector. I recognize a lot of indicators, which are currently in our PMS at Essent. I don’t see any indicators which shouldn’t be in there or any missing
indicators. I do however wonder what kind of useful measures could be found for some indicators in the Internal Processes perspective and the Learning and Growth perspective. I think it will be hard to find any useful indicators for the manage innovation indicator”. 5.4) Final changes
Both Fens and Rodgers suggested changing the name of the Customer Perspective into Market Perspective, since people in the electricity generation business tend to talk in terms of the market, rather than the customer. In the electricity sector the term customers refers to the consumers of the electricity instead of the Trading Department, which is referred to in this research. For this reason the term ‘Customer Perspective’ will be changed into ‘Market Perspective’.
Rodgers suggested changing the name of the ‘Internal Processes perspective’ into the ‘Core Processes perspective’. The indicator ‘Manage Operations & Maintenance’ definitely covers the core business of the electricity generation sector. Not all the indicators however are covering the core processes in a power plant. Managing innovation or managing stakeholder processes for example are important processes for a power plant but can’t be addressed as the core process for a power plant. For this reason this suggestion is not taken over and the name ‘Internal Processes perspective’ will be kept in this research.
Rodgers also suggested adding the ‘Cost/Value for money management’ and ‘Budgetary discipline’ indicators to the Internal Processes perspective, since these are core processes in the context of the commercial management of the business. The author agrees that these processes are core processes to the business, but does not agree that these elements should be in the internal processes perspective. The costs also have a link with the other elements, for example the aspects from the Learning and Growth Perspective. Since an indicator should not be more than ones in the framework and the author feels that the costs and budgets are a part of the financial perspective it is chosen to keep all the financial indicators collected in the financial perspective. All the perspectives have a mutual dependence and the perspectives are closely interlinked. Therefore the core processes can perfectly be managed with the costs placed in the Financial Perspective.
