Performance Measurement Systems: A Critical Review
of Design, Implementation and Use that Affect
Performance of Small and Medium-Sized Enterprises
R. van Lenthe Wolfkampstraat 29 7722 TB Oudleusen s1681052@student.rug.nl
Abstract
This paper reviews the organizational context (implementation), performance measurement content (design) and process (use) of performance measurement systems in relation to the performance of small and medium-sized enterprises by the assessment framework of Pettigrew (1989). The design of performance measures are not fully understood that result in a negative performance of both studied organizations. This paper shows important dimensions that need consideration when implementing such a system in small and medium-sized enterprises. Research findings calls for further investigation on the basis of large scale survey instruments to include a large population.
Word count: 25.646
II
Performance Measurement Systems: A Critical Review
of Design, Implementation and Use that Affect
Performance of Small and Medium-Sized Enterprises
R. van Lenthe Pioenstraat 139 8713 XW Groningen s1681052@student.rug.nl Master Thesis MSc Business administration
Specialization Small Business & Entrepreneurship
University of Groningen
Faculty of Business & Economics
1th Supervisor University of Groningen: Dr. Chris Streb
III
ACKNOWLEDGEMENTS
This paper is the final phase in finishing my Master of Business Administration, a step on my educational path that has provided me more knowledge and insights in the way that small and medium-sized enterprises use performance measurement systems and the relation to the performance of these organizations. Due to earlier education in the area of information science, my personal interest on performance measurement systems and the special focus of this Master, on small and medium-sized enterprises has resulted in this research topic. I would like to use this acknowledgement to thanks the persons that are involved during this research. Supervisor Dr. C. Streb for his critical and exhaustive feedback during the meetings that took place, Dr. C. Lutz for his role as second supervisor and Mr. Gerard van Workum of Quirius NV for his facilitating role as provider of organizations that have been cooperating during this research.
I hope you will enjoy reading this thesis.
IV INDEX 1. INTRODUCTION ...1 2. BACKGROUND ...6 2.1 Content ... 11 2.2 Context ... 15 2.3 Process ... 18
2.4 Effects on Business Performance ... 21
2.6 Literature summary ... 24 3. GENERATION OF HYPOTHESES ... 26 3.1 Research methodology ... 28 3.2 Research organizations ... 34 3.2.1 Organization X ... 34 3.2.2 Organization Y ... 37 4. RESEARCH FINDINGS... 40 4.1 Discussion of findings ... 45 4.1.1 Content ... 45 4.1.2 Context ... 47 4.1.3 Process ... 49
5. CONCLUSIONS & RECOMMENDATIONS ... 51
REFERENCES ... 55
APPENDIX 1 – INTERVIEW DATA OF ORGANIZATION X ... 61
APPENDIX 2 - INTERVIEW DATA OF ORGANIZATION Y ... 70
APPENDIX 3 – SCREENSHOTS OF QLIKVIEW ... 78
1
1. INTRODUCTION
Managerial tools that include a large number and broad group of performance measures to improve managerial decision-making are called performance measurement systems. A number of studies has been carried out on the impact of performance measurement systems on business performance in large organizations whereas little research is carried out on small and medium sized enterprises and that will become the special focus of this research due to their specific organizational characteristics. On the one hand, organizational researchers and managers claim that performance measurement systems have a positive impact on business performance. On the other hand, there is little research carried out that test this hypothesis (Kennerley and Bourne, 2004; Davis and Albright, 2004; Schiemann and Lingle, 1999; Franco et al., 2004). Besides this, less research has been carried out on the identification of factors that should result to a positive impact of performance measurement systems on business performance (de Waal, 2003; Hoque and James, 2000; Vasconcellos, 1988; Fisher, 1995; Ketelhohn, 1998). Therefore, this study will test the impact of performance measurement systems on business performance of small and medium sized enterprises in the Netherlands. The framework of Pettigrew et al., 1989 on organizational context, performance measurement content and process, will guide this research. The reason to choose explicitly for the aforementioned framework will be explained in detail in the next background section.
2 The quantification and the unanticipated consequences of measurement systems lead to one potential solution to introduce ‘balanced’ sets of measures. “Market standing, innovation, productivity, physical and financial resources, profitability, manager performance and development, worker performance and attitude, and public responsibility” are appropriate performance criteria says Drucker in his publication, The Practice of Management (Drucker, 1954). Turning forward the clock almost a half a century, shows that the same themes are still being discussed.
In practice, the organizational context, performance measurement content and process will all affect the performance of small and medium-sized enterprises. A research conducted by Franco et al., (2004) states that evidence in existing literature contradict which resulted in this research topic by trying to discontinue this existing research gap. Their study on a single organization has taken the proposition that the manner in which the organizational data are acquired, analyzed, interpreted, communicated and acted upon has an impact on business performance but revealed that this is over simplistic. The organizational context as well as performance measurement content also impact on business performance but these two major important factors are neglected in this study because they focus only on the stable system, the status-quo of a performance measurement system. This study will be more exhaustive and explicit by concentrating on all three factors that impact on performance by assessing the organizational context, performance measurement content besides performance measurement process, which was the only focus of the study of Franco et al. (2004).
3 performance measurement systems are linked to compensation systems, a significantly boost of employee satisfaction can occur. Besides these results, Ittner and Larcker (2003) came up with evidence to the contrary. Their study tested customer satisfaction and quality, which has increased more rapidly one year after the introduction of a performance measurement system in relation towards a year before. A result of this improvement was a decrease in profit. The positive effects on customer satisfaction as well as quality after the implementation has resulted in an increase in turnover which has not recovered investments associated with these improvements and therefore this research revealed that performance measurement systems impact negative on business performance. Besides this, another study revealed that operational measures within performance measurement systems are mostly ad-hoc and informal with no real understanding of key performance drivers (Greatbanks and Boaden
1998) which explain why these systems are ineffective in supporting the achievement of strategic
goals, which in turn leads to a decrease in performance of these small and medium-sized organizations
(CIMA 1993, Barnes et al. 1998, Hudson et al. 2001b, Veitch and Smith 2000). Another finding presented by Forza and Salvador (2000) is that employee communication based on performance measurement systems increases collaboration and facilitates buy-in of strategic objectives. Ketelhohn (1998) and Vasconcellos (1988) revealed that when choosing for appropriate measures will improve the implementation and recognition of business strategy and staff its business understanding.
4 organization, the content of performance measures are well designed and will therefore be proofed. This research is besides a theoretical, also interesting from a practical point of view due to the fact that many practitioners embarking upon a redevelopment of their performance measures within their system assuming that this effort will have a positive impact on business performance (Bourne et al., 1999) but this is questionable until now.
Although, a significant stream of research is carried out on the impact of performance measurement systems on business performance in large organizations; this is not reflected in small and medium-sized enterprises (Franco and Bourne, 2004). Hudson et al., (2001b) states that small and medium-sized enterprises require an alternative approach to the use of performance measurement systems
because of the characteristics of limited resources, limited cash flow coupled with reliance on few
customers, and potential staff turnover coupled with a flat organizational structure. An almost similar
stance is being made by Garengo (2005). He revealed that the specific characteristics of small and
medium-sized enterprises can be obstacles of the effectiveness of performance measurement systems
which is an important aspect that need to be considered. Garengo (2005) states that there are basically
two main obstacles for small and medium sized-enterprises, e.g. the lack of financial and human
resources, and endogenous barriers, e.g. short-term strategic planning and the perception of
performance measurement systems as bureaucratic systems that cause rigidity (Garengo et al., 2005).
Therefore, the research question that needs to be addressed is;
- What is the impact of performance measurement systems on business performance of SMEs?
5 background section. One of these reasons is that there is significant acceptance of this framework in recent literature.
This paper proceeds as follows. Section two presents an elaborated review of the literature from which
I work towards the factors that will impact on the outcome of performance measurement systems, the
content, context and process. Section three presents the formulation of hypotheses, methodology and a
description of the studied organizations as well as interviewees. Section four presents the research data
and a discussion of findings. This research paper ends up with a conclusion of findings and the
recommendations towards further investigation. Furthermore, interview questions as well as the
6
2. BACKGROUND
7 reflects causality, emphasizing that the results that has been obtained are results of business performance in the past in relation to specific determinants. Another measurement framework that has appeared is the one of Lynch and Cross (1991) and is called the performance pyramid. This pyramid provides an explicit link between strategy and operations, and encourages a user-centered design. A summary of the above mentioned balanced performance measurement frameworks have been adapted in table 1.
Balanced performance measurement frameworks
Author(s) Dimensions
Balanced Scorecard Kaplan and Norton (1992) Financial perspective Customer perspective Internal perspective
Learning and growth perspective Performance measurement matrix Keegan et al., (1989) Costs
Non-costs Internal External The results-determinants
framework
Fitzgerald (1991) Results (financial performance, competitiveness);
Determinants (quality, flexibility, resource utilization, innovation)
Performance pyramid Lynch and Cross (1991) Vision, market, financial, customer
satisfaction, flexibility, productivity, quality, delivery, cycle time, waste
Table 1. Overview of balanced performance measurement framework
Besides the availability of the above mentioned balanced performance measurement frameworks, a significant stream of work on design and deployment of performance measures is studied (Bitton, 1990; Bourne et al., 2000; Dixon et al., 1990). The problems that arose with the implementation of these systems have also been studied widely (Bourne et al., 1999, 2000, 2002, 2003). Research conducted by Franco et al., (2004) state that nowadays many researchers focus on how performance measurement systems are used (Lipe and Salterio, 2000, 2002; Kalagnanam, 2001; Vakkuri and Meklin, 2001; Barsky and Marchant, 2001; Malmi, 2001; Malina and Selto, 2002; Epstein, 2002). Each of the above mentioned areas contains a collection of research variables that impact on business
performance. These variables of design, implementation and use will be nurtured by already
8 deployment of performance measurement systems as outlined above, which is related to the content of
performance measurement systems in this research. Content refers to what is being measured and the
design of performance measurement systems couples with the question of what should be measured.
Therefore, these factors of content and design can be related to each other. The context that influence
on the outcome of performance measurement deals with an inner and outer context. The inner context
refers to the existing strategy, structure, culture, management and political processes of the
organization, which will influence on the outcome. The outer context is more widely, national, social,
political and economic context and the interpretation of local and national policies and events. The
context factor is related to the implementation of performance measurement systems due to the fact
that the contextual factors affect the implementation of performance measurement systems (Bourne et
al., 1999, 2000, 2002, 2003). The use of performance measurement systems (Lipe and Salterio, 2000,
2002; Kalagnanam, 2001; Vakkuri and Meklin, 2001; Barsky and Marchant, 2001; Malmi, 2001; Malina and Selto, 2002; Epstein, 2002) can be related to the process factor due to the fact that process deals with actions, reactions, responses and interactions of performance measurement systems.
9 information systems measures and is therefore, at this moment of writing, not appropriate to conduct in this research as evaluation framework. Seddonm (1997) concludes that the aforementioned evaluation success framework of Delone and McLean (1992) leads to so many potentially confusing meanings that the value of the model is diminished. Because of the confusion that this overloading of meaning can cause, they come up with a respecified and slightly extended version of the framework of Delone and McLean (1992). They clarify the meaning of information systems use, introducing four variables (expectations, consequences, perceived usefulness, and net benefits to society), and reassembling the links between the variables. By doing so, it become possible to develop a respecified and slightly extended model that provides a clearer, more theoretically sound conceptualization of relationships between the various information system success constructs identified by Delone and McLean (1992). Hirschheim and Smithson (1998) states that evaluating information systems, as a technical problem leads to no significant result, that overlook the social activities inherent in the evaluation process and, neglect the political-social environment of organizations. This is in line with arguments of Guba and Lincoln (1989) in recognizing the importance of stakeholders and the need to look beyond evaluation producing an answer. Walsham (1993) states that evaluation is a dynamic process within the social context. Recognition of the socio-political contexts and role of stakeholders demands an interpretive approach to allow for understanding and generation of motivation and commitment to an evaluation (Irani et al., 2008).
11
2.1 Content
The content of performance measurement systems deals with what is being measured and how this is
formulated into an appropriate set of measures. Inadequately designed performance measures can
result in dysfunctional behavior because of the method of calculating performance support individuals
to pursue inappropriate courses of action that, impact negative on business performance (Neely et al.,
1997). He also states that one of the key problems with performance measurement systems is that
these have traditionally adopted a narrow, or unidimensional focus. This problem can overcome if an
organization adopts a balanced set of measures which addresses the questions of how the organization
looks to its shareholders (in a SME perspective this could be the business owner), what must the
organization excel at, how do customers see the organization and to what extent the organization is
able to continue to improve and create surplus value. These four questions addresses the dimensions of
Kaplan and Norton (1992) its balanced scorecard on financial, internal, customer and innovation and
learning perspective. Therefore, it is necessary that the organizations that are willing to cooperate in
this research makes use of a performance measurement system that reflects the perspectives of a
balanced scorecard, the balanced scorecard of Kaplan and Norton (1992) for example or any similar
recognized framework as outlined in table 1.
Numerous authors have studied the design of performance measures but the most exhaustive literature
review is conducted by Neely (1997) on the design of performance measures. Due to the fact that this
research is based on ten different papers and books it elaborates further on earlier scientific work and
is therefore exhaustive and worthwhile to adapt to guide this research on the design of performance
measures.
Dixon et al., 1990 states that performance measures within the system should, be derived from strategy
and provide timely and accurate feedback. Kaplan and Norton (1992) support this recommendation
that performance measures should be derived from strategy but also recommend that it should be part
of a closed management loop. A significant stream of writing in the literature is that of Globerson
12 measurement systems. Globerson (1985) states that; performance criteria must be chosen from
organizational objectives, the purpose of performance measures must be clearly defined, the collection
of data must be clear, performance measures should be based on ratios instead of numbers and,
objective in comparison to subjective. Similarly, Maskell (1989) suggests the following principles on
the design of performance measures; measures should, be aligned to organizational strategy, be
non-financial, be adjustable during the lifecycle, be simple, clear and provide fast feedback and, stimulate
continuous improvement. Lea and Parker (1989) and Lynch and Cross (1991) argue that performance
measures should focus on improvement, besides this recommendation, Lynch and Cross (1991) argue
that performance measures should be; derived from strategy, simple, relevant and consistent, based on
quantities, reflects the business process and, should be based on trends rather than snapshots. A
slightly different stance is recommended by Azzone et al., (1991) who assumes that, measures should
be; derived from strategy, simple to understand and, relevant. Goold (1991) only recommends deriving
measures from strategy and, formulating these, as simple as possible.
The conclusions on the design of performance measures, made by Fortuin (1988) has been almost
identical to those of Lynch and Cross (1991) but is more extended. Fortuin (1988) suggests, besides
the previously mentioned recommendations that measures should; provide information, be precise and,
be objective and not based on opinions. The discussion above has resulted in a set of 22 characteristics
and is summarized in Table 2.
Performance measures should…. Authors
1 be derived from strategy Dixon et al., 1990; Kaplan and Norton, 1992; Globerson, 1985; Lynch and Cross, 1991; Fortuin, 1988; Maskell, 1989; Azzone, 1991; Goold, 1991
2 be simple to understand Lea and Parker, 1989; Lynch and Cross, 1991; Fortuin, 1988; Maskell, 1989; Azzone, 1991; Goold, 1991; Goold and Quinn, 1990
3 provide timely and accurate feedback Dixon et al., 1990; Globerson, 1985; Fortuin, 1988
4 be based on quantities that can be influenced, or controlled, by the user alone or in co-operation with others
Globerson, 1985; Lynch and Cross, 1991; Fortuin, 1988
5 reflect the “business process” – i.e. both the supplier and customer should be involved in
13
the definition of the measure
6 relate to specific goals (targets) Globerson, 1985; Fortuin, 1988; Goold and Quinn, 1990
7 be relevant Lynch and Cross, 1991; Fortuin, 1988; Azzone, 1991
8 be part of a closed management loop Kaplan and Norton, 1992; Globerson, 1985
9 be clearly defined Globerson, 1985; Fortuin, 1988
10 have visual impact Lea and Parker, 1989; Fortuin, 1988
11 focus on improvement Lea and Parker, 1989; Lynch and Cross, 1991
12 be consistent (in that they maintain their significance as time goes by)
Lynch and Cross, 1991; Fortuin, 1988
13 provide fast feedback Fortuin, 1988; Maskell, 1989
14 Performance measures should have an explicit purpose
Globerson, 1985
15 Performance measures should be based on an explicitly defined formula and source of data
Globerson, 1985
16 employ ratios rather than absolute numbers Globerson, 1985
17 use data which are automatically collected as part of a process whenever possible
Globerson, 1985
18 be reported in a simple consistent format Lynch and Cross, 1991
19 be based on trends rather than snapshots Lynch and Cross, 1991
20 provide information Fortuin, 1988
21 be precise – be exact about what is being measured
Fortuin, 1988
22 be objective – not based on opinion Fortuin, 1988 Table 2. Characteristics of performance measures
By reviewing table 2, the conclusion is made that the majority is duplicated, not relevant or deemed to
be desirable. Therefore the consideration to remove duplication has made, which results in an adjusted
set of recommendations towards the design of performance measures that in turn results in an
increased performance of small and medium sized enterprises. This adjusted set will guide this part of
the research on the content of performance measurement systems. An adjusted set of recommendations
14
Performance measures should… Authors
1 Be derived from strategy Dixon et al., 1990; Kaplan and Norton, 1992; Globerson, 1985; Lynch and Cross, 1991; Fortuin, 1988; Maskell, 1989; Azzone, 1991; Goold, 1991
2 Be clearly defined with an explicit purpose Kaplan and Norton, 1992; Globerson, 1985; Fortuin, 1988; Lynch and Cross, 1991;
3 Be relevant and easy to maintain Lynch and Cross, 1991; Fortuin, 1988; Azzone, 1991; Globerson, 1985
4 Simple to understand and use Lea and Parker, 1989; Lynch and Cross, 1991; Fortuin, 1988; Maskell, 1989; Azzone, 1991; Goold, 1991; Goold and Quinn, 1990
5 Provide fast and accurate feedback Dixon et al., 1990; Globerson, 1985; Fortuin, 1988; Maskell, 1989
6 Link operations to strategic goals Globerson, 1985; Fortuin, 1988; Goold and Quinn, 1990
7 Stimulate continuous improvement Maskell, 1989; Lea and Parker, 1989; Lynch and Cross, 1991
15
2.2 Context
Pettigrew et al., 1989 has made the distinction between the external (such as economical and political
factors) and internal (such as organizational structure and resources) organizational context. Published
research about the external context is scarce, but by reviewing the small number of literature, this
study revealed that differences of impact of external context exist. Research conducted by Smith and
Goddard (2002) shows that the contingent external factors like, demand uncertainty, supplier
characteristics, the nature and relative power of stakeholders, technology employed and the economic
situation influences performance measurement systems but do not take a side whether these factors
impact positive or negative on business performance. Otley (1999) argues to concentrate on explicit
the external context rather than just being concerned with internal activities. Competitor analysis is
important, but even more central is the value that an organization is delivering to its customers (Otley,
1999). Uncertainty in the economic environment and the pressure of socioeconomic-political
institution and regulatory regimes influences performance measurement systems, suggested by
Hussain and Hoque (2002). This is party in line with Smith and Goddard (2002) its argument of
economic situations, which impacts on performance measurement systems. Managers in businesses
that face turbulent and rapidly changing environments (Lawrence and Lorsch, 1969) derive less benefit from performance measurement systems than managers in more stable or mature businesses;
therefore the effectiveness of performance measurement systems is subject on the speed to adapt to
changes in the external environment. As the discussion shows, research is available but scarce on
external environmental influencing factors on performance measurement systems. From this minority
of published research the conclusion can be made that the external environment does impact on
performance measurement systems but there is no descriptive framework available that highlights the
relationship between the external context and performance measurement systems (Franco and Bourne,
2004). Therefore, the external context is worthwhile to mention but, neglected and will not guide this
research on the impact of performance measurement systems on business performance of small and
16 The internal context of performance measurement systems is studied more extensively. An exploratory
study of performance measurement systems, conducted by Evans (2001), has revealed significant
differences in the maturity of performance measurement systems to assess business performance.
Evans (2001) suggests a positive relation between the maturity of performance measurement systems
and customer, financial and market results. This view is supported by Martins (2002) who argue that
more mature systems are more effective. De Waal (2003) states that an organizational culture focused
on using performance measurement systems will improve effectiveness when managers of these
systems, trust the generated information and clearly see the promoter using the performance
measurement system. An empirical study on the effectiveness of a performance measurement system,
based on the principle of the balanced scorecard of Kaplan and Norton (1992), is conducted by Malina
and Selto (2002). They suggest that an alignment between the cultural elements embedded in
performance measurement systems and, the users of this system its cultural preferences, are beneficial.
A slightly difference stance is made by Lockamy and Cox (1995) by studying six world-class
manufacturing firms. Lockamy and Cox (1995) argue that the development of an organizational
culture is a prerequisite to integrate effectively all the organizational levels into business performance
assessment processes. The style of management used in organizations will impact on performance
measurement systems. Numerous authors call for an appropriate style but this is unclear in my opinion
because an appropriate style is never feasible. Different settings and phases of implementation and use
of performance measurement systems call for different styles of management. For example, suppose a
manager with considerable experience. In general, he or she should makes less use of performance
measurement systems due to his managerial experience but a research conducted by Gelderman (1998)
revealed the opposite; they conclude with a positive influence to the number of information used when
managerial experience has increased. Bititci et al., (2002) and Eccles (1991) calls for high data
integrity and a low burden of data capture within performance measurement systems. These
infrastructure aspects are reflected by the studies of Lingle and Schiemann (1996) and Manoochehri
(1999). De Toni and Tonchia (2001) and Kaplan and Norton (2001) argue for the alignment between
17 improve performacne. A study conducted by Moon and Fitzgerald (1996) on the role of the
performance measurement system at TNT (UK) ltd. revealed that it is a good point to relate
performance measurement systems to reward systems.
A summary of the above literature discussion is adapted in table 4.
Internal context Authors
1 System Maturity
More mature systems are more effective
Evans (2001) and Martins (2002)
2 Organizational structure
Importance of aligning structure and measurement
Hendricks et al. (1996) and Bourne et al. (2002)
3 Organizational size
Measurement is easier in large organizations and more problematic in smaller ones
Hoque and James (2000) and Hudson et al. (2001a, b)
4 Organizational culture
Alignment between the cultural elements embedded in the measurement system and the users its cultural preference is beneficial
De Waal (2003), Gates (1999), Johnston et al. (2002), Lingle and Schiemann (1996), Lockamy and Cox (1995), Maisel (2001), Malina and Selto (2002) and Bititci et al. (2004)
5 Management style
Appropriate style is important, appropriate style may be different in different settings and phases of implementation and use
Gelderman (1998), Libby and Luft (1993), Hunton et al. (2000), Simon (1987) and Bititci et al. (2004)
6 Competitive strategy
Measures should be aligned to strategy
Kaplan and Norton (1996, 2001), Lockamy (1998), Mendoza and Saulpic (2002), McAdam and Bailie (2002) and Neely (1998a)
7 Resources and capability
Companies need resources and capabilities to implement and refresh their measurement
Franco et al., (2004) and Kennerley and Neely (2002)
8 Information systems infrastructure
High data integrity and a low burden of data capture are important
Bititci et al. (2002), Eccles (1991), Lingle and Schiemann (1996) and Manoochehri (1999)
9 Other management practices and systems
There should be alignment between measurement and other systems
(e.g. budgeting, compensation)
De Toni and Tonchia (2001), Eccles (1991), Eccles and Pyburn (1992), Kaplan and Norton (1966), Kaplan and Norton (2001), Moon and Fitzgerald (1996) and Otley (1999)
18
2.3 Process
The main processes of performance measurement systems are the design, implementation and the use
of measures within performance measurement systems. As stated above, there is a significant stream
of work carried out on design and deployment of performance measurement systems (Bitton, 1990; Bourne et al., 2000; Dixon et al., 1990). Also the problems that can occur during the implementation phase have been studied widely (Bourne et al., 1999, 2000, 2002, 2003). The design and implementation processes will change the measurement system over time. In the situation whereby the system is stable, the status-quo, less research is carried out on the use of performance measurement systems and how this affects business performance.
19 that one of the processes of performance measurement systems deals with the linking to strategic objectives. This finding contrast recommendation made by Neely (1998) who neglect linking to strategic objectives underlying process.
The term “organization control” is used by Flamholtz (1983) in its full sense of any actions or activities taken to influence the probability that people will behave in ways which lead to the attainment of organizational objectives (Flamholtz, 1979 and Otley and Berry, 1980). To help gain control over the behavior of people, organizations can make use of performance measurement systems. Simons (1991) also call for action taking within organizations. The performance measurement systems should be used to stimulate face-to-face dialog and build information bridges among hierarchical levels and departments. By using performance measurement systems, managers can signal where organizational attention and learning should be focused. Simons (1991) conclude that by systemic focusing, managers are allowed to guide the emergence of action plans and new strategic initiatives. Therefore it is important to take action based on these signals to improve organizational performance.
As stated previously, Neely (1998) suggest investigating the use of performance measures, within performance measurement systems, through the stages underlying process; data capture, data analysis, interpretation, communication and decision-making. Atkinson (1998) and Otley (1999) calls for the linking to strategic objectives and the approach of taking action is suggested by Flamholtz (1983), Flamholtz et al., (1985) and Simon (1991). A summary of the process stages of linking to strategic objectives and action taking, as well as the five stages supposed by Neely (1998) of data capture, data analysis, interpretation, communication and, decision-making are adapted in table 5. These seven process stages will guide this part of the research on how performance measurement systems are used.
Process stages Authors
1 Alignment with strategic objectives Atkinson (1998) and Otley (1999
20 3 Data analysis Lynch and Cross (1991) and Neely (1998a)
4 Interpretation and evaluation Simons (1991), Neely (1998a), Ittner et al. (2003) and Kerssens-Van Drongelen and Fisscher (2003
5 Communication and information provision Bititci et al. (1997), Forza and Salvador (2000), Kerssens-Van Drongelen and Fisscher (2003), Lebas (1995), Lynch and Cross (1991), Simons (1991), McGee (1992), Neely (1998a) and Otley (1999)
6 Decision-making Ittner et al. (2003) and Neely (1998)
7 Taking action Flamholtz (1983), Flamholtz et al. (1985) and Simons (1991)
21
2.4 Effects on Business Performance
In the introduction section, I mentioned studies of Lingle, Schiemann (1996) and Gates (1999) besides that of others that organizations managed through performance measurement systems perform better than organizations that did not. I also mentioned a study conducted by Ittner and Larcker (2003) that came up with evidence to the contrary. The question that arose is how business performance should be made operational within this study, to make viable conclusions because business performance is a multi-dimensional construct, argued by academics (Fitzgerald and Moon, 1996). Analysis of performance measurement systems on business performance has resulted in an overview (table 6.) of results that highlights the positive and negative effects after the implementation of a performance measurement system.
Author(s) Business performance Impact
Lingle and Schiemann (1996)
Gates (1999)
Organizations managed through performance measurement systems perform better than organizations that did not.
Positive
Lawson, Stratton and Hatch (2003)
Use of performance measurement systems as a management control-tool will reduce overhead costs by 25 percent and sales and profits will increase.
Positive
Dumond (1994) Use of performance measurement systems will contribute to decision-making performance of managers as well as employees.
Positive Lawson, Stratton and
Hatch (2003)
When performance measurement systems are linked to compensation systems, a significantly boost of employee satisfaction could occur.
Positive
Forza and Salvador (2000) Employee communication based on performance measurement systems increases collaboration and facilitates buy-in of strategic objectives.
Positive
Ketelhohn (1998) and Vasconcellos (1988)
When choosing appropriate measures will improve the implementation and recognition of business strategy and staff its business understanding.
Positive
Ittner and Larcker (2003) Their study has tested quality and customer satisfaction which increased more rapidly one year after the implementation of the performance measurement system in relation towards a year before. A result of this improvement was that profit decreased. The positive effect on customer satisfaction as well as quality, after the implementation, has resulted in an increase in turnover
22
which not recovered investments associated with these improvements and therefore the overall performance was decreasing.
Greatbanks and Boaden (1998)
CIMA (1993)
Barnes, Mills, O'Brien, and Rowbotham (1999) Hudson, Smart and Bourne (2001b)
Veitch and Smith (2000)
Their study revealed that operational measures within performance measurement systems are mostly ad-hoc and informal with no real understanding of key performance drivers which explain why these systems are ineffective in supporting the achievement of strategic goals, which in turn leads to a decreased performance of these small and medium-sized organizations.
Negative
Table 6. Business performance
Table 6 shows that a large majority of studies revealed that the impact of performance measurement systems on business performance of small and medium-sized enterprises was positive besides a small number of two studies that came up with evidence to the contrary. Therefore, in this research, the assumption is made that performance measurement systems impact positive on business performance of small and medium-sized enterprises but to support this assumption it is necessary to test the design of the performance measurement system, the implementation of it, as well as the use of such a system. By making use of the research framework of Pettigrew et al., 1989 on content, context and process it is possible to test the factors that cause the use of performance measurement systems to an increased performance of small and medium-sized enterprises. Moreover, to assess the performance of the cooperating organizations, it is necessary to come up with a general construct that measures the performance of these small and medium-sized enterprises.
23 Cowling, 1997). However, there is harmony that no single financial indicator can precisely and comprehensively capture business performance, particularly in small and medium sized enterprises (Begley and Boyd, 1987; Daily and Dollinger, 1992). Therefore, it is preferable to develop a various measure of financial performance and interpret the results based on one indicator in combination with others. Multiple financial measures are consistent with the theoretical point of view that business performance, as a construct, should be viewed as being multi-dimensional to face diverse purposes and types of businesses (Levin and Minton, 1986). To determine business performance, Daily et al., (1992) suggested sales growth, rate of improvement of net margin and operating margin as measures. Jennings and Beaver (1997) calls for sales growth, profitability, return on capital employed, cash flow, employee productivity and job creation to measure performance of small firms. Morris et al., (1997) utilizes sales growth, profit margin, employment growth, asset growth and new markets its products to study the success of small and medium sized organizations.
In this research, a combination of indicators as outlined above, to assess business performance of small and medium sized enterprises, is used. The choice have been made for multi-dimensional indicators to show a more comprehensive view of performance and this will help decrease a possible bias. The elements of this multi-dimensional construct are profitability of the business, the potential of the business to develop further, and the effectiveness of the business, which will be nurtured by five indicators based on earlier research.
To assess the profitability of the business, the financial variables: profit margin and return on capital employed are used (Jennings and Beaver 1997; Morris et al., 1997). Sales growth as well as employment growth (Daily et al., 1992 and Morris et al., 1997) is used to measure the potential of the business to develop further. The last variable, to asses the effectiveness of the business, is employee productivity (Jennings and Beaver., 1997). A summary of the variables in conjunction with the elements of a multi-dimensional construct is displayed in figure 2.
24
2.6 Literature summary
On the one hand, organizational researchers and managers claim that performance measurement systems have a positive impact on business performance. On the other hand, there is little research carried out that test this hypothesis (Kennerley and Bourne, 2004; Davis and Albright, 2004; Schiemann and Lingle, 1999; Franco et al., 2004) and as a results, a mixed and confused picture exist whether these systems impact positive or not. Lingle and Schiemann (1996) and Gates (1999) besides others argue that organizations managed through performance measurement systems perform better than organizations that did not. Ittner and Larcker (2003) besides others came up with evidence to the contrary. Published research is scarce, by reviewing the literature along the framework of Pettigrew et al., 1989), the assumption is made that well designed performance measures, an excellent implementation as well as a correct way of using a performance measurement system contribute to an increased performance of small and medium-sized organizations. This assumption is made due to the fact that the majority of studies revealed that performance measurement systems impact positive on business performance.
Although, a significant stream of research is carried out on the impact of performance measurement systems on business performance in large organizations; this is not reflected in small and medium-sized enterprises (Franco and Bourne, 2004). Research revealed small and medium-medium-sized enterprises exhibit different characteristics that differentiate them from the majority of large organizations. Limited resources, limited cash flow, few customers, a flat organizational structure are some of the aspects that need to be considered. Besides this there are basically two main obstacles; the lack of financial and human resources, and endogenous barriers, short-term planning and the perception of performance measurement systems as bureaucratic systems that cause rigidity (Garengo et al., 2005)
25 proposed for his work on organizational change but it makes a distinct contribution and offers two salient advantages. First, there is a significant acceptance of content, context and process among leading contributions to information systems evaluation theory. Second, it provides the content, context and process factors, which need to be considered in evaluating performance measurement systems due to the fact that these factors impact on the outcome and therefore worthwhile to consider. This parsimonious framework of Pettigrew et al., 1989 consisting of three variables, offers me room for building my own individual evaluation model based on research findings of others on design, implementation and use.
26
3. GENERATION OF HYPOTHESES
The assumption is made that performance measurement systems impact positive on business performance of small and medium-sized enterprises which is supported by studies of Lingle and Schiemann (1996), Gates (1999), Lawson, Stratton and Hatch (2003), Dumond (1994), Forza and Salvador (2000), Ketelhohn (1998) and Vasconcellos (1988). A conceptual model (Figure 3) will help me explain the factors that might affect performance of small and medium-sized enterprises. The content, context and process of performance measurement systems all influence the outcome and therefore these three factors are distinguished in the conceptual model. Due to the fact that no single financial indicator can precisely and comprehensively capture business performance I have developed a various measure of business performance and interpret the results based on one indicator in combination with others to test business performance.
Figure 3 Conceptual model
Based on the literature review and the conceptual model the following research question is formulated:
27 To operationalize the testing, three dimensions of the framework of Pettigrew et al., 1989 on process, content and, context are distinguished and integrated, into the proposition to test the effect of performance measurement systems on the performance of small and medium-sized, generating three hypotheses.
H1. The content of performance measurement systems impact positive on business performance.
H2. The context of performance measurement systems impact positive on business
performance.
28
3.1 Research methodology
In order to progress the research, access to small and medium-sized enterprises was needed. However, defining the small and medium-sized enterprise sector was not easy, as there are differences in what is appropriate to describe as small or medium-sized enterprise in different industries (Burns, 2001; Storey, 1994). Therefore, a definition established by The European Union for small and medium-sized enterprises is utilized, shown at table 7. This table revealed that access to organizations with more than 10 and less than 250 employees is needed to fit the definition of small and medium-sized enterprises during this study.
Criteria Micro Small Medium
Maximum number of employees 9 49 250
Table 7. European Union definitions of micro, small- and medium-sized enterprises
Besides that, performance measurement systems should be operational for more than two years, so that it is embedded and not a new system (Bourne et al., 2000; Evans, 2001; Martins, 2002). Another important element is that performance measurement systems should consist of a number of both, financial and non-financial criteria, ideally that represents the perspectives of a balanced scorecard. Very few performance measurement systems are developed academically, which not reflect the perspectives of a balanced performance framework. One of the expectations is the balanced scorecard, which reflects both, financial and non-financial criteria. Therefore, it is necessary to execute this study in organizations that make use of performance measurement systems that are based on balanced performance measures, due to their academic relevancy. Balanced performance measures are preferable because organizations have traditionally adopted a narrow or unidimensional focus and are able to overcome these problems by adopting a balanced set of measures.
29 designed to minimize external contextual differences by executing this study in the same sector, that of production. To exclude the interference of other factors that might affect business performance it was necessary that no significant changes occurred when such a system was operational. Changes in the organization its internal as well as external environment refers to significant alterations in the composition of management and management style, new markets and products, new technologies, regulation, policies etc. Due to the aforementioned hard conditions, only two organizations fit the conditions on size, sector, system maturity, balanced measurement systems and, did not perceive a major shift in their internal and external environment which makes it possible to drawn conclusions by eliminating the interference of these factors that might affect business performance. By doing so, it was possible to study the effect of performance measurement systems initiatives on business performance of small and medium-sized enterprises.
30 questions are answered proper. So, during this second round of interviews, only verification of answers has been conducted to test whether my interpretation of transcript data was right transformed digitally. As a result of these two phases, with twelve interviews in total, a clear picture emerge of how these two organizations cope with performance measurement systems. An elaborated version is adapted in appendix 1 and 2. Information about the organizations, the involved employees as well as the systems that are in use, are adapted in paragraph 3.2.
Because it is hard to draw conclusions from interview results, I have based my research approach on Miles and Huberman (1994) its view of qualitative analysis. They focus on three phases; data reduction, data display and conclusion drawing and verification. Data reduction refers to the process of selecting, focusing, simplifying, abstracting and, transforming the data that appear during the interviews. As a result of the conducted phone calls and interviews, rough data appear that is adopted in appendices 1 and 2. The second major flow of analysis activity is data display. Generically, a display is an organized, compressed assembly of information that permits conclusion drawing and action taking. Looking at the displays helps me to understand what is happening and to do something. After the analysis of rough data a summary out of the interview results is established that reflects how these organizations design their measures, how they implement such a system and how they communicate upon these systems which is displayed in table 9, 10 and, 11. Besides that display of factors, that relate towards the dimensions within each area of investigation, scores to each dimension are allocated. The third stream of analysis is conclusion drawing and verification. Verification of interview data takes place by a discussion of these findings in which I compare the displayed interview results of the organizations as shown in table 9, 10 and 11 to the dimensions as outlined in the background section of the framework of Pettigrew et al., 1989 in the three areas of investigation.
31 these dimensions. For example, the content of performance measures consists of seven recommendations towards the design of performance measures. Inadequately designed performance measures results in dysfunctional behavior that pursue inappropriate courses of action that have a negative effect on business performance. Therefore, it is necessary to score the dimensions that are in line with the recommendations and those that did not. To proof of falsify the formulated hypotheses H1, H2 and H3 on content, context and process it is necessary to measure all of the dimensions within the participating organizations and score them. To make it more tangible for the reader, figure 3 is displayed as an example of how the different dimensions will be measured by the icons plus and minus. A plus score means that one of the dimensions is in line with the recommendations based on earlier scientific research. A minus score means that one of the dimensions is not in line with the recommendations made by these authors. The authors and their recommendations are reviewed exhaustive by making use of the framework of Pettigrew et al., 1989 as showed in the background section.
Performance measures should…
Derived from strategy Clearly defined with an explicit purpose
Organization 1 Measures derived from strategy
All measures drive/support strategy Automatically collected from source databases
All of the measures are objective rather than based on opinion
Not all measures are clearly defined, elimination of measures will take place in the near future
Display right info to support strategy All based on snapshots rather than trends
Organization 2 Measures derived from strategy and yearly
revision of measures due to market circumstances take place
Clearly defined
Majority defined with explicit purposes or goals
Measures are consistent over time Measures employs ratios and numbers Measures are exact about displayed info Scores
Figure 4. An example of how to measure different dimensions of content, context and, process
32 assigning a percentage to each dimension that is in line with the recommendations. In theory, a total percentage to all of the dimensions of 100 means that all of the dimensions are understood and therefore the hypothesis should be proofed. This study assumes that in practice it will not be feasible that the organizations gain a 100 percent score on each area and therefore all of the formulated hypothesis should be falsified which is not a fair conclusion. Besides this measure of 100 percent, it is also not fair to conclude once 51 percent or more is achieved that all of the dimensions within each area of investigation are well understood and therefore impact positive on the performance of small and medium-sized enterprises. Therefore, in this study is assumed that once 75 percent or more of the dimensions are in line with the recommendations, than this area of content, context or process is proofed and have therefore a positive impact on the performance of small and medium-sized enterprises. For example, the area of investigation on content consists of seven recommendations towards the design of performance measures. One hundred percent divided by seven dimensions equals 14.28 percent per dimension. This percentage will be allocated to each dimension that is in line with the recommendations towards design. Once the final percentage is calculated, it becomes possible to conclude whether each area of content, context and, process is well understood. For example, once 75 percent is achieved this study assumes a positive impact on performance. This is because the literature discussion and as a result of this discussion, hypotheses H1, H2 and, H3 revealed that the organizational context, performance measurement content and process impact positive on the performance of small and medium-sized enterprises.
34
3.2 Research organizations 3.2.1 Organization X
The first organization is a cable producer in the Netherlands who would like to remain unknown due to confident information that has published in this research. This organization made a shift to a technological leading producer of cable solutions with customers all across Europe. This organization has access to various international marketing, purchasing, sales and research groups with diverse specialisms. Reliability is one of the most import aspects to this organization; this applies for its technology, products, logistics and services. Thinking along with its customers and flexibility are essential for the success of this organization. They focus on a number of market segments; broadband, energy, marine & offshore, railinfra, home and, utility. Together with its customers, they continue to develop the way to distinguish itself now, and in the future.
35 project at the beginning because he saw no advantage in such a system however, once he was involved he saw benefits that were interesting to him too. By the use of some simple keyboard and mouse clicks, up-to-date data about the processes appears what resulted in increased communication provision to lower level employees as well as communication to same level employees. Further information about the interviewees cannot be provided because all of the interview questions are straight forwarded and established to gather information about their performance system whereas detailed information about the interviewees is are irrelevant for the purpose of this research and therefore not asked.
37 achieving the budgeted turnover in conjunction with the wages when achieving high levels of production.
3.2.2 Organization Y
The second organization provides production, marketing, sales and distribution of medicines and self-care products in the Benelux. The establishment of this organization thirty years ago resulted in a second CEO of this organization that direct this organization to its goals. The CEO fulfills an executive function to almost ninety employees along three separate business units. This organization has different offices across with management and warehouses centralized at one place. They focus on specialized niche products, with medical specialists as its target group. It represents the interests of internationally respected pharmaceutical companies. It also markets its own produced products across Europe.
38 has implemented such systems too. Minor information about the personal motivation towards the introduction of a performance measurement applies for this organization too. This is because that all of the semi-structured interview questions are related to factors that need to be studied on the basis of the framework of Pettigrew et al., 1989 rather than their personal career because this will not impact on the performance of such a system.
40
4. RESEARCH FINDINGS
Gathering up threads from different organizations was a difficult process due to the fact that a significant number of what was happening across the two organizations was equivalent. Therefore, the assumption is made to reduce repeating similarities and focus more on the differences across the two organizations. A compressed assembly of research findings out of the interview data is displayed in tables 9, 10 and, 11 that is in line with Miles and Huberman (1994) its second phase of data display. Hereby, the focus is mostly on the differences rather than repeating similarities. A more detailed and elaborated version of the research findings are adopted in appendix 1 and 2 where interview results are reflected to the dimensions that need to be tested, based on the framework of Pettigrew et al., 1989 on content, context and, process. Insights of the performance of each individual organization become clear by the adoption of tables 8a/b. A discussion of how the organizational context, performance measurement content and, process relate to the performance of these organizations and besides this, to earlier studies on this topic, are described in the discussion of findings.
Year 2005 Profit margin Return on capital employed
Employment growth
Sales growth Employee productivity Organization X 53.579 / 685.492 = 7.8 % 35.366 / 631.913 = 5.6 % 243 € 653.699 € 2690,- revenue per employee Organization Y 12.133 / 186.743 = 6.5 % 7.351 / 162.158 = 4.6 % 104 € 171.547 € 1649,- revenue per employee Table 8a. Business performance results before the implementation of a performance measurement system
Year 2008 Profit margin Return on capital employed
Employment growth
Sales growth Employee productivity Organization X 73.172 / 997.036 = 7.3% 50.316 / 923.864 = 5.4% 225 € 964.952 € 4288,- revenue per employee Organization Y 11.745 / 191.157 = 6.1% 8.141 / 214.623 = 3.8% 91 € 246.535 € 2709,- revenue per employee Scores
41 Performance measures
should
Derived from strategy Clearly defined with an explicit purpose
Relevant and easy to maintain
Simple to understand and use
Provide fast and accurate feedback
Link operations to strategic goals
Simulate continuous improvement
Organization X Measures derived from
strategy All measures drive/support strategy Automatically collected from source databases All of the measures are objective rather than based on opinion
Not all measures are clearly defined, elimination of measures will take place in the near future
Display right info to support strategy All based on snapshots rather than trends
All users responsible for establishment of measures Able to maintain in conjunction with others Garbage in – Garbage out Not all relevant Easy to maintain
Univocal to all users Consensus about formulation of measures Differences in
interpretation of data Visual impact to all of the users by traffic light Combination of ratios and numbers
Measures provide timely feedback
Transactional data not older than 3 days Precise information – sometimes adjustments
All measures are set up to drive strategy Reflects necessary operational information by traffic lights
Redesign of measures every year on basis of customer survey and supplier performance
Established to focus on improvement, in practice, means of controlling and not for maximizing improvement
Organization Y Yearly revision of
measures due to market circumstances
Clearly defined Majority defined with explicit purposes or goals
Measures are consistent over time
Measures employs ratios and numbers Measures are exact about displayed info
Majority of measures are relevant and easy to maintain Beginning of each year formulation of measures or adjustment during the year Automatically collected as part of a process
Interpretable to all users Users able to readjust measures on their own All measures have visual impact by traffic lights
Simple and consistent format
Timely and accurate feedback on data of max 1 day old
System designed to display real fast analysis
All display snapshots rather than trends Performance measures reflects the process and link to strategic goals All the measures are objective rather than based on opinion
Continuous monitoring of suppliers and customers measures used as means of controlling rather than maximize performance
Scores
42 System
maturity
Organizational structure
Organ. size Organizational culture
Management style Competitive strategy
Resources and capabilities
IS-infrastructure Other systems
Organization X Qlikview is operational for approx. 2 ½ years Alignment between organizational structure and measures Professional organization with top management and staff member to lower-level management and blue color workers (operational users) Individuals with strong control over their work
225 employees divided over 2 locations in the Netherland with central headquarter 25 privileged balanced scorecard users Strong culture Staff responds to stimulus because of alignment to organizational values Entrepreneurial culture of doing the right thing Commitment and responsibility Attention to basics
Work hard; play hard Fun and action Hands-on-mentality A little shift in style before and after use of PMS-system.
After: hard qualitative data Before: mental models to manage Strong alignment between measures and strategy Measures derived from strategy at initial implementation phase Championed by CEO Lower-level managers fully involved during implementation Every year readjustment of measures take place Clickview as balanced scorecard tool on Windows 2003 network No garbage in, garbage out Different sources, different views. High data integrity
Performance measurement system is aligned to HRM, ERP, quality, planning and scheduling and, reward systems. Organization X Business Analytics operational for 2 years
Top CEO and 3 autonomous business units with their own managing directors. BU1 = 30 FTE, BU2 = 25 FTE, BU3 = 20 FTE 15 managers across the organization
No significant alignment between hierarchical levels and performance measures In total 90 employees operating in 3 different business units 20 privileged scorecard users from which 50% has a management function
Values are clearly defined and aligned with the
measurement system
Owner managers are seen as role models
Employees allowed doing it their own way with supervision of higher level managers. Weekly and Monthly reports and meetings are communicated across the organization Walks the talk
Management style has been changed after the
implementation towards qualitative data mngt.
Remain important market share in the Netherlands Vision: being a reliable partner, now and in the future Mainly derived from strategy. Yearly revision of measures to support strategy Project champion (CEO) Project team of 3 employees from different disciplines Consultant guide the process of implementation and offers training possibilities
Business Analytics on a Navision ERP system based on Windows NT4 Reliable data out of Navision into PMS due to extensive testing before implementation So, high integrity of data PMS aligned to ERP, HRM system. Budget practices available in Navision. Scores
43 Alignment with strategic
objectives
Data capture Data analysis Interpretation and
evaluation
Communication and information provision
Decision-making Taking-action
Organization X Rework reflects training and personal issues
Employees are important besides insights in their performance
All the formulated measures are established to drive
performance
Source performance data max 3 days old Rely on automatic data acquisition
Occasional down drill to business level
Analysis of profit and losses with staff and higher-level managers Evaluation against formulated budgets Information is evaluated against targets Means as controlling standards and not for maximizing profit. Performance reports are send weekly to the mailboxes of employees
Decided to eliminate some products in the products portfolio due to the fact that some did not make profit
The associated costs were higher in relation to the generated turnover
Action is based on the gathered data. See example of products that are eliminated due to no contribution to profit
Action is based on trends rather than snapshots to be more concise.
Organization Y During implementation all
measures have been derived from strategy
Mental models besides hard qualitative data are used to manage the organization Well trained employees make significant use of the system
Daily mirror of source database
Garbage is minimal due to extensively testing. Displaying right performance data Occasional drill down to business level
Old employees uses manual workload displays with no connection to performance system Analysis of weekly and monthly reports Evaluation against formulated budgets Information is evaluated against targets Communication of gathered performance through business unit on a daily basis
Profit and loss account information is reviewed weekly
When necessary one-to-one meetings with sales representives
Decided on scorecard information to go ahead with another supplier of packages due to poor quality of the present supplier
On the basis of budget sales with sales representatives when necessary
On ‘red’ traffic lights meetings are planned one to one and on departmental level
On daily received performance mail is action taken when necessary
Scores
45
4.1 Discussion of findings
A distinction between three areas of investigation of content, context and, process is made to discuss the findings in relation to the performance of small and medium sized enterprises.
4.1.1 Content
