Research in Performance Measurement:
Comparing the Private Sector and the Public Sector
/literature review/
Stanimira Stoyanova
Master Thesis
University of Groningen
Faculty of Economics and Business
MSc Business Administration
Specialization Organizational & Management Control
August 2012
Supervisor: Dr. S. Tillema
Research in Performance Measurement:
Comparing the Private Sector and the Public Sector
/literature review/
Master Thesis Stanimira Stoyanova
University of Groningen
Faculty of Economics and Business
MSc Business Administration
Specialization Organizational & Management Control
August 2012
Stanimira Stoyanova
Address: Friesestraatweg 286, Groningen, 9718 NS
Email Address: Stanimira.Stoyanova@yahoo.com
MSc Business Administration
Specialization Organizational & Management Control
Student Number: s2189631
Supervisor: Dr. S. Tillema
Preface
The paper You are about to read is the final step to receiving my master’s degree in Business Administration: Organizational & Management Control at the Faculty of Economics and Business, University of Groningen.
I was inspired for this research during my study in the University of Groningen, when I discovered that little attention is paid to the public sector in our study programme. By means of a literature review of studies on the topic under consideration some interesting results and directions for future research were established.
Hereby, I take the opportunity to thank my family and friends who believed in me and were
extremely supportive while I was studying and writing this paper. I am most grateful to my supervisor Dr. S. Tillema for providing me with professional advice and feedback. I would also like to thank to Prod.dr. H.J. ter Bogt for co-reading my thesis. Last but not least, I am thankful to all the lecturers I had who made my study here a great experience. I am certain that all I learned during the past year will be an important advantage for my future career.
Table of contents
Abstract 1
1. Introduction 2
2. Theoretical background 4
2.1. Public versus private characteristics related to ownership, funding and control 4 2.1.1. Differences in ownership, funding and control between the public and the
private sector organizations 4
2.1.2. The idea of convergence and The New Public Management reforms 8
2.2. Content-related dimension 9
2.2.1. Stages of performance measurement systems development 9 2.2.2. Types of performance indicators 10
2.3. Research-related dimension 12
2.3.1. Types of methodologies and their link to the types of research 12
2.3.2. Types of theories 13
3. Goal of the research, research questions and method 15
3.1. Goal of the research 15
3.2. Research questions 15
3.3. Research methodology 15
4. Results from the literature review and analysis 18 4.1. Categorization according to the stage of PM system development discussed 18 4.2. Characteristics of public and private organizations emphasized in the studies 19 4.2.1. Implications for the design stage 19 4.2.2. Implications for the implementation stage 21
4.2.3. Implications for the use stage 22
4.3. Categorization according to the types of performance measures discussed 26 4.4. Categorization according to the research method applied 29 4.5. Categorization according to the type of theory represented 30 4.5.1. Categorization according to the type of high-status theory represented 32
5. Answers to the research questions, directions for future research and limitations 34
References 36
Tables:
Table 4.1. Stages of PM system development 18 Table 4.2.3. Negative effects of PM systems 25 Table 4.3. Types of performance measures discussed 26
Table 4.4. Research methods applied 29
Table 4.5. Types of theories represented 31
Research in Performance Measurement:
Comparing the Private Sector and the Public Sector
/literature review/
Abstract
The measurement of the performance in public and private organizations has received prominent attention in Management Control literature. This paper discusses whether significant differences and similarities exist between research approaches of studies on performance measurement practices in the public and the private sectors. For a long time, it has been claimed that the different
characteristics of public and private organizations have implications for performance measurement and performance measures (Drake, 1972; Rainey et al., 1976). More recently, however, some researchers have argued that the boundaries between the sectors are blurring (Bozeman, 1987; Boyne, 2002). The aim of this study is to explore which of these statements is supported. Drawing on a literature review of 130 studies, equally distributed between the public and the private sector literature stream, some significant differences are discovered in respect to the topics discussed, the methodologies and the theoretical backgrounds represented by the two types of studies.
Key words
Performance management, performance measures, performance indicators, public sector, private sector
Supervisor: Dr. S. Tillema
1. Introduction
The measurement of the performance in public and private organizations has long been of central interest, not only to managers, but also to governments, accounting researchers and different groups of stakeholders. What is exactly meant by the term “performance” frequently lacks an explicit explanation, but we will assume in this thesis that it refers to the degree of effective and efficient attainment of organizational goals (Otley, 1999). Then, the naturally divergent goals and mission of public and private sector organizations will give rise to a different context for performance
management. The main objective of business organizations is to maximize the value of the firm, or to create wealth for its owners. In contrast, public organizations usually do not have any outside equity interests and their primary purpose is to provide some kind of public service (Merchant and Van der Stede, 2007). For this reason, measurement in public organizations should not be constrained to financial aspects, but should be in terms of meeting the expectations to all the interested groups – the people to whom the “performance” should be ultimately delivered. Thus, measuring
organizational performance while striving to achieve overall objectives is not a straightforward task, especially not in the public sector where the degree of achievement of the provision of intangible services can hardly be measured in accounting terms. Nevertheless, the need for measuring performance and evaluating activities is essential for management control in each type of organizations.
For a long time, it has been argued that public and private sectors are inherently different, and that this has implications for the performance measures (Drake, 1972; Rainey et al., 1976). In the last few decades, however, the boundaries between the sectors are said to be blurring (Golembiewski, 1985; Bozeman, 1987; Boyne, 2002). Public management has gained prominent attention in recent years, particularly since the concept of New Public Management (NPM) reform which was first introduced by Hood in the 1990s. It has been a highly debatable subject whether transplanting successful business practices to government settings (a central element of the reform programme) is
reasonable as this means that an assumption about public and private organizations being similar is made. If this is indeed the case, there is no need for differences in terms of research focus, methods and outcomes. However, if some persistent differences appear to exist, either the two streams of literature can learn from each other, or the blurring is not as strong as suggested, implying that the streams cannot be integrated.
and how are they accounted for in the academic literature. If indeed a convergence of the sectors proves to be taking place, they can learn from each other’s successful practices.
2. Theoretical Background
In order to see if there are some distinctive characteristics between the literature in the private and in the public sector I will compare them according to three main dimensions: a dimension, related to the differences of ownership, funding and control between public and private organizations and whether they have implications for the performance management; a content-related dimension, exploring the types and the stages of development of PM systems, and finally, a research-related dimension, categorizing the studies on the basis of their theoretical foundation, research
methodology and the main focus of the studies. In this way a more comprehensive and systematical comparison is intended.
2.1. Public versus private sector characteristics related to ownership, funding and control: I will first investigate a number of propositions about the similarities and differences between public and private organizations and their implications for the performance management. For this purpose definitions of the two types of organizations will be useful.
A number of approaches have been used in the literature to draw a distinction line between them. Rainey et al. (1976) argue that a main difference between public and private organizations is their ownership. Private firms, on the one hand, are owned by entrepreneurs or shareholders, and public agencies, on the other hand – by collective political communities. Another frequently cited
differentiation between the two types of organizations is on the basis of their primary goal. It is widely agreed that the main objective of business organizations is to maximize the value of the firm, or to create wealth for its owners, whereas public organizations usually do not have any outside equity interest and their primary purpose is to provide some kind of public service or product (Merchant and Van der Stede, 2007). These distinctions, however, are seen as insufficient by a number of authors who include other internal and external variables in the public-private
categorization to create a more complex, multidimensional continuum (Wamsley and Zald, 1973; Bozeman, 1987; Golembiewski, 1987; Rainey, 1989; Boyne, 2002).
2.1.1. Differences in ownership, funding and control between the public and the private sector organizations
he presents to support his view that organizations are never wholly public or private. Instead, they should be positioned along the complex dimensions through an empirical study of the extent of publicness. Wamsley and Zald (1973) build a distinction upon ownership and funding. Their book leads to a similar conclusion: although most organizations can be categorized as either public or private, there is some “blurring” of the boundaries, because a number of organizations have a mixed status – for instance, government agencies funded by customer charges or operated for profit and private organizations funded by the government.
These dimensions of publicness have important theoretical and practical implications for
organizational behavior and management control. One significant contrast stemming directly from the form of funding, for example, is the presence or absence of economic markets as a source of resources and constraints to the operating efficiency and effective performance. In private organizations the presence of the market enforces a relatively direct relationship between
performance and rewards (or punishments), and hence, provides incentives for the shareholders to control and monitor the performance of the managers, responsible for cost reductions, operating efficiency and effectiveness (Boyne, 2002). Some economic theorists (Clarkson, 1972) suggest that the collective political ownership results in lower efficiency of public providers. Organizations which obtain resources through political budget allocations instead of market performance emphasize less on cost reductions and according to Drucker (1973) tend to be more ineffective. Boyne (2002) attributes this to the fact that managers usually do not receive direct financial benefits from better organizational performance. Lindblom (1977) recognizes two alternatives for control over the process of decision making: “poliarchies” (which means political or hierarchical controls) and markets. Organizations that are subject to poliarchical constraints have different control practices from those subject to markets, although he argues that empirical comparisons in this respect are insufficient.
Another consideration associated with the property rights that Boyne (2002) cites is that public agencies face political rather than economic controls as well as multiple sources of authority that are sometimes conflicting. As Wamsley and Zald (1973) note, public organizations are perceived as owned by the state and the citizens, which consist of various groups: voters, consumers, political groups and others. As a result the mix of objectives is believed to be more complicated because of the multiple stakeholders as the public ownership and funding implies that citizens place
organizations as diverse, vague and conflicting (Lyden, 1975; Rainey et al., 1976). The distinctive for the public sector goals such as equity and accountability are claimed to require specific management practices (Ranson and Steward, 1994). The contradicting interest of the different groups of
stakeholders and the resulting more complicated mix of objectives according to Baker (1969) and Rainey et al. (1976) lead to a more complex mix of performance evaluation criteria in public than in business organizations.
A study by Rainey (1983) emphasizes differences in various aspects of the internal processes of public agencies and private firms, namely organizational goals, incentive structures and individual roles. A first consideration is that public sector entities face more formal external constraints due to their ownership, making them less flexible and imposing a greater need for accountability. Political scientists and legal scholars frequently pay attention to how laws regulate behavior in various sectors; to the legal communities which require compliance; to the press and the general public opinion, to individual clients and other interest groups (e.g., Stigler, 1975; Zald, 1978). Considerable evidence exists to show that these factors exert greater pressure on public managers than on their private counterparts. Bozeman (1987) also sees political influence as essential for his model of publicness.
Arguments such as these are the basis for the hypothesis that perceived levels of formalization are higher for public managers (Bozeman and Kingsley, 1998). This is relevant to incentive structures because it limits the discretion of public managers due to various procedures and rigid rules
concerning personnel practices such as hiring, pay rises and promotions. Some researchers state that personnel rules contribute to the weak link between individual performance and extrinsic incentives (Savas and Ginsburg, 1973) and that this in turn weakens individual motivation and organizational commitment (Perry and Porter, 1982).
Moreover, Drake et al. (1972) express their concerns about applying quantitative methods in the public sector in association with the vague and diverse objectives. The multiple aspects of
government activities make quantitative analysis and specifying performance measures problematic. The goals in the public sector are perceived as more complex and ambiguous by managers and Drake et al. (1972) argue that this leads to multiplicity, conflict and imprecise definition and measurement of performance.
ambiguity and role conflict, and clarity of organizational goals. The proposition that public
organization tend to have higher levels of perceived formalization is confirmed though. We may infer from Rainey’s study that even though the literature frequently advances assertions concerning the greater multiplicity, vagueness and conflict of public organizations objectives, statistical evidence of such propositions are insufficient.
Propper and Wilson (2003) argue that because of the multiplicity of goals and principals
(stakeholders) in the public sector, performance management will not necessarily lead to the same results as in private sector organizations, where individuals typically pursue clearly defined tasks and goals. High-powered incentives are less likely to be suitable if individuals found in the public sector are less materialistic, more strongly motivated by non-pecuniary benefits and desired to serve the public interests as it is asserted by some researchers (Posner and Schmidt, 1982; Rainey, 1983; Gabris and Simo, 1995;).
Business organizations, on the other hand, rely on the market as a source of clear quantitative targets, goals and performance measures. Such information facilitates performance evaluation and formulating clear objectives. Quantitative demand indicators aid operating efficiency, allocational efficiency and effectiveness, because, facilitated by the existence of marker mechanisms,
organizations can more adequately react to consumer preferences by matching supply and demand (Davies, 1971; Savas 1974).
2.1.2. The idea of convergence and the New Public Management reforms
In the last few decades a new body of literature has advanced the idea that there has been a convergence between the sectors. On this assumption the idea of general management has
developed. In the words of Murray (1975): “Boundaries between public and private activity seem to be blurring. At any rate, in areas such as attitudes and values it will require a good deal more
behavioral research to establish the fact of difference as opposed to the sentiment of difference.“ In his exploratory study he brings into question the assertion which was supported in the literature for a long time that public and private management are inherently different. He stresses that one should distinguish between distinctions and differences: for example, much like their public counterparts, private managers are also very sensitive to the levels of perceived ethical behavior although
pressures for them are usually informal. Even if legal or formal constraints exist, this is a “superficial” difference with no significant impact on practice.
Weinberg (1983) argues that, when one is looking for similarities and differences, it is very important that similar activities are compared. She states that managers in both sectors may have difficulties measuring the quality or quantity of a product, especially in service providing organizations. She proposes a research agenda going beyond the conventional public-private segregation, because common pressures and problems may exist in both sectors. If this proves to be the case public and private managers may learn from each other and borrow techniques of one sector for use in the other.
During the 1980s the so called New Public Management (NPM) reform programme became a popular theme in public policy. The basic assumption behind it is that public and private organizations are alike so successful business practices (such as performance-related pay and management by
objectives) should be transferred to public activities (Hood, 1991; Ferlie et al., 1996; Box, 1999). NPM implies a move toward more specific emphasis on measuring performance to assess more accurately efficiency and effectiveness. Hood (1995) observes the various descriptions of NPM, and records several features of it, persisting in the literature, among which are: a shift in focus from process to output, from fixed pay to pay based on job performance, from hierarchies toward a more
competitive style of providing public services and from policy making to management skills.
The adoption of business-like models of management has been criticized on the grounds that public organizations differ in a lot of crucial respects from their private counterparts. If the distinctions explained above turn out to have a significant impact on PM, the criticism would be reasonable. That is why the question I will address in chapter 4 of my study is: Do these distinctions have important
influence? For this purpose I will analyze two streams of literature on PM: literature concerning the
public sector and literature concerning the private sector. I will analyze if there is a good
communication between the two fields – can they learn from each other and are there some gaps in the academic literature. On the grounds of this analysis I will make propositions for future research. 2.2. Content-related dimension
2.2.1. Stages of PM systems development
Another criterion for categorization that I will use is related to the content of the studies. I will use the stages identified by Bourne et al. (2000) who propose that the development of PM systems can be divided into three main phases:
1/ The design of performance measures;
The design stage consists of two important elements: identifying the strategic objectives to be evaluated and designing the measures that encourage behavior in conjunction with the strategy. There has been wide consensus in the literature that measures of performance should effectively deploy strategy (Globerson, 1985; Eccles and Pyburn, 1992; Kaplan and Norton, 1992; Carrie and Macintosh, 1992; Bititci et al, 1997).
2/ The implementation of performance measures;
Implementation is defined as the process of putting in place the systems that collect the information needed for a comprehensive measurement. This may involve changes in practices for analyzing data already being used or entirely new procedures may be introduced to gather currently unavailable information.
3/ The use of performance measures;
The use of performance measures serves two key purposes together, creating a so-called “proactive closed loop control system” (Bititci et al., 1997). First, as performance measures deploy strategy they provide feedback if the implementation of that strategy is successful. Second, the information from the measures should be used to assess the effectiveness of the organization and to challenge the validity of the existing strategy in order to make continuous improvements (Grady, 1991; Kaplan and Norton, 1996-a; 1996-b).
implemented before the design stage is completed or the phases of implementation and use can overlap.
In chapter 4 of my paper I will examine if there are some tendencies in public and private sector literature regarding their focus on the different stages. Bourne et al. (2000) point out that most of the recent literature has been dedicated to initial stages of development such as conceptualization and design, but few studies have gone as far as successful implementation and regular review of PM systems. I will check if this is the case and whether it holds true for the public sector stream of literature too, because it is worth taking into consideration that this study basically refers to manufacturing enterprises.
2.2.2. Types of performance indicators
The second content-related dimension I will be using is the type of performance measures that the different studies place emphasis on.
A wave of criticism leveled at traditional performance indicators, which were developed from costing and accounting systems, has risen in the 1970s and 1980s. Some of the main shortcomings discussed in the academic literature are their short-term oriented view - as managers face a trade-off, they may prefer to enhance current profits at the expense of long-term strategic goals (Banks and
Wheelwright, 1978; Kaplan, 1986); their inadequate strategic focus and inability to provide feedback about quality and responsiveness of strategy implementation (Skinner, 1974); their internal rather than external focus taking little account of customers’ needs and competitors’ performance (Kaplan and Norton, 1992; Neely, 1995); their encouragement of local optimization (Hall, 1983; Fry and Cox, 1989) and their focus on minimization of variance from standards instead of pursuing continuous improvement (Johnson and Kaplan, 1987; Turney and Anderson, 1989).
The idea of developing “multi-dimensional” PM frameworks in which non-financial, future-oriented and external performance indicators play a dominant role (or at least supplement traditional ones) was born in the late 1980s and early 1990s after authors expressed their discontent with traditional backward-looking accounting-based performance measurement and demanded an alternative, more balanced view (Bourne et al., 2000). I will examine whether this process of change takes place in an analogical manner in both of the sectors. I will pay attention to questions such as:
When did this development start to take place in the public and in the private sectors respectively?
The concept of the Balanced Scorecard (BSC), for instance, was introduced by Kaplan and Norton in 1992 as a means of measuring performance in business units. It consists of four perspectives, interconnected by cause-and-effect relationships perspectives: financial, customer, internal business process, and learning and growth. The Balanced Scorecard is supposed to provide managers with a mixture of financial and non-financial measures derived from the business unit’s strategy. Therefore, it helps them to link the long-term business goals with a set of strategic performance measures that include not only outcome measures but also the drivers for these outcomes. As the idea of the BSC was initially intended for business settings, I will examine whether it is discussed in public sector literature stream, too.
Economic Value Added (EVA) is another measure of performance that has often been discussed recently. It was developed by the founders of Stern Steward & Company, who argue that this residual income measure aligns managers’ incentives to that of the shareholders and it encourages managers to undertake only projects that will increase shareholder wealth (Lovata and Costigan, 2002). Thus, EVA is purported to relieve the moral hazard problem. In addition to the EVA, Merchant and Van der Stede (2007) discuss other types of accounting measures of performance, such as return on
investment (ROI) and return on capital employed (ROCE). I will check if these financial measures of performance have been discussed only in private sector literature.
Furthermore, the performance measures in the studies can be typified according to financial – non-financial classification. One might intuitively believe that non-financial measures are prevailing in the private and not so much in the public sector taking into consideration the goal of profit maximization in business enterprises. Nonetheless, this intuition can be misleading, because we should not neglect the inevitable need of accountability and control of expenditures in public entities. Some authors propose that PM systems integrate and balance between external and internal measures, financial and non-financial indicators (Cross and Lynch, 1988/89; Kaplan and Norton, 1992). Perera et al. (1997) stress the increasing importance of non-financial measures in manufacturing firms in respect of the growing globalization, advances in technology and new management approaches. In such circumstances enterprises can no longer rely on stable demand for their products but are forced to maintain a strategy that includes customer-focused factors such as high quality, flexibility of product characteristics and dependability of supply along with the traditional low cost aspect (Johnson, 1990). He argues that measures derived from financial accounting systems are inappropriate for managing performance because they cannot capture these factors which are so important for the competitiveness. The shift toward customer-focused manufacturing strategies requires a
flexibility, dependability and quality of a product and customer service (Banker et al., 1993; Abernethy and Lillis, 1995).
The world of global economy has implications for public sector management, too. Reforming governments turn to NPM for solutions how to make public services as efficient and effective as possible (Lapsley, 2008). Much of the ideas of NPM are oriented toward results, outcomes, clear statement of goals and accountability (Hood, 1991). The scrutiny of expenditure plans and
accounting information is supposed to be facilitated by “attributing costs to outputs and measuring outputs by qualitative performance indicators” (Hood, 1996) or in the words of Lapsley (2008) “quantification is at the heart of NPM”. How has this affected the PM and whether the types of performance measures used in the public sector resemble these in the private sector is to be discussed in the comparative analysis of Chapter 4.
2.3. Research-related dimension
2.3.1. Types of methodologies and their link to the types of research
I will use the study by Hopper and Powell (1985) as a starting point to outline a framework according to which studies can be classified. The authors argue that different theoretical and philosophical assumptions underlie any piece of research. On the basis of assumptions regarding the nature of social science and the nature of society, they divide the studies among two dimensions. The social science dimension consists of assumptions about ontology (the nature of reality), epistemology (the nature of knowledge) and human nature (the relationship between individuals and their
environment) which, taken together, have a direct impact on the methodology used. Different methodologies are typical for each type of research:
Methodologies based on experimental design and statistical survey that treat the world as objective and measurable are consistent with the functionalist/positivist type of research. Reality is viewed in an objective manner; knowledge consists of established laws; people are seen as being constrained by the world they inhabit. The aim of this kind of studies is to test hypotheses in order to make generalizable assertions about causal relationships and turn them into laws. Data from questionnaires and structured interviews are analyzed by statistical techniques to establish relationships between variables within a population.
own perceptions of reality. Therefore, the focus is on individual meaning and people’s perceptions of reality. Understanding of the conduct of the others is obtained by a process of interpretation rather than direct observation. Research questions that are not
predetermined in the beginning of the study aim to gain insight into specific situations in actual practice and to make sense of the world.
By means of case research or collecting data on a limited number of subjects through various sources – interviews, questionnaires, document studies - assumptions about possible
relationships among subjects are developed in exploratory studies. They aim to explore a certain phenomenon about which little is known in advance and the researcher has an open mind toward possibly relevant aspects or causal relationships in regard with this
phenomenon. The researcher formulates hypotheses and states in advance clear
expectations about causal relationships. These hypotheses can be tested and turned into laws in a later stage.
I will try to find which kind of research is most typical for each sector. Since some studies lack a clear indication of the assumptions that underlie them, and therefore do not clearly pertain to any of the groups above, I will categorize them on the grounds of the methodology common for each of the types.
2.3.2. Types of theories
In addition to the methodology, I will take notice of the differences in the theoretical background represented in the studies.
Malmi and Granlund (2009) recognize two types of theory in management accounting which will serve as a classification basis:
Theories that have high status (called positive theories by Watts and Zimmerman, 1986), which are not theories of management accounting per se, but are “borrowed” from other disciplines such as economics or psychology; they are criticized for not providing clear insights to practitioners’ problems;
Low-status theories (normative theories), which are not regarded as theories by many researchers, but are more practitioner-oriented; they are usually associated with
3. Goal of the research, research questions and method
3.1. Goal of the research
The objective of this research is to gain an in-depth understanding of the subtle similarities and differences in the research literature about the two sectors in the light of the dimensions discussed in Section 2. The focus will be on revealing the aspects according to which we can categorize the studies, and whenever some dissimilarities appear I will examine the possible explanations for and implications of them.
3.2. Research questions
The main research question that the thesis will be based upon is the following:
What different approaches have the researchers in the two fields – the public and the private sector- used to conduct their studies, are there any important distinctions between their findings and what are the explanations for the differences?
To answer this question several sub-questions will be addressed:
What is the focus of the studies – do they discuss the specific characteristics of the two types of organizations, which stage of PM system development do they emphasize and which types of measures are discussed?
What methodology do they employ and what are the implications for the type of research conducted?
Which theories are represented?
Do studies provide support for practitioners?
On the basis of the answers to these questions I will try to divide the studies into different categories to see if there are some tendencies: for example, more explorative empirical research in one sector and more positivist research in the other, or if some trends develop differently over time. If such tendencies exist, I will analyze what are the possible explanations for them.
3.3. Research Methodology
A literature review appears to be the appropriate mode of research for the intended study, because it would enable a comprehensive approach to investigate the questions posed above. It will not only provide a comparative logic of analysis, but it will also help to find what is already known and what is missing from the studies on the topic, thereby providing directions for future research.
that sufficient evidence is assumed to exist in order to make assertions about common characteristics or disadvantages of the studies.
The review consists of a selection of papers from three journals devoted to management accounting research: ‘Management Accounting Research’, ‘Accounting, Organizations and Society’ and ‘Financial
Accountability & Management’. The former two will be a source of data on PM mainly in the private
sector, but also contain a few studies about the public, and the latter deals primarily with the public sector. In this manner, the studies about the private and the public sectors which will be included in the literature review receive equal weights. The analysis will be restricted to empirical research. It is considered the most plausible approach to provide evidence on how the theoretical background in Chapter 2 links to PM systems and practices in the public and the private sectors. Empirical
knowledge is gained by means of experience, observations or experiments, thus, it represents a viable approach to the study of organizational practice.
In addition to the pragmatic reasons for limiting the scope of the review, I have chosen these three journals because they are among the academic accounting journals of top quality according to the Academic Journal Quality Guide.
The first studies on PM in public organizations included in the journal Financial Accountability &
Management date back to 1985. However, for the sake of comparability I will exclude the first five
volumes, since the journal of Management Accounting Control begins in 1990. In this way I will control for some external environmental factors (e.g. legal environment) that could account for additional differences in the studies I will compare. The final year for all three journals is 2012, so studies until the current year will be included. Nevertheless, it should be noted that some important issues discussed in earlier studies may be omitted.
The number of public sector related studies is 65, of which the majority (46 studies) is from the journal Financial Accountability & Management, only 5 studies are found in Accounting,
Organizations and Society, and the rest are extracted from Management Accounting Research. Only
17 articles were written in the 1990s, and the rest date back to the period 2000-2012. The selection of private sector related studies consists of 34 articles found in Management Accounting Research and 31 studies included in Accounting, Organizations and Society. 24 studies on the topic under consideration appeared in the 1990s, and again the majority of articles were written during the period from 2000 until the current year.
The public sector literature stream discusses various types of public organizations, such as
in both public and private organizations, but some types of public entities, such as charities, churches and business public organizations (i.e. nationalized industries made up of commercial, financial and industrial organizations) will be excluded, since they do not fully satisfy the characteristics of a typical public organization that I described in Section 2.1.
The review will be restricted to PM research. This implies that other aspects of management control (for example, general management and finance) will be disregarded. However, studies
simultaneously discussing PM issues and other management control issues will be included. If some of the terms such as “performance indicators”, “performance assessment”, “performance
measurement”, “performance management” and “performance measures” serve as key words in an
4. Results from the literature review and analysis
This chapter builds on a comparative analysis and discusses the results from the conducted literature review.
4.1. Categorization according to the stage of PM system development discussed
I will classify the studies according to the dimensions discussed in Chapter 2, beginning with the stages of development of PM systems, on which they focus. The results are summarized in Table 4.1 in a chronological order. When a study discusses more than one stage, for example both the
implementation and the use phases, I have included it in both of the rows. Table 4.1: Stages of PM systems development
Stage of PM system development
Public Sector Private Sector
- Design Dunk and Lysons, 1997; Ballantine et al., 1998; Kloot and Martin, 2000; Otley and Pollanen, 2000; Rutherford, 2002; Abernethy et al., 2005; Catasus and Gronlund, 2005; De Bruijn and van Helden, 2006; Woods and Gribnic, 2008; Aidemark and Funck, 2009; Cuganesan and Lacey, 2011
Dunk, 1990, 1993; Merchant and Riccaboni, 1990; Archer and Otley, 1991; Brownell and Dunk, 1991; Coates et al., 1992;1995; Harrison, 1992; 1993; Vaivio, 1999; Ahrens and Chapman, 2002; Lillis, 2002; Malina and Selto, 2004; Hyvonen, 2007; Dossi and Patelli, 2008; Jansen et al., 2009; Lee and Yang, 2011; Groen et al., 2012
- Implementation Johnsen, 1999; Brignall and Modell, 2000; Modell, 2000-a; Jones, 2002; Rutherford, 2002; Cavaluzzo and Ittner, 2004; Dooren, 2005; Legreid et al. 2006; Norreklit et al., 2006; Ostergren, 2006; Siverbo and Johansson, 2006; Modell et al., 2007; Kasperskaya, 2008; Conrad and Uslu, 2011; Dyball et al., 2011; Hoque and Adams, 2011
Odgen, 1997; Kasurinen, 2002; Speckbacher et al., 2003; Davis and Albright, 2004; Andon et al., 2007; Groen et al., 2012
- Use Gietzmann, 1990; Abernethy and Stoelwinder, 1991; Boyne and Law, 1991; Pina and Torres, 1992, 1996; Cutt et al., 1993; Hyndman and Anderson, 1995; Osbourne et al. 1995; Thompson, 1995, 2001; Mol, 1996; Garcia et al. 1999; Scott and Tiessen, 1999; Modell, 2000-b; Rutherford, 2000; Aidemark, 2001; Rutherford, 2002; Modell, 2003; Vakkuri, 2003; Budding, 2004; ter Bogt, 2003, 2004; Carlin, 2004; Cavaluzzo and Ittner, 2004; Jacobs et al., 2004; Catasus and Gronlund, 2005; De Bruijn and van Helden, 2006; Chang, 2006, 2009; Kurunmaki and Miller, 2006; Legreid et
al., 2006; Norreklit et al., 2006; O’Connor et al., 2006; Pizzini, 2006; Siverbo and Johansson, 2006; Adolfson and Wikstrom, 2007; Modell et al., 2007; Tooley et al, 2007, 2010; Cohen, 2008; Jansen, 2008; Lee, 2008; Woods and Gribnic, 2008; Aidemark and Funck, 2009; Chang, 2009; Beeres et al., 2010; Sholihin and Pike, 2010; Cuganesan and Lacey, 2011; Hoque and Adams, 2011
Costigan, 2002; Ittner et al., 2003; Malmi and Ikaheimo, 2003; Halkos and Salamouris, 2004; Malina and Selto, 2004; Chenhall, 2005; Marginson and Odgen, 2005; Tuomela, 2005; Henri, 2006; Widener, 2006 –a, b; Busco et al., 2008; Dossi and Patelli, 2008; Hall, 2008; Burney et al., 2009; Hartmann and Slapnicar, 2009; Jansen et al., 2009; Wiersma, 2009; Cardinaels and van Veen-Dirks, 2010; Grafton et al., 2010; Hansen, 2010; Kraus and Lind, 2010; Dambrin and Robson, 2011; Speckbaher and Wentges, 2012
It is easily noticeable from Table 4.1 that in both streams of literature most attention is paid to the stage of use. Slightly more than two thirds of the total number of studies are focused on this phase of PM system development. In the public sector literature stream the rest of the articles are
approximately equally distributed between the other two stages. In contrast, private sector related studies devoted to the design stage are three times as many as those exploring the implementation phase (18 and 6, respectively).
Another point worth mentioning is that unlike the private sector literature stream where the design related studies are similarly spread between the two periods (before and after 2000), in the other stream of literature we can see only two design related articles written in the 1990s. Similarly, the stage of implementation is discussed in 1990s in only one study per stream.
4.2. Characteristics of public and private organizations emphasized in the studies
In this section I will discuss the characteristics described in Section 2.1 and their implications for the phases of PM systems development. Most of the studies in both fields of literature take into consideration the specific features of public and private entities in connection to the phases of PM systems development.
4.2.1. Implications for the design stage
does not provide a meaningful measure of organizational effectiveness and efficiency in public organizations (Ballantine et al., 1998; Aidemark and Funck, 2009).
In order to reflect the interests of the broad range of stakeholders and the multiplicity of objectives, some authors have proposed and demonstrated a multidimensional approach to PM (Ballantine et al., 1998; Kloot and Martin, 2000; Aidemark and Funck, 2009). According to them, to evaluate the performance of public organizations it is necessary either to adopt multiple measures, which are often based on the subjective judgment of the various groups of information users, or to assign some weights to the goals and make a single summary measure. Aidemark and Funck (2009), for instance, examine how professional context in health care organizations with their “multi-dimensional goals, democratic control and partly conflicting interests” influences the design and use of the BSC. In that context the BSC was introduced and considered not as a management or control system, but as a tool for quality improvement.
In recent years a topic that has been discussed in the private sector literature concerns the effects of employee participation in the design of PM systems. For example, Dossi and Patelli (2008) find a positive correlation between subsidiaries’ participation in the design of PM systems and the influence of PM systems on subsidiaries’ decisions to initiate and implement local strategies. The findings of Dunk’s study (1990) indicate that when budgetary participation and agreement on evaluation criteria are both high, managerial performance is low. Brownell and Dunk (1991) argue that budget participation and budget emphasis in performance evaluations should match (either both high, or both low) under conditions of low task uncertainty, and where task difficulty is high, participation is always preferred. Harrison (1992) asserts that matching high levels of participation and budget emphasis is associated with reduced job-related tension. Dunk (1993) focuses on the relationship between budgetary participation, job-related tension and managerial performance and concludes that participation does not moderate the negative relation between job-related tension and managerial performance.
the studies by Brownell and Dunk (1991) and Harrison (1992), in publicly funded organizations. The results are partially consistent with those reported by Brownell and Dunk and by Harrison, which comes to show that findings should be generalized with caution, because practices and their effects differ among organizational contexts and cultures.
Another often recurring subject in private sector literature is the influence of culture. For example, Coates et al. (1992; 1995) argue that the diversity of PM practices in U.S., U.K. and German
companies is due to cultural and structural differences, with U.K. and U.S. companies being more predisposed to short-termism, compared to their German counterparts. Dossi and Patelli (2008) find support for their hypothesis that collectivism, masculinity, power distance and uncertainty avoidance are negatively correlated with the influence of PM systems on subsidiaries’ decisions. Harrison (1993) finds evidence for the hypotheses that national culture influences the design of PM systems. His study shows that high reliance on accounting PM and budgets in superiors’ evaluative style is associated with higher job satisfaction and lower job-related tension in a high power distance and low individualism society, and in case of low power distance and high individualism society, reliance on such measures should be low. Jansen et al. (2009) provide support that national setting, including cultural and institutional factors, does matter in the design of PM systems and compensation
schemes.
4.2.2. Implications for the implementation stage
Cultural differences between countries and their influence on PM systems are barely discussed in public sector literature stream, and it is with regard to the implementation stage that they are taken into account. One of the few studies (Norreklit et al., 2006) considers whether US “fair contract” based PM models consistent with Danish ideology, especially with Danish way of creating order. Ostergren (2006) aims to explore the political-cultural context surrounding the introduction of national quality strategy including quality indicators.
Johansson (2006) discuss plenty of obstacles and reasons for unsuccessful implementation and use of relative performance evaluation model in Swedish municipalities, dividing them in three categories: unwilling, unable and prevented, the most common of which is unwillingness. Dooren (2005) finds that organizations which perceive a lack of political involvement have a significantly lower degree of adoption and implementation of PM systems.
Likewise, in private sector literature Kasurinen (2002) explores how the lack of a clear business unit strategy impedes the implementation of the BSC. Davis and Albright (2004) provide evidence for the proposition that financial performance is greater in organizations that have implemented the BSC compared to similar settings, where traditional financial measures were employed to measure performance. Speckbacher et al. (2003) also investigate the implementation of the BSC in business settings.
4.2.3. Implications for the use stage
Most of the studies take into consideration the specific features of public and private organizations in connection to the use stage of PM system development.
The need for greater social responsibility in public entities, which I already mentioned, is supported by other studies, related to this stage (Boyne and Law, 1991; Cutt et al., 1993; Hyndmann and Anderson, 1995; Rutherford, 2002; Jansen, 2008; Lee, 2008; Chang, 2009; Tooley et al., 2010). The diverse and often conflicting interests of the stakeholders, together with the absence of competition, market mechanism and profit motive (Gietzmann, 1990; Aidemark, 2001; Thompson, 2001; Lee, 2008; Beeres et al., 2010) mean that a single bottom-line measure – like “net income” that measures profit maximization in the private sector - is inappropriate in public organizations, where objectives are multiple (Pina and Torres, 1992, 1996; Osbourne et al., 1995; Aidemark, 2001; Chang, 2009) and vague (Modell, 2003; Budding, 2004). Lee (2008) argues that judgment on which performance indicators are relevant is often necessary. Jansen (2008) demonstrates that politicians have a different perspective on performance and use of performance information. Unlike production managers who have an internal perspective, politicians seem to focus on a citizen perspective and financial perspective, and Jansen sees this as a possible hindrance to the use of more businesslike practices of running public organizations.
singular PM, namely the ROI, can be used to balance different information needs and to incorporate both efficiency and effectiveness considerations. Although Rutherford (2002) recognizes that public sector organizations are rarely subject to competitive forces, he believes that ROCE is a useful measure of performance, because it can be applied and compared across all the operations within a body, and it also indicates the efficiency of the generated services.
Considerable evidence exists that pressures by funding agencies and political bodies play a significant role in this stage of PM systems development. Cutt et al. (1993), for example, finds evidence that there is steadily increasing pressure by funding and legislative bodies for universities in Canada to report on efficiency and effectiveness. Aidemark (2001) argues that the Balanced Scorecard (BSC) is very appropriate in Health Care organizations, because in the medical professional context, the administrators, controlled by the politicians cannot construct output or behavior measures. In contrast, Budding (2004) states that environmental uncertainty - including political influences - made no difference in the use of accounting measures for performance evaluation of managers in public organizations, compared to private ones. Chang (2009) argues that PM in National Health Service (NHS) organizations are not neutral, but are conditioned by the political context, because in order to obtain a good performance rating under the star rating system, local NHS organizations might be forced to comply with the political targets, imposed by the government. According to Chang, this comes to show that multi-dimensional PM systems may show distorted information, as they are subject to the influence of institutional interests of the government. Lee (2008) claims that the greater extent of development of financial compared to non-financial indicators in Australian public organizations could be attributed to the traditional financial focus of public organizations - to be in compliance with budget appropriation (to receive funding), reinforced by the efficiency focus of NPM. According to Legreid et al. (2006), the use of Management-by-Objectives-and-Results adjusted to the complex political and administrative context is a reasonable PM method to deal with
efficiency, effectiveness and accountability problems in the public sector.
based on principles from business administration in the profit sector, but output-oriented performance indicators are not always successfully introduced, due to the disregarded political dimension in public management, difficulties of measuring governmental outputs, bureaucratic and change resistant culture and other possible reasons. He finds that because of the task complexity aldermen prefer to use rich verbal information as a source of performance valuation rather than formal measures. Vakkruri (2003) points to the problem of measuring performance in organizational context with multiple objectives, inputs, outputs and outcomes and proposes Data Envelopment Analysis (DEA) as a solution. Around 40% of Norwegian public agencies included in the survey by Legreid et al. (2006) reported that their performance indicators gave little or no information about outcomes – the societal effects, because it is difficult to observe and measure outcomes and the nature of their tasks makes it difficult to get reliable information about the societal impacts of agencies’ efforts. Modell (2003) notices that in the education organizations he examined there were considerable difficulties in establishing relevant performance indicators and tension between competing views of which quality control practices are good.
In private sector literature stream, similarly to the public sector one, researchers have recognized the importance of examining the influence of organizational context on PM systems, rather than
exploring them in isolation. However, the variables stated to have influence on the PM practices differ between the two streams of literature. Contextual factors like strategy, mission, structure, culture, overall organizational objectives or operational goals are discussed in numerous articles (Lillis, 1992; Euske et al., 1993; Moon and Bates, 1993; Fullerton and McWatters, 2002; Lovata and Costigan, 2002). The results provided by Euske et al. (1993), for example, show that PM reflects both customer and corporate influences, particularly the corporate strategy and goals. Lovata and
Costigan (2002) find that firms adopting defender strategy are more inclined to use EVA. Fullerton and McWatters (2002) state that PM and incentive systems play an important role in implementing just-in-time (JIT) strategies by finding significant statistical relationship between the implementation of JIT strategy and the adoption of appropriate measures to assess its effectiveness. Ahrens and Chapman (2002) observed that considerable diversity of PM practices existed in the restaurant chain they studied, reflecting different local and central factors (e.g. locations and local events and
different managerial standards of service and quality).
measures). Although no significant association between these variables is found in the study, the results suggest that a good fit of organizational structure with PM systems determines their effective use. In the case study by Vaivio (1999) the uncertainty in general economic conditions and the intense competition were claimed to be the main catalyst for the increasing use of the new “non-financial” practices in management accounting.
Some recent studies discuss the linking of PM systems to employee incentive and reward systems in business organizations (Speckbacher et al., 2003; Widener, 2006 – b; Tuomela, 2010; Andon et al., 2007; Dossi and Patelli, 2008; Burney et al., 2009; Jansen et al., 2009; Grafton et al., 2010; Kraus and Lind, 2010; Dambrin and Robson, 2011; Lee and Yang, 2011; Speckbacher and Wentges, 2012). This is an aspect that is missing from public sector literature stream.
Negative behavioral effects that result from the use of PM systems are a topic which receives considerably more attention in the private than in the public sector literature (15 studies in the private compared to only 4 in the public literature). Table 4.2.3 presents a summary on the articles that discuss these effects.
Table 4.2.3: Negative effects of PM systems
Public sector Private sector
Negative effects of PM systems: Chang, 2006; 2009; Norreklit et al., 2006; Modell et al., 2007
Briers and Hirst, 1990; Merchant and Riccaboni, 1990; Brownell and Dunk, 1991; Lillis, 1992; Ross, 1994, 1995; Lau et al., 1995; Chow et al., 1996; Ghosh and Lusch, 2000; Hartmann, 2000; Lovata and Costigan, 2002; Rouse et al., 2002; Tuomela, 2005; Burney et al., 2009; Hansen, 2010
It was only in the last few years that negative effects were considered in the public sector literature. Chang (2006) observes that in order to satisfy central government’s requirements imposed to public entities, managers at NHS organizations attempt to manipulate figures of some performance
intended to have a citizen orientation and focus on long-term outcomes may in fact reinforce myopic managerial behavior. Norreklit et al., 2006 argue that if PM models are not consistent with Danish organizations ideologies, they are expected to lead to “dysfunctionalities”, by which they mean unintended negative control effects.
A greater variety of negative unintended effects of PM practices is discussed in private sector related articles. These include job-related tension, interpersonal relations and dysfunctional behavior and performance (Briers and Hirst, 1990). They analyze case evidence of dysfunctional behavior as a result from subordinates’ performance evaluation on the basis of budgetary information. In response to “budget pressure”, they engaged in numerous unintended attitudes including increased tension, resentment, suspicion, fear and decline of long-term performance. The results from Merchant and Riccaboni’s field study (1990) indicate that financial controls encouraged managers to engage in actions that improve their short-term financial reports. The side effects of the financial control systems took various forms – manipulation of financial PM, discouragement of new ideas and myopic orientation – and were not in the companies’ best long-term interests. Brownell and Dunk (1991) and Ross (1994, 1995) examine the relationship between budget emphasis in performance evaluation and job-related tension. Hartmann (2000) states that empirical studies contain contradictory results on the relationship between the use of accounting PM indicators and related tension, job-attitudes, data manipulation and dysfunctional decision making. Thus, the call for further empirical investigation in this area led researchers to seek contingent explanations of the differences, to take into consideration various variables that might have impact on that relationship: national culture characteristics (Harrison, 1992, 1993; Lau et al., 1995), environmental characteristics (Ross, 1995), task characteristics (Brownell and Dunk, 1991; Lau et al., 1995) and budget participation (Brownell and Dunk, 1991; Lau et al., 1995).
4.3. Categorization according to the types of performance measures discussed
The second content-related dimension, besides the stages of PM system development, is the type of performance indicators that the different studies investigate. The results are presented in Table 4.3. Table 4.3: Type of performance measures discussed
Type of PM discussed: Public sector Private sector - financial measures Abernethy and Stoelwinder, 1991;
Rutherford, 2002; Budding, 2004; Pizzini, 2006; Cohen, 2008; Cuganesan and Lacey, 2011
Malmi and Ikaheimo, 2003; Halkos and Salamouris, 2004; Hansen and van der Stede, 2004; Marginson and Odgen, 2005; Hyvonen, 2007; Jansen et al., 2009; Seal, 2010
- non-financial measures Osbourne et al., 1995; Ostergren, 2006; Wiersma, 2008
Harrison and Poole, 1997; Vaivio, 1999; Wouters, 1999; Hansen, 2010 - both discussed or
integrated/multi-dimensional system
Gietzmann, 1990; Boyne and Law, 1991; Pina and Torres, 1992,1996; Cutt, 1993;Hyndmann andAnderson, 1995; Thompson, 1995,2001; Mol, 1996; Bullantine, 1998; Garcia et al., 1999; Kloot and Martin, 2000; Modell, 2000-a,b; Brignall and Modell, 2000; Rutherford, 2000; Aidemark, 2001; Modell, 2003; Carlin, 2004; ter Bogt, 2003,2004; Abernethy et al., 2005; Chang, 2006, 2009; De Bruijn and van Helden, 2006; Kurunmaki and Miller, 2006; Legreid et al., 2006; Modell et al., 2007; Tooley and Guthrie, 2007; Jansen, 2008; Kasperskaya, 2008; Lee, 2008; Woods and Gribnic, 2008; Aidemark and Funck, 2009; Li and Tang, 2009; Sholihin and Pike, 2010; Tooley et al., 2010; Dyball et al., 2011; Hoque and Adams, 2011
Merchant and Riccaboni, 1990; Coates et al. 1992,1995; Euske et al., 1993; Nixon, 1998;Malmi, 2001; Ahrensand Chapman, 2002; Kasurinen, 2002; Lipe and Salterio, 2002; Rouse et al., 2002; Ittner et al., 2003; Speckbacher et al., 2003; Davis and Albright, 2004; Malina and Selto, 2004; Chenhall, 2005; Tuomela, 2005; Henri, 2006; Widener, 2006 – a,b; Andon et al., 2007; Dossi and Patelli, 2008; Burney et al., 2009; Wiersma, 2009; Cardinaels et al., 2010; Kraus and Lind, 2010; Dambrin and Robson, 2011; Lee and Yang, 2011; Hartmann and Slapnicar, 2012; Speckbacher and Wentges, 2012
It is evident form Table 4.3 that most of the studies are related to multidimensional PM systems that integrate both financial and non-financial indicators. Around 85 per cent of the studies on the public sector discuss both types of performance indicators. This trend seems to be persistent not only in the 1990s, but also in the years after 2000. In the other stream of literature the percentage is
Ballantine, 1998; Brignall and Modell, 2000; Kloot and Martin, 2000; Rutherford, 2000; Chang, 2006; Jansen, 2008; Lee, 2008; Woods and Gribnic, 2008; Li and Tang, 2010; Tooley et al., 2010). Two other reasons mentioned by researchers are the specifics of organizational goals, which in the public sector are typically expressed in non-financial terms (Kloot and Martin, 2000), and the political pressures and reforms to improve accountability, quality of public services or the performance of their
providers (Modell, 2000-a,b, 2003; Thompson, 2001; Carlin, 2004; Chang, 2006, 2009; Legreid, 2006; Modell et al., 2007; Tooley and Guthrie, 2007; Kasperskaya, 2008; Hoque and Adams, 2011).
Osbourne et al. (1995) acknowledges the multiple objectives and the related complexity of tasks in public service organizations, but discuss mainly non-financial indicators in their case study. Ostergren (2006) examines how non-financial indicators in Norwegian health care sector are implemented to measure quality as a part of a national quality strategy in order to give information to different interests groups. Cuganesan and Lacey (2011) maintain that rather than a multiple PM system, a singular PM, namely the ROI, can be used to balance the various information needs and stakeholders’ interests. Similarly, Rutherford (2002) contends the usefulness of ROCE in public sector bodies. In private sector organizations, as typically their single goal is profit maximization, one might expect a greater focus on financial indicators. This is correct if we compare the number of studies on financial measures in the public and in the private sectors, but, as I already indicated, over time more
emphasis seems to be placed on multidimensional PM systems.
Private sector studies usually discuss the relationship between corporate strategy, objectives and PM systems (Coates et al., 1992, 1995; Malmi, 2001; Fullerton and McWatters, 2002; Kasurinen, 2002; Lovata and Costigan, 2002; Ittner et al., 2003; Malmi and Ikaheino, 2003; Speckbacher et al., 2003; Davis and Albright, 2004; Hansen and van der Stede, 2004; Malina and Selto, 2004; Chenhall, 2005; Tuomela, 2005; Andon et al., 2007; Hyvonen, 2007; Burney et al., 2009; Lee and Yang, 2011; Speckbacher and Wentges, 2012). Nixon (1998) does not mention this association, but claims that competitive pressures are the reason why not only cost and productivity information is necessary in R&D divisions, but also establishment of whether these costs are appreciated by customers. Rouse et al. (2002) also consider competition as the main pressure for the adoption of an integrated PM system as a means to improve efficiency.
Riccaboni (1990), and Ahrens and Chapman (2002) demonstrate that managers exercise discretion in shaping and using PM system according to their specific situations and needs.
Hartmann and Slapnicar (2012) find that diverse (multidimensional) PM systems have positive effects on managers’ attitudes in private enterprises. Although some studies recognize multiple PM systems as superior to relying on financial indicators only, others (Kraus and Lind, 2010) provide empirical evidence for the little importance these multidimensional systems have for the corporate control and for the existence of clear focus on financial measures.
4.4. Categorization according to the research method applied
This research-related dimension categorizes the studies according to the types of methodology distinguished in Section 2.3. The categorization is presented in Table 4.4. On the grounds of this classification, I will infer what type of research predominates in each of the literature streams. Table 4.4: Research methods applied
Method applied Public sector Private sector - statistical surveys and
hypothesis testing
Abernethy and Stoelwinder, 1991; Hyndman and Anderson, 1995; Pina and Torres, 1996; Dunk and Lysons, 1997; Garcia et al., 1999; Rutherford, 2000; Thompson, 2001; Rutherford, 2002; Modell, 2003; ter Bogt, 2003, 2004; Carlin, 2004; Cavalluzzo and Ittner, 2004; Dooren, 2005; Legreid et al., 2006; O’Connor et al., 2006; Pizzini, 2006; Siverbo and Johansson, 2006; Cohen, 2008; Lee, 2008; Beeres et al., 2010; Sholihin and Pike, 2010; Tooley et al., 2010; Dyball et al., 2011; Hoque and Adams, 2011
Briers and Hirst, 1990; Brownell and Dunk, 1991; Dunk, 1993; Harrison, 1992,1993; Ross, 1994; Coates et al., 1995; Lau et al., 1995; Ross, 1995; Chow et al., 1996; Harrison and Poole, 1997; Stark and Thomas, 1998; Scott and Tiessen, 1999; Wouters et al., 1999; Otley and Pollanen, 2000; Fullerton and McWatters, 2002; Lovata and Costigan, 2002; Ittner et al., 2003; Speckbacher et al., 2003; Hnasen and vand der Stede, 2004; Chenhall, 2005; Henri, 2006; Widener, 2006 – a, b; Hyvonen, 2007; Dossi and Patelli, 2008; Hall, 2008; Burney et al., 2009; Hartmann and Slapnicar, 2009; Jansen et al., 2009; Wiersma, 2009; Grafton et al., 2010; Hartmann and Slapnicar 2012; Speckbacher and Wentges, 2012
- case/ field research on a limited number of subjects –interviews, observations,
questionnaires;
Gietzmann, 1990; Boyne and Law, 1991; Pina and Torres, 1992; Cutt et al., 1993; King et al., 1994; Lapsley and Pettigrew, 1994; Jacobs, 1995; Osborne et al., 1995; Thompson, 1995; Mol, 1996; Ballantine et al., 1998; Johnsen, 1999; Kloot and Martin, 2000; Modell, 2000 – a, b; Aidemark, 2001; Jones, 2002; Vakkuri, 2003; Budding, 2004; Jacobs et al., 2004;
Merchant, 1990; Merchant and Riccaboni, 1990; Archer and Otley, 1991; Coates et al., 1992; Lillis, 1992; Euske et al., 1993; Moon and Bates, 1993; Merchant et al., 1995; Odgen, 1997; Nixon, 1998; Vaivio, 1999; Ghosh and Lusch, 2000; Malmi, 2001; Ahrens and Chapman, 2002;
Abernethy et al., 2005; Gatasus and Gronlund, 2005; Chang, 2006, 2009; de Bruijn and van Helden, 2006; Kurunmaki and Miller, 2006; Ostergren, 2006; Adolfsson and Wikstrom, 2007; Modell et al., 2007; Tooley and Guthrie, 2007; Jansen, 2008; Kasperskaya, 2008; Wiersma, 2008; Woods and Gribnic, 2008; Aidemark and Funck, 2009; Conrad and Uslu, 2011; Cuganesan and Lacey, 2011
Davis and Albright, 2004; Halkos and Salamouris, 2004; Malina and Selto, 2004; Marginson and Odgen, 2005; Tuomela, 2005; Andon et al., 2007; Busco et al., 2008; Hansen, 2010; Kraus and Lind, 2010; Dambrin and Robson, 2011; Groen et al., 2012
- Others (experiments) Lipe and Salterio, 2002; Cardinaels et al., 2010
Table 4.4 shows some interesting results with respect to the research methodology. Case studies, in which semi-structured interviews play a dominant role, are the most important research technique in public sector literature stream. Among the studies reviewed, 40 articles use case research, and 25 studies apply statistical surveys. In the public sector literature stream, on the other hand, the number of the studies using survey data to test hypotheses (35 studies) is slightly higher than those using case research. There are two studies of experimental character.
Since it is very hard to recognize what assumptions underlie each study, I will use the methodology as an indication of the type of research conducted. As I already explained in Section 2.3 of my study, surveys including relatively numerous samples of objects are typical for functionalist research, while case studies are associated to either explorative or interpretive research. Only few researchers explicitly specify the type of research conducted. For example, Modell (2000-a) collects data through field research and states that his study is explorative. Chang (2009) provides interpretive research by means of a case study. Both of the examples are in compliance with the relationship between the methodology and the type of research stated above. However, there might be some studies which present results from survey data, but make assumptions typical for the interpretive type of research, or vice versa – utilize case research in a study with functionalist characteristics. For this reason, the inference that the majority of public sector research is interpretive or exploratory, and that in the private sector literature stream the interest toward positivist research is slightly higher, should be viewed with caution.
4.5. Categorization according to the type of theory represented
