The relation between strategy, PMSs, and charity performance : survey evidence from Dutch charities

(1)

Student: Robin Kras (10419748) June 23, 2014 Supervisor: Prof. Dr. F.H.M. Verbeeten MBA

MSc. Accountancy & Control, control track Amsterdam Business School

Faculty of Economics and Business, University of Amsterdam

The relation between strategy, PMSs, and charity

performance: Survey evidence from Dutch

(2)

2

Summary

The aim of this thesis is to examine whether a relationship exists between the performance of a charity, its strategy, and its performance measurement system (PMS). For this I analyze a survey conducted among the financial managers of charities in The Netherlands. The results of a total of 63 questionnaires were used. A literature review results in the hypotheses that alignment between strategy and PMS leads to better performance, and that having a broad set of performance measures increases performance.

The results of this research were partially consistent with expectations. The alignment hypothesis is tested both in a positive sense (i.e. too much measurement emphasis) and in a negative sense (i.e. too little measurement emphasis). Contrary to the normative contingency view, having too much measurement focus does not lead to a decrease in performance. But I find, as expected, that too little measurement focus is negatively related to performance. Thus, a lack of alignment between strategy and the PMS does not decrease performance per se, but only having too little measurement focus decreases performance.

The measurement-diversity hypothesis can’t be supported by the evidence of this

research, and thus having a broad set of performance measures does not increase performance. On the contrary, there is weak evidence implying that the more charities diversify and

intensify the use of their performance measures, the less their performance will be. Exceptions to this are the largest charities; they do perform better with a broad set of performance measures. This unexpected result can possibly be explained by the distinct characteristics of charities as compared to for-profit organizations.

(3)

3

1. Introduction

1.1. Background

This research will investigate whether a relationship exists between the performance of a charity, its strategy, and its performance measurement system. For this I analyze a survey conducted among charities in The Netherlands.

Performance measurement systems (PMSs) have become increasingly more important for charities over the years. Committee Wijffels developed a governance code for charities in The Netherlands because of an increasing focus on integrity, transparency, and leadership within charities. One of the guidelines is that charities should report their expected and actual results (Wijffels, 2005). To address higher expectations of accountability, charities need to measure their performance (Speckbacher, 2003). Eisinger (2002) argues that nonprofits progressively engage in the provision of state funded services which results in increased pressure from the government. Furthermore, pressure comes from within nonprofits

themselves to improve performance (Cairns et al., 2005). Hoefer (2000) states that funders, donors, the public, agency boards of directors, staff members, and program clients have made calls for greater accountability and increasingly wish to ensure that resources allocated to programs achieve something worthwhile and measurable. Another reason for the increased importance of measuring performance within charities is a movement named New Public Management (NPM). NPM is a movement advocating the implementation of managerial processes and behavior from the private sector by the public sector (Box, 1999; Keen and Murphy, 1996; Hood, 1991; Lapsley, 2008). Considering the tenets of NPM, performance measurement within charities becomes increasingly important since performance

measurement is a key practice of the private sector.

There are generally two perspectives on the relationship between strategy, PMSs, and performance: a contingency/alignment perspective and a measurement-diversity perspective. Proponents of contingency theory argue that the PMS should be aligned to the strategy in order for the performance to be higher. (Otley, 1980; Chenhall, 2003; Fisher, 1998).

On the other hand, proponents of the measurement diversity perspective assert that diversity in performance measures will be most beneficial (e.g. Ittner et al., 2003). Thus, according to them no alignment between strategy and the PMS is necessarily needed. There are doubts about contingency theory. For example, Schoonhoven (1981) states that

(5)

5

discrepancies between contingency theory and the extent of empirical support for it. Contingency theorists adopted a normative stance (i.e. how things should be), without

sufficient empirical support (i.e. how things actually are). Langfield-Smith (1997) conducts a literature review covering papers regarding management control systems and strategy. She argues that the merits of contingency theory are based on intuitive arguments rather than empirical evidence and that most studies on contingency theory are published in professional journals rather than academic journals.

An alternative to the contingency perspective is the measurement-diversity

perspective. The popularity of this approach is gaining more and more support, especially in recent years. Researchers find empirical evidence in favor of the measurement-diversity perspective (e.g. Ittner et al., 2003). At first sight, this seems to be grave news for the contingency camp; has their research been for naught? It seems that the two approaches are mutually exclusive since it is stated that the PMS should be derived from strategy, or based on diversity of measurement independent from strategy.

But Van der Stede et al. (2006) find something peculiar. Their results support the view that performance measurement diversity benefits performance regardless of strategy, since firms with more extensive PMSs have higher performance. But their findings also partly support the contingency approach as they find that a mismatch between PMS and strategy is associated with lower performance only when firms use fewer measures than firms with similar strategies, but not when they use more. Van der Stede et al. (2006) show us by finding empirical evidence for both views that we might be overlooking something; there could be a research gap. It might not be so simple to state that the two approaches are mutually

exclusive.

Kaplan (2001) published one of the most well-known performance measurement concepts: the balanced scorecard. Some studies, like Ittner et al. (2003), place the Balanced Scorecard (BSC) under the measurement-diversity approach. This is somewhat complex as they state that the original balanced scorecard article argues that specific measures should be aligned to strategy, yet the ‘philosophy’ of the BSC is grounded in the four different

perspectives. Many non-financial measures are added with these perspectives, to be used by all firms. This proposition is important since it contradicts the alignment perspective in that the diverse set of performance measures should be used by all firms instead of certain performance measures needing to be aligned to certain firms. These four perspectives are financial, customer, internal business process, and learning and growth perspectives. Thus, the measures within these perspectives should be aligned to the firm’s strategy according to

(6)

6

Kaplan (2001), yet the broad perspectives with their corresponding measures, to be used regardless of strategic choice, are the foundation of the BSC. Greatbanks and Tapp (2007) find empirical support for Ittner et al.’s (2003) assessment as the public organization in their case study uses the BSC from a measurement-diversity perspective without aligning the performance measures to their strategy. Therefor I will follow Ittner et al.’s (2003) reasoning and categorize BSC-focused studies under the measurement-diversity approach.

The balanced scorecard continually increases in popularity (Davis and Albright, 2004), and with this popularity the measurement-diversity approach seems to be ever more important. There is no consensus yet regarding the debate on the effectiveness of a

contingency approach versus the effectiveness of a measurement-diversity approach. Thus, the current research base is not sufficient and future research should expand on the subject. With this thesis I aim to contribute to this ongoing debate. Besides this lack of research, the findings of the common for-profit research are not always applicable to a nonprofit situation. It is often assumed that results of research focused on for-profit firms are applicable to

nonprofit firms, but this assumption has been viewed with skepticism (Pollitt, 1986; Ittner and Larcker, 1998). Chapter 2 will extensively cover the applicability of for-profit centered

research on nonprofit organizations.

Literature concerning the relation between strategy, PMSs, and performance has not been extensively researched in specifically a charity context. For example, Chew and Osborne (2009) argue that strategy literature recognizes the strategic importance of positioning for for-profit organizations, but the literature offers conflicting arguments within a charity context. They find that charities have begun to focus more on strategic positioning. Charities have other agency problems compared to commercial organizations (Gjesdal, 1981). Market based performance measures are absent since capital is donated (Williamson, 1983). Charity

performance is more difficult to measure than market-based for-profit performance and other criteria of performance are needed (Baber et al., 2002). Considering all these difficulties, it is yet not clear whether alignment of strategy with the PMS or adopting a broad set of

performance measures will contribute to charity performance.

1.2. Research question

Considering the previous review, I formulate the following research question:

(7)

7

1.3. Contributions

This research contributes to the academic literature since literature in a charity context is scarce, especially concerning topics as strategy, PMSs, and performance (Moxham and Boaden, 2007; Dart, 2004). For example, Chapman (1997) states that the organizations which form the basis of most contingency studies, are business organizations with unambiguous goals. Furthermore, regarding the debate between contingency and measurement-diversity proponents, this study provides the academic field with additional theoretical insight through empirical evidence. On a more detailed level, this research provides insight in the use of strategy and PMSs in a charity context, as well as the (lack of) alignment between these concepts and its effect on performance.

The results of this thesis are contrary to the normative contingency theory stating that all misalignment between strategy and PMS negatively influences performance, but they do partially support contingency theory. The results are consistent with Van der Stede et al. (2006), and thus reinforce the contingency view. More specifically, the assertion that having too much measurement emphasis does not affect performance negatively, but having too little measurement emphasis does affect performance negatively, gains more credibility because the results of Van der Stede et al. (2006) are being reinforced. On the other hand, the results did not support the measurement-diversity approach, which provides a fruitful ground for future research. Furthermore, this thesis provides a literature review on how to measure performance of nonprofit organizations (NPOs), and on the applicability of for-profit centered research on NPOs.

From a societal perspective, this research will have implications for the design of PMSs within charities. This research will provide guidance to practitioners in improving their performance, by providing food for thought on developing their PMS. For example, the finding that there is a lack of alignment because of a too small measurement focus leading to a decrease in performance, could urge practitioners to put more emphasis on performance measures.

1.4. Reading guide

In the second chapter of this thesis I will discuss the literature regarding the relation between strategy, PMSs, and performance, from both the contingency and the measurement-diversity perspective. Furthermore, a discussion on measuring charity performance and the

(8)

8

discussed. The third chapter will discuss the research methodology. The results of the

analyses will be discussed in the fourth chapter. Finally, the discussion and conclusion will be given in the last chapter.

(9)

9

2. Theoretical background

2.1. Charity strategy and performance measurement

2.1.1. Introduction

In this section I will discuss strategic priorities within charities, why charities measure performance, difficulties with measuring the performance of charities, and whether regular research aimed at the for-profit sector can be applied to the nonprofit sector.

To clarify certain terminology I will provide definitions before I continue. Baber et al.’s (2002) definition of charities is: “organizations that broker philanthropic resources from donors to beneficiaries.” A similar definition of charity is an independent organization which has a nonprofit distribution policy, is reliant on volunteers, and its actions are beneficial to society (Garton, 2005; Tokeley, 1991). In this thesis, I will often use the terms charity and nonprofit organization (NPO) interchangeably.

2.1.2. Strategy

Chandler (1962, p. 13) defines strategy as ‘the determination of the basic long-term goals of an enterprise, and the adoption of courses of action and the allocation of resources necessary for carrying out these goals’. This definition will be used throughout this thesis.

Wilson and Butler (1986) argue that NPOs are more dependent on interorganizational relations compared to for-profit organizations. This leads to less flexibility in strategy. For example, NPOs need to employ cooperative strategies when that NPO receives government grants. They find that NPOs also need to manage the interdependence between donors and beneficiaries. This could lead to a multi-focused strategy. Stone et al. (1999) argue that NPOs pursue both competitive strategies (e.g. reputation safeguarding) and cooperative strategies (e.g. conforming to government wishes). Thus NPO strategy can be aimed at different constituencies. Contingency theory, which will be discussed later, states that a proper fit between strategy and the PMS should lead to greater performance. The described ambiguity of NPO strategy could lead to a decreased usefulness of contingency theory.

Jenster and Overstreet (1990) find that many NPOs (65%) do not use strategic planning. But larger NPOs, not necessarily older ones, are more likely to use strategic planning (Odom and Boxx, 1988). This could mean that the results of an alignment analysis are driven by the size of the NPO rather than the PMS.

(10)

10

There are strategic frameworks, such as Miles and Snow’s prospector, defender, and reactor model. But Andrews et al. (2009) test the Miles and Snow model among 90 public service organizations, and find that the model is not entirely applicable to NPOs. Moore (2000) argues that the private sector strategy models fail to take into account two important aspects of NPOs: (1) the value produced by NPOs lies in the achievement of social purposes rather than in generating profit; and (2) NPOs receive revenues from sources other than customer purchases. It could therefore be more useful not to categorize NPOs in distinct strategic orientations but recognize that NPOs have multiple strategic priorities (e.g. reputation safeguarding, conforming to government wishes, satisfying donors, satisfying beneficiaries).

2.1.3. The necessity of measuring performance

In contrast to the interest in business and public sector performance measurement, the voluntary and public sectors are low on the academic and practitioner agenda (Moxham and Boaden, 2007; Moxham, 2009). In this thesis I will mainly focus on the voluntary sector, but also on the public sector, because both sectors do not have profit as their primary goal and that distinguishes them from for-profit companies.

But why do nonprofit organizations (NPOs) need to measure performance in the first place? NPOs need to measure their performance to be accountable since, among other reasons, pressure from government is becoming more significant as nonprofits progressively engage in the provision of state funded services (Speckbacher, 2003; Eisinger, 2002).

Furthermore, pressure comes from within nonprofits themselves to improve performance (Cairns et al., 2005), and performance has to be measured in order to know whether it has improved. Hoefer (2000) states that funders, donors, the public, agency boards of directors, staff members, and program clients have made calls for greater accountability and

increasingly wish to ensure that resources allocated to programs achieve something worthwhile and measurable. Thus, it becomes increasingly more important for NPOs to measure and disclose performance.

2.1.4. Applicability of for-profit centered research on NPOs

Financial accountability is the key driver for measuring performance, and there have been calls for NPOs to be more ‘business-like’ in their operation and attitude. Unfortunately, research literature examining business-like activities in a nonprofit context have not been very

(11)

11

detailed (Dart, 2004). Since NPO-centered research is lacking, it would be helpful if the body of knowledge aimed at the for-profit sector can be applied to the nonprofit sector. It is often assumed that this is the case. Yet, this assumption has been viewed with skepticism (Pollitt, 1986; Ittner and Larcker, 1998). Thus, can the research regarding PMS design principles focused on for-profit organizations be applied to NPOs? This is an important question since in this thesis I conduct research from a contingency and measurement-diversity perspective. The academic underpinnings of both these perspectives are mainly based on studies that focus on for-profit organizations.

New Public Management (NPM) is a movement which advocates business-like management styles being practiced by public companies. Hvidman and Andersen (2004) argue that the rationale underlying NPM is that public organizations will gain some of the presumed efficiency of the private sector when they introduce private sector performance measurement techniques. But this rationale is based on a number of seldom-tested

assumptions. They argue that the primary assumption is that private management techniques can be transferred successfully across sectors. But new theories of public and private

management suggest that there are fundamental differences between the sectors. For example, Moynihan et al. (2012) state that a lack of goal clarity affects the use of performance

information. And as Hvidman and Andersen (2004) argue, a lack of goal clarity is prevalent within NPOs. They researched the Danish lower secondary school system and use survey data sent to school principals, from which they received performance information of 683 students. They find that performance management in public organizations does not improve

performance. This finding implies that the relation between strategy, PMS, and performance is different for charities than for for-profit firms, which could lead to different results of this thesis compared to research using a sample of for-profit organizations.

Fryer et al. (2009) investigate whether expected improvements in accountability, transparency, performance, quality of service and value for money have been established in the public sector. They find that there are three main classes of problems regarding the use of PMSs by NPOs: technical, system, and involvement. Technical problems consist of the collection, interpretation, and analysis of data and performance measures. System problems refer to problems such as a lack of strategic focus, and ambiguity of performance objectives. The last class of problem is of a ‘softer’ nature; they consist of cultural issues, people

problems, and a lack of involvement of the whole organization. The conclusion of their research is that the expected improvements have not yet been established in the public sector, thus suggesting that research findings aimed at for-profit organizations are not easily

(12)

12

applicable to NPOs. A limitation of Fryer et al.’s (2009) research is the fact that it is a theoretical research; these are not empirical findings. Verbeeten (2011) finds similar results regarding cost management systems in the public sector.

On the other hand, Moxham (2009) studies the private and public sector literature and finds that the drivers for, and the assessment of, performance in NPOs were similar to those identified in the private and public sector literature. This suggests that for-profit centered research is applicable to the nonprofit sector. Carter (1991) conducts case studies and a comparative literature research by comparing the public sector with the private sector. His conclusion is that performance measurement techniques could be translated from the private to the public sector. Boyne (2002) reviews the differences between private and public firms through analysis of evidence from 34 empirical studies. His conclusion is that the available evidence does not provide clear support for the view that public and private organizations are fundamentally dissimilar in all important respects. He substantiates this by saying that there is no empirical evidence for some differences, and that the evidence that exists for other

differences is problematic. It is problematic since most studies measure only the ownership dimension (the governance structure), and omit the dimension of governmental funding and political control; and few studies control statistically for other variables that may explain the differences between public and private organizations. For example, the lack of controlling for the size of the organizations might explain the increased evidence found for bureaucracy among public organizations. Boyne (2002) provides with his research additional evidence that empirical results of studies focused on the private sector can be used for the nonprofit sector.

Greatbanks and Tapp (2007) research the use of balanced scorecards in a single longitudinal case within a New Zealand public organization. Twelve people of the

management team were individually interviewed. Notes from meetings were taken in addition to these interviews. From their literature study they find that, with some necessary

modifications, the experiences from the for-profit sector are applicable to the public and nonprofit sectors. The primary reasons for these necessary modifications are the multiplicity of customers and stakeholders, and the disparate nature of public and nonprofit organizations regarding their strategic focus. The applied BSC is not the same as Kaplan and Norton originally intended who presented their BSC from a contingency approach where the measures were aligned with strategy. The examined organization used the BSC from a

measurement-diversity approach since only the diverse performance measurement aspect was adopted, not the linking of the strategy to the measures. The empirical evidence suggests that the use of the BSC has a positive impact on performance by enabling employees to appreciate

(13)

13

their role, and by providing transparency and focus. A limitation of this study is the small sample of one case organization with twelve interviews. Despite being a case study, these results suggest that the measurement-diversity approach within a NPO could positively affect performance. Another interesting piece of information is the fact that the NPO was able to adapt the BSC which was originally developed for for-profit organizations. This suggests that the results of studies focused on for-profit organizations can be applied to NPOs.

Some authors claim that models and frameworks that are successful in the for-profit sector can be adapted in such a way that it can be successfully used in the nonprofit sector (Osborne et al, 1995; Van Peursem et al, 1995; Gooijer, 2000). Furthermore, there is much support for the use of PMSs adopted by NPOs from the for-profit sector (see for example Goddard et al, 1999; Curtright et al, 2000). Especially the BSC (Inamdar et al, 2002; Wachtel et al, 1999) is deemed to have a positive effect on performance. What all these adoptions of ‘for-profit frameworks’ by NPOs have in common, is that they can easily be adapted, as is needed (Micheli and Kennerley, 2005). In conclusion, there is sufficient empirical evidence for stating that the results of studies researching for-profit organizations can, with the necessary adaptions, be applied to NPOs.

2.1.5. Difficulties in measuring performance of NPOs

We have discussed whether literature regarding for-profit organizations is applicable to NPOs, now the difficulties regarding the measurement of performance of NPOs will be discussed. We have seen that it is necessary for NPOs to measure performance, and that the measuring of performance will possibly have the same effects as in for-profit organizations. But for NPOs there are still some difficulties regarding the mere act of measuring

performance. Thus the question remains, how should one measure and evaluate the performance of a NPO?

Performance evaluation of NPOs is by many scholars viewed as problematic. The existence of a large number of stakeholders often results in the simultaneous use of different performance measures for different purposes (e.g. business plans, star rating systems and service agreements) to ensure that all parties are addressed (Radnor and McGuire, 2004); just a financial report is not enough. Radnor and McGuire (2004) argue that the use of different measures simultaneously leads to extra overhead, and could lead to information overload. They define PMS as a control system using Simons’s (1995) classification: as (1) belief systems (e.g. mission statements), (2) boundary systems (e.g. ethical codes and rules), (3)

(14)

14

diagnostic control systems (e.g. budgets), and (4) as interactive control systems (e.g. profit planning and project management). The intensive use of these different aspects of a PMS could lead to the information overload. Furthermore, Di Francesco (1999) argues that there are various problems relating to the measurement of performance in the public sector (i.e. problems with output specification, quality and effectiveness measurement, and client identification).

Since charities do not have the bottom line as primary goal, and market-based

performance measures are absent, it is difficult to measure performance (Williamson, 1983). Speckbacher (2003) therefor states that non-financial measures are easier to use by nonprofit organizations. But does that mean that we should use non-financial measures to evaluate the performance of the NPO? Since the bottom line is not the primary goal, Moxham and Boaden (2007) warn that wrong performance measures could not be representative of the activities of the organization. This could lead to “means-end inversion” where managers focus on certain measures while ignoring the actual goal of the charity. Stakeholders evaluating the NPO have to take this into account. Therefore, choosing a performance measure to evaluate the NPO with should be done in careful consideration. Thus, what is a good variable to measure the performance of a charity?

There is a belief that what an organization should measure depends on what it is trying to achieve (Johnston and Pongatichat, 2008). Oversight agencies that report whether charity organizations comply with standards, along with the popular press and individual donors, frequently use the program spending ratio (i.e. the ratio of program expense to revenue/total costs) as a performance indicator for charities (Baber et al., 2002; Schuman, 1993).

Furthermore, the program spending ratio is for practitioners an important indicator of performance since research by Weisbrod and Domingues (1986) shows that donation levels increase with program spending ratios.

Baber et al. (2002) investigate whether the program spending ratio causes management to seek what is measurable rather than what is important (means-end-inversion). They find no evidence that the use of program spending ratios by outsiders distorts the charities’ incentives, thus it seems that the program spending ratio is a good measure of performance for charities and for stakeholders.

Kähler and Sargeant (2002) state that the administration costs to expenditures (ACE) ratio is deemed to be an inefficient measure of performance. They argue that in general ACE is driven by the size of the charity. Small charities tend to have a higher share of

(15)

15

administration costs in their total expenditure than larger charities. It is argued that larger charities benefit from economies of scale.

Regarding the relation between charity performance and PMSs, Moxham and Boaden (2007) find evidence that NPOs don’t make extensive use of PMSs. They state that NPOs regard the reporting process as being incredibly time consuming. In their case research, one of the participating organizations stated that to make effective use of a PMS “an administrative secretary would be needed which we cannot afford.” Additionally, Moxham (2009) finds in her study that the majority of examined NPOs in her sample had underdeveloped PMSs. In conclusion, it remains difficult to measure the performance of a NPO, but there are effective ways of overcoming these difficulties such as the use of the program spending ratio.

2.2. Theoretical perspectives on performance measurement

2.2.1. Contingency theory: alignment approach

Contingency theorists argue that the PMS should be aligned with strategy in order for the performance to be higher (Langfield-Smith, 1997; Chenhall, 2003; Fisher, 1998).

Otley (1980) argues that, according to contingency theory, an appropriate accounting system for a company is dependent, or contingent, on specific circumstances. The idea behind contingency theory is that different operating environments result in different strategic

initiatives, which in turn may require different management information systems (Miles et al., 1978). The measures of performance are thus determined by the level of environmental uncertainty, which in turn is determined by the competitive strategy (Hoque, 2004). Thus, the accounting system needs to be tailored to the situation and needs of the organization.

Neely et al. (1995) conduct a literature review and find that it is clear that the PMS should be aligned to strategy. But they do not refute arguments of opponents of contingency theory that these studies are based on assumptions rather than empirical results. Pun and White (2005) also conduct a literature review on the topic of integrating performance measurement into strategy formulation. They conclude, based on their literature review, that there must be alignment between strategy and PMS in order for the organization to be successful.

Hoque (2004) investigates from a contingency framework the determinants and

consequences of performance measures through a questionnaire survey targeted at 52 New Zealand manufacturing firms. He finds a significant positive association between

management’s strategic choice and performance. This relation is activated through management’s high use of non-financial measures for performance evaluation.

(16)

16

Govindarajan and Gupta (1985) investigate the relation between the control system, strategy, and performance. They collect data from 58 general managers of strategic business units (SBUs) in diversified firms through interviews. They find that greater reliance on non-financial compensation criteria is stronger positively related to performance in firms with a ‘build’ strategy than firms with a ‘harvest’ strategy. This shows us that the same criteria has a different impact dependent on the strategy of the firm. This research provides with these results support for the contingency approach.

Abernethy and Guthrie (1994) investigate the characteristics of the information required for firms pursuing different strategies. They base their findings on survey responses from 49 business unit general managers. They argue that the effectiveness of business units is

dependent on a match between the design of the information system and the firm’s strategic position. In particular, information systems which have the characteristics of broad scope systems are more effective for prospector than for defender firms. This shows us again support for contingency theory. Some limitations of the study were the small sample, an incomplete specification of the strategy variable, and the measurement of performance was based on self-ratings. Especially this last limitation is important as Ittner et al. (2003) point out that most contingency studies base their research on self-ratings which are inherently subjective.

Abernethy and Lillis (1995) provide additional evidence for contingency theory. They collect data through semi-structured interviews from 81 general managers in Australian manufacturing firms. Furthermore, they conduct several site visits. Their study provides empirical evidence that the appropriate match between control system design and flexibility enhances performance. But they do warn the reader that the qualitative data collected in the field suggest the need for caution in interpretation.

Perego and Hartmann (2009) study the concept of alignment between strategy and PMS in an environmental strategy context. They collect data through a mail survey targeted at

financial managers of manufacturing companies in The Netherlands. The sample consists of 285 respondents. Their study provides evidence that strategic alignment impacts the use of PMSs. The more proactive a strategy is, the more reliant the organization is on the PMS that systematically report performance measures. This indicates that strategy directly impacts the PMS, thus providing support for contingency theory. An important limitation of their study is that, in order to receive neutral feedback, they contacted financial managers instead of

employees who deal with environmental management. Financial managers know less about environmental management which could lead to more shallow data.

(17)

17

Gani and Jermias (2012) investigate whether a lack of alignment between the strategy and management control systems has an effect on performance. They state that the proposition that a strategy control system misfit will have a negative effect on performance is intuitively appealing, but little research has been conducted to support this proposition. Therefore, they conduct their research. For their research method they sent out a questionnaire survey to executives of 109 banks. They find evidence that a strategy-control system misfit has a significantly negative correlation with both self-rated and publicly available performance measures. Thus they provide evidence for contingency-theory. A limitation of their study is that their data was obtained exclusively from the banking sector in Indonesia; a highly deregulated sector. The data might not represent the adopted MCSs and strategies by companies in other situations.

On the other hand, contingency theory also received some critique (Otley, 1980; Miller, 1981; Schoonhoven, 1981). Miller (1981) argues that contingency theorists are torn between two unrealistic and opposite extremes which are supposed to depict reality. On the one hand, theorists work with the simple assumptions of their statistical methods. These are assumptions regarding sample-wide consistency in relationships, the adequacy of linear statistical models, and deterministic thinking. On the other, even if theorists are convinced of the simplicity of this approach, they still choose to work with them because they fear chaos when they abandon these assumptions. Miller (1981) argues that the simplistic assumptions of contingency

theorists are lacking, and he points to conflicting findings in contingency related research. Verbeeten and Boons (2009) investigate, based on a survey of 201 medium-sized Dutch firms, whether there is a relationship between the strategic priorities of an organization and the use and effectiveness of specific performance measures. They find that firms for which financial performance priorities are more important use accounting performance measures more intensively. But no relation is found between economic profit measures (i.e. shareholder value analysis, Economic Value Added©) and strategic priorities. But these are associated with industry, organizational size and culture. Therefore, the use of performance measures is affected by both internal characteristics (e.g. strategic priorities, size, organizational culture) and external characteristics (e.g. industry, competitiveness). This shows that it is important to control for other factors like organizational size, otherwise the relation between strategy and PMS could be driven by another variable. In conclusion, Verbeeten and Boons (2009) find no support for the claim that aligning the PMS to the strategic priorities of the firm positively affects performance.

(18)

18

Pongatichat and Johnston (2008) investigate whether a lack of alignment between strategy and performance measurement could have some benefits. They conducted a four-year

empirical study at four central government agencies of the Thai government. Data was collected from 30 semi-structured interviews and from documentation. They find evidence that misalignment could even have some benefits. In their study, senior managers understood that strategy should be aligned with the used measures but they accepted, and sometimes even created, misalignment. They had clear reasons for doing this (e.g. to deal with requirements from other stakeholders, to enhance personal promotion prospects, and to demonstrate performance regarding other requirements not included in strategic objectives). Furthermore, they state that the process of alignment is continual; because of external and internal

environment changes, organizations will always experience some misalignment. With their research, Pongatichat and Johnston (2008) show us that the usefulness of the contingency approach could be understood in theory; in practice the approach may not be as useful as expected.

To better understand the relation between strategy, PMS, and performance, I will look at the measurement-diversity perspective next.

2.2.2. Measurement-diversity approach

According to the measurement-diversity perspective, the performance of a firm is best

serviced through having a broad set of performance measures, independent of strategy. There is also some theoretical and empirical support for this measurement-diversity perspective.

Measurement-diversity proponents argue that solely financial performance measures are suboptimal because it instills a short-term focus. Non-financial performance measures are deemed to indicate progress towards long-term goals, and should complement financial performance measures (American Accounting Association, 1971; Johnson and Kaplan, 1987, p. 259). Kaplan and Norton (1992) argue that another reason to adopt a broader set of

performance measures is that profit measures and other financial performance measures only partially reflect the effects of past and current actions; they are ‘lagging’ indicators.

Complementing these measures with non-financial measures (e.g. customer satisfaction, internal process improvement) will reflect the effect of current activities that will show up in the future; they are ‘leading’ indicators. Kaplan and Norton (1992) specifically warn that non-financial measures should not replace non-financial measures, but that an organization needs a broad set of performance measures.

(19)

19

Rees and Sutcliffe (1994) state that non-financial measures are less susceptible to manipulation since the measures are not subject to the potentially distorting influences of some accounting procedures (e.g. allocation of costs and valuation issues). On the other hand, Ittner et al. (1997) claim that non-financial performance measures (e.g. customer satisfaction survey results) are prone to managerial manipulation and are rarely subject to public

verification because non-financial measures are not audited.

Another reason why an organization should adopt a broad set of performance

measures is because financial measures may be imperfect and noisy signals of effort (i.e. they ‘lag’), while on the other hand non-financial measures induce long-term focused effort (i.e. they ‘lead’) (Feltham and Xie, 1994; Hemmer, 1996; Thevaranjan et al., 1996). Banker et al. (2000) support these findings. Using time-series data for 72 months from 18 hotels managed by a hospitality firm, they find that non-financial measures (i.e. customer satisfaction

measures) are significantly associated with future financial performance. They argue that customer satisfaction is associated more with long-term rather than immediate financial performance. Thus financial performance measures should be complemented with non-financial performance measures, effectively creating a more diverse set of measures.

Using data from US financial services firms, Ittner et al. (2003) examine the relationship between measurement system satisfaction, economic performance, and two general approaches to strategic performance measurement (i.e. the alignment/contingency approach and the measurement-diversity approach). They find consistent evidence that firms making more extensive use of a broad set of financial and particularly non-financial measures than firms with similar strategies or value drivers have higher measurement system

satisfaction and higher stock market returns. They find little support for the alignment hypothesis that more or less extensive measurement than predicted by the firm’s strategy or value drivers adversely affects performance. A greater measurement emphasis and diversity than predicted by their benchmark model is associated with higher satisfaction and stock market performance.

Furthermore, Van der Stede et al. (2006) examine the relationship between (quality-based manufacturing) strategy and the use of different types of performance measures, including their effects on performance. They find support for the view that measurement-diversity benefits performance as they find that, regardless of strategy, firms with more extensive PMSs have higher performance. Nevertheless, their findings also partly support the view that alignment of strategy with PMS affects performance. They state that a mismatch between PMS and strategy is associated with lower performance only when firms use fewer

(20)

20

measures than firms with similar strategies, but not when they use more. Thus, they find more evidence for the measurement-diversity perspective than for the contingency perspective. They come to this conclusion by explaining that using more objective and subjective non-financial measures appear to enhance performance, and that firms with similar strategies that use less measures have lower performance. They state that the limited support for the

contingency perspective could be caused by the fact that during the study firms were in the process of modifying their strategy and PMS. Firms that were in this process might have achieved a better match between their strategy and PMS which resulted in a higher performance.

As explained before, the balanced scorecard uses a measurement-diversity framework in the sense that, following Ittner et al.’s (2003) line of reasoning, its primary attribute is the (diversity in) measurement of multiple aspects of an organization rather than only the financial aspect.

Davis and Albright (2004) state that even though the BSC increases in popularity, there is little empirical evidence of its effectiveness. Therefore, they investigate in their research whether bank branches implementing the BSC outperform bank branches within the same banking organization, which use traditional PMSs, on key financial measures. They find evidence that the BSC can be used to improve financial performance. The BSC branches in their sample outperformed the non-BSC branches. Davis and Albright’s (2004) research method and design allow for a causal statement concerning the association between the implementation of the BSC and the corresponding financial improvement of performance. They used the actual targeted financial measure of the BSC at the business-unit level as the dependent variable which gives a more direct test of whether the BSC achieved its purpose rather than using corporate-wide measures. Furthermore, they had a control group. With their results, Davis and Albright (2004) provide evidence for the measurement-diversity approach. One downside of their research is their low sample. Their sample consisted of four BSC branches with a total of 75 employees and five non-BSC branches with a total of 71 employees.

Further support for the measurement-diversity approach comes from Hoque and James (2000). They conduct a survey among 66 Australian manufacturing companies, and find that greater BSC usage is associated with increased organizational performance. This is

independent of organizational size, product life cycle, or market position. In other words, the increased performance is not related to strategic characteristics of the firm. An implication of their results is that alignment between the degree of BSC usage and organizational

(21)

21

characteristics have less practical significance compared to measurement-diversity. The results of this study thus provide evidence for the measurement-diversity approach. A potential problem with Hoque and James’ (2000) results is that they are contrary to the findings of Ittner et al. (2003). Ittner et al. (2003) find that although implementation of BSC increases measurement system satisfaction, they are not associated with higher economic performance.

Additionally, Hoque and James’ (2000) research is more difficult to compare with other studies since they make a distinction between firms that should use the BSC and firms that shouldn’t use the BSC, while other studies have a more ‘absolute’ approach; the

measurement-diversity approach is best for every company. Another limitation of their research is that they mainly focus on the diversity of measurement aspect of the BSC; not the strategic focus of the measures and causal linkages. Their statement that alignment has less practical significance compared to measurement-diversity loses value because of this.

Scott and Tiessen (1999) find additional evidence in their study on teamwork within managerial teams. They state that the measuring of team work that is measured more

extensively, and with more diverse measures (i.e. financial and non-financial measures), will have a positive impact on team performance.

2.3. Hypothesis development

The measurement-diversity view asserts that a broad set of performance measures increases the performance of an organization. The idea behind the positive effect of having a broad set of performance measures is that having solely financial performance measures leads to a short-term focus and they ‘lag’ as indicators. Adding non-financial performance measures creates a long-term focus and they ‘lead’ as indicators (American Accounting Association, 1971; Johnson and Kaplan, 1987, p. 259; Kaplan and Norton, 1992; Feltham and Xie, 1994; Hemmer, 1996; Thevaranjan et al., 1996; Banker et al., 2000). Kaplan and Norton (1992) specifically warn that non-financial measures should not replace financial measures, but that an organization needs a broad set of performance measures.

Many studies find theoretical and empirical evidence for this assertion (Ittner et al., 2003; Van der Stede et al., 2006; Scott and Tiessen, 1999). Therefore, I put forth the following hypothesis.

(22)

22

The idea behind contingency theory is that competitive strategy determines the level of environmental uncertainty, which in turn determines the measures of organizational

performance (Hoque, 2004). For example, Miles et al. (1978) argue that because companies have different operating environments, they will have different strategic initiatives, and thus may require different management information systems.

Based on contingency theory and the evidence found that alignment between strategy and PMS is positively related with performance (Pun and White, 2005; Hoque, 2004;

Govindarajan and Gupta, 1985; Abernethy and Guthrie, 1994; Abernethy and Lillis, 1995; Perego and Hartmann, 2009; Gani and Jermias, 2012), I expect that this is the case for this research as well. These studies have mainly been focused on for-profit organizations rather than NPOs, but research suggests that the studies focused on for-profit organizations can be applied to NPOs. Therefore, I put forth the following hypothesis.

Hypothesis 2: Alignment between strategy and performance measurement system increases charity performance.

(23)

23

3. Research methodology

3.1. Sample

As part of a wider research project examining the measurement of performance in the charitable sector, I have a survey database at my disposal. I have added to this database by surveying Dutch charities. I have contacted 50 charities, largely by phone. Most of these charities were already contacted before in the past two years (2012-2013) by other students. Upon request extra information was sent by e-mail. A risk inherent in surveys is that the respondents are not representative of the population and that the questions may be misunderstood. To mitigate these risks, the surveys were sent to the most knowledgeable person about the subject (i.e. the controller or the financial director), and the charities have been contacted multiple times by different students over a long period of time. Of these 50 contacted charities, 20 expressed that they wanted to cooperate, 17 declined, and 13 doubted. All who said yes and who doubted have been sent a questionnaire survey. Besides these additional responses, the database already consisted of charities that have cooperated and were contacted by other students. The charities that I have contacted were contacted in the period of February-March 2014. The charities that responded earlier were contacted in three periods: November 2012, April 2013, and June 2013. In the end a total of 73 surveys have been completed. The response rate is 53%. Of these 73 completed surveys, 66 are usable. The data will be analyzed with SPSS, since this software is helpful especially with small data samples.

3.2. Research design

To determine the effect of alignment and measurement-diversity on performance, I will use the same method as Ittner et al. (2003), Van der Stede et al. (2006), and Verbeeten and Boons (2009) used in their papers. I will explain this method next.

First of all, I will run a factor analysis to determine the strategic priorities of the charities. I will do the same for the performance measures to see if there is any correlation between the measures. This is useful for reducing the amount of data.

To measure the diversity of measures, I will compute a scale for measurement-diversity (DIV_VALUE) from the scores resulting from the questions. This scale will be used in a regression with charity performance as the dependent variable, and with DIV_VALUE as the independent variable together with the control variable SIZE.

(24)

24

To measure the extent of alignment, I will first measure the relationship between the strategic priorities and performance measures with regression analyses. The absolute values of the residuals will be saved and summed up and I will run a correlation between the sum of these residuals and the performance of the charity. Any deviations from the estimated model (i.e. the measurement emphasis is too small or too great) should be negatively related to performance.

Furthermore, to get a better insight into the results of the alignment test, I will follow Ittner et al. (2003) and Van der Stede et al. (2006) in their additional analysis. This means that the residuals will be divided in a positive portion where all positive residuals (i.e. too much measurement emphasis) will be retained and all negative residuals will be given the value of zero, and in a negative portion where all negative residuals (i.e. too little measurement emphasis) will be retained and all positive residuals will be given the value of zero. Then a Pearson Correlation will be run with each of these new residual variables individually, SIZE as control variable, and the performance of the charity.

3.3. Measurement of variables

3.3.1. Performance

As a reminder, I will use Baber et al.’s (2002) definition of charities: “organizations that broker philanthropic resources from donors to beneficiaries.” This means that the brokering process consumes a part of the contributed capital, but the objective is to maximize spending on program activities. Oversight agencies that report whether charity organizations comply with standards, along with the popular press and individual donors, frequently use the

program spending ratio as a performance indicator for charities (Baber et al., 2002; Schuman, 1993). The program spending ratio will be used in this thesis as an indicator of charity

performance. Let the program spending ratio be PRATIO = PSPENDING / TC, where PRATIO is the ratio of program expense (PSPENDING) to total costs (TC). Another reason for using the program spending ratio as an indicator for performance is that it is an objective measure; the program spendings and expenditures are audited by independent auditors. This is important as Ittner et al. (2003) point out that most contingency studies base their research on self-ratings which are inherently subjective.

I will not use another common measure of performance, the administration costs to expenditures (ACE), because this is deemed to be an inefficient measure of performance (Kähler and Sargeant, 2002). Kähler and Sargeant (2002) argue that in general ACE is driven

(25)

25

by the size of the charity. Small charities tend to have a higher share of administration costs in their total expenditure than larger charities. It is argued that larger charities benefit from economies of scale.

The performance data is hand-collected from the charities’ 2012 annual reports. The year 2012 was chosen because it is the most recent available financial information, and it overlaps the 2012-2013 research period when most of the surveys were completed. The expenditures on programs have been divided by the total costs of the organization. This program spending ratio (PRATIO) is used as a measure of the performance of the charity. See Appendix B for a full overview of the program spending ratios of the charities used in the sample.

3.3.2. Measurement-diversity

All the measurement-diversity questions will be answered on the following scale: “Not applicable, daily, weekly, monthly, quarterly, semiannually, per year”. These questions will be answered regarding both the board of directors and the employees. This means that for each question two answers are given; one with the board of directors in mind, the other with the employees of the charity in mind. The answers have been recoded to 1, 7, 6, 5, 4, 3, and 2 respectively. To determine the value of measurement diversity (DIV_VALUE), all 7, 6, and 5 answers have been given the value of ‘1’. All other answers have been given the value of ‘0’. All scores were summed up to total DIV_VALUE. Thus, a higher value means that the charity uses performance measures to a greater extent.

3.3.3. Normality testing and data trimming

To ensure legitimate use of a linear regression, the assumption of normally distributed data must be met. To be more specific, the residuals of the regression plot must be normally distributed. To ensure this is the case, a linear regression has been run by letting PRATIO be explained by DIV_VALUE. The residuals of this regression have been saved and a normality test is performed on the residuals.

Table 1. Tests of Normality of Residual

Kolmogorov-Smirnova Shapiro-Wilk Statistic df Sig. Statistic df Sig. Standardized Residual ,177 66 ,000 ,881 66 ,000

(26)

26 a. Lilliefors Significance Correction

Table 1 (but also the histogram and Q-Q plot) indicates that the residual values of PRATIO and DIV_VALUE are not normally distributed (p < .05) (Shapiro and Wilk, 1965). After experimenting with different transformations of data, a Log10 transformation of both PRATIO and DIV_VALUE give the best results. A constant of 1 was added at the DIV_VALUE transformation, LG10(1 + DIV_VALUE), since the Log10 transformation requires data to be above 0. This resulted in the variable DIV_VALUE_LG10. In addition to transformation, PRATIO also had to be reversed (or reflected) because of a negative skew. It was reversed as follows: LG10((Maximum value PRATIO + 1) – PRATIO), resulting in the variable PRATIO_RLG10. This also means that this reversed data must be interpreted differently; a high PRATIO_RLG10 score does now not mean high performance, but exactly the opposite.

Again a linear regression was run between the two transformed variables (i.e.

DIV_VALUE_LG10 and PRATIO_RLG10) with the following results regarding normality of the residuals.

Table 2. Tests of Normality of Residual of Transformed Data

Kolmogorov-Smirnova Shapiro-Wilk Statistic df Sig. Statistic df Sig. Standardized Residual ,107 66 ,059 ,924 66 ,001 a. Lilliefors Significance Correction

Table 2 shows that the data is normally distributed according to the Kolmogorov-Smirnov test, but not according to the Shapiro-Wilk test. The skewness is -.930 (SE = .295) and the kurtosis is 2.240 (SE = .582). These values exceed the acceptable -1.96/1.96 interval (i.e. -.930 / .295 = -3.153 and 2.240 / .582 = 3.849) (Field, 2009). The Q-Q plot, Boxplot, and histogram show that the data approximate normality more closely, but there are 3 distinct outliers (see

Appendix C). After deleting these outliers, the results are as follows.

Table 3. Tests of Normality Residuals without Outliers

Kolmogorov-Smirnova Shapiro-Wilk Statistic df Sig. Statistic df Sig. Standardized Residual ,126 63 ,014 ,963 63 ,054 a. Lilliefors Significance Correction

(27)

27 Table 16 indicates that the Kolmogorov-Smirnov result now states that the data is non normal, while the Shapiro-Wilk test states that the data is normally distributed. On average the results have improved. The skewness is now .384 (SE = .302) and the kurtosis is now -.536 (SE = .595). These values do not exceed the acceptable -1.96/1.96 interval (i.e. .384 / .302 = 1.272 and -.536 / .595 = -.901). Besides these formal tests, the Q-Q plot, histogram, and boxplot also provide evidence that the data does not significantly deviate from normality (Razali and Wah, 2011). The Q-Q plot, Boxplot, and histogram show that the data now approximate normality even better (see Appendix C). Therefor I will continue this research without the outliers; resulting in a final sample of n = 63.

3.3.4. Strategy

Stone et al. (1999) have shown us earlier that NPOs tend to have different strategic priorities focused on several stakeholders. Also Andrews et al. (2009) find that the Miles and Snow model is not entirely applicable to NPOs. Moore (2000) argues that this holds for all for-profit strategic models. Therefore, I will not use such models nor will I try to categorize the NPOs in my sample into distinct strategic orientations. Through factor analysis I will try to find out the strategic orientations ‘as they are’.

First I will conduct an exploratory factor analysis (EFA) on the questions in the survey regarding the strategic priorities of the charity. The goal of this is to find underlying

relationships between the questions; to find whether specific questions can be traced back to one or a limited number of aspects. This way the number of strategic priorities can be reduced. I will perform a reliability test to see whether this is a legitimate action.

There is a difference in calculation between factor analysis and principal components analysis but these differences are small. Guadagnoli and Velicer (1988) conclude from their research that the solutions generated from principal component analysis differ little from those derived from factor analytic techniques. There is some critique on this statement since there are some differences arising from the calculation, but these differences are not too large; and thus factor analysis and principal component analysis are often used interchangeably (Field, 2009).

A question that remains is how many factors one should extract. Cattell (1966) advocates the use of a scree plot. The scree plot graphs each eigenvalue, a value that

(28)

28

Cattell (1966) argues that the cut-off point for selecting factors should be at the point of inflexion of the curve. Figure 1 and Figure 2 show the scree plot of the unmodified data and one in which the point of inflexion is visualized by the crossing of red lines.

Figure 1. Scree Plot Strategy

Figure 2. Scree Plot Strategy with point of inflexion

The point of inflexion occurs at the second data point (factor), which means that only one factor should be extracted. Only factors left of the point of inflexion should be extracted without including the factor at the point of inflexion itself (Field, 2009). One downside of the scree plot is that it provides a fairly reliable criterion for factor selections with a sample of

(29)

29

preferably more than 200 participants (Stevens, 2002, in Field, 2009). This is not the case in this thesis. Thus, factor selection should not be based on this criterion alone.

Kaiser (1960) states that the factors with an eigenvalue higher than 1 should be extracted, SPSS uses the same criterion. Since the sample of this research is rather small, Kaiser’s criterion will be applied. The actual factor analysis is depicted in Table 4.

Table 4. Factor Analysis Strategy

Component

1 2

A. Increasing national reputation. ,722 ,290

B. Improving reputation. ,594 ,470

C. Avoiding negative publicity. ,813 ,396

D. Satisfying donors. ,850 ,336

E. Acquire additional funds (collections). ,638 ,393 F. Acquire long-term funds (legacies etc.). ,630 ,632 G. Increasing income from investments. ,599 ,085 H. Improving the efficiency of projects. ,768 ,358 I. Increasing the average donation amount. ,761 ,392 J. Increasing the number of donors. ,861 ,278 K. Improving the effectiveness of projects. ,634 ,447 L. Increasing the number of program spendings. ,702 ,246 M. Decreasing the costs of the organization. ,546 ,565 N. Increasing the number of volunteers. ,134 ,927

O. Satisfying volunteers. ,291 ,850

P. Satisfying employees. ,572 ,701

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization. a. Rotation converged in 3 iterations.

The variables that load on both factors are marked green. These will be removed in order to generate ‘clean’ factors.

After removing the variables that load on both factors, a Kaiser-Meyer-Olkin measure of sampling adequacy (KMO) has been conducted. The KMO statistic varies between 0 and 1 where a value close to 0 means that factor analysis is likely to be inappropriate, and a value close to 1 means that factor analysis should yield distinct and reliable factors (Kaiser, 1970). The KMO test shows in Table 5 a result of .870 which is higher than the necessary .5, meaning that we may use the factor analysis (Field, 2009). Kaiser (1974) states that values below .5 should be rejected, in this case more data should be collected. Hutcheson and Sofroniou (1999, p. 225) argue that scores below .50 are unacceptable, in the .50’s they are

(30)

30

miserable, in the .60’s mediocre, in the .70’s reasonable, in the .80’s great, and in the .90’s they are marvelous.

Table 5. KMO and Bartlett's Test Strategy

Kaiser-Meyer-Olkin Measure of Sampling Adequacy. ,870

Bartlett's Test of Sphericity

Approx. Chi-Square 473,153

df 45

Sig. ,000

Removing the variables that load on both factors results in the following factor loadings depicted in Table 6.

Table 6. Factor Analysis Strategy (trimmed)

Component

1 2

A . Increasing national reputation. ,743 ,270 C . Avoiding negative publicity. ,839 ,319

D . Satisfying donors. ,874 ,263

G . Increasing income from investments. ,587 ,121 H . Improving the efficiency of projects. ,793 ,265 I . Increasing the average donation amount. ,791 ,403 J . Increasing the number of donors. ,883 ,202 L . Increasing the number of program spendings. ,723 ,204 N . Increasing the number of volunteers. ,196 ,926

O . Satisfying volunteers. ,349 ,858

The factor loadings of variables A, C, D, G, H, I, J, and L show us that they have a common underlying factor. These variables cover topics such as keeping donors satisfied, improving reputation, and increasing income (primarily from donors). Therefor I will name the first factor ‘DONOR’, meaning that the strategic priority of a charity scoring high on this factor is focused on donors. The variables N and O also have a common factor. I will name this factor ‘VOLUNTEER’ since its focus (i.e. satisfying volunteers and increasing the number of volunteers) is on the charity’s volunteers.

A Cronbach’s alpha test (Cronbach, 1951) is performed to test the reliability of the above assertions. Cronbach’s alpha tests the internal reliability of the constructs (i.e. DONOR

(31)

31

and VOLUNTEER) that have been created. One person has been left out of the test (n = 62) because this person did not answer all questions. The reliability test of the underlying

variables of the DONOR construct shows that the Cronbach’s alpha is .928 as shown in Table 7. This excellent Cronbach’s alpha supports the view that the variables can be combined into one construct.

Table 7. Reliability Test of DONOR construct

Cronbach's Alpha

N of Items

,928 8

The reliability test of the underlying variables of the VOLUNTEER construct shows that the Cronbach’s alpha is .855 as shown in Table 8. This good Cronbach’s alpha supports the view that the variables can be combined into one construct.

Table 8. Reliability Test of VOLUNTEER construct

Cronbach's Alpha

N of Items

,855 2

3.3.5. Alignment

First I will run regression analyses between performance measure factors, to be discussed hereafter, and strategy together with a control variable. The error terms, or residuals, of these regression analyses will be saved and summed up. Then a Pearson Correlation between the sum of these error terms will be run with PRATIO_RLG10. The results will be discussed in chapter 4.

But first I will conduct an exploratory factor analysis (EFA) on the questions in the survey regarding the use of performance measures. This way the number of variables can be reduced. I need these variables in order to test the extent to which performance measures are being driven by strategy. I will use the same process as described in 3.3.4 Strategy. The survey question providing these results comes from question 18 of the survey. All these questions are answered on the following scale: “Not applicable, daily, weekly, monthly, quarterly,

semiannually, per year”. These questions regard the use of performance measures, and had to be answered for both the board of directors and for the employees. There are in total 11 sub

(32)

32

questions, but because they are answered regarding two user groups, there are 22 variables. The tag (RVB) means that the question is answered with the Board of Directors (Dutch: Raad van Bestuur) in mind. They indicate that the measures are used by the Board of Directors. The tag (P) means that the question is answered with the personnel in mind. The results of the EFA are depicted in Table 9.

Table 9. Factor Analysis of Performance Measures

Component 1 2 3 4 Budgets (RVB) -,006 ,211 ,094 ,829 Budgets (P) ,546 -,006 ,041 ,613 Returns (revenue/profit) (RVB) ,302 -,067 ,275 ,589 Returns (revenue/profit) (P) ,679 ,123 ,185 ,289

Program spending ratio (program expense/total revenues) (P) ,014 -,047 ,090 ,837 Program spending ratio (program expense/total revenues) (RVB) ,774 ,145 ,153 ,062 Program spending ratio (program expense/total costs) (P) ,208 ,266 -,139 ,818 Program spending ratio (program expense/total costs) (RVB) ,862 ,180 ,052 ,206

Costs of fundraising (P) ,361 -,041 ,164 ,676

Costs of fundraising (RVB) ,848 ,151 ,061 ,193

Reputation and/or brand strength (RVB) -,015 ,561 ,585 ,261 Reputation and/or brand strength (P) ,354 ,726 ,269 -,017 Customer satisfaction of donor (giver) (RVB) ,095 ,295 ,808 ,158 Customer satisfaction of donor (giver) (P) ,417 ,313 ,752 -,032 Customer satisfaction of beneficiary (recipient) (RVB) ,046 ,265 ,892 ,113 Customer satisfaction of beneficiary (recipient) (P) ,324 ,315 ,781 ,021

Employee satisfaction (RVB) -,026 ,529 ,426 ,224 Employee satisfaction (P) ,451 ,586 ,180 -,003 Volunteer satisfaction (RVB) -,145 ,710 ,370 ,054 Volunteer satisfaction (P) ,287 ,651 ,160 -,165 Process improvement (RVB) ,008 ,678 ,329 ,279 Process improvement (P) ,408 ,731 ,119 ,084

Table 9 shows again that certain variables load on multiple factors, they are marked green. These conflicting variables will be removed from the analysis to provide ‘clean’ factor loadings. After removing these variables, Table 10 shows the KMO and Bartlett’s Test.