How fit between the performance appraisal system and organizational culture is related to employee satisfaction with the performance appraisal system

(1)

How Fit between the Performance Appraisal System

and Organizational Culture is related to Employee

Satisfaction with the Performance Appraisal System

Final thesis: June 26, 2014

Annemieke Koning (10296077)

MSc. in Business Studies - Leadership & Management Track

Amsterdam Business School (UvA)

(2)

2

3 4.3.3. Market values ... 29 4.3.4. Hierarchy values ... 30 4.3.5. Summary hypothesis 1 ... 31 4.4. Hypothesis 2 ... 31 4.5. Hypothesis 3 ... 31 4.5.1. Clan values ... 33 4.5.2. Adhocracy values ... 34 4.5.3. Market values ... 35 4.5.4. Hierarchy values ... 36 4.5.5. Summary hypothesis 3 ... 37

5. Discussion

...

37

5.1. Practical implications ... 40

5.2. Limitations and recommendations for future research ... 41

6. Conclusion

...

43 7. Acknowledgements

...

44 8. Reference list

...

45 9. Appendices

...

51

9.1. Appendix 1: Survey ... 51

9.2. Appendix 2: Complete correlation matrix ... 59

(4)

4

Abstract

Employee performance plays an important role in organizational success, which is why

organizations invest in performance management practices, such as performance appraisal

systems. However, employees are often unhappy with the performance appraisal system. This

study investigates to what extent fit between the performance appraisal system and

organizational culture is related to performance appraisal satisfaction as perceived by the

employee. Two types of fit were assessed in this regard: indirect (hypothesis 1) and direct fit

(hypothesis 2). Furthermore, this study looked into the relationship between indirect and

direct fit (hypothesis 3). The sample consisted of people from the employed working

population in The Netherlands (n = 178). For indirect fit, the four types of values as

determined by the Competing Values Framework were used. Hypotheses 1 and 3, involving

indirect fit, were tested using polynomial regression analysis and response surface analysis,

whereas the relationship between direct fit and performance appraisal satisfaction (hypothesis

2) was tested by means of a linear regression. The results showed support for all three

hypotheses. Clan and adhocracy values have the most explanatory power in regard to

performance appraisal satisfaction, having more impact when they are present in the

performance appraisal system rather than the organizational culture. The values of the

Competing Values Framework do not seem to capture the essence of what is it that employees

take into account when assessing direct fit. Furthermore, practical implications, and

limitations of the current study were discussed, and recommendations for future research were

(5)

5

1. Introduction

Steve Jobs once said "I make 50 cents for showing up (...) and the other 50 cents is based on

my performance” (Isaacson, 2011, p.479). However, when speaking of the exception to the rule, Steve Jobs may have been just that, as unfortunately, for most of the working population

this particular 50/50 situation does not seem to apply. The performance of an employee plays

a key role in organizational success (Pfeffer & Veiga, 1999); and fifty percent will not be

enough. Performance matters and it is for this reason that organizations invest in performance

management systems to manage the performance of their employees. Performance

management is a ‘continuous process of identifying, measuring and developing the

performance of organization members and aligning performance with the strategic goals of

the organization’ (Aguinis, 2007, in Biron, Farndale & Paauwe, 2011, p. 1295).

The pressure of the current economy is forcing organizations to be more efficient and

effective and therefore emphasis on performance management is more important than ever.

However, although organizations might see the importance of performance management, the

subjects of the performance management (i.e. the employees) often do not. Whereas almost

every organization implements some type of system to evaluate the performance of their

employees (Aguinis, Joo & Gottfredson, 2011; Cascio, 2006), employees are often unhappy

with these performance management practices, especially in regard to performance appraisal

(Abu-Doleh & Weir, 2007; Molleman & Timmerman, 2003; Noe, Hollenbeck, Gerhart &

Wright, 2010; Zedeck, 2011). This is worrisome, as it is suggested that employee opinions of

the performance appraisal system are equally as important as the system’s validity and

reliability (Lawler, 1967; Hedge & Teachout, 2000). Organizations should pay attention to

performance appraisal and taking an employee perspective may offer the organization an

(6)

6 Not only is the working environment changing due to current economic conditions and

the pressure that this brings along, the context of organizations is changing as well.

Traditional ways of working are increasingly making way for new ways of working,

supported by new technology. The latter makes it possible to work outside the office and

currently in The Netherlands almost 1 out of 3 workers is working from home during a part of

the time (Centraal Bureau voor de Statistiek [CBS] & TNO, 2012). Also, the number of

organizations making their office facilities remote is growing rapidly (CBS, 2013a). These

new ways of working are an important issue in today’s organizations and will only become more visible as generation Y begins to take over. It is clear that the organizational context is

changing.

Will this have implications for performance management practices adopted by the

company? From the definition of performance management described above it can be derived

that strategic and organizational factors permeate performance management (Aguinis, 2007,

in Biron et al., 2011). Performance management practices cannot be separated from the

organizational context. However, although organizations often implement the most

sophisticated performance management practices, they often fail to take into account the

organizational context (Haines & St. Onge, 2012, p.1169). What will this mean for the

effectiveness regarding positive organizational outcomes of performance management

practices? How important is it that performance management practices and the organizational

context are aligned? The aim of the current study is to try and find an answer to these

questions, with a specific focus on performance appraisal, which is a very important aspect of

performance management (Noe et al., 2010).

This paper will be structured as follows. First, having already introduced the topic

shortly in the above section, a more detailed literature review will describe and discuss the

(7)

7 research methods will be described, followed by an overview of the results. The paper will be

concluded by a discussion and conclusion, in which the results, practical implications, and

limitations of the study, as well as suggestions for future research will be discussed.

2. Literature Review

2.1. Performance appraisal satisfaction

Performance appraisal is a part of the broader process of performance management and is

described as ‘the process through which an organization gets information on how well an employee is doing his or her job’ (Noe et al., 2010, p. 351; Zedeck, 2011). Performance

appraisal is described as one of the most important human resource practices and has to be

performed correctly in order to achieve competitive advantage (Boswell & Boudreau, 2002;

Noe et al., 2010). According to Noe et al. (2010), performance management can serve several

purposes: strategic, administrative, and developmental. The performance appraisal system

plays an important role in all three purposes and perhaps that is why performance appraisal is

one of the most researched topics within the field of performance management (Fletcher,

2001). When considering the outcomes of performance appraisal, often outcomes are

measured on an organizational level (e.g. influence of performance appraisal on

organizational performance). However, the reactions of the ratees (i.e. the employees) should

not be disregarded (Keeping & Levy, 2000; Zedeck, 2011). Perceptions of performance

appraisal may influence people’s judgment and attitudes towards the process, and in turn

influence organizational performance (Purcell & Hutchinson, 2007). This particular aspect of

performance appraisal research focuses on employee reaction to performance appraisal.

Dissatisfaction with the performance appraisal systems in place may signal a lack of success

(8)

8 suggested that favorable employee perceptions of performance appraisal are needed in order

to positively influence employee behavior and future development (Fletcher, 2001). The most

frequently measured appraisal reaction is performance appraisal satisfaction (Giles &

Mossholder, 1990; Keeping & Levy, 2000), which suggests that it is important. Levy and

Williams (2004) pointed out that although a lot of research is done on factors that may

contribute to performance appraisal satisfaction, a lack of empirical evidence exists on the

why and how performance appraisal satisfaction matters and more field research on this

subject is needed. The empirical research that has been done, however, suggests a positive

association between performance appraisal satisfaction and various positive work outcomes.

In a longitudinal study, Blau (1999) showed performance appraisal satisfaction to influence

overall job satisfaction. Years later, an empirical study by Kuvaas (2006) showed that

performance appraisal satisfaction was directly related to affective commitment as well as

intention to leave, and indirectly to work performance, providing empirical support that

performance appraisal satisfaction may lead to positive work outcomes.

Performance appraisal satisfaction can be divided into two components: satisfaction

with the appraisal system and satisfaction with the review session, hereafter referred to as

system satisfaction and session satisfaction respectively (Giles & Mossholder, 1990). System

satisfaction concerns satisfaction with those aspects of the performance appraisal system that

primarily exist independently of the actions of the raters, making it objective (Giles, Findley

& Feild, 1997). Session satisfaction on the other hand, can be found on the more subjective

end of the spectrum (Giles et al., 1997). The current study will only focus on the objective

system satisfaction component, as it was not possible within the boundaries of this study to

(9)

9 2.2. Organizational culture

The environment in which people work is important. A work environment variable that can

have a major impact on the members of the organization is organizational culture.

Organizational culture is a ‘pattern of basic assumptions, invented, discovered, or developed by a given group, as it learns to cope with its problems of external adaptation and internal

integration, that has worked well enough to be considered valid and, therefore is to be taught

to new members as the correct way to perceive, think, and feel in relation to those problems’

(Schein, 1990, p.111). Although there is a widespread claim among many researchers that

organizational culture is linked to the performance of the organization, there seems to be no

consensus regarding the mechanisms through which this linkage occurs. For instance, some

researchers claim that the strength (i.e. how widely the values of the organization are shared

within the organization) of the culture is key (e.g. Smart & John, 1996), whereas others adapt

the resource based view, stating that the organizational culture should be valuable, rare,

inimitable, and non-substitutable in order for an organization to be successful (Barney, 1991;

Ogbonna & Harris, 2000). Regardless of the lack of universality in regard to the mechanism

that links organizational culture and performance, the evidence does generally highlight the

importance of organizational culture within an organization (Ogbonna & Harris, 2000).

In regard to the measurement of organizational culture, an extensive amount of

different scales and models exist (Scott, Mannion, Davies & Marshall, 2003; Yu & Wu,

2009). This study will adopt the Competing Values Framework, developed by Cameron and

Quinn (2011), based on several reasons. First of all, in order to determine which measurement

scale is appropriate, it is important that the measurement captures the essence of the various

organizational culture aspects (Ralston, Terpstra-Tong, Terpstra, Wang & Egri, 2006). In an

extensive review of literature, Detert, Schroeder and Mauriel (2000) identified eight common

(10)

10 dimensions to a certain degree (Ralston et al., 2006). Secondly, the Competing Values

Framework is the measurement that is most often used in quantitative research concerning

organizational culture (Kwan & Walker, 2004). Thirdly, the focus of the framework is on the

individuals that comprise the organization (Cameron & Quinn, 2011), which is in line with

the focus of this study. Another important factor in the choice for this particular model is that

a large amount of empirical research has established the validity and reliability of this

framework (Yu & Wu, 2009).

According to the Competing Values Framework, organizational culture varies along

two scales: in terms of focus (i.e. internal or external) and degree of control (i.e. flexibility or

stability), which results in four types of organizational culture: clan, adhocracy, market, and

hierarchy (Quinn, 1988, in Hooijberg & Petrock, 1993; Cameron & Quinn, 2011). Clan

culture can be described as a very personal place, it is like an extended family. Collaboration

and participation are key in this culture, and the leaders fulfill a mentor role. Adhocracy

culture is characterized by a dynamic and entrepreneurial workplace in which people are

willing to take risks. Innovation and experimentation are important. Market culture is

results-oriented, it is about getting the job done. Competition and achievement are very important in

this culture. Hierarchy culture is about structure and control. In this, coordination and

efficiency are important, as well as following the rules and procedures (Cameron & Quinn,

(11)

11 Figure 1: Competing Values Framework (retrieved from: Cameron & Quinn, 2011, p. 39).

The type of organizational culture that is dominant within the organization is determined by

using six key dimensions: dominant characteristics, leadership, the management style, means

of bonding, strategic plans, and criteria of success. The first dimension, dominant

characteristics of the organization, describes what the overall organization is like. The second

dimension is about the organizational leadership style and approach that are present in the

organization. Thirdly, the management style dimension refers to the style that characterizes

how employees are treated and what the working environment is like. The fourth dimension

describes the means used by the organization to hold the organization together. The fifth

dimension is concerned with the drivers of organizational strategy. The criteria of success, the

sixth and final dimension, determine what is important to the organization in terms of what

(12)

12 2.3. Performance appraisal system and organizational culture

Although organizational contextual factors have always been recognized as being important

(Jackson & Schuler, 1995) not much empirical research into the effect of contextual factors on

performance management has been done (Haines & St. Onge, 2012; Levy & Williams, 2004).

Cawley, Keeping and Levy (1998) suggest that the performance appraisal system is dependent

on and must be consistent with the larger organizational context. Moreover, it is suggested

that because of interdependency, ignoring the impact of organizational culture on

organizational practices such as performance appraisal could even result in counterproductive

outcomes (Magee, 2002, in ul Mujeeb & Ahmad, 2011). This highlights the importance of

examining the relationship between organizational culture and performance appraisal into

more detail (Fletcher, 2001). Taking into account the statement made by Posthuma and

Campion (2008, in Haines & St. Onge, 2012, p. 1159) that “too much attention has been

placed on the design of a [performance management] system, and not enough on how it works

when implemented”, Haines and St. Onge (2012) proposed that a collaborative organizational culture (i.e. clan culture) will lead to more positive performance management outcomes, as

this type of organizational culture will value employee engagement. However, the authors did

not find evidence supporting their statement that this particular type of organizational culture

moderates the relationship between performance management practices and effectiveness.

This does not mean that the impact of organizational culture on performance management

outcomes should be disregarded, but a different perspective may offer better insight. Overall,

there seems to be a consensus that performance management practices should be congruent

with the culture of the organization (Aguinis et al., 2011). For example, in an organization

with a dominant hierarchic organizational culture, it is very likely that a 360-degree feedback

(13)

13 work with, will not result in positive organizational outcomes, as this will not ‘fit’ together

(Aguinis et al., 2011).

This suggestion is in line with contingency theory, that describes organizational

performance as resulting from a fit between characteristics of the organization and

environmental aspects (Fredericks, 2005). More specifically, in regard to human resource

management, it is about ensuring that human resource practices are appropriate for the

circumstances of the organization, including the culture (Delery & Doty, 1996; Wright,

Gardner, Moynihan & Allen, 2005). Drawing on contingency theory, outcomes of the

performance appraisal system will depend on the fit between the performance management

system and the organizational culture (Chenhall, 2003; Gerdin & Greve, 2008). Whether this

will be the case in relation to the outcome of employee satisfaction with the performance

appraisal system is the subject of this study.

To what extent is fit between the performance appraisal system and organizational

culture related to employee satisfaction with the performance appraisal system?

2.4. Performance appraisal system – organizational culture fit

In order to assess the fit between the performance appraisal system and the organizational

culture, it is important to determine what fit entails exactly. Although there seems to be a

consensus in regard to the importance of performance appraisal system – organizational

culture fit, no previous empirical research on this concept is available to build on. Therefore,

this study will borrow from a research field that is somewhat related: person-organization fit.

In this particular field, extensive research has been done and person-organization fit is

determined in various ways. One of these methods consists of assessing value congruence

between the person and the organization. Value congruence is considered the most consistent and effective predictor of employee outcomes (Westerman & Cyr, 2004). Measuring values of

(14)

14 employee and organization along the same scale is a so called commensurate measure

(Schneider, Goldstein & Smith, 1995). This type of measure allows to assess fit between the

performance appraisal system and the organizational culture by directly comparing the two

variables, and thus achieve high predictive validity. Furthermore, value congruence can be

combined with the values as described by the Competing Values Framework (Kalliath,

Bluedorn & Strube, 1999; van Vuuren, Veldkamp, de Jong & Seydel, 2007). Hence, although

it is recognized that person-organization fit and performance appraisal system-organizational

culture fit differ from one another in some aspects, this study will adopt value congruence

from the field of person-organization fit as a method to assess the fit in the current study.

2.5. Direct and indirect fit

Also drawing on the field of person-organization fit, a distinction can be made between

assessing fit directly and indirectly. Direct fit basically entails asking a person to judge the

extent to which they believe there is a fit between two variables, in this case between the

performance appraisal system and the organizational culture. Because of the subjective nature,

this type of fit is also referred to as perceived fit (Kristof, 1996). Indirect fit, on the other

hand, is also referred to as actual fit and can be described as the empirical relationship

between separate assessments of the underlying values of the two variables (Kristof, 1996).

Within this type of fit, a distinction can be made between subjective and objective indirect

assessment of fit. Indirect subjective fit compares the values of the performance appraisal

system with the values of the organizational culture as perceived by the employee, whereas

indirect objective fit compares the values of the performance appraisal system with the values

of the organizational culture as determined by other sources (Kristof-Brown, Zimmerman &

Johnson, 2005). In the person-organization fit field there seems to be little consensus on

(15)

15 measures of fit in order to better assess the relative effect of each (Kristof, 1996). This study

will follow this recommendation, insofar that it will assess both the direct fit, and the indirect

subjective fit. Due to the time and resource constraints linked to this particular study, it was

decided not to include indirect objective fit. It has to be noted that from this point forward,

indirect subjective fit will be simply referred to as indirect fit, because indirect objective fit is

not included in this study.

2.6. Hypotheses and conceptual model

The purpose of this study is to analyze the influence of the fit between the performance appraisal system and the organizational culture, which is operationalized as value congruence between the performance appraisal system and the organizational culture, on employee satisfaction with the performance appraisal system, using direct as well as indirect fit measures.

Based on contingency theory (Fredericks, 2005), as well as general consensus that

management practices should be congruent with the culture of the organization in order to

achieve favorable organizational outcomes (Aguinis et al., 2011), it is expected that this will

also be the case in relation to the performance appraisal system, for both direct and indirect

fit.

 Hypothesis 1: A better indirect fit between the performance appraisal system and the organizational culture will be positively related to satisfaction with the performance appraisal

system.

 Hypothesis 2: A better direct fit between the performance appraisal system and the organizational culture will be positively related to satisfaction with the performance appraisal

system.

Furthermore, although both direct and indirect fit are hypothesized to lead to more

satisfaction towards the performance appraisal system, it is expected that a relationship

(16)

16 based on perceptions of the employee, and therefore not objective in nature, it can be assumed

that indirect fit is relatively more objective than direct fit. This is based on the fact that

indirect fit captures the perception of values that were operationalized independently from the

employee (i.e. by the researcher and the Competing Values Framework) and can therefore be

called objective. Assuming that the more objective perceptions of these operationalizations

will to a certain extent influence the more subjective perceptions, it is expected that the

indirect fit between the performance appraisal system and the organizational culture will

influence how this fit is perceived directly by the employee.

 Hypothesis 3: A better indirect fit between the performance appraisal system and the organizational culture will be positively related to the direct fit between the performance

appraisal system and the organizational culture.

Figure 2 shows the conceptual model, and the hypotheses.

(17)

17

3. Methods

3.1. Research population

This study was conducted among the employed working population of The Netherlands,

which, translated from the Dutch definition, consists of ‘all people in the age from 15 till 65 who live in The Netherlands and have paid employment for twelve or more hours per week’ (CBS, 2013b). This population consists of approximately 7.2 million people, of which

approximately 1.1 million are self-employed (CBS - StatLine, 2013a). As people who are

self-employed are not the subject of a performance appraisal, they were not eligible for the

current study and therefore excluded, leaving a research population of approximately 6.1

million people. No sampling frame could be retrieved for this research population, therefore a

non-probability sampling method was used. The main goal was to achieve a sample as large

as possible, which would increase the chance of having a representative sample and being

able to generalize research outcomes to the entire research population.

3.2. Research instruments and procedures

The data were collected through an online survey, which was completely anonymous. First of

all, the questionnaire was distributed within the researcher’s personal network (i.e. primary

respondents) by means of social media and email. Then, snowball technique was used to

reach further respondents, asking the primary respondents to spread the questionnaire in their

personal network. Using this method provided the opportunity to reach a large group of

people. Unfortunately, it also made it impossible to provide an exact response rate, as it is

unknown how many respondents were reached beyond the primary network. There was no

(18)

18 The questionnaire started with four questions to determine whether someone was

eligible or not to participate in the study (i.e. whether they are a part of the research

population). For example, they were asked if they live in The Netherlands. If eligible,

respondents were forwarded to the research part of the questionnaire, and if not, they were

forwarded to a thank you message. The research part of the questionnaire contained questions

regarding the independent (i.e. organizational culture, performance appraisal system, and

direct fit) and dependent (i.e. performance appraisal satisfaction) variables, as well as a

number of demographical questions, such as age (see Appendix 1 for the complete

questionnaire). Indirect fit between the performance appraisal system and organizational

culture was calculated using polynomial regression after the data were collected (Edwards &

Parry, 1993; Shanock, Baran, Gentry, Pattison & Heggestad, 2010).

3.2.1. Organizational culture measure

Organizational culture is a ‘pattern of basic assumptions, invented, discovered, or developed

by a given group, as it learns to cope with its problems of external adaptation and internal

integration, that has worked well enough to be considered valid and, therefore is to be taught

to new members as the correct way to perceive, think, and feel in relation to those problems’

(Schein, 1990, p.111). Organizational culture was assessed with the Organizational Culture

Assessment Instrument (OCAI) that consists of 24 items total (Cameron & Quinn, 2011). The

OCAI differentiates four culture types, which were measured separately: clan, adhocracy,

market, and hierarchy. Reliability analyses showed a Cronbach’s alpha of .86, .91, .89, and

.77, respectively. An example item of the OCAI scale describing clan culture is “The

management style in the organization is characterized by teamwork, consensus, and

(19)

19 3.2.2. Performance appraisal system measure

The performance appraisal system was assessed by means of a self-developed scale based on

the values and dimensions of the OCAI items. The performance appraisal system scale

consists of 24 items and was developed with the help of subject matter experts, as well as

respondents representative of the intended sample. In line with the organizational culture

measure, the performance appraisal system scale consisted of four subscales measuring the

clan, adhocracy, market, and hierarchy values of the performance appraisal system. Reliability

analyses showed a Cronbach’s alpha of .86, .85, .77, and .67 respectively, with the latter value

being debatable, as the rule of thumb indicates that alpha should be higher than .70. In regard

to this issue, the reliability analysis showed that the first item did not correlate well with the

composite score from the other five items of the hierarchy subscale (i.e. item-total correlation

was .19, which is well below the proposed number of .30), and that removing the first item

would increase reliability to an alpha of .71, providing clear statistical reason to delete the

first hierarchy item from this scale. This statistical reason was in this case deemed sufficient

to decide to remove the item, because of the fact that the scale was self-developed, and has

not been validated before. Analysis was therefore continued with a 23 item-scale.

Furthermore, an exploratory factor analysis was conducted, which, in combination with a

thorough investigation of the content of the items, did not provide sufficient reason to remove

any more items. An example item, translated from the Dutch item, of the performance

appraisal system scale is “I feel that the performance appraisal system is focused on

teamwork, consensus, and participation”.

3.2.3. Performance appraisal system satisfaction measure

Performance appraisal system satisfaction was assessed by means of the three-item System

(20)

20 satisfaction with those aspects of the performance appraisal system that primarily exist

independently of the actions of the raters (Giles et al., 1997). Reliability analysis showed a

Cronbach’s alpha of .90. An example item of the System Satisfaction scale (1990) is “In general, I feel the company has an excellent performance appraisal system”.

3.2.4. Direct performance appraisal system – organizational culture fit measure

Direct fit between the performance appraisal system and organizational culture was assessed

by means of an adapted version of the Person-Organization fit scale (Cable & DeRue, 2002).

The adapted scale describes an individual’s overall judgment about the extent to which he or

she perceives the performance appraisal system and organizational culture to be congruent in

terms of underlying values (Kristof, 1996). Reliability analysis showed a Cronbach’s alpha of

.85. An example item of the adapted Cable & DeRue scale for direct performance appraisal

system-organizational culture fit (2002) is “The values of the performance appraisal system

match my organization’s values and culture.”.

For all measures, items were rated on a Likert scale ranging from 1 (strongly disagree) to 7

(strongly agree) and translated from English to Dutch (with exception of the self-developed

Performance Appraisal System scale, which was developed in Dutch). For the translation a

method of forward- and back-translation was used to avoid a loss in content validity. The

complete questionnaire in Dutch can be retrieved from Appendix 1.

3.3. Sample

The final sample consisted of 178 respondents, of which 75 were men (42.1%) and 103 were

women (57.9%). The mean age of the respondents was 40.50 years (SD = 12.22, MIN = 21,

MAX = 65). The average tenure with the current employer was 7.35 years (SD = 6.69; MIN =

(21)

21 followed by higher professional education (37.1%), senior secondary vocational education

(7.1 %), and secondary education (1.7%). Respondents were employed in a wide range of

sectors, with the majority of the respondents working in business services (40.4%). Other

sectors that provided a substantial amount of the respondents were ‘other services’ (21.3%), government organizations (14.0%), and health care (9.6%).

Comparing the abovementioned sample descriptives to CBS statistics of the research

population provides a preliminary insight into the representativeness of the sample. In terms

of average age, the mean of the sample (40.50 years) lies very close to the mean of the

research population, which is 41.70 years (CBS - StatLine, 2013b). In regard to gender, the

sample is slightly reversed in the male/female distribution, as the research population consists

of slightly more men (52.5%) than women (47.5%), whereas the sample consists of more

women than men (CBS - StatLine, 2013a). Education wise, approximately 36% of the

research population has either finished university or higher professional education, and

approximately 43% has completed senior secondary vocational education or secondary

education (CBS - StatLine, 2013a). It is clear that in terms of education level, the research

population statistics do not match the sample.

4. Results

4.1. Descriptive statistics and correlations

Table 1 contains the mean scores, standard deviations, correlations, and reliability coefficients

of the independent and dependent variables that were measured in the survey. The complete

(22)

22 Table 1: Means, standard deviations, correlations, and reliability coefficients

Variables M SD 1 2 3 4 5 6 7 8 9 10 1. PA system - Clan 4.48 1.19 (.86) 2. PA system - Adhocracy 3.90 1.17 .71** (.85) 3. PA system - Market 4.53 1.00 .22** .33** (.77) 4. PA system -Hierarchy 4.33 1.02 -.01 .00 .33** (.71) 5. OC - Clan 4.37 1.19 .65** .48** .05 -.06 (.86) 6. OC - Adhocracy 4.23 1.33 .51** .56** .22** -.19* .64** (.91) 7. OC - Market 4.33 1.33 .14 .15* .67** .12 .11 .38** (.89) 8. OC - Hierarchy 4.31 1.05 .01 -.13 .19* .52** .12 -.10 .31** (.77) 9. Direct PA system – OC fit 4.93 1.11 .32** .23** .22** .12 .35** .27** .27** .26** (.85) 10. PA satisfaction 4.49 1.37 .67** .54** .24** .07 .56** .42** .28** .20** .43** (.91)

PA = performance appraisal; OC = organizational culture ** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed).

Looking at table 1, it is clearly visible that almost all correlations show a significant positive

result. In regard to the measurement of the four types of values of the performance appraisal,

(i.e. variable 1 to 4), as well as the four types of organizational culture (i.e. variable 5 to 8),

this is not entirely surprising, because the four quadrants of the Competing Values Framework

are connected to each other (see figure 1, p. 11). For instance, in terms of degree of control,

clan and adhocracy values are both flexible, and in terms of focus, clan and hierarchy values

share an internal orientation. The significant correlations in table 1 that may be explained by

the overlap in terms of degree of control or focus between the four types of values of the

(23)

23 .33), and market and hierarchy (r = .33). In regard to the organizational culture types, this

overlap may explain the significant correlation between clan and adhocracy (r = .64), and

market and hierarchy (r = .31). Taking this into account, this leaves one significant correlation

that cannot be explained. Clan and market values show a positive correlation in regard to the

four types of values of the performance appraisal system (r = .22).

Furthermore, is stands out in table 1 that each of the four types of organizational

culture correlates positively and significantly with the value of the performance appraisal

similar to it (clan r = .65; adhocracy r = .56; market r = .67; hierarchy r = .52). Other

significant correlations regarding the organizational culture types and values of the

performance appraisal system exist between clan and adhocracy (r = .48), adhocracy and clan,

market, hierarchy (r = .51; r = .22; r = -.19), market and adhocracy (r = .15), hierarchy and

market (r =.19), respectively.

With the exception of the hierarchy values of the performance appraisal system, there

were significant positive correlations between the values of the performance appraisal system

as well as the types of organizational culture, and the direct fit between the performance

appraisal system and organizational culture (see table 1). Similar to this, all values of the

performance appraisal system, with the exception of the hierarchy values, and all

organizational culture types correlated positively and significantly with performance appraisal

satisfaction (see table 1). Direct fit between the performance appraisal system and

organizational also correlated positively and significantly with performance appraisal

satisfaction (r = .43).

4.2. Hypotheses testing

Hypotheses 1 and 3 were tested by means of polynomial regression and response surface

(24)

24 hypotheses testing will be discussed, polynomial regression and response surface analysis will

be explained further, as this method is not as well-known as linear regression.

4.2.1. Polynomial regression analysis and response surface analysis

Polynomial regression analysis provides an opportunity to examine the extent to which a

combination of two independent variables relate to the dependent variable (Shanock et al.,

2010). This type of analysis has been used before in various fit studies somewhat similar to

the current study, also including the four types of values as described by the Competing

Values Framework (Cameron & Quinn, 2011; Kalliath et al., 1999; van Vuuren et al., 2007).

Polynomial regression was proposed by Edwards (1993, 1994) as an alternative to the

traditional congruence tests, such as difference scores, and offers a solution for the numerous

methodological flaws that come with those particular tests. For example, difference scores

combine two measures into one score, and by doing this, do not offer insight into the extent to

which each of the components contribute to the outcome, whereas polynomial regression does

provide this option (Shanock et al., 2010).

Response surface analysis was conducted to aid interpretation of the results. For this,

the Excel spreadsheet made available by Shanock et al. (2010) was used, calculating the

slopes and curvatures and providing three-dimensional plots of the variables involved in the

hypotheses. Furthermore, the article by Shanock et al. (2010) was used as a guide to interpret

the results.

4.3. Hypothesis 1

Hypothesis 1 was tested with the use of polynomial regression analysis and response surface

analysis. Table 2 shows the explained variance (R2) of the polynomial regression analyses.

(25)

25 Performance appraisal satisfaction = X + Y + X2 + XY + Y2

In this equation X represented the underlying values of the performance appraisal system (e.g.

clan values of the performance appraisal system), and Y represented the type of

organizational culture (e.g. clan culture).

Table 2: R2 and significance (p) of the regressions of performance appraisal system values and organizational culture values in relation to performance appraisal satisfaction.

R2 Sig. (p)

Clan .531 .000

Adhocracy .355 .000

Market .169 .000

Hierarchy .069 .030

In line with the hypothesis, it was expected that the highest level of performance appraisal

satisfaction would be reached when there is perfect agreement between the values of the

performance appraisal system and the organizational culture, so when X equals Y. This line of

perfect fit X=Y should be straight and horizontal, because the level of value agreement should

not influence the level of the outcome value (e.g. a Likert scale score of five for both X and Y

is expected to show the same level of outcome value as a Likert scale score of three for X and

Y). The line of misfit (X = -Y) should show an inverted U-shape, as it is expected that the

outcome value will decrease when there is a higher degree of discrepancy between X and Y.

For every regression equation of the four in total, slopes and curvatures of the regression

(26)

26 Table 3: Performance appraisal satisfaction as predicted by indirect PA system – OC fit.

Clan Adhocracy Market Hierarchy

Slope X = Y 0.86** 0.60** 0.59** 0.25* Curvature X = Y -0.11* -0.12* -0.31** -0.15 Slope X = -Y 0.48** 0.32* 0.26 -0.33 Curvature X = -Y -0.26* -0.20 -0.64** -0.31 * p < .05, ** p < .01 4.3.1. Clan values

The regression of the clan values and performance appraisal satisfaction shows a significant

result, as well as reasonably high explained variance (R2 = 0.531, see table 2). The graph of

this regression is shown in figure 3.1.

Figure 3.1: PA satisfaction as predicted by the indirect fit of the clan values of the performance appraisal system and the organizational culture. Note that PA system and OC values were centered.

Table 3 shows that the slope as well as the curvature of the line of fit (X = Y) were

significant. The slope of X = Y is positive, indicating that performance appraisal satisfaction

P

A Satis

fa

cti

(27)

27 increases as the clan values of the performance appraisal system and organizational culture

increase. The significance of the curvature of the fit line shows that this relationship is

non-linear. As this curvature is negative, this means that there is a downward curving of the fit

line. This is visible in figure 3.1.

In regard to the misfit (X = -Y) line, also both the slope and curvature showed

significant results, indicating the proposed inverted U-shape, which is somewhat visible in

figure 3.1. However, as can be seen in the graph, this U-shape is not perfectly balanced. The

negative significant curvature of the misfit line (X = -Y) indicates that performance appraisal

satisfaction decreases more sharply as the discrepancy between the clan values of the

performance appraisal system and the organizational culture increases. The value of the slope

of X = -Y dictates the direction of the discrepancy. The positive significant value of this slope

indicates that performance appraisal satisfaction is affected more when the clan values of the

performance appraisal system are higher than the organizational culture, than when the

opposite is the case.

4.3.2. Adhocracy values

The regression of the adhocracy values and performance appraisal satisfaction shows a

significant result, as well as reasonably high explained variance (R2 = 0.355, see table 2). The

(28)

28 Figure 3.2: PA satisfaction as predicted by the indirect fit of the adhocracy values of the performance appraisal system and the organizational culture. Note that PA system and OC values were centered.

Table 3 shows that the slope as well as the curvature of the fit line (X = Y) are significant.

The slope of the fit line has a positive coefficient, indicating that in this case performance

appraisal satisfaction increases as the adhocracy values of the performance appraisal system

and organizational culture increase. The significance of the curvature of the fit line indicates

that this relationship is non-linear, and because the curvature has a negative coefficient, this

means that there is a downward curving of the fit line. This is supported by the graph in figure

3.2.

In regard to the line of misfit (X = -Y), only the slope showed a significant result. The

positive significant value of this slope indicates that performance appraisal satisfaction is

affected more when the adhocracy values of the performance appraisal system are higher than

the organizational culture, than when the opposite is the case.

P

A Satis

fa

cti

(29)

29 4.3.3. Market values

The regression of the market values and performance appraisal satisfaction shows a

significant result, but the explained variance was relatively weak (R2 = 0.169, see table 2).

The graph of this regression is shown in figure 3.3.

Figure 3.3: PA satisfaction as predicted by the indirect fit of the market values of the performance appraisal system and the organizational culture. Note that PA system and OC values were centered.

Both the slope and the curvature of the fit line (X = Y) show significant results (see table 3).

The slope has a positive coefficient, which indicates that in this case performance appraisal

satisfaction increases as the market values of the performance appraisal system and

organizational culture increase. The significance of the curvature of the fit line indicates that

this relationship is non-linear, and because the curvature has a negative coefficient, this means

that the line of fit shows a downward curving, as is visible in figure 3.3.

In regard to the line of misfit (X = -Y), only the curvature showed a significant result

(see table 3). This result has a negative coefficient, indicating that when the discrepancy

between the market values of the performance appraisal system and the organizational culture

P

A Satis

fa

cti

(30)

30 increases, the performance appraisal satisfaction will decrease more sharply. This is also

clearly visible in figure 3.3.

4.3.4. Hierarchy values

The regression of the hierarchy values and performance appraisal satisfaction shows a

significant result, but the explained variance was very weak (R2 = 0.069, see table 2). The

graph of this regression is shown in figure 3.4.

Figure 3.4: PA satisfaction as predicted by the indirect fit of the hierarchy values of the performance appraisal system and the organizational culture. Note that PA system and OC values were centered.

Only the slope of the line of fit (X = Y) showed a significant result, indicating a straight line

of fit. As visible in figure 3.4, this line rises as the value of fit is higher. The positive

coefficient of the slope supports this notion.

P

A Satis

fa

cti

(31)

31 4.3.5. Summary hypothesis 1

The regression analyses (see table 2) supported hypothesis 1, for each of the four types of

values. However, for all four value types, the fit lines did not show the straight and horizontal

shape that was expected. In all four cases, the fit line rose as the level of the values got higher.

Also, for the clan, adhocracy, and market values, the fit line was not linear. Furthermore,

coefficients indicate that in regard to clan and adhocracy values, the performance appraisal

system has a stronger effect on performance appraisal satisfaction than organizational culture.

Hypothesis 2 was tested with the use of linear regression. Table 1 shows that there is a

significant positive correlation (r = .43, p < 0.01) between direct performance appraisal

system – organizational culture fit and performance appraisal satisfaction. This direct fit

explained approximately 18% (R2 = 0.182) of the variance in performance appraisal

satisfaction and was found to be a significant predictor of performance appraisal satisfaction.

This result supports hypothesis 2: a better direct fit between the performance appraisal system

and the organizational culture is positively related to satisfaction with the performance

appraisal system. In other words, when employees perceive a better fit between the values of

the performance appraisal system and the organizational culture, they will be more satisfied

with the performance appraisal system.

Hypothesis 3 was tested with the use of polynomial regression analysis and response surface

analysis. Table 4 shows the explained variance (R2) of the polynomial regression analyses.

The general regression equation was as follows:

(32)

32 In this equation X represented the underlying values of the performance appraisal system (e.g.

clan values of the performance appraisal system), and Y represented the type of

organizational culture (e.g. clan culture).

Table 4: R2 and significance (p) of the regressions of performance appraisal system values and organizational culture values in relation to the direct fit.

R2 Sig. (p)

Clan .155 .000

Adhocracy .112 .001

Market .123 .000

Hierarchy .109 .001

Similar to hypothesis 1, hypothesis 3 also predicted that the highest level of the outcome

variable (here: perceived direct fit) would be reached when there is perfect agreement

between the values of the performance appraisal system and the organizational culture, so

when X equals Y. This line of perfect fit X = Y should be straight and horizontal, because the

level of value agreement should not influence the level of the outcome value (e.g. a Likert

scale score of five for both X and Y is expected to show the same level of outcome value as a

Likert scale score of three for X and Y). The line of misfit (X = -Y) should show an inverted

U-shape, as it is expected that the outcome value will decrease when there is a higher degree

of discrepancy between X and Y. For every regression equation of the four in total, slopes and

curvatures of the regression equations were calculated. These calculated values are shown in

(33)

33 Table 5: Direct PA system – OC fit as predicted by indirect PA system – OC fit.

Clan Adhocracy Market Hierarchy

Slope X = Y 0.37** 0.32** 0.41** 0.21* Curvature X = Y 0.09 0.11* -0.09 0.20** Slope X = -Y -0.02 -0.04 0.39 -0.27 Curvature X = -Y -0.01 0.06 -0.53** 0.00 * p < .05, ** p < .01 4.5.1. Clan values

The regression of the clan values and direct fit shows a significant result, but the explained

variance was relatively weak (R2 = 0.155, see table 4). The graph of this regression is shown

in figure 4.1.

Figure 4.1: Direct PA system – OC fit as predicted by the indirect fit of the clan values of the performance appraisal system and the organizational culture. Note that PA system and OC values were centered.

Dire ct P A s y stem – OC f it

(34)

34 In regard to the clan values, only the slope of the fit line (X = Y) showed a significant result

(see table 5). Although figure 4.1 shows that the fit line is quite horizontal, there is a slight

rise visible as well. The positive coefficient of the slope supports this notion, indicating that

when the clan values of the performance appraisal system and the organizational culture were

in agreement, direct fit increased as the level of the clan values of the performance appraisal

system and the organizational culture increased. The non-significant result of the curvature of

the fit line indicates that this relationship is linear.

4.5.2. Adhocracy values

The regression of the adhocracy values and direct fit shows a significant result, but the

explained variance was relatively weak (R2 = 0.112, see table 4). The graph of this regression

is shown in figure 4.2.

Figure 4.2: Direct PA system – OC fit as predicted by the indirect fit of the adhocracy values of the performance appraisal system and the organizational culture. Note that PA system and OC values were centered.

Somewhat similar to figure 4.1, figure 4.2 shows the fit line (X = Y) to be quite horizontal, as

was predicted. However, as the slope of the fit line is significant and has a positive

D ire ct P A s y ste m – O C f it

(35)

35 coefficient, this indicates that direct fit between the performance appraisal system and the

organizational culture increases as the levels of adhocracy values of the performance appraisal

system and the organizational culture increase. The significant curvature of the fit line

indicates that the relationship between the independent and dependent variables is non-linear.

In line with the positive coefficient of the curvature, figure 4.2 shows a slight upward curving

of the fit line.

4.5.3. Market values

The regression of the market values and direct fit shows a significant result, but again the

Figure 4.3: Direct PA system – OC fit as predicted by the indirect fit of the market values of the performance appraisal system and the organizational culture. Note that PA system and OC values were centered.

The slope of the fit line (X = Y) shows a positive significant result. This means that the direct

fit between the performance appraisal system and the organizational culture increases as the

levels of market values of the performance appraisal system and the organizational culture

(36)

36 increase. The non-significant result of the curvature of the fit line indicates that this

relationship is linear.

In regard to the misfit line (X = -Y), table 5 shows a significant curvature, indicating

that direct fit between the performance appraisal system and the organizational culture will

decrease more as the discrepancy between the market values of the performance appraisal

system and the organizational culture increases.

4.5.4. Hierarchy values

The regression of the hierarchy values and direct fit shows a significant result, but the

Figure 4.4: Direct PA system – OC fit as predicted by the indirect fit of the hierarchy values of the performance appraisal system and the organizational culture. Note that PA system and OC values were centered.

In regard to the hierarchy values, table 5 only shows significant results for the fit line (X = Y).

Both the slope and curvature of this line are significant. The coefficient of the slope is

positive, indicating that the direct fit increased as the hierarchy values of the performance

(37)

37 appraisal system and the organizational culture increased. The significant positive curvature

of the fit line indicates a non-linear relationship with upward curving. This is visible in figure

4.4.

4.5.5 Summary hypothesis 3

The regression analyses (see table 4) supported hypothesis 3 for each of the four types of

values. However, for all four value types, the fit lines did not show the straight and horizontal

shape that was expected. In all four cases, the fit line rose as the level of the values got higher.

Also, for the adhocracy, and hierarchy values, the fit line was not linear. Furthermore, direct

fit between the performance appraisal system and the organizational culture seems to suffer

more as the discrepancy between the market values of the performance appraisal system and

the organizational culture increases.

5. Discussion

The aim of this study was to investigate the effects of the fit between the values of the

performance appraisal system and the organizational culture on the satisfaction with the

performance appraisal system. In this regard, both indirect (hypothesis 1) and direct fit

(hypothesis 2) were examined, as well as the relationship between the two types of fit

(hypothesis 3). The results showed support for all three hypotheses. However, with regard to

the results some remarks need to be made. In the next section the outcomes of each of the

hypotheses will be discussed.

The first hypothesis expected indirect fit to be positively related to performance

appraisal satisfaction. Although the significant amount of explained variance for all four types

(38)

38 relationship was stronger when the levels of the values were higher rather than lower. This is

a remarkable outcome, as this was not anticipated. The first explanation that comes to mind is

that this would have been caused by a lack of differentiation in the answers of the

respondents. However, closer examination of the dataset indicates that this most likely would

not have been the case, as the range of means was broad (see Appendix 3). Thus, the

explanation has to come from a different perspective. The higher people rated the values of

the performance appraisal system, and the higher they rated the organizational culture values,

the more satisfied they were with the performance appraisal system. This phenomenon could

be a result of people wanting to reduce uncertainty (Kalliath et al., 1999). It is suggested that

people dislike ambiguity and uncertainty and will try to reduce it as much as possible. The

stronger (i.e. higher) the perceptions of values are, the more a situation is perceived as certain

(Kalliath et al., 1999). This could explain why the relationship between performance appraisal

satisfaction and the fit of the performance appraisal system with the organizational culture

was stronger when the levels of the values were higher rather than lower. Especially in current

times where alternative job opportunities are difficult to find, having a desire to reduce the

uncertainty and ambiguity of the situation seems plausible (Kalliath et al., 1999).

Furthermore, the results regarding hypothesis 1 indicated that the clan and adhocracy values

have more explanatory power than the market and hierarchy values (see table 2). This

suggests that a focus on flexibility is a better predictor of performance appraisal satisfaction

than a focus on stability. Change and adaptability are important factors in this case (Cameron

& Quinn, 2011). It could be argued that these findings are in line with the things that are

valued most by organizations nowadays. The clan values strongly emphasize the human

values, focusing on aspects such as personal development and training, and in the adhocracy

values, there is a strong emphasis on innovation and thinking outside of the box. All of which

(39)

39 is not a complete surprise that these values are represented in the performance appraisal

system as well as the organizational culture, and have relatively more explanatory power.

Moreover, the results showed that when the clan and adhocracy values were present in the

performance appraisal system rather than the organizational culture, satisfaction with the

performance appraisal system was higher. This indicates that it is even more important for a

performance appraisal system to be focused on flexibility than it is for an organizational

culture to be flexible. It is likely that the clan and adhocracy values of the performance

appraisal system are easier to observe for the employee than are the clan and adhocracy values

of the organizational culture. This may have resulted in the performance appraisal system

values having a stronger direct effect on the performance appraisal satisfaction, and thus

being more important in determining fit.

Hypothesis 2 expected direct fit to be positively related to performance appraisal

satisfaction. Results indicate that this is the case. When employees perceive the performance

appraisal system and the organizational culture to be a good fit, they will be more satisfied

with the performance appraisal system. This highlights that, in addition to the importance of

certain values of the performance appraisal system and the organizational culture in relation to

the performance appraisal satisfaction that was mentioned above, employee perception of how

well a performance appraisal systems fits within an organization is important as well. The

better the performance appraisal system is perceived to fit with the organizational culture, the

more satisfied employees are with the performance appraisal system itself.

The third and final hypothesis investigated the relationship between the indirect and

direct fit. As expected, the results indicated that indirect fit was positively related to direct fit.

However, similar to the relationship between indirect fit and performance appraisal

satisfaction, fit between the performance appraisal system and the organizational culture was

(40)

40 this may be explained by the tendency of employees to rate values higher in order to reduce

ambiguity and uncertainty (Kalliath et al., 1999). Furthermore, results indicated that direct fit

suffers the most when there is a misfit between the market values of the performance

appraisal system and the organizational culture. This may suggest that, with the exception of

market values, the values as operationalized by the Competing Values Framework do not

entirely capture the aspects which employees take into consideration when asked for fit

directly (van Vuuren et al., 2007). The fact that employees do take into account the

results-oriented aspects (i.e. the market values) when evaluating how well a performance appraisal

system fits within the organization seems plausible, as a focus on results will most likely be

something that people will always find important when evaluating a performance appraisal

system.

In all, the results offer useful insights into the relationship between the values of the

performance appraisal system and the organizational culture, and performance appraisal

satisfaction.

5.1. Practical implications

The results that were discussed above do not only offer useful insights for scholars, but also

for practitioners. Some practical implications will therefore be addressed. Organizations have

to recognize the importance of how their performance appraisal system is perceived by their

employees. The better employees perceive the system to fit within the organization, the more

satisfied they are with it. Ultimately, this will help organizations to achieve positive outcomes

(Blau, 1999; Kuvaas, 2006). Thus, it is critical that organizations do not ignore the

organizational context when implementing a performance appraisal system, and feedback

from employees plays an important role in this. Organizations should specifically pay

How fit between the performance appraisal system and organizational culture is related to employee satisfaction with the performance appraisal system