The influence of gender differences on subjective performance evaluation

(1)

The influence of gender differences on subjective performance

evaluation

Student name: Robert van Grol Student number: 11384522 Date: June 25, 2018

Word count: 14053

Thesis supervisor: dr. V.S. Maas

MSc Accountancy & Control, specialization Control Amsterdam Business School

(2)

Statement of Originality

This document is written by student Robert van Grol who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(3)

Abstract

Subjective performance measures are used to address shortcomings of objective performance measures. However, one important disadvantage of using subjectivity in evaluations is that managers can be biased towards their employees. Prior literature has shown that managers divide their subordinates in an ''in-group'' and an ''out-group'', whereby members of the first group receive more time, attention and better rewards from their supervisors. In addition, it has been found that employees with the same personal characteristics as the manager will be placed in the ''in-group''. The focus in this thesis is on gender differences between a supervisor and a subordinate in relation with subjective performance evaluations. It is predicted that supervisors give higher subjective performance evaluations to employees with the same gender as themselves. Findings of a case-based experiment indicate that there are no significant interaction effects between the gender of the supervisor and the subordinate. Therefore, both hypotheses are rejected, despite the fact that male participants gave higher evaluations to male subordinates compared to female. It is remarkable that female participants also gave higher subjective performance evaluations to male subordinates compared to female. This could mean that participants in general have a gender stereotype that male subordinates perform better than female and should therefore receive a higher bonus. These findings contribute to the existing literature by indicating that in this setting there is no significant evidence for the existence of a pro-male or pro-female bias. Further, this thesis shows that female managers have less preference for working with same-gender employees than male. Future research should indicate if the same results will be found with a larger sample and in a field setting with real managers.

Key words: subjective performance evaluation, performance measurement, LMX-theory,

(4)

Contents

1 Introduction ... 1

2 Literature review ... 4

2.1 Subjective performance measurement ... 4

2.2 LMX-theory ... 5

2.3 Gender bias and stereotypes ... 8

3 Hypothesis development ... 12

4 Research method and design ... 14

4.1 Participants ... 14

4.2 Experimental design ... 15

4.3 Procedures ... 15

4.4 Experimental case ... 16

4.5 Manipulations ... 17

4.6 Treatment of participants and expected outcome ... 18

5 Empirical results ... 19

5.1 Descriptive statistics ... 19

5.2 Preliminary analysis ... 19

5.3 Hypotheses tests ... 20

5.4 Additional analysis ... 23

6 Discussion and conclusion ... 28

References ... 32

Appendices ... 38

Appendix 1: Case description ... 38

(5)

1 Introduction

Gender discrimination within companies is a hot item in the news headlines. Examples of commotion about this subject are the differences in payment between male and female employees for the same job, better positions for male employees and higher bonuses for males. However, the focus of the media and the general public is mainly on top and higher functions in firms, for example the difference in salary between female and male CEO's or CFO's. In contrast with this, the purpose of this thesis is to look at a lower firm-level and find out how gender differences between a middle manager and a subordinate could have an influence on the subjective performance evaluation of the subordinate. Previous research called this phenomenon a 'gender bias' and made a difference between a pro-male bias and a pro-female bias towards the subordinates of the manager. Historically, a manager was practically always a man, and thus a pro-male bias was likely to exist. More recently however, encouraged by feminism, woman are increasingly promoted or hired as manager but yet little is known about the level of the existence of a pro-female bias. Moreover, it is unknown which kind of bias will be stronger currently. Therefore, the research question of this thesis could be formulated as follows:

RQ: Does gender differences between a subordinate and a supervisor lead to lower subjective performance evaluation, and is this effect stronger for male or female managers?

Answering this research question is important, because it can give insight how gender differences play a role in the subjective performance of a subordinate in lower levels of a firm. Previous literature has shown mixed results about the existence of a rating bias related to gender in performance evaluations settings. The aim of this study is to test with a case-based experiment whether a gender bias exists or not and for which gender this bias is stronger, i.e. is the pro-male bias or pro-female bias stronger? The general public and the media shares the view that managers in organizations can be discriminating people from the opposite gender, this experiment investigates whether this opinion is true or not. So this study is first of all important for the general public. Secondly, the results of this study can be considered important by top management of companies, because they can investigate whether middle managers in the firm give fair performance ratings to their subordinates or not. Performance evaluations should be a fair representation of the real performance of the employee and should not contain a pro-male or pro-female bias. If there is low or no evidence

(6)

for the existence of gender bias, top management can have more trust in their middle managers that in this regard supervisors give a fair evaluation.

Varma and Stroh (2001) found that both male and female supervisors exhibit a positive bias toward their same-gender subordinates, but they did not explicitly indicate for which gender this effect was stronger. The current paper further differentiates from the mentioned one by having a different experimental design, which is more focused on the link between objective results and subjective evaluation rather than only a subjective evaluation. In addition, the present paper is more actual than the approximately 15 years old paper of Varma and Stroh (2001). In the last 15 years, women are increasingly been promoted or hired in higher and managerial functions. Therefore, this replication in a new setting is very interesting. Further, as Levy and Williams (2004) suggest, additional research is necessary to investigate whether any potential rating bias due to gender exist or not.

Most previous literature suggests that supervisors have a gender bias towards their subordinates when making a subjective performance evaluation, except a few other studies which conclude that equality of gender between supervisor and subordinate has nothing to do with subjective performance evaluations. In addition, discrepancy exists whether this gender bias is stronger for male or female employees. Based on the LMX-theory (Graen, 1976), which suggests that supervisors divide their employees in an ''in-group'' and an ''out-group'', various studies are reviewed. Members of an ''in-group'' receive most of the time, support and trust of the supervisor, whereas members of the ''out-group'' will not. In addition, ''in-group'' members are likely to get higher subjective performance evaluations from their supervisor. Evidence has been found that subordinates with the same gender as the supervisor are likely to be placed in the ''in-group''. Related to this are the underlying gender stereotypes supervisors can have about their employees, which also will be reviewed. In the current research, two related hypotheses suggested that when a supervisor has the same gender as their subordinate, he or she will receive a higher subjective performance evaluation, compared to a subordinate with the opposite gender.

To answer the research question and test the hypotheses, data was collected using a case-based experiment. Participants in this experiment acted as a supervisor of a subordinate in a hypothetical managerial setting. The design of the experiment was a 2*2 between subjects design. According to Sprinkle (2003), an experiment is best suited as research method when studying whether and how management accounting practices affect the behaviour of individuals in an organization. The present paper copies the experimental design

(7)

from Bol, Kramer, and Maas (2016) with the main difference that the participant has to allocate a bonus to one subordinate rather than two and in the present study other independent variables will be used. In their experiment, the authors find that two specific design elements of an MCS have an influence on the subjective performance evaluation of employees, as measured by the degree of compression. More specific, they find that information accuracy and outcome transparency will reduce compression, but only when they are both in place. The concept gender differences as stated in the research question will be explained in this paper as differences between male and female employees, the group of transgender people will be left out of consideration to keep things clearly and easily.

The results of the experiment indicated that there was no significant support for both hypotheses in the main analysis, nor in the additional analysis. Although male participants gave higher subjective performance evaluations to male subordinates, compared to female, these results were statistically not significant. Contradictory with the expectation, female participants gave higher subjective performance evaluations to male subordinates, compared to female. This could mean that the participants in general possess a gender stereotype that male employees perform better and therefore deserve a higher amount of bonus. In line with the main results, answers on the process variable question indicated that women stronger disagreed with the statement whether they prefer to work with same-gender colleagues.

This study contributes to the subjective performance evaluation literature by giving an actual insight in the link between gender differences in a supervisor-subordinate relationship and subjective performance evaluations. More specific, it shows that there is in this setting no significant evidence for the existence of a pro-male or pro-female bias. In addition to this, the study contributes to practice by showing that male and female managers have a different rewarding style related to subjective performance evaluations. Particularly, the current research shows that female managers have less preference for working with same-gender employees, whereas male managers have a stronger preference for doing so. Because the results are not significant, we can conclude that top management can have trust in their middle managers with regard to the fact that they evaluate their employees in a fair way.

In order to answer the research question, there will be a literature review in the next section, which will result in the formulation of two hypotheses in section 3. In section 4 the research method will be described, followed up with the results of the research in section 5. Section 6 concludes.

(8)

2 Literature review

In this thesis, there is particularly a link with the performance appraisal literature and more specific, literature that deals with gender-related issues like gender bias, pro-male bias and pro-female bias in objective and subjective performance appraisals and evaluations. The most important theory which will be discussed in this literature review is the LMX-theory (Graen, 1976). First of all, a short introduction will be given to subjective performance measurement. After that, the variables and theoretical perspectives will be explained and in addition the current state of knowledge with regard to this subject will be described.

2.1 Subjective performance measurement

In general, objective performance measures are more accurate and precise in measuring the performance of employees, compared to subjective performance measures (i.e. judgments based on personal impressions and opinions). Although the use of objective measures can have significant benefits, there are also some disadvantages of using only objective performance measures. For example, the existence of uncontrollable events in a managerial setting will lead to the use of a subjective performance evaluation, because measuring only objectively in this kind of situations would be regarded as unfair (Baker, Gibbons, and Murphy, 1994). Subjectivity is used to complement perceived weaknesses in objective performance settings and to provide employees insurance against downside risk in their pay (Gibbs, Merchant, Van der Stede, Vargus, 2004). Therefore, subjective and objective performance measures are often used in combination.

Notwithstanding the benefits of subjective performance evaluation, there are also some disadvantages of using subjectivity, for example the bias that supervisors can have towards their employees or the fact that supervisors can be self-interested. This is particularly a disadvantage in combination with the fact that subjective performance evaluations cannot be verified by third parties. When supervisors are biased, they do not give a performance evaluation related to the real performance of the employee, but instead they base their evaluation on other (personal) characteristics and they have a tendency to compress evaluations by making use of leniency and centrality bias (Bol, 2011). Prior research has also shown that subjective performance measurement is influenced by cognitive limitations. For example, the halo-effect in managerial settings means that supervisors rely heavily on general impressions when they have to evaluate their employees (Nisbett and Wilson, 1977). Lipe

(9)

and Salterio (2000) have shown that in the case of different performance measures, supervisors will underweight unique measures and have more focus on common measures.

Another disadvantage of subjective performance measurement is that it is in general conducted by middle managers, whose goals could be not in line with the organization's goals (Prendergast, 1999). They are self-interested and pay attention to their personal benefits and costs (Prendergast and Topel, 1993). A personal benefit for a manager is for example the amount of bonus he or she will receive due to the work of his or her subordinates. So from that point of view, the manager tries to motivate his or her employees to work hard, because it could lead to a higher personal bonus. According to Bol et al. (2016), an example of personal costs are confrontation costs. Managers would like to avoid these costs because they are time-consuming, psychologically painful, it leads to ongoing tension and the reputation of the manager could be damaged. In conclusion, middle managers can base the performance measurement on personal benefits and costs rather than the real performance of their subordinate.

The last and most important disadvantage in this thesis of using subjective performance evaluations is the fact that employees can perceive subjective performance evaluations as inaccurate and unfair. Prior research has shown that unethical behaviour of a supervisor leads to perceived unfairness from employees, for example the bias from managers related to demographic characteristics of employees (Arvey and Murphy, 1998). Examples of those characteristics are gender, age, or race, on which supervisors can base a biased performance evaluation (Demuijnk, 2009). This could lead to unfair decisions and rankings, when employees who objectively actually score worse than their colleagues, receive higher performance evaluations because they posses certain demographic characteristics.

2.2 LMX-theory

Empirical evidence about perceived unfairness in subjective performance evaluations was given by Ostroff, Atwater and Feinberg (2004). They found that individuals tend to have a preference to classify them and work together with people with a similar background, for example gender or race. In addition to this, they found that supervisors would give a higher rating when they feel they are more similar to a subordinate (the so-called self-other

agreement). Although this basically is a psychological process, this can be regarded as unfair

(10)

found that male supervisors as well as female supervisors have a positive bias to subordinates of the same gender and that those supervisors give higher subjective evaluations to subordinates with the same sex. In their study, the authors used the leader-member exchange (LMX) theory as explanation which was originally developed by Graen (1976). This theory suggests that managers categorize their subordinates in a so-called ''in-group'' or ''out-group'', rather than have a close look at the performance of each subordinate. Liden and Graen (1980) found that supervisors put a subordinate in an ''in-group'' if he thinks that the subordinate would like to assume a greater responsibility in the firm. Therefore, ''in-group-members'' will receive a lot of time, interaction, support, trust and rewards of the supervisor, whereas the employees in the ''out'' group will not (Dienesch and Liden, 1986). This is because of limited time and resources of a manager for doing a extensive subjective performance evaluation. Heneman, Greenberger and Anonyuo (1989, p. 468) described in-group members as

''individuals from who you would most likely welcome suggestions, who you would most likely assist if they have a problem, who you would most likely to turn when you have a problem, and who you most likely depend upon to get things done''.

The LMX-model has been empirically tested (e.g. Dienesch and Liden, 1986) and they found evidence that managers place subordinates in the two different groups based on personal characteristics rather than real performance. They further found that two specific factors determine the quality of the supervisor-subordinate relationship, namely on the one hand individual characteristics such as gender and race and on the other hand nonperformance behaviors such as personal relationships. This is in line with the similarity-attraction paradigm (Byrne, 1971) which propose that people with the same characteristics have positive relationships with each other. Critical to the development of this relationship are liking and interpersonal attraction (Varma and Stroh, 2001). Empirical evidence has shown that supervisors perceive subordinates with the same gender more similar to themselves and therefore place those employees in their ''in-group'' and subordinates with the opposite gender as members of their ''out-group'' (Deluga, 1998; Engle and Lord, 1997; Phillips and Bedeian, 1994). In addition to this, Ibarra (1992) found that this phenomenon was stronger for males than for females. Thus, according to Ibarra (1992), male supervisor have a stronger favorability to male subordinates, compared to the relation between a female supervisor and subordinate. In contrast with this, Furnham and Stringfield (2001) found not only that male employees in general received lower ratings than female employees, but they also found that female supervisors rated female subordinates more favourably than male

(11)

subordinates, whereas this effect was not found by male supervisors. Still, Varma and Stroh (2001, p. 309) found that ''both male and female supervisors exhibit a positive bias toward

subordinates of the same sex and rate members of the same gender higher.''

Support for the LMX-theory suggests that the quality of the leader-member exchange has a significant impact on the work place experience of the subordinate, but also on their performance ratings (Vecchio and Gobdel, 1984). Varma and Stroh (2001) investigated which determinants of quality exist in leader-member relationships and found that individual characteristics (sex, race and educational background), task behaviour and liking are important determinants of the quality of LMX relationships. Therefore, they argue that women may prefer working with other women and men may would like to work with other men, because the communication will be easier. In addition, they proposed and found support for a gender-dyad model which suggests that the gender composition has an influence on the degree of liking, i.e. supervisors will report considerably higher liking toward subordinates with the same gender, which leads to a higher quality of the LMX-relationship. All these things together in this sequence can lead to a higher performance rating of a subordinate.

Maas and Torres-González (2011) builds further on this and they found that female participants in their experiment have a more positive response to situations with subjective evaluation by a female supervisor, compared to objective evaluations. This is in contrast with the results of male participants, where similar results were not found. So they conclude that it is important to have a female supervisor when a firm attracts women to jobs with subjective performance evaluation and to communicate this fact to candidates. Notwithstanding the fact that this paper gives a good insight in the differences between men and women with regard to the response to subjective evaluations, the results were written from the employee-perspective. The current paper looks at the opposite side and investigates whether or not supervisors are biased in subjective performance evaluations towards their employees.

An interesting side-step in the performance appraisal literature is the study of Varma, Stroh, and Schmitt (2001). They focused on the differences in the amount of international assignments between men and women, and found that male subordinates will receive more international assignments, despite the fact that women are more successful on this journeys (Adler, 1987). They also use the LMX-theory to explain this and argue that the supervisor-subordinate relationship is extremely important for getting those assignments. The authors argue that because the majority of the supervisors is male and therefore the subordinates in their ''in-group'' are likely to be also males, men will get more chances to get on international

(12)

trips. However, the authors give also some suggestions for interventions by women and they argue that upward-influencing behaviour is the most effective manner for females to prevent gender bias in this type of work settings. Two particular forms of upward-influencing

behaviour are given: on the one hand self-enhancement by making the supervisor aware of

your competencies and achievements and on the other hand opinion conformity whereby the subordinate needs to make effort to find common areas of work interests with their supervisor. Further, Colella and Varma (2001) found that people with disabilities can also improve the quality of the LMX relationship by engaging in upward influencing tactics, so Varma et al. (2001) suggest that female employees who perceive low quality leader-member exchanges should also involve in these type of tactics to improve the quality.

In summary, the LMX-theory proposes that managers divide their subordinates in an ''in-group'' and an ''out-group'', and prior research has indicated that the decision in which group an employee is divided can be depending on several demographic characteristics like gender, age, or race. This can influence the subjective performance evaluation of employees, because employees in the ''in-group'' are favoured by the manager and have better chances on higher subjective performance evaluations, relative to employees in the ''out-group'' of the manager. Prior research indicates that upward-influencing behaviour is the most effective manner to improve the quality of the leader-member exchange.

2.3 Gender bias and stereotypes

Another stream of literature related to this subject is research to gender bias and the underlying stereotypes. This is also linked to the LMX-theory because the more negative bias a supervisor has towards his or her subordinate, the greater the chance that this subordinate will be placed in the ''out-group'' of the supervisor. Gender bias can be divided into a pro-male bias and a pro-fepro-male bias. Nieva and Gutek (1980) defines a pro-pro-male bias as a situation when men get more favourably ratings than women with exactly the same performance. A pro-female bias exists when the opposite situation is true and females are rated more favourably compared to males. Early literature has shown mixed results whether the existence of a gender bias, because some studies show a pro-male bias (Goldberg, 1968; Fidell, 1970; Pheterson, Kiesler, and Goldberg, 1971; Rosen and Jerdee, 1974; Pazy, 1986), whereas other studies show pro-female bias (Taynor and Deaux, 1973; Jacobson and Effertz, 1974; Bigoness, 1976; Mobley, 1982). Finally, some early studies indicated neither no pro-male nor pro-fepro-male bias (Heilman, 1975; Hall and Hall, 1976; Wexley and Pulakos, 1982).

(13)

Shore (1992) argues that the scarcity of women in management functions is mainly because of the gender stereotypes about the qualities of women as leaders. Because the majority of managers is still male, female subordinates do not get the chance to promote to higher functions. He further argues that these gender stereotypes are deeply rooted because the results of two similar studies (Schein, 1973; Heilman, Block, Martell, and Simon, 1989) does not change in 16 years time. Bauer and Baltes (2002) supports this finding and argues that raters with traditional stereotypes of women will give them lower ratings compared to raters who do not have traditional stereotypes. The authors give the structured free recall

intervention as a solution to reduce gender bias in performance ratings in situations with little

information, such as job interviews. More recently, Booth and Leigh (2010) investigated whether gender discrimination exist or not by sending fake CVs to apply for entry-level jobs. They found that in their setting a pro-female bias exist, but only when the occupation is female-dominated. The authors conclude that this finding is due to gender stereotyping: the society has prescriptions about which jobs are appropriate for men and women.

Building further on the discrepancy of the existence of a male bias or a pro-female bias, Sackett, DuBois, and Noe (1991) found something interesting, namely that women received lower ratings than men when they are underrepresented in a work group (i.e. less than 20% women), but in contrast they receive higher ratings than men when the work group consisted of more than 50% women. Gender bias can have some consequences for both the firm and employee. More specific, according to Bowen, Swim, and Jacobs (2000), gender bias could lead to less accuracy and potential unfair discrimination. They define unfair discrimination as gender-segregated work force and a different status of women in organizations.

Prior literature suggests that gender stereotypes are the driving force behind gender bias (Biernat, Tocci, and Williams, 2012). According to Biernat (2003, p. 1019), the definition of a stereotyping effect is: ''a finding that individual group members (comparable

in all ways except their category membership) are judged in a direction consistent with group-level expectations or stereotypes''. Both men and women do have certain gender

stereotypes about the same and the opposite gender. According to Koch, D'Mello, and Sakket (2015), there are two categories of gender stereotypes: communal and agentic. The first stereotype is in general more related to females and concerns things for helping other people, like being kind, helpful, emotionally expressive and affectionate (Eagly and Karau, 2002). The second stereotype is in general more attributable to males and this can be summarized as

(14)

being assertive and controlling, with characteristics like dominant, ambitious, confident and independent (Eagly and Karau, 2002). According to Shields (2002), the strongest gender stereotype is the belief that women are more emotional. Another distinction that can be made between gender stereotypes is on the one hand the descriptive ones and on the other hand the prescriptive ones (Eagly and Kite, 1987; Heilman, 2001; 2012). The latter means how men and women should be, and the first one describes how men and women are in general.

Gender bias in managerial settings can arise when people judge males and females differently because of the gender stereotypes described above. Koch et al. (2015) gives two possible explanations why gender bias exists. First, they give the role congruity theory (Eagly and Karau, 2002) as an explanation. This theory defines bias as congruence between two different types of stereotypes: job requirements on the one hand and gender groups on the other hand. The greater the dissonance between these two, the greater the chance on gender bias. This theory suggest that there are typically male and female occupations, which determines whether an employee will be selected or not. Another explanation for gender bias according to Koch et al. (2015) is the lack-of-fit model, developed by Heilman (1983,2001). This model is in line with the role congruity theory and explains why women will be in general not selected for typically masculine jobs and men will be not selected for typically feminine jobs. Davison and Burke (2000) supports this notion and found in addition that the gap between the evaluations of men and women decreased when there was more job-relevant information at hand. A certain form of self-selection will also apply. With this two explanations as starting point, Koch et al. (2015) hypothesized and found that men are rated more favourably in male-dominated jobs and women are rated more favourably in female-dominated jobs. In addition, a stronger gender-role congruity bias has been found for male raters compared to female raters.

Pulakos and Wexley (1983) found similar results, and they conclude that perceptual similarity between a supervisor and a subordinate accounted for a great part of the variance in the performance rating. In their study, they found significantly lower performance ratings in cases where managers and subordinates have a different gender. The major finding of this study is that the similarity hypothesis, which has been confirmed in laboratory studies, can also be generalized into a field setting. In both experimental settings and field studies there is evidence of gender bias. To give an example, Bowen et al. (2000) did a field study and found significant evidence for a pro-male bias when only men served as raters. Measure-specific gender stereotypicality was the cause of gender bias in performance appraisal. Gender bias

(15)

both exists in the business sector (Gutek, Cohen, and Tsui, 1996) as well as in the non-profit sector (Ngo, Foley, Wong, and Loi, 2003).

Green and Mitchell (1971) proposed in their leadership model that performance by subordinates in the first phase of their model leads to the formation of attributions. This leads to specific managerial behaviour towards the subordinates in the second phase. The attributions of the first phase can be attributed to internal and external causes. Examples of internal causes are effort and ability, whereas examples of external causes are luck or task difficulty (Heneman et al., 1989). In the latter study, the authors focused on this first phase and they tried to link this with the LMX-theory. They found that the internal attributions were higher for in-group members in a situation of effective performance, and external attributions were higher for out-group members. They found no support for the opposite situation, where an ineffective performance leads to higher external attributions for in-group members and higher internal attributions for out-group members. Only the internal attributions were significantly related to the leader-member exchange and critical performance incidents. Their data suggest that managers assign attributions and show behaviour towards their subordinates which are not based on performance, but on the in-group or out-group status of them.

As described above, prior gender bias literature has shown very mixed results about the existence of a gender bias in general, a pro-male bias or a pro-female bias. The majority of the research on this subject has been done a long time ago. However, Koch et al. (2015) showed new interest in this subject by making a meta-analysis. In their study, they have reviewed all the literature about gender bias and similar subjects. They found that male employees were favoured for male-dominated occupations and that male supervisors have a greater role-congruity bias. The existence of this gender bias is driven by the gender stereotypes that supervisors can have towards their subordinates. The consequence of gender bias is that supervisors place subordinates with the same gender in their ''in-group'' and subordinates with the opposite gender in their ''out-group'', according to the LMX-theory.

In sum, both men and women have certain gender stereotypes about the opposite gender, and those stereotypes can lead to a gender bias, which can be divided into a pro-male and a pro-female bias. Prior literature has shown mixed results which form of gender bias is stronger. However, the majority of these kind of studies have been done a long time ago. Gender bias is related to the LMX-theory, because managers are likely to place subordinates with the same gender as themselves in their ''in-group''. Based on this literature review, the hypotheses for this thesis will be formulated in the next section.

(16)

3 Hypothesis development

Based on previous literature as described in section 2, the current section presents the hypotheses which will be tested with a case-based experiment.

Despite the fact that subjective performance evaluations are used to complement objective performance measures, employees can regard them as unfair, because those

evaluations can be based on personal characteristics of the subordinate and supervisors have a tendency to compress evaluations by using leniency and centrality bias (Bol, 2011).

Examples of personal characteristics which can be involved in performance evaluations are age, education level, race, or gender.

Focusing on the gender characteristic, it can be assumed (based on prior literature) that supervisors give higher performance evaluations to same-gender employees, compared to opposite-gender employees. However, subordinates with the same gender as the supervisor can actually be worse performers than the subordinates with the opposite gender (based on objective measures).

Prior literature (Varma and Stroh, 2001) suggests that supervisors will give higher ratings to subordinates with the same characteristics as themselves, because the supervisors place those subordinates in their ''in-group'', according to the LMX-theory (Graen, 1976). This theory suggests that managers do not place their subordinates based on real performance in their favorite group, but rather on personal characteristics. In addition, Ostroff et al. (2004) found that both subordinates and supervisors have a preference to classify themselves and work together with people with a similar background.

In contrast with this previous results, other researchers found no significant support for the proposition that leaders give always higher ratings to employees with the same gender (Davison and Burke, 2000). The latter study suggests that male subordinates always get higher performance evaluations, both from male and female supervisors. Underlying gender stereotypes can lead to gender bias, which is linked to placing employees in ''in-groups'' and ''out-groups''. Out-group members are stereotyped as the inferior group and could therefore experience gender discrimination (Jussim, Coleman, and Lerch, 1987). In the end, this can have negative consequences for the subjective performance evaluation of subordinates who have a different gender compared to their supervisor. More important, to reduce

discrimination in work settings, performance evaluations should not be based on personal characteristics, but only on the real performance of the subordinate.

(17)

Despite a few exceptions, the literature in general support the idea that supervisors have a preference group of employees based on personal characteristics, like gender. The question remains however, for which gender this effect will be stronger. Ibarra (1992) found a stronger pro-male bias compared to a pro-female bias, whereas Furnham and Stringfield (2001) found support for the opposite case. Yet, Varma and Stroh (2001) found strong and significant support that both male and female supervisors have a positive bias toward subordinates of the same gender and consequently rate those subordinates higher. Gender stereotypes are deeply rooted in work settings and lead to a gender bias from supervisors towards their employees. Historically, managers were generally male, therefore, female employees were always in a inferior position.

Assuming that it is true that supervisors give better evaluations to subordinates with the same personal characteristics, one might expect that subordinates with the same gender as the supervisor will get higher subjective performance evaluations, compared to subordinates with the opposite gender. Taken all these things together, this leads to the following

hypothesis:

Hypothesis 1: Despite similar objective results, male supervisors will give higher subjective performance evaluations to male subordinates, compared to female

Hypothesis 2: Despite similar objective results, female supervisors will give higher subjective performance evaluations to female subordinates, compared to male

(18)

4 Research method and design

In order to test the hypotheses, data was gathered with a case-based experiment, whereby the respondents were mainly students from the university and people from the personal network of the author. No specific knowledge or education level was needed to fill in the survey. However, there was one control question in the end to check whether the participants understood the case or not. The respondents received an anonymous link via e-mail, Whatsapp or LinkedIn to participate in this experiment. One important element in this experiment was that approximately the one half of the participants should be male and the other half female. In this section, there will be first some general information about the participants. After that, the experimental design and procedures will be explained. Then, a detailed description of the experimental case will be described. This section concludes with describing the manipulations, treatment of participants and a graph with the expected outcome of the experiment.

4.1 Participants

Table 1 shows some general information about the participants in the experiment. This table indicates that the group of participants consisted of 47 male respondents and 41 female respondents. Almost all participants (86%) were Dutch, the remaining 14% had a different nationality. The average age of the participants was 26, ranging from 15 till 71. The education level of the participants ranged from high school till university master level, but more than half of the participants (57%) was educated at university level. The level of work experience was in general low (57% of the participants had less than three years of work experience), but ranged from 0 till 43 years and the average amount was 5 years. The last demographic question that participants answered was about a managerial position. Almost every participant answered that they never had been in a managerial position (86%), whereas the other participants answered that they are currently in this position (8%) and the remaining participants answered that they held a managerial position before (6%).

(19)

Table 1: Descriptive statistics participants 4.2 Experimental design

The hypotheses have been tested using a case-based experiment with a 2 * 2 between-subjects design, because there was manipulation in one variable. The participant filled in his or her gender and after that he or she was randomly assigned to the condition of a male or a female subordinate. This means the experiment consisted of total four conditions, namely male supervisor with male subordinate, male supervisor with female subordinate, female supervisor with female subordinate and lastly female supervisor with male subordinate.

4.3 Procedures

The experiment was filled in during the period of the 16th of May 2018 till the 29th of May 2018 using Qualtrics software. Because the experiment was sent by e-mail, Whatsapp or LinkedIn, respondents were able to fill in the experiment whenever and wherever they liked. The average time of completion was 4,95 minutes (without outliers above 30 minutes; when those are included, the average time of completion was 106,21 minutes. This is because participants could save their progress in the survey and go further with it whenever they

Gender Frequency Percent Nationality Frequency Percent

Male 47 53,40% Dutch 76 86,36%

Female 41 46,60% Other 12 13,64%

Total 88 100,00% Total 88 100%

Age Frequency Percent

Education Frequency Percent <21 12 13,64%

Elementary school 0 0,00% 21 - 30 64 72,73%

High school 3 3,41% 31 - 40 3 3,41%

Secondary vocational education 11 12,50% 41 - 50 5 5,68% Higher professional education 24 27,27% 51 - 60 3 3,41% University bachelor 16 18,18% >60 1 1,14% University master 34 38,64% Totaal 88 100% University PhD 0 0,00%

Totaal 88 100% Work experience Frequency Percent

<1 year 19 21,59%

Managerial position Frequency Percent 1 year 18 20,45%

No 76 86,36% 2 years 13 14,77%

Yes, but not at this moment 5 5,68% 3 years 5 5,68%

Yes, currently 7 7,95% 4 years 3 3,41%

88 100% 5 years 4 4,55%

6 years 10 11,36% >6 years 16 18,18% 88 100%

(20)

want). The complete experiment lasts four phases, which were: pre-experimental questionnaire, general case information, the actual experiment and the exit questionnaire. In the pre-experimental questionnaire the participants were asked some general demographic information (gender, age, nationality etc.). However, the question related to the gender is of great importance for the allocation of the participants in the specific condition in the third phase, which is the actual experiment. After this first phase, the participants were given general information about the case and what the expectation was of the participant. Below, a detailed description of the case will be shown. The last phase of the experiment was the exit questionnaire, which contains of manipulation checks, the experience of the participants and in the end a question related to process variables. The latter was to analyze whether the hypothesized cause-effect relationship was actually the one that drove the decisions of the participants. Please see appendix 1 for the full case description.

4.4 Experimental case

This experiment followed the prescription of Nieva and Gutek (1980), which explains that the most used form in studies of evaluation bias involves the description of two identical hypothetical persons, except their gender, for who evaluation judgments and personnel decisions need to be made. The participants in the hypothetical case setting were given a role of one of the six middle managers of a retail company (RVG NV). The participants had to review the performance of one subordinate in their region. The participants were given information about six KPI's, whereby both the target and actual measures were given. Comparing targeted KPI's with actual KPI's is the most reliable and fair method to give subjective performance evaluations, because then non-controllable circumstances do not exist. The set of performance indicators included sales revenue per square foot (euro amount), return on sales (percentage), customer satisfaction rating (1 - 100 scale), mystery shopper program rating (1 - 5 scale) and average percentage markdowns (average percentage of markdown from original retail price). Based on this objective performance indicators, participants had to subjectively weight this information and decide how much bonus a subordinate should receive. The aim of the experiment was namely that the participants should determine an annual bonus for the store manager in the end of the experiment.

After answering the question related to their gender, participants were randomly assigned to a condition with a male or female subordinate, what makes a total of four conditions. All the participants will get the same objective performance information of a male

(21)

or a female subordinate, depending on the condition they are divided in. To give more guidance in this experiment, the case description gave more information about the retail company. The organizational strategy that was used in the hypothetical firm is a form of regional management. The participant in the experiment had to assume the role of a regional manager who had a few store managers under his or her management. In the third phase of the experiment, the participant had to determine an annual bonus amount for the allocated supervisor, which can be a man or a woman. To make a good comparison between male and female subordinates, the objective results of them were fully identical. This means that all participants got the same objective performance information from a subordinate, with the only difference in the name of the subordinate. The male subordinate was called John and the female subordinate was called Anna. Furthermore, in the description, a few times John is called he and consequently Anna is called she. Therefore, the participants should in the end of phase three of the experiment know which gender the subordinate has. In the beginning of phase four, a control question existed whereby the question was what gender the subordinate had. The results of this manipulation check will be discussed in the next section. Furthermore, in the exit questionnaire there was a process variable question and the possibility to give remarks and suggestions about the survey.

4.5 Manipulations

The case has been manipulated in the following ways: after the gender was determined in the pre-experimental questionnaire the participants were allocated in different groups. Approximately fifty percent of the female group evaluated a female subordinate, the other half a male subordinate. For the male group, also approximately fifty percent of the group evaluated a female subordinate and the other half a male subordinate. In table 2 the distribution of the participants over the 4 conditions is presented.

Supervisor

Male Female Overall

Employee

Male (John) 21 23 44

Female (Anna) 26 18 44

Overall 47 41 88

(22)

As an copy of the experiment of Bol et al. (2016), in the end of the experiment, the participants were asked to allocate a bonus amount between €0 and €10,000 to the subordinate. For the sake of complexity, the participants were asked to allocate the bonus amount directly instead of providing performance ratings which should be translated into a monetary amount of bonus. It was important to know for the participants that the annual bonus was part of a new bonus system, so there was not a possibility to compare bonuses with the previous year.

4.6 Treatment of participants and expected outcome

In total, there were 101 participants who started the survey. However, 13 of them did not completed the survey and will therefore be excluded from further analysis. This leads to a total of 88 valid respondents. Using an option in Qualtrics, it was prevented that the same people filled in the survey twice. Furthermore, participants could not go back to the previous page in the experiment. However, they could save their progress and go further with the survey whenever they wanted. All these things together increased the validity of the research. Based on the hypotheses as described in the previous section, the expectation is that male participants give higher subjective performance evaluations to male employees compared to female employees. Secondly, it is expected that female participants give higher subjective performance evaluations to female employees compared to male employees. This relationship is shown in figure 1. In the next section, the results of this experiment will be discussed.

M F M (John) Gem € - € -F (Anna) Gem € - € -Employee Supervisor

(23)

5 Empirical results

When the phase of data gathering was finished, the data analysis started. An important validity control was the presence of a manipulation check in the end of the experiment. In all cases, participants had to recall whether the evaluated subordinate was a female or a male employee. The result of the manipulation check indicate that 81 participants answered the manipulation check right and 7 participants answered this question wrong. However, in order to be conservative, all recorded data will be used in the main analysis. In the additional analysis, there will be one analysis that exclude the participants who answered the manipulation check wrong to find out if this makes sense for the overall findings. First, some descriptive statistics will be shown in this section. After that, the preliminary analysis will be described, including an outlier analysis. Then, the two hypotheses will be tested, using SPSS software. In the end, there will be some additional analysis. This also includes an analysis of the process variable included in the experiment. To give some guidance in reading this section, an overview of the variables is included in appendix 2.

5.1 Descriptive statistics

In table 3 the descriptive statistics are shown. The mean scores of the bonus amount, standard deviations and the number of all observations are shown for every condition. From this table, you can already see some differences between the conditions. For example, it shows that male participants gave an higher subjective performance evaluation to same-gender employees compared to woman. However, for female participants this effect cannot be seen. As described before, table 1 shows some general descriptive statistics about the participants in this experiment. Later in this section, the hypotheses will be tested and this will make clear whether the results are significant or not. However, the next paragraph discusses first a preliminary analysis of the data.

5.2 Preliminary analysis

Before the hypotheses are tested, there will be a preliminary analysis. As the data indicates, there are some outliers at first sight, which can be interesting for the main results of this thesis. The outliers related to demographic variables will be excluded from the preliminary analysis, those outliers will be discussed in the additional analysis paragraph. However, there are also some outliers in the amount of bonus the participants gave to the hypothesized subordinates. Some participants gave an very low or high bonus amount (probably because

(24)

Supervisor

Male Female Overall

Employee Male (John) N = 21 Mean = €5,241 SD = €2,376 N = 23 Mean = €5,224 SD = €2,910 N = 44 Mean = €5,232 SD = €2,638 Female (Anna) N = 26 Mean = €4,617 SD = €3,184 N = 18 Mean = 4,094 SD = €3,065 N = 44 Mean = €4,403 SD = €3,111

Table 3: Descriptive statistics

they didn't read the case or they didn't understand the case). However, an outlier analysis in SPSS did not indicate there were any outliers in the data set. Therefore, in order to be conservative with the data set, all the results will be included in the hypotheses tests. After that, the additional analysis includes some tests to find out whether those observations make a difference or not. Thus the main hypotheses tests are conservative and includes the full data set.

5.3 Hypotheses tests

Because both hypotheses can be tested in the same manner, this specific statistical test will be first described. In this research, there are two categorical independent variables with both two levels (firstly the gender of the supervisor, which can be either male or female and secondly the gender of the employee, which also can be either male or female). Further, there is one quantitative dependent variable, namely the amount of bonus. Therefore, a (factorial) two-way ANOVA is particularly suitable in this research. First of all, a new variable (BONUS) was created to merge the two different variables (BONUS_JOHN and BONUS_ANNA). After that, the two-way ANOVA could be done. However, it is first important to check whether the criteria for this statistical test are satisfied. This means that the populations need to have the same variances. A Levene's test in SPSS could give an answer whether this is true or not. The null hypothesis related to this test specifies that all conditions have an equal variance, therefore it is important that the p-value of this test is not significant. Then, we can assume equal variances in all conditions.

Hypothesis 1 predicts that male supervisors give higher subjective performance evaluations to male subordinates, compared to female. To test this hypothesis the average

(25)

annual bonuses of the two following conditions are analyzed: on the one hand a male supervisor with a male subordinate and on the other hand a male supervisor with a female subordinate. This hypothesis is tested using a two-way ANOVA, but it is important that equal variances in all conditions can be assumed. Therefore, a Levene's test has been conducted. Table 4 gives the results of this test and indicates a non-significant result (Levene's Statistic 1.917; p-value > 0.05).

Table 4: Levene's test variable bonus

Therefore, the conclusion here is that equality of variances can be assumed. The next step was to conduct the two-way ANOVA test. Because this research is about multiple comparisons, it is important to correct for errors with the Bonferonni-correction in SPSS. The descriptive statistics in table 3 showed that there was a difference in the amount of bonus between the two conditions related to male supervisors. However, as table 5 indicates, the main effects are not significant, but more important, the interaction effect is neither significant (F = 0.163; p-value > 0.05)

Table 5: Results ANOVA main analysis

The results of the two-way ANOVA test indicate that the interaction effect is not significant, therefore a follow-up test was not relevant. These results therefore do not support the first hypothesis, suggesting that male supervisors give not significant higher subjective performance evaluations to employees with the same gender, compared to female employees.

Levene's test for Equality of Variances; Test variable: Bonus

F Sig.

Bonus 1,917 0,133

Type III Sum of Squares df Mean square F Sig. Partial Eta²

Corrected model 18.020.469 3 6.006.823 0,708 0,550 0,025 Intercept 1.986.492.824 1 1.986.492.824 234,219 0,000 0,736 Gender_Supervisor 1.569.471 1 1.569.471 0,185 0,668 0,002 Gender_Employee 16.608.826 1 16.608.826 1,958 0,165 0,023 Interaction 1.383.256 1 1.383.256 0,163 0,687 0,002 Error 712.433.331 84 8.481.349 Total 2.772.890.737 88 Corrected total 730.453.801 87

(26)

Hypothesis 2 predicts that female supervisors give higher subjective performance evaluations to female subordinates, compared to male. To test this hypothesis the average annual bonuses of the two following conditions are analyzed: on the one hand a female supervisor with a female subordinate and on the other hand a female supervisor with a male subordinate. This hypothesis has been tested in the same manner as hypothesis 1, which is a two-way ANOVA test. This test is the same as the one described above, because this includes all data. Therefore, we can assume equality of variances in all conditions (see table 4). Because the ANOVA test suggested that there is no significant interaction effect (see table 5), a follow-up test was not needed. These results do not support the second hypothesis, suggesting that female supervisors give not significant higher subjective performance evaluations to employees with the same gender, compared to male employees.

In the end, a graph was made to show the difference in the average amount of bonus that was given by the participants to their hypothesized subordinate. Figure 2 shows that male participants gave on average an higher bonus amount to male subordinates compared to female subordinates. However, as this section describes, this difference was not significant, and therefore no conclusion can be made based on this graph. A notable outcome is the difference in the average amount of bonus that was given by female participants. As opposed to the expectation, they gave a higher subjective performance evaluation to male subordinates and not to female subordinates. Because the lines in figure 2 do not cross each other, we have extra support that there was no interaction effect in this setting. The implications of this results will be discussed in the next section.

(27)

5.4 Additional analysis

Now, the main results are clear. However, some additional tests could indicate further interesting differences in the data set. This can give more insight in this topic and possibly give significant results. Because the main analysis used all the data, it can be interesting to find out whether the results are the same when some participants are excluded or not.

First of all, it is interesting to look only at the participants who answered the manipulation check right. When people answered this question wrong, this could indicate that they have not read the case study at all or very quickly and not accurate enough. After exclusion of these participants, the same statistical test has been performed, as described above. Table 6 gives the results and this indicates that there is still no significant interaction effect in place. The group of participants who answered the manipulation check question wrong consisted also of some participants who filled in an unrealistic bonus amount. In this first additional analysis, they are also excluded. When compared to the main analysis, it becomes clear that the F-score related to the interaction effect is much greater when we exclude those participants. Therefore, the p-value decreases from 0.687 in the main analysis to 0.417. However, this is still not significant with an alpha level of 0.05. Thus, the hypothesis are still rejected.

Table 6: Results ANOVA test without participants who answered manipulation check wrong

Furthermore, it can be interesting to find out whether age of the participant has an effect on the results. As described earlier, the average age of the participants is 26 years. Because the major part of the participants is still student, they are younger than 30 years. The participants are divided into a group of people who are younger than 30 years and a group of people who are 30 years and older. Table 7 gives the results of the two-way ANOVA test and this indicates that there is neither in the age group of below 30 or above 30 a significant interaction effect. However, it is interesting to see that the F-score of the group of participants who are 30 years or older is much higher than the F-score of the group of participants who are below 30 years old. This indicates that age has an effect on the subjective performance evaluation in this setting, but still not significant.

F Sig.

Gender_Supervisor 0,013 0,909

Gender_Employee 2,873 0,094

(28)

Table 7: Results ANOVA test divided in two age groups

The third control variable is nationality of the participant. The respondents are divided in the group of Dutch people and in the group of other nationalities. This is because the major part (86%) of the participants is Dutch. As table 8 indicates, no significant interaction results can be found. However, the score of non-Dutch participants is slightly higher than the F-score of Dutch participants, which means that the results of Dutch participants are more opposed to the expectations.

Table 8: Results ANOVA test divided in two nationality groups

Fourthly, education level can be an interesting factor, therefore the participants are divided into a group of higher educated people (only people with university and university of applied sciences) and lower educated people. The first group covered 84% of the whole data set. Table 9 gives the results of the two-way ANOVA test and this indicates also that there is still no significant interaction effect. However, it is interesting to see the difference in the F-statistic between the two groups. When we only look at the group of participants with an education level below Higher Professional Education, the interaction effect is stronger than

Panel A: only participants who are younger than 30 years old

F Sig.

Interaction 0,499 0,482

Panel B: only participants who are 30 years old or older

F Sig.

Panel A: only Dutch participants

F Sig.

Panel B: only other-nationality participants

F Sig.

(29)

in the group with participants with an education level equal to or higher than Higher

Professional Education. However, these results are still not significant.

Table 9: Results ANOVA test divided in two education level groups

In addition to this, work experience is also an interesting factor to find out whether this makes any difference related to the main analysis. Because the major part of the participants is still student, they do not have a lot of work experience. Two groups were made, one which existed of participants with more than 3 years of work experience and the other with participants with equal or less than 3 years of work experience. As table 10 indicates, there are still no significant interaction effects. However, the F-score for the group with participants with equal or less than 3 years of work experience is very low, compared to the F-score of the other group. This indicates that work experience has an effect on the way a subjective performance evaluation is made.

Table 10: Results ANOVA test divided in two groups related to work experience

Panel A: only University (of applied sciences) education level

F Sig.

Panel B: only < University of applied sciences education level

F Sig.

Panel A: only participants with <= 3 years work experience

F Sig.

Panel B: only participants with > 3 years work experience

F Sig.

(30)

Lastly, one question in the experiment was related to a managerial position of the participant. The majority of the participants (86%) never held this position and the minority was currently a manager or was in this position before. It is interesting to look if there are differences between those two groups. Table 11 gives the results of the two-way ANOVA test. Because there were such a few participants with (previous) managerial position, there was no condition in which a female supervisor evaluated a male subordinate. Therefore, no interaction effect could be determined. But table 11 also indicates that the F-score in the group with participants with no managerial position is very low.

Table 11: Results ANOVA test divided in two groups related to a managerial position

Finally, the results of the process variable will be discussed. In the survey, participants were asked how much they agree with the following statement: ''In work-related

settings, I prefer to work with same-gender colleagues''. They gave answers on a 5-point

Likert scale, whereby 1 indicates that they strongly agreed with the statement, and 5 indicates that they strongly disagreed with this statement. The Likert scale is treated as an ordinal scale. As table 12 shows, male participants gave on average a score of 3,17 and female 3,46. The middle score on a 5-point Likert scale is 3. With a one-sample t-test, it has been tested whether the scores given on this scale are significantly different from the midpoint or not. As table 12 further indicates, this is indeed the case for female participants and all participants together, but not for only male participants. This means that female participants are in general do not prefer to work with same-gender colleagues, because they stronger indicate that they disagreed with the statement. Male participants have a less strong opinion about this, but in general also disagree with the statement.

This is in line with the results of the main analysis. Although not significant, male participants gave on average an higher subjective performance evaluation to male

Panel A: participants with no managerial position

F Sig.

Panel B: participants with a (previous) managerial position

F Sig.

(31)

-subordinates compared to female. This is in contrast with female participants, where similar results were not found. Moreover, female participants gave male subordinates even higher evaluations, compared to female. Therefore, we can conclude that female managers less prefer to work with same-gender subordinates compared to male managers. However, both male and female participants on average disagreed with the statement, which means that in general all participants do not have a strong preference to work with same-gender colleagues. But the results from male participants are less strong.

Table 12: Process variable

To sum up, this section discussed the empirical results of the experiment. First of all, the main analysis included all data but do not indicate significant interaction effects. Therefore, based on an alpha level of 0.05, we reject both hypotheses, because the results are not significant. It is however noteworthy that on average the male employee (John) received from both male and female supervisors an higher subjective performance evaluation. After the main analysis, some additional analysis were conducted to find significant interaction effects. Although those were not found, it is noticeable to see that especially age, education level and work experience have an impact on the F-score of the interaction effect from the ANOVA-test. Finally, the answers on a process variable were discussed, which indicates that female participants stronger disagreed with the statement that they prefer to work with same-gender colleagues. This is in line with the prior results. The next section contains the discussion and the conclusion of this study.

Strongly Agree Somewhat agree Neither agree nor disagree Somewhat disagree Strongly

disagree Total Mean

Male 5 4 23 8 7 47 3,17

Female 4 2 16 9 10 41 3,46

9 6 39 17 17 88 3,3

Condition Variable N Test value Mean Mean diff t df Sig. (2-tailed)

Male PROC_VAR1 47 3 3,17 0,170 1,034 46 0,307

Female PROC_VAR1 41 3 3,46 0,463 2,460 40 0,018