The interacting effect of gender and performance report layout on managers’ subjective evaluation judgements

The interacting effect of gender and performance report layout

on managers’ subjective evaluation judgements

Name: Alexandra-Diana Dobrin Student number: 10856595 Thesis supervisor: Razvan Ghita Date: June 25, 2018

Word count: 19942,0

MSc Accountancy & Control, specialization Control

Statement of Originality

This document is written by student Alexandra-Diana Dobrin who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

Abstract

In the context of subjective performance evaluations, managers tend to offer performance ratings which are lenient (relatively high) and compressed (not very dispersed). This paper reports on the effect of performance reports containing either two-dimensional or three-dimensional bar charts on the leniency and compression of managers’ subjective evaluations. In addition, it investigates if this effect depends on the evaluator’s gender. Firstly, it is predicted that performance ratings are less lenient and less compressed when performance reports contain 2D bar charts than when performance reports contain 3D bar charts. Secondly, it is expected that performance ratings are more lenient and more compressed when performance reports are evaluated by female managers than by male managers. Lastly, it is predicted that the effect of 2D bar charts as compared to 3D bar charts on the leniency and compression of performance ratings is stronger when the evaluation is performed by male managers. The results show that female managers offer more compressed performance ratings than male managers. The implications of this study for research and practice are discussed.

Keywords: subjective performance evaluation; leniency; compression; 2D bar charts; 3D bar

Contents

1 Introduction ... 5

2 Literature review ... 8

2.1 Subjective evaluation judgements ... 8

2.1.1 Performance evaluation leniency and compression ... 9

2.2 Performance report layout ... 11

2.2.1 Information organization – balanced scorecard format ... 12

2.2.2 Presentation format - 2D and 3D bar charts ... 13

2.3 Female managers versus male managers – psychological and cognitive traits ... 17

3 Hypotheses development ... 22

3.1 The impact of 2D and 3D bar charts on leniency and compression ... 22

3.2 The influence of gender on the relation between presentation format and evaluation judgements ... 25

3.3 Overview of the Hypotheses ... 28

4 Experimental design and method ... 29

4.1 Experimental design ... 29

4.2 Participants ... 31

4.3 Task and manipulations ... 32

4.4 Experimental procedures ... 38

5 Results ... 40

5.1 Preliminary analyses and measures ... 40

5.2 Hypotheses test ... 43

5.3 Supplemental analyses ... 47

6 Discussion and conclusion ... 50

References ... 56

1 Introduction

A large body of literature comes to argue that the way in which information is presented can have an impact on the judgement and the processes and outcomes of decision-making (e.g. Cardinaels 2008, Kramer et al. 2016). In addition, it can be stated that when it comes to the design of management accounting information, there are no standard setters or regulators to specifically influence it and as a consequence, there is no prescribed form about its content, how this is to be developed and presented (Atkinson et al. 2011).

Starting from the fact that managers’ judgements and decisions can be influenced by the design of the information they are presented and adding the fact that there is no specific rule for the layout, the question that quickly arises regards what is the most effective format that would lead to more accurate evaluations. I found this particular topic interesting for an in-depth study as I was also discovering that the presentation and visualization of management accounting information is part of a small but growing literature with some relevant future research avenues.

Further on, within any type of organization, an important aspect regards allowing some discretion to the managers in assessing their subordinates’ performance. This process is linked to the reward system and therefore, has implications for employees’ motivation which is highly relevant for any company as a way of aligning behaviors towards better results. While some research tried to have a look on the managers’ subjective performance evaluations and its influences (Bol and Smith 2011), there is not much known about the factors that determine the rating behavior (Golman and Bhatia 2012).

Linking the two aspects presented earlier on regarding the presentation of management accounting information and the subjective performance evaluation process, an interesting and relevant research path is investigating the effects of performance reports layout on the subjective evaluation judgements. This comes as highly needed in the literature taking into account that the impact of managerial accounting reports’ design on decision-making was very little researched (Kramer et al. 2016).

This paper, after a comprehensive review of the research niche previously introduced, sets to investigate the effect of performance reports containing either two-dimensional or three-dimensional bar charts on the leniency and compression of managers’ subjective evaluations. In addition, it investigates if this effect depends on the evaluator’s gender.

The investigation is essential and it sets to fill some important gaps in knowledge as there is no study I am aware of in this particular research context. Firstly, in order to make accurate subjective evaluations, managers that are provided with performance reports might face some costly cognitive effort to extract the relevant information. This might lead to managers giving high ratings and not clearly differentiating between the best and worst employees in order to keep away any conflicts (Bol 2011, Bol et al. 2016). As leniency and compression are costly for a company, finding the right layout for the performance reports is of high relevance and importance. Secondly, it is also highly essential for both theory and practice to have knowledge of the rating behavior of female and male managers as research points towards gender differences (Furnham and Stringfield 2001). As a consequence, finding the reason behind these might also help explaining the biased ratings.

There is no direct literature guidance related to the study’s investigation. Despite this, a mix of knowledge from various related fields brings important insights and help defining the predictions. The literature on visualization suggests that overall, 2D bar charts as compared to 3D bar charts seem to lead to better decision making, making data easier to interpret by offering visual clarity. Further on, with regards to female and male managers, literature provides some interesting insights for gender’s psychological and cognitive traits.

To answer the study’s research question, data was collected using an online experiment. As the purpose of the study relates to the managers’ evaluation judgements, the online setting offered the great chance of gathering managers from various parts of the world to engage in performing the evaluation task within the scenario-based experiment. The main empirical findings show that overall performance ratings are more compressed when performance reports are evaluated by female managers than by male managers.

Further on, this study contributes to the management accounting theory and practice with regards to the design of effective performance reports which can lead to more accurate subjective evaluations and reduce the costs determined by lenient and compressed ratings. It brings elements of novelty with regards to the presentation format of a performance report as the investigation of the effects of 3D bar charts in the context of the current research is something new. In addition, it offers perspectives on how female versus male managers perceive the information displayed in the performance reports containing information in either 2D bar charts or 3D bar charts and sheds light over what might explain their rating behavior.

The study adds knowledge to the subjective performance evaluation literature, the presentation and visualization of management accounting information literature, introducing and using notions from psychology and cognitive studies. For instance, it introduces the stereotype threat and the Role Congruity theory to explain the rating behavior of female managers. A considerable implication of the study’s findings is that it might offer a relevant explanation for the fact that some companies have decided to move aside from the very common performance rating process to reinventing their performance evaluation systems.

This paper is structured as follows. Next, the Literature review section provides a broad overview of the literature that offers some valuable insights. These perspectives and other theories are used in the Hypotheses development section to formulate the main predictions of the study. Further on, the Experimental design and method section introduces the research method used and some other important aspects related to how the experiment was designed. After this, the Results section reports the findings that result as a consequence of the analyses conducted. Last but not least, the Discussion and conclusion section brings everything in one place and mainly discusses the implications for theory and practice while also providing suggestions for future research. The References and Appendices sections close the paper.

2 Literature review

The current section introduces the main concepts and variables that are investigated in this study. Moreover, it provides a broad overview of the literature that offers valuable insights for getting a more complete picture over the potential answers to the research question.

2.1 Subjective evaluation judgements

Management literature and practice place a high importance on measuring and evaluating performance as companies strive for better personnel motivation, a more goal-oriented behavior and better decision making. The emphasis is put on the performance measurement procedures’ effectiveness in terms of facilitating the process of alignment and communication of objectives, supporting monitoring and providing insights into deviations from goals (Drongelen and Fisscher 2003).

In most of the cases, a performance measurement system includes besides objective measures also subjective evaluations (Gibbs et al. 2004). The subjective evaluations are often present in the evaluation process as companies allow some discretion to their managers in assessing their subordinates’ performance (Maas and Verdoorn 2017).

Research states that discretion might lead to a more accurate and complete picture of employees’ performance by allowing managers to make use of information about efforts and actions that the objective measures do not capture (e.g. Bol and Smith 2011, Gibbs et al. 2004, Murphy and Cleveland 1995).

Further on, it is argued that firms can introduce discretion in the evaluation process in different ways (Bol 2008, Hoppe and Moers 2011, Van Rinsum and Verbeeten 2012). Regarding this, a frequently used procedure would be to let managers use their subjective judgement in determining an overall performance score, based on a report which contains the target and actual values related to a number of selected performance measures (Lipe and Salterio 2000, Ittner et al. 2003, Liedtka et al. 2008).

Research on performance appraisal has been giving overwhelming attention to performance ratings. In particular, human resource management and industrial-organizational psychology research consider performance evaluation and performance ratings as synonyms (Brutus 2010).

The fact that companies require managers to capture the subordinates’ evaluation in one key number while this is based on a formal report with a collection of figures might have both positive and negative consequences. This process can have a positive impact in the sense that it contributes to reducing the impression management and the evaluation ambiguity (Marginson 2006, Brutus 2010). Regarding the negative consequences, it can be argued that more complex and less tangible features of an employee’s performance are not observed in the process (Brutus 2010).

There is a constant need for companies to make sure that their procedures and information systems facilitate managers to make unbiased appraisals (Maas and Gonzales 2011). In addition, there is a moral responsibility for managers to provide accurate evaluations of their subordinates (Kerssens-van Drongelen and Fisscher 2003).

With all these, subjective evaluation might open the way towards intentional or unintentional biases (Harris 1994, Jawahar 2005, Prendergast and Topel 1993). Intentional biases in a subjective evaluation setting reflect the fact that managers might try to advance their own position or reward and promote their favourite subordinates (Prendergast and Topel 1993). On the other hand, with regards to unintentional biases, research suggests that raters have the tendency to inflate ratings (i.e. leniency bias) and to restrict the provided range of the ratings (i.e. centrality bias) (Jawahar and Williams 1997, Moers 2005, Murphy and Cleveland 1995).

Therefore, even though the literature generally perceives subjectivity in performance evaluation as a beneficial supplement to the objective measures, there is considerable evidence of biases (Chen 2014).

Further on, as the focus of this study is towards the unintentional biases, an elaboration on the main aspects to be considered will be provided.

2.1.1 Performance evaluation leniency and compression

The particular features to have a thorough look on are represented by the leniency and compression of evaluation judgements. In the performance evaluation process, leniency reflects how high or low the ratings are, whereas compression refers to the variance within the given scores (Maas and Verdoorn 2017).

Prior research showed that there is a tendency for managers to offer higher ratings in the situation of subjective performance evaluation. In addition, it is argued that subjective ratings usually exhibit less variance which means they are more compressed (Bol 2011, Chen 2014).

Leniency of performance ratings reflects the case of managers having the tendency to assess subordinates as above average (Moers 2005). In this way, employees might be offered ratings which are higher than is demonstrated by their performance (Chen 2014).

Further on, Bol et al. (2015) claims that compression of performance ratings occurs when managers have the tendency of increasing the ratings of the weaker performers above what they truly believe those subordinates deserve. Thus, it results a compressed performance evaluation distribution across performers. This situation clearly reflects how supervisors do not sufficiently make a difference between top and weak subordinates when appraising their performance (Ahn et al. 2010).

Within companies, subjective performance evaluations are performed by managers that have interests which might not be perfectly aligned with those of the organization’s owners (Prendergast 1999). Therefore, these managers might consider their own personal costs and benefits when conducting subjective evaluation decisions and in this way not necessarily rate subordinates’ performance towards strictly enhancing firm value (Prendergast and Topel 1993). While companies can influence managers’ rating behavior by making some specific choices regarding the design of the control system (Bol et al. 2015), in general managers have incentives to reduce the time and effort which is invested in the process of performance evaluation, avoid confrontations, limit criticism and prevent damaging their relations with the subordinates (Harris 1994).

In order to make unbiased subjective evaluation judgements, managers have to invest time and effort in order to gather information regarding the subordinates’ performance (Bol 2011). As there are opportunity costs linked to spending time and effort on performance evaluations, providing accurate ratings will not always be in managers’ personal interest. In this regard, managers will have the tendency to offer ratings which do not require a lot of extra work, are acceptable for the assessed subordinates and also easy to provide an explanation for to both subordinates and superiors. These stated criteria are often accomplished by high and not too dispersed ratings (Bol et al. 2016). Following the same line of reasoning, Bol (2011) concludes that the lack of complete information, taking into account the managers’ preference to reduce their time and efforts in the evaluation process, will lead to higher and more compressed scores. In this regard, compression increases the chance that managers’ estimation based on imperfect information is close to the real performance level while leniency facilitates avoiding any confrontations with unsatisfied subordinates. In addition, by

providing higher and less varied scores managers can limit the psychological burden that evaluating performance put on their shoulders (Villanova et al. 1993).

For an organization, the fact that managers offer more lenient and compressed ratings can be costly (Maas and Verdoorn 2017). Leniency is costly as it might lead to increased costs on employee compensation when, for instance ratings are linked to variable pay. In addition, it will put less pressure on low performers to improve their work as an increase in performance is not clearly linked to better ratings. Moreover, top performers will not feel incentivized to keep their efforts at a high level because they obtain a favorable score even if they have a lower level of performance (Bol et al. 2016). Compression is also costly for a company as there is not a clear identification of exceptionally talented and motivated employees. The lack of visible distinction between top performers and the other employees is detrimental for an organization in the sense that the best workers will seek for other job opportunities that better recognize their efforts while the firm might move towards mediocrity (Lazear 2000, Bol et al. 2016).

It has previously been stated that managers have a tendency to offer higher and more compressed scores when there is a lack of information about the subordinates. Maas and Verdoorn (2017) come to argue that even in the case in which managers are provided with relatively much information about subordinates in the form of performance reports, this will still require time and effort in order to identify, analyze and compare data. With regards to this, it is claimed that the performance report layout might have an influence on the leniency and compression of performance ratings. Therefore, the authors state that if the performance report is designed in such a way that it makes it easier for managers to observe and compare the different subordinates’ performance levels, the accuracy of subjective performance evaluations can be improved.

2.2 Performance report layout

The presentation of data in a visual form in order to facilitate understanding and exploration is defined in literature by the concept of visualization. In this regard, visualization reflects the utilization of visual cues such as charts, for instance to make the users’ information processing easier (Tang et al. 2014).

In order to get more accurate subjective performance evaluations, the performance report layout that is used shall be an effective one. According to Taher et al. (2015) effective visualizations have the purpose of improving the reasoning about data and help users to process information as this is transformed into visual structures that increase the individuals’ biological abilities to recognize patterns and trends.

The effects of various presentation formats have been studied by researchers from different disciplines (e.g. Blanco et al. 2010). Moreover, early research in this field primarily had a focus on the comparison between graphs and tables (Dull and Tegarden 1999, Kelton et al. 2010).

Existing literature claims that in order to ease managers’ tasks in the evaluation process by reducing their efforts and the time spent on it, performance reports shall comprise information items which are categorized in meaningful ways that reduce cognitive processing. In addition, it is argued that graphical formats shall be used to present information as it makes it easier to notice extreme values and compare values across various subordinates’ performance (Maas and Verdoorn 2017).

2.2.1 Information organization – balanced scorecard format

Research argues that the way information is organized influences the information processing and judgements. It has been suggested that information shall be grouped in meaningful ways to the decision maker. In accordance with the “dive and conquer” method, described by Shanteau (1988) which reflects the information organization into groups, the balanced scorecard format leads in itself to this mental approach (Lipe and Salterio 2002).

The Balanced Scorecard (BSC), a management and measurement tool developed in the early 1990s by Kaplan and Norton (1992) lists a diverse set of performance measures which are grouped in four categories represented by financial performance, customer relations, internal business processes and learning and growth activities. In each category it was encouraged the inclusion of 4-7 measures (Kaplan and Norton 1996). However, companies that adopt the balanced scorecard usually increase the number of performance measures they use (Lipe and Salterio 2002).

In prior judgement and decision making research was provided evidence of human information processing limitations and decision strategies. Cognitive psychology research suggests that people are, in general not able to process more than 7 to 9 items of information simultaneously. It is argued that humans have the ability of retaining and using only a small

number of items in working memory (Baddeley 1994). The four categories of the balanced scorecard suggest a way for managers to mentally organize the large number of performance measures that may help diminishing this cognitive difficulty (Lipe and Salterio 2002).

Literature argues that in order to improve their performance measurement system, many companies adopted the balanced scorecard concepts as these provided: focus on the desired improvements in performance, clarification and consensus (Kaplan and Norton 2007).

The findings of Lipe and Salterio (2002) suggest that the balanced scorecard has predictable and understandable effects on managerial performance evaluation judgements. The authors concluded that in the situation of balanced scorecard’s information organization, performance measures were perceived to be more logically structured and usefully categorized than in the situation of without a BSC format.

Recent literature shows that managers provide less lenient ratings if they are provided with reports in a balanced scorecard format. This comes to add credibility to the effectiveness of balanced scorecard as a contemporary performance measurement system as it is well adapted to the human information processing’s limitations (Salterio 2012, Maas and Verdoorn 2017). Therefore, taking into account all aspects previously mentioned, this study uses the balanced scorecard format as information organization of the performance report.

2.2.2 Presentation format - 2D and 3D bar charts

Prior research has emphasized the presentation format’s importance with regards to decision making (O’Donnell and David 2000).

Kelton et al. (2010) presents a framework which is rooted in Shaft and Vessey (2006)’s extended cognitive fit model and it suggests that the presentation format can have an effect on decision making. In this regard, the authors claim that the user’s ability to process the information and the decision processes used to reach a decision outcome are influenced by the manner in which information is presented.

Decision making research claims that the presentation format might have an influence on decision-making outcomes such as accuracy and confidence. The task characteristics, such as task difficulty and the presentation format such as numeric tables, bar charts etc. can impact an individual’s certainty to act on a judgement or decision. It is argued that a difficult task tends to lower people’s confidence. With all these, when information is presented in a graphic format, such as bar charts and not in a numeric table, users will be more confident (Tang et

al. 2014). In addition, an earlier research found that evaluators provided lower scores (i.e. more accurate) for the assessed employees when the information was presented in graphs. In this regard as they had to deal with much variance, it was easier for evaluators to recognize the patterns from the time series data and perform the performance appraisal when they had access to a graph (Johnston and Shields 1983).

According to Kelton et al. (2010), graphs represent spatial problem representations as they focus on and offer visualizations of the data relationships among the data and help the user viewing it as an integrated unit. It is argued that graphs’ efficiency and effectiveness is determined by the suitable construction of them.

In organizational decision making, compared to the more traditional tabular representations, the graphical representations proved to be a superior reporting technique (Jarvenpaa and Dickson 1988). When it comes to presenting data, there are multiple approaches that can be used considering the vast possibilities of chart types (Schwabish 2014). As early research came to argue, when displaying the data it is highly important to choose the correct chart type and let go of the temptation of creating something “pretty”, but totally illegible (Coles 1997). One of the most common visualization types is represented by bar charts. Research conducted on how different bar chart designs might impact the viewers’ accuracy of completing perceptual tasks (i.e. estimation of the relative heights in various bar chart designs, for instance) highlighted the fact that small design changes might significantly affect the way bar charts are perceived. In addition, it has been argued that there is still a need for future investigation in order to have a comprehensive understanding on the matter (Talbot et al. 2014).

When it comes to performance reports, there is a large variety of graphs management accountants can choose from. In addition, future research avenues suggest the need of studies to compare different types of charts (Maas and Verdoorn 2017). Thus, as literature emphasizes bar charts as a basic and widely used graphic representation (Zhou and Tan 2000) and there is limited research regarding multiple dimensions graphs (Kelton et al. 2010) this study is considering 2D (i.e. two-dimensional) and 3D (i.e. three-dimensional) bar charts as presentation format for performance reports.

In this study, it is important to mention that the concepts of two-dimensional and three-dimensional bar charts are in line with the literature (Coles 1997, Schwabish 2014). In this regard, it is stated that the creation of a chart involves some data and a graph based on that

data. Therefore, the concept of graph describes the medium for presenting the data while the concept of chart indicates the type of representation (i.e. bar charts, line charts etc.). With all these, in most studies the concepts of graphs and charts are interchangeable.

2D bar charts are characterized as the simplest graphic presentations, being the most in line with the call for efficient charts (Tractinsky and Meyer 1999). These types of charts show values in bars across categories and one of the axes reflects those compared categories while the other axis shows the numerical values of the data (Coles 1997).

3D bar charts, in comparison with 2D bar charts make use of the Z axis which adds the appearance of perspective or depth. The title is justified as these charts use all three dimensions leading to a stylistically and technically more complex charts (Tractinsky and Meyer 1999).

According to Carswell et al. (1991) the use of the “3D look” became widely spread as the graphics software started to be more and more available. The authors encouraged giving more attention on understanding the effect of adding such a decorative depth on charts’ comprehension.

When it comes to the impact of performance reports having information presented in 3D bar charts on leniency and compression of managers’ subjective evaluations, there is no research that I am aware of. However, there are a series of studies that had a look on 2D and 3D bar charts’ impact on comprehension and efficient use under different tasks.

According to Maas and Verdoorn (2017) managers have the tendency to provide more lenient and compressed scores when there are costly cognitive efforts in the evaluation process. In this regard, it is essential and interesting to see which type of bar chart (i.e. 2D versus 3D bar chart) facilitates an easier extraction and “digestion” of information and a better analysis of the performance levels.

Early research suggested that decision accuracy gets better when using multidimensional graphs. It was argued that there is a need for further studies with regards to three-dimensional graphs in order to determine the circumstances in which these formats are suitable (Dull and Tegarden 1999).

The findings of Siegrist (1996) reflect that when it comes to 3D bar charts, they usually take more time to evaluate, however the accuracy of those evaluation being the same as for 2D bar charts. In the study the participants had the task of comparing the bars from the chart and estimating the relative size of the second or third bar in comparison with the first bar. The

author concluded that the 2D bar charts were not superior to the 3D bar charts. A similar study conducted by Hughes (2001) showed that there is an advantage in accuracy for 2D bar charts comparing to 3D bar charts.

According to Tractinsky and Meyer (1999), the use of perspective in the case of 3D bar charts might lower the accuracy when it comes to reading and analyzing graphs. The authors conducted a study in which subjects had to take different roles and based on the provided information that was reflected in 2D, 3D bar charts and tables to perform some decision-making. The research’s findings show that there is a significant effect of presentation format on decision-making. Moreover, 2D bar charts lead to better decision making, being at the same time the most preferred presentation format in comparison with tables and 3D bar charts.

It seems as the three-dimensional graphs might have a more professional look, however the use of this particular 3D “style” tends to have a negative influence on the accuracy of the reader’s perception with regards to the presented information. Research suggests that 3D graphs are difficult to read comparing to 2D graphs mainly due to the 3D depth cues which might lead to severe errors of interpretation (Russell and Bielewicz 2005). In addition, Carswell et al. (1991) found that 2D charts performed better than 3D charts when subjects had the task of making relative magnitude estimations. It is argued that 3D designs have an impact on performance and they can also influence the attitudes formed by participants toward the information which is presented in the graphs.

Further on, a recent online study supports the use of 2D charts to make data easier to interpret as 2D charts outperformed 3D charts across all measures (i.e. speed, efficiency, ratings and accuracy). The participants were provided with various types of 2D and 3D charts and had to answer certain questions regarding the information presented in the graphs. The purpose of the research was to determine the performance and reading perception of a certain chart type. With regards to 2D and 3D bar charts, the findings showed that people were faster with 2D bar charts while the level of accuracy was the same for both 2D and 3D bar charts. Overall, 2D charts were perceived as easy, appealing, confident, effective, easy to learn and professional (Tullis and Case 2017).

Schwabish (2014) argues that effective visualizations reflect the data which tell the story while reducing clutter in order to keep the emphasis on the important points and integrating the text with the graphs in order to efficiently transfer information.

It is therefore, highly relevant for this study to keep in mind literature’s suggestions related to effective visualizations and moreover, the knowledge gathered from the previously described studies with regards to the various perceptual aspects involved in 2D and 3D bar charts.

In the subjective performance evaluation context, as it has already been argued, the presentation format might have an impact on the leniency and compression of ratings related to how much cognitive effort 2D or 3D bar charts imply. However, in order to have a complete understanding on the matter, it is relevant to have a closer look at the gender’s influence in the subjective performance evaluation process as research argues that when it comes to rating, women rate differently than men (Furnham and Stringfield 2001).

2.3 Female managers versus male managers – psychological and cognitive traits

Prior research reflects that gender, indeed plays a role in the performance evaluation process and it is relevant to have some insights on the issue as this might also help explaining the biased ratings (Murphy and Cleveland 1995). In addition, early literature from various areas of academic research indicates the existence of both psychological and cognitive differences between females and males (Chung and Monroe 1998).

According to Gabdreeva and Khalfieva (2014), psychological traits reflect characteristics such as confidence, autonomy, organizational skills and level of emotional intelligence. It is argued that the traits attributed to females or males compose gender stereotypes. Thus, the discussion on psychological traits will be closely connected to the influence of gender stereotypes.

Cognitive traits involve thinking, reasoning and describe cognitive processes such as information processing, visual processing, perception, attention and memory (Chung and Monroe 1998, Halpern and Tan 2001, Li 2014).

Research claims that the professional activity of managers is characterized by cognitive complexity, the need of adaptation to problematic situations and decision-making under uncertainty and risk (Gabdreeva and Khalfieva 2014).

Therefore, psychological and cognitive traits will be analyzed as follows to get a more comprehensive view on gender’s characteristics and in this way, a better understanding on how gender can have an impact in the context of this study.

Psychological traits

In the article of Maas and Verdoorn (2017) the evaluator’s gender was included as a covariate in the study and results have shown that female participants offered higher and more compressed ratings than their male counterparts.

According to the Social Role Theory and empirical evidence to support it, the differences that exist when it comes to women and men’ aptitude and personality traits usually express the traditional gender roles that are present in society. Following this line of reasoning, it is argued that men are mostly guided by agentic goals (i.e. controlling tendencies) and are characterized by assertiveness, self-efficacy and mastery. On the other hand, women are influenced by communal concerns which refer to harmonious relationships and interpersonal affiliation. In this regard, the women gender role emphasizes sensitivity to others’ concerns (Putrevu 2001). Extant literature claims that even though male managers might not suffer from not following their gender role, female managers might be more inclined to negative evaluations when their leadership is not in line with their role expectations (Luthar 1996).

Further on, more recent studies (e.g. Moczydlowska and Szydlo 2017) come to enforce the aspects mentioned by early literature with regards to gender stereotypes and features of females and males’ management styles. It is claimed that based on psychological research gender stereotypes are quite similar all over the world. In this regard, men are described by dominance, self-confidence, independence with a focus on tasks and actions while women are characterized by gentleness with a focus on emotions and social relations. When it comes to management styles, male managers have the tendency to put pressure and handle resistance in a forcible way, while female managers have the tendency to use negotiations, consultations and persuasion regarding the relations with their subordinates. It is argued that female and male management styles differ due to the fact that the ways they think and process data are dissimilar. In this regard, male managers have a preference for a more authoritative style, while female managers make use of their emotional intelligence more often and therefore, they use their empathy and intuition when taking decisions (Moczydlowska and Szydlo 2017).

Mandell and Pherwani (2003) found that female managers had a higher score than the male managers when it came to measuring their level of emotional intelligence. The authors claim that emotional intelligence is a concept which defines not only the ability of self-monitoring

but also using the information about the others and taking into account their feelings and emotions.

Cognitive traits

According to performance appraisal research, performance ratings are perceived as a complex information-processing task, while the evaluator has the role of rationale information processor. In addition, models regarding the appraisal process acknowledge the impact and importance of non-cognitive factors as well. Research argues that in subjective appraisals, it is necessary to take into account besides cognitive determinants also behavioral, affective and social factors (Lefkowitz 2000).

Brutus (2010) claims that in performance evaluations, there are three types of processes which describe the assessor: cognitive processes, motivational processes and normative processes. The normative influences refer to the feedback environment which describes the performance evaluation process. Further on, when people tap into their memory and try to integrate information in order to assess someone, there are two separate cognitive processes that come into play according to social cognition research. These are represented by: online processing and memory-based processing. The first process reflects acquiring performance information and integrating it in order to form an overall impression of the assessed person. On the other hand, memory-based processing occurs when raters have to draw on past episodes about the ratees to provide their evaluation.

Regarding motivational processes, it is argued that evaluators provide inaccurate ratings, not because they are incapable of accuracy but due to their unwillingness to do that (Murphy and Cleveland 1995). The evaluators might avoid in this way to deal with uncomfortable situations such as criticism, ill-feelings and other social costs related to evaluatees’ reactions (Smith et al. 2000). The motivational processes might be very well connected with what has previously been presented in the section of psychological traits that describe female and male managers.

Prior literature extended its investigation to many areas of human behavior including the human information processing. It is argued that when it comes to information processing, education, psychology and marketing research shows evidence of gender differences. In this regard, men are considered to be selective information processors and they have a tendency to focus on single cues. On the other hand, women are considered to be comprehensive processors and they have a tendency to process all or most of the information that is

available. Thus, women are more carefully paying attention to details comparing to men and they manage to store more information in their long-term memory (Chung and Monroe 1998).

Halpern and Tan (2001) argue that the gender patterns of cognitive abilities have important implications for varied backgrounds and theoretical orientations. It is argued that men are particularly better at visual-spatial tasks that necessitate using the visual-spatial working memory. In this regard, males tend to have superior performance to mathematical tasks when these are more visuospatial (i.e. geometry, trigonometry etc.).

A study conducted by Lowrie and Diezmann (2011) had a look on the capacity of solving tasks that contained high concentrations of visual-spatial information including bar charts. The research considered the performance of students in the context of solving graphics tasks. These tasks referred to the use of particular encoding tools for understanding the representation of mathematical relationships among perceptual elements. In the context of the apposed-position language (i.e. encoding information displayed in bar charts) gender differences were revealed by an item reflecting a two-dimensional horizontal bar chart. The results showed that males outperformed females with regards to accurately processing the information. The authors suggested that the performance differences across gender might be caused by the tasks’ level of complexity as they found that males were better than females at more difficult tasks that required decoding the representational forms and forming an opinion by going beyond the graphic.

Even though the study of Lowrie and Diezmann (2011) examined the cognitive abilities of males and females within the educational context, Halpern and Tan (2001) argue that the males’ superior performance with regards to visuospatial tasks is maintained into old age as well.

Miller and Halpern (2014) had also a look at the cognitive sex differences and came to argue that a large gender difference relates to spatial tasks. In this regard, the authors claim that men are better at mental rotation (i.e. mentally visualizing figures in various positions) when it comes to 3D figures in comparison with 2D figures. These cognitive gender differences regarding mental imagery skills are confirmed by studies of brain imaging providing evidence of different cognitive strategies for females and males (Li 2014).

More comprehensive studies into gender cognitive functions (e.g. Zaidi 2010, Upadhayay and Guragain 2014) state that cognitive abilities reflect attributes such as visual and spatial

processing, working and long-term memory, perception, attention, motor, language and executive functions. It is argued that women perform better in perceptual speed, accuracy, verbal fluency and fine motor skills (e.g. handwriting) while men show advantages in spatial, mathematical abilities and working memory (i.e. short-term memory).

It is important to mention that research highlights the unconscious impact of gender stereotypes on performance, altering in this way a great set of biological systems (Lowrie and Diezmann 2011). It seems that gender stereotypes as they were described earlier on might influence the gender cognitive abilities, this phenomenon being known as the stereotype threat. Literature on the matter shows how gender stereotypes can have immediate situational impact on cognitive performance (Miller and Halpern 2014).

All things considered, in this section, the discussion reflecting the context of subjective performance evaluation has touched upon performance ratings characterized by leniency and compression, performance report layout and managers’ psychological and cognitive traits. Starting from the knowledge previously described, the following section focuses on the hypotheses development. In this regard, insights from the literature review section and theoretical perspectives are used to make predictions about the impact of performance reports having information presented in 2D or 3D bar charts on the leniency and compression of performance ratings. In addition, it is predicted the gender’s influence on the relation between the variables of interest.

3 Hypotheses development

3.1 The impact of 2D and 3D bar charts on leniency and compression

As it has already been stated in the Literature review section, the more cognitive effort managers have to put in understanding the information they are provided, the more they are inclined to give higher and more compressed ratings for their subordinates.

Further on, an important aspect to enforce is that leniency and compression of performance ratings are much more present in the case of subjective performance evaluation. This might also be the case as the supervisors can have the tendency to exploit the power which is given to them by acting in a way that is in their own interest (Giebe and Gürtler 2012). The situation is detrimental to a company as leniency and compression’s persistence might hurt the employee performance and also leads to a lower firm productivity (Golman and Bhatia 2012).

Therefore, the performance report layout shall be created in such a way that it is easy for managers to “digest” the information.

Cognitive fit theory highlights the influence of information presentation on judgement and decision making. The figure presented below, displayed by Kelton et al. (2010) and rooted in Shaft and Vessey (2006) reflects the information presentation’ s effects and role in the decision-making context. It is argued that information presentation has an impact on the mental representation of the issue. In this regard, the layout affects the information processing in working memory and the decision processes that are used to reach a decision outcome. Further on, the format can also influence the decision maker characteristics, for instance the user’s involvement. It is important to mention that in the presented figure, the arrow that links decision maker characteristics and information presentation reflects interactive formats in which the user has the possibility to manipulate these formats. Also, the arrow that links decision maker characteristics and task characteristics shows that during decision making the user has the possibility to choose the information cues to use. With regards to the previously mentioned aspects, this study focuses on a static presentation format describing the situation in which the user does not influence the nature of the presentation.

Figure 1: Decision-Making Framework for Information Presentation Effects According to Kelton et al. (2010)

The cognitive fit theory suggests that cognitive fit occurs in the situation of a match between the decision processes that are required by the task and those supported by the problem representation. In this regard, the cognitive fit leads to an accurate mental representation and to performing the task in a more effective and efficient way. The situation in which cognitive fit does not occur reflects the fact that the decision maker has to handle additional cognitive efforts which will lead to an increase in the task time and/or a decrease in accuracy (Kelton et al. 2010).

In the evaluation process, managers will have the tendency to inflate and compress performance ratings in case of high cognitive efforts to extract the relevant information from the performance report. In this way, clearly presented information is linked with less leniency and compression (Maas and Verdoorn 2017).

Literature provides some clues with regards to 2D and 3D bar charts’ impact in decision-making processes, even though research has not touched upon the effects of performance reports having information presented in 3D bar charts on the leniency and compression of evaluation judgements.

As it was also mentioned in the literature review section, Carwell et al. (1991) found that 3D bar charts were used more quickly than 2D bar charts. However, at the expense of speed, a decrease in accuracy has been noticed. On the other hand, a similar study conducted by Siegrist (1996) concluded that 3D bar charts were not inferior to 2D bar charts with regards to

the accuracy of graph evaluation and participants to the experiment needed more time to evaluate 3D bar charts.

Further on, in a study conducted by Tractinsky and Meyer (1999), participants strongly preferred 2D bar charts for decision making and it was argued that the use of perspective in the 3D bar charts ‘case might lower the accuracy of reading and analyzing these particular charts.

In favor of 2D bar charts is also the article of Fischer (2000). The study investigated the influence of depth cues on the bar charts’ comprehension. The research results show that the 3D bar charts are hard to understand and take more time to process. In addition, it is argued that the use of depth cues leads to a memory load, processing costs and a slower encoding of the chart into working memory.

In general, the three-dimensional bar charts are considered to have a more professional look, but research (e.g. Hughes 2001, Robbins 2005, Russell and Bielewicz 2005, Skau et al. 2015, Tullis and Case 2017) argues that this particular style might have a negative impact on how accurate the user’s perception of the presented information is. Therefore, it is claimed in unison that 3D bar charts might be attractive but they seem more difficult to read and comprehend. Moreover, empirical evidence shows strong performance differences in favor of 2D bar charts and a high preference for this particular type.

Schwabish (2014) talks about effective graphs and claims that a chart shall tap into the pre-attentive processing part of the brain to be a successful one. This mentioned type of processing is described as allowing the user to simultaneously perceive multiple basic visual elements. The author suggests the use of certain pre-attentive attributes (i.e. color, size, orientation, shape) and the removal of unnecessary clutter which is referred to as the 3D effect. In this way, charts are more clear, compelling and able to effectively communicate the information to the user.

Overall, taking into account all aspects previously mentioned in the literature review and the current section, 2D bar charts seem to provide an advantage in accuracy, lead to better decision making, make data easier to interpret and are perceived as effective charts. Therefore, as 2D bar charts offer visual clarity and understanding, it is expected that performance reports that have information presented in these types of charts will necessitate less cognitive efforts. In this way, managers will find it easier to extract the relevant

information from the reports and as a consequence, performance ratings are expected to be less lenient and compressed in the case of 2D bar charts comparing to 3D bar charts.

The stated predictions are summarized by the two hypotheses below:

H1a: Overall performance ratings are less lenient when performance reports contain 2D bar charts than when performance reports contain 3D bar charts.

H1b: Overall performance ratings are less compressed when performance reports contain 2D bar charts than when performance reports contain 3D bar charts.

3.2 The influence of gender on the relation between presentation format and evaluation judgements

It has already been stated that research points towards a different rating behavior for female managers compared to male managers (Furnham and Stringfield 2001, Maas and Verdoorn 2017). In this regard, it is expected that gender has an influence on the relation between the presentation format of the performance report and the leniency and compression of overall performance judgements.

In the literature review section, gender’s psychological and cognitive traits were presented in order to have a better understanding of how female and male managers might perform their evaluation task.

Following the predictions of the first set of hypotheses, it is expected that performance reports that contain information presented in 2D bar charts will lead to less lenient and compressed performance ratings than in the case of 3D bar charts. However, the relation between the presentation format and the subjective evaluation judgements is predicted to be conditional on the manager’s gender as it will be further elaborated.

Literature (Mandell and Pherwani 2003, Moczydlowska and Szydlo 2017) suggests that the management style of women is characterized by gentleness, a high use of empathy and intuition while the management style of men is more authoritative and characterized by dominance, self-confidence, a tendency to put pressure and handle resistance in a forcible way. In addition, the Social Role theory argues that, in general women tend to be sensitive to others’ concerns while men tend to be highly assertive (Putrevu 2001).

Byron (2007) claims that gender stereotypes prescribe emotional sensitivity and supportiveness to female managers. The Role Congruity theory states that women do not only have the tendency to be caring and interpersonally sensitive but they are also highly expected to act in this way. It is argued that several studies found that female managers, who behaved stereotypically feminine, had a higher level of likeability and were perceived as more influential. Thus, expectations are supported by reality. When it comes to male managers, the stereotyped attributes refer to being logical, analytical and good at reasoning. Moreover, being persuasive is more valued in male managers while providing comfort more valued in female managers.

In the study of Brutus (2010) it is argued that performance appraisal might be characterized as being a highly emotionally charged activity in the business life. In this regard, the evaluation tasks that managers have to perform might lead to a certain level of discomfort and reluctance.

Following the aspects mentioned so far, it can be argued that female managers as being more sensitive to others’ concerns, understanding, kind, supportive and more focused on their subordinates’ emotions, they will try as much as possible to keep everyone satisfied with the provided ratings and avoid any conflicts. On the other hand, male managers might not engage in the same behavior as they are, generally more authoritative and do not fear expressing their own views and managing the potential resistance from their subordinates.

Therefore, taking into account the more empathetic nature of women and the more assertive nature of men, it is expected that in the performance evaluation process, female managers will have the tendency to provide more lenient and compressed ratings than male managers.

In order to have a complete picture it is essential to take into account also the gender’s cognitive traits. According to Conati et al. (2014), user characteristics such as cognitive abilities can have a noticeable impact on visualization.

The literature review pointed towards some gender differences when it came to the cognitive aspects. In this regard, it has been argued that in general, men are at an advantage in terms of visual-spatial tasks (Zaidi 2010, Lowrie and Diezmann 2011, Upadhayay and Guragain 2014). In addition, research provides evidence of differences with regards to information processing which is essential to understand considering the fact that performance rating is perceived as a complex information-processing task (Lefkowitz 2000).

Having in mind all these, the visuospatial advantage of men is mostly referred to their superior capability of mentally rotating 3D figures which is relevant as it shows a good visual perception (Xue et al. 2017). However, it might not be of such high use in the context of this study as for the rating task, managers do not necessarily have to deal with a figure’s rotation, but more with an ability to accurately observe and compare the subordinates’ performance levels. The task requires an accurate processing of the information from the bar charts and moreover, high attention to details. With regards to the first aspect, literature claims a superior performance for men when dealing with accurately decoding two-dimensional bar charts. Related to the later aspect, it is argued that men are not as carefully paying attention to details as women are.

All things considered, in the case of male managers having the task to evaluate subordinates, it can be reasoned that the performance ratings will highly depend on the cognitive efforts that the presentation format implies. This is also to be claimed as it has previously been stated that male managers have a tendency to be assertive and not fear conflicts. Thus, the layout is expected to have a higher influence on them comparing to women who are expected to have a certain management style regardless of any other aspects. Therefore, as 2D bar charts are expected to exert less cognitive efforts compared to 3D bar charts, male managers are predicted to offer less lenient and compressed ratings when using 2D bar charts than 3D bar charts.

Further on, in the case of female managers having the task to evaluate subordinates, following the stereotype threat and the Role Congruity theory that were previously elaborated upon, they are expected to provide more lenient and compressed ratings irrespective of the performance report’s presentation format (2D bar charts or 3D bar charts).

The below hypotheses summarize the stated predictions:

H2a: Overall performance ratings are more lenient when performance reports are evaluated by female managers than male managers.

H2b: Overall performance ratings are more compressed when performance reports are evaluated by female managers than male managers.

H3a: The effect of 2D bar charts as compared to 3D bar charts on the leniency of performance ratings is stronger when the evaluation is performed by male managers.

H3b: The effect of 2D bar charts as compared to 3D bar charts on the compression of performance ratings is stronger when the evaluation is performed by male managers.

3.3 Overview of the Hypotheses

Here is presented an overview of the previously stated predictions:

H1a: Overall performance ratings are less lenient when performance reports contain 2D bar charts than when performance reports contain 3D bar charts.

H1b: Overall performance ratings are less compressed when performance reports contain 2D bar charts than when performance reports contain 3D bar charts.

H2a: Overall performance ratings are more lenient when performance reports are evaluated by female managers than male managers.

H2b: Overall performance ratings are more compressed when performance reports are evaluated by female managers than male managers.

H3a: The effect of 2D bar charts as compared to 3D bar charts on the leniency of performance ratings is stronger when the evaluation is performed by male managers.

H3b: The effect of 2D bar charts as compared to 3D bar charts on the compression of performance ratings is stronger when the evaluation is performed by male managers.

4 Experimental design and method

This section elaborates upon the experimental design, provides information about the sample, the task and manipulations and last but not least touches upon the experimental procedures.

4.1 Experimental design

In order to test the hypotheses mentioned in the previous section, an experiment is conducted. According to Bryant et al. (2004), an experiment is defined by the systematic manipulation of stimulus variables (i.e. independent variables), the empirical observation of response variables (i.e. dependent variables) and the control of one or more unrelated variables.

In this particular study, the independent variable is reflected by the presentation format of the performance report and has two levels: 2D bar charts and respectively, 3D bar charts. In the experiment, the chart type is manipulated. The bar charts contain the targets and the actual results for the performance measures used in the evaluation process. The type of experiment is the scenario-based experiment and the participants were randomly assigned to either performance reports in the form of two-dimensional bar charts or three-dimensional bar charts.

Further on, the dependent variables are represented by the leniency and compression of managers’ subjective evaluation judgements. The two concepts have been defined in the Literature section and their measurement will be discussed later on.

In addition, the relation between independent and dependent variables is expected to be influenced by gender. This is the moderating variable in the research model and is measured.

In the model, there were considered some control variables as alternative factors might have an impact on the variables of interest. In this regard, the control variables are represented by age, nationality and experience in evaluating subordinates. The variable age indicates the potential differences with regards to evaluations between the younger and older evaluators. Further on, as the pool of participants included various countries and research has emphasized the potential effect of culture on judgement and decision-making (e.g. Nutt 1988, Hofstede et al. 2010), nationality variable is considered. Lastly, a study conducted by Hirsch & Bassler (2010) argues that experience does not have a significant influence on visualization success, however it might still be claimed that a higher level of experience in evaluating subordinates might lead to different outcomes.

All these being said, a summary of the aspects presented above is displayed in the following conceptual model:

`

Figure 2: Conceptual model

In order to test the hypotheses, the following four conditions were necessary:

Female managers – 2D bar charts Female managers – 3D bar charts

Male managers – 2D bar charts Male managers – 3D bar charts

The experiment was created with Qualtrics, a software which facilitated the assignment of participants to one of the four conditions mentioned before. Moreover, Qualtrics is an efficient and effective way to design and administer the experimental instruments as also research studies (e.g. Brandon et al. 2014) came to argue.

Further on, as the aim of this study is to see the impact of performance reports’ presentation format on managers’ subjective evaluation judgements, getting access to a category of fairly hard to reach participants was definitely a challenge to rise to. In this regard, the option of

Independent variables Presentation format:  2D bar charts  3D bar charts Dependent variables  Leniency of performance ratings  Compression of performance ratings Moderating variable Gender  Female managers  Male managers Control variables  Age  Nationality  Experience H1a,H1b H2a, H2b H3a, H3b

conducting an online experiment and recruiting subjects via an online platform has proved to be the best fit for the situation. According to a study conducted by Palan and Schitter (2018), within the wide pool of platforms, Prolific, a graduate startup of Oxford University Innovation incubator programme, stands out as it provides a subject pool for research, in particular and it is highly suitable for conducting social and economic science experiments. Furthermore, Prolific offers the researchers the possibility of pre-screening participants that are eligible for the study and it facilitates sound scientific research as the subjects are well aware of being recruited for research and in this regard of expected payments for their involvement but also of their obligations in such a context.

4.2 Participants

The platform gave the possibility of pre-screening the participants that were going to participate in the experiment. In this regard, as the purpose of the study was to see the effects of the performance report layout on managers’ subjective evaluation judgements, a pre-requisite for participating in the experiment was the management position. Thus, the criteria used for pre-screening were a full-time employment status and leadership, position of power, supervisory duties.

A more thorough elaboration on the demographics of the sample is provided in the table below.

Table 1. Demographics of the participants

N Min Max Mean

Std. Deviation Age 220 18 66 35.42 9.507 Nationality 220 1 26 4.55 5.714 Industry 220 1 38 19.5 10.966 Gender 220 Females 110 Males 110 Experience 220 1 3 2.10 .904 0 - 2 years 80 2 - 4 years 39

more than 4 years 101

The age of the participants varied between 18 and 66 years old with a mean of 35.42 and standard deviation of 9.507. The sample included 26 nationalities, representatives of

Organization for Economic Cooperation and Development (OECD) Member countries. Further on, the sample consisted of mangers working in a great variety of industries.

The majority of participants (i.e. 126 managers) were British. Furthermore, the platform used for recruiting the subjects allowed for an equal split between females and males. Thus, in the experiment there were 110 female managers and 110 male managers who participated. With regards to the experience in evaluating subordinates, there were 80 managers with less than 2 years of experience, 39 managers with 2 to 4 years of experience and 101 managers with more than 4 years of experience.

4.3 Task and manipulations

Participants to the experiment assumed the role of a regional manager and their task was to evaluate the performance of eight store managers. In this regard they had to provide an overall performance score based on eight different performance measures that were organized in a balanced scorecard format. Furthermore, the performance measures used were the same for all store managers. The decision to use eight performance measures and eight store managers to be rated was not at random. This particular number was chosen in order to make sure that the evaluation process is cognitively difficult. To enforce this, research suggests that when individuals have to deal with seven or more information cues in a judgment task, cognitive limitations manifest themselves (e.g. Baddeley 1994). Therefore, the number of eight is suitable with regards to the cognitive effort implied and the timeframe to complete the task.

The full version of the experiment can be found in the Appendix section. A more comprehensive elaboration on the experiment will be provided in the following subsections.

Scenario

After being welcomed to the experiment, the participants were introduced to the context in which they had to perform their task.

The subjects to the experiment had to take the role of a regional manager for DFO Inc. The organization was described as a global retail company, specialized in clothing and fashion accessories. The scenario is based on previous studies that touch upon the same research area and it ensures that a realistic context is provided for framing the subjects in their role as