Faculty of Economics and Business
Bachelor Thesis | Business Studies Adrien Gassama | BSc Bèta-Gamma | 6121225
dr Maarten de Haas | Supervisor
The Spillover Effect
of a bias in Performance Measures:
A case study of the Self-Assessment of Employees in a salesforce
with an Internal Benchmark System
Abstract
Performance appraisal is a significant part of a performance management system and has the
potential to motivate employees and improve performance. Research has yet to determine the ideal
balance between objective (e.g., total sales) and subjective performance measures (e.g., customer satisfaction). Internal benchmarking systems can rank staff according to performance and
efficiency in order to narrow performance gaps. But these systems tend to focus on objective
measures, potentially biasing both management and staff to undervalue subjective performance. I
investigate the effect of internal benchmarking of total sales in a recruitment salesforce in
Amsterdam. Expected is a correlation between performance ranking and an employee’s general
self-evaluation (measured in Core Self Evaluations; Judge, 1997) and a correlation between ranking
and a bias towards objective performance measures. The appraisal by team managers is expected to
moderate these relationships. Research is a mixed design of surveys used to analyze correlation
and moderation effects, with additional interviews to obtain further explanations for the results.
Results suggest manager emphasis on financial performance might moderate objective
performance measures bias in employees with lower total sales rank. Implications of these results
are discussed and suggestions for further research are made.
Keywords: performance management, performance appraisal, internal benchmarking, core self evaluations, salesforce, human resources, rater bias
Statement of Originality
This document is written by Student Adrien Gassama who declares to take full responsibility for the contents of this document.
I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.
The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.
“The best form of motivation is progress. If you see the work you are doing progresses you, then you’re motivated to keep doing it. Every athlete has a point at which they stop. They stop pushing. Could be a wet and rainy day, could mean getting out of bed early in the morning, could be the pressure. It could be tiredness, could be fatigue. Your role is to help that athlete delay his stopping point, until he’s rowed the last stroke of that Olympic final.”
Mike Spracklen Head-coach Canadian Olympic rowing crew
Foreword
My experiences as a varsity rower have shaped my view of what motivation, performance and progress mean to me. And particularly this quote by Mike Spracklen, one of the most renowned coaches in the world of rowing, reflects what that is. Long winter trainings prepare us for a season of only a few races that are just a couple of minutes long. During these trainings, progress is what motivated us. Sometimes progress as small as being able to row a distance one second faster than before. But it was still progress.
It made me wonder how my colleagues in Amsterdam, in the office where I worked part-time alongside my studies, felt about their progress that was projected on the total sales ranking list on the wall. At the rowing club our rowing scores were not listed publicly. But in the office they were.
It was part of their performance management system, with the ultimate goal of motivating everyone to reach the top, and to improve everyone’s performance. With this research paper I want to investigate whether or not this system had that desired effect.
I’m grateful for the help of my very dependable and helpful supervisor, dr Maarten de Haas, and everyone else who helped me through the mysteries of SPSS. Hopefully you, the reader, will find this thesis as insightful as I intended it to be!
Adrien Gassama
Table of contents
Introduction
PurposeTheoretical Framework
Performance appraisal & core Self Evaluations Internal Benchmarking Performance appraisal: objective (financial) vs subjective Rater bias in performance appraisal DesignMethod
Design Measurements Analyses Research ProposalsResults
Participants Correlations Regressions InterviewsDiscussion
Summary Unpredicted results and Alternative Explanations Limitations Practical implications and Future research Conclusion Literature Appendix ……… 5 ……… 7 ……… 8 ……… 8 ……… 9 ……… 9 ……… 11 ……… 13 ……… 14 ……… 14 ……… 14 ……… 15 ……… 15 ……… 16 ……… 16 ……… 16 ……… 17 ……… 19 ……… 20 ……… 20 ……… 21 ……… 22 ……… 23 ……… 26 ……… 27 ……… 29INTRODUCTION
For every salesforce, there are desired results and realized results. As the maximum potential of the salesforce is yet to be reached, management is challenged with narrowing the performance gap through checks and balances, measures, coaching and feedback.
Performance management refers to a range of activities applied by organisations to improve the performance of individuals and teams, with the ultimate purpose of improving organisational effectiveness and efficiency (DeNisi 2000). Activities that fit with this definition are Performance Planning; the setting of corporate, departmental, team, and individual objectives; Performance Coaching, where performance is adjusted according to intervention, coaching and feedback from managers and Performance Appraisal ; the use of appraisal systems, reward strategies, training schemes and individual career plans (Roberts 2001). The overall goal of performance management is, therefore, to ensure that the subsystems of the organization (processes, units and employees) are working together in an optimal fashion to achieve the results desired by the organisation (Castka, Bamber & Sharp, 2003). According to surveys among 278 multinational organizations in 15 different countries 90% of the companies indicated using a formal performance management system (Cascio, 2006).
Notwithstanding of its apparent popularity among corporations, the degree to which these elements should be applied is still a topic of much debate and research. Outside of research literature performance management is not thoroughly understood or appreciated, particularly not by employees (Aguinis, Joo, Gottfredson, 2011).
On the individual level the relevant part of performance management is the appraisal , where direct measurement of effectiveness and efficiency is performed, a performance gap is identified, rewards are given for outstanding performance or improvements, and further guidance and career planning is considered. Developing appropriate appraisal systems of individuals is crucial, as the performance of the individual employee is essential and a formal management system is seen as one of the top most important human resource management instruments in an organization (Liu, Combs, Ketchen, & Ireland, 2007; Platts & Sobo ́tka, 2010).
Aguinis, Joo & Gottfredson (2011; 2012) suggest “(in) a well designed and executed performance management system employees experience increased self-esteem, better understand the behaviors and results required of their positions and better identify ways to maximize their strengths and minimize
However, 30% of employees in a survey by Holland (2006) indicated they doubted performance management interventions indeed improved their performance. Often “performance management” is reduced to performance appraisal , and is over-generalized as simple performance reviews with little to no useful contribution to their motivation or personal development, and a waste of resources (Aguinis, Joo, Gottfredson, 2011). Performance appraisal may either support or spoil efforts to manage performance in an effective manner (Haines Illa & St-Onge, 2012).
A complication in finding appropriate performance appraisal scoring systems is measuring the different types of performance. According to Churchill, Ward and Ford (1990) (and Landy & Farr, 1983)) performance can be divided into objective performance and subjective performance.
Objective performance appraisals are more concerned with tangible measures of performance. These can be expressed in output measures (e.g., sales targets or quota attained), the input of the employee (efficient use of time, number of calls a day, etc.) or relative/ratio measures where input is compared to output (e.g., amount of calls to definite sales ratio) (Churchill et al, 1990).
Subjective performance appraisals are more concerned with the quality of the employee’s efforts rather than the quantity achieved. Examples are customer satisfaction, knowledge of affairs, client relationships, and many more aspects that cannot be quantified like sales quota can be.
Whereas figures like total sales can be an easy measure helping management identifying a performance gap between sales and the sales quota (performance planning ), this is a short-term view of profitability, which may not be in the best interest of the long term growth of the organization.
Thus, emphasis on customer satisfaction and developing long-term relationships has increased since the end of the last century to ensure salespeople are delivering more and better service to their respective accounts and clients (Crosby, Evans, Cowles, 1990). Total sales and profit are more indicative of past performances, but not predictive of future performances some suggest (Kaplan & Norton, 1992; Singleton-Green, 1993; Cross & Lynch, 1992). Future profitability has been shown to depend largely on non-financial performances such as customer satisfaction (Hauser, Semester, Wernerfelt, 1994). Employees doing well in subjective performance could be regarded as developing their objective performance (e.g., output, total sales) in the long run, in turn allowing organizations to reach long term strategic goals like expanding networks and market share, relationships, etc. (American Accounting Association, 1971; Johnson & Kaplan, 1987).
Therefore we can see the importance of a balance between different performance measures in performance appraisal, objective as well as subjective, considering that both contribute to an increase in output (total sales).
Internal benchmarking is a potentially effective way of measuring the performance (gaps) of individuals within an organization (Bielecki, Albers, Mantrala, 2012). Through relative ranking, top performers can easily be identified, as well as lower performers, allowing managers to distribute guidance, feedback and rewards based on how someone performs relative to the total salesforce (Bielecki, Albers, Mantrala, 2012). Performance gaps can then be bridged by looking at what makes top performers more efficient than bottom performers (Parsons, 2004).
In practice, commonly found internal benchmarking systems (e.g., for sales force efficiency) use very simple input-output ratios of objective performances (e.g., sales per call, hires per interview) to determine and compare individual performance efficiency (Johnston and Marshall 2010, p184-213).
This emphasis on objective performance measures can blind management to the way employees perform in other tasks that are not recorded in such a benchmarking scorecard. It’s unknown if this type of system biases management or employees to overvaluing objective performance to subjective performance, disallowing a proper balance between performance measures.
Purpose
Considering the relative importance of performance appraisal within the performance management cycle, this thesis will further investigate what makes effective performance appraisal. Specifically the use of objective versus subjective performance measures in the form of an internal benchmarking system will be considered.
The purpose of this thesis is to investigate the effect of internal benchmarking on core Self Evaluations of employees (Judge et al., 1998) (e.g., feelings that contribute to motivation, self-efficacy, locus of control, etc.). I will also investigate whether such a ranking - if it is merely based on objective performance measures - further influences employees, biasing them towards financial performance.
Additionally, management might further influence the effect of performance ranking by the emphasis that is placed on the ranking, potentially increasing the significance the employee assigns to financial performance, and therewith strengthening their bias.
Future financial performance has been demonstrated in the past the be linked to not only short-term financial performance, but especially subjective performance. A bias towards short-term performance is therefore potentially harmful, signifying the importance of this research question.
Based on the outcomes organizations can fine tune their internal benchmark appraisal systems to assess how an individual is performing besides simply an output figure such as total sales. Management could
positively contribute to how a person feels about their own performance even if they are on the lower end of the benchmark ranking.
To investigate this, this question is posed: Does high performance on a total sales scorecard in an internal benchmarking system correlate with a higher level of core Self Evaluations, and does it influence how important recruitment employees consider financial performance rather than other complementary performances? How do the focus of managers on objective performance measures influence how employees view their performance?
THEORETICAL FRAMEWORK
Performance appraisal & Core Self Evaluations
Considerable research has indicated that particularly four personality dimensions - locus of control , neuroticism, generalized self-efficacy , and self-esteem, part of a construct known as core self evaluations (Judge, Locke, & Durham, 1998) - are motivators to achieve higher performance in employees. While CSE have been shown to be related to employee motivation and future performance (Bono & Judge, 2003; Erez & Judge, 2001; Judge, Erez & Bono, 1998; Kacmar, Harris, Collins & Judge, 2009), a great degree in variability has been unaccounted for (Judge & Bono, 2001).
Theory suggests that employees with a higher CSE tend to be more motivated due to their belief in their abilities to take on future challenges. But it remains unknown whether this goes both ways - meaning, whether performing well also gives employees higher CSE, and thus more motivation, etc.
Whereas some suggest (e.g., Johns, 2006) moderators are involved in the variability of employee CSE, it is unknown what type of factors might be involved. It might be so that CSE is influenced by performance appraisal.
Considering that motivation and improving performance is the intention of performance appraisal, it is important to investigate this relationship.
Based on these findings and this gap in the literature I propose performance appraisal can affect a person’s core self evaluations, meaning that performance appraisals could affect how employees vary in feelings of locus of control, neuroticism, generalized self-efficacy, and self-esteem.
Internal Benchmarking
Some analysts found an increase in productivity of over 200% by narrowing the performance gap between the top 15 to 20% and the rest of the sales force (e.g., Ledingham, Kovac, and Simon 2006).
Successful practices of top performers could be incorporated into company best practices, training regiment, and even company culture (Rigby & Bilodeau, 2007). While cost effective, it also has the potential to create a narrowed scope about what is the best approach for reaching the top.
Even though there is no research about the relationship between internal benchmarking and core self evaluations, some research suggests that when performance appraisals do not include aspects of performance employees consider important, it affects their job satisfaction negatively (e.g., Saari & Judge, 2004). Also, as Rackham and DeVincentis (1999, p. 281) suggest, if performance measures consist of things salespeople cannot fully control, it will not result in improved sales. Perhaps because it negatively impacts CSE, as locus of control is a part of the construct, and positively impacts motivation when high.
I propose a possible link between internal benchmarking - a form of appraisal system - based on narrowed scope of performance measures and the core Self Evaluations of employees in the workplace. I hypothesize that:
Hypothesis IA: Higher ranking on total sales scoring correlates with higher level of CSE
If employees consider an appraisal system fair (e.g., measuring controllable input, encompassing employee contribution to the company at large), their satisfaction will be higher. High performance in such a system would result in a higher CSE. If employees do not consider such a system fair, both high and low performers would either be unaffected or have a low CSE, and HIa would not be supported.
Performance appraisal: objective (financial) vs subjective
Both subjective and objective performance measures have been suggested to be valuable in evaluating individual performance accurately (e.g., Kren & Tyson, 2009). However, in previous a meta-analysis interchangeability has not been confirmed (Bommer et al., 1995). Therefore, when setting up a balanced performance measurement system and allocating performance-dependent compensation, research suggests using both measures as complements, in order to both avoid and ensure the respective disadvantages and
advantages of both measures (Rajan and Reichelstein 2009, 2006; Zhao and Yu 2007; Van der Stede et al. 2006; Moers 2005; Ittner et al., 2003; Baiman & Rajan 1995; Baker et al. 1988).
It is in the best interest of a performance management system to be unbiased towards each of the two measures when assessing overall performance. Subjective judgments can be influenced by information of other, unrelated information, meaning they could incorporate correlating but n0t causal input related to an employee’s performance. Someone could be rated qualitatively well, due to their financial (objective) performance (Nisbett et al. 1981; Bond et al. 2007). Thus, rater bias cannot only affect the appraisal of an item, but also the balance or relevance of different dimensions of performance.
Ittner & Larcker (1998) and - in a more recent experiment - Bol & Smith (2011) speak of a “spillover effect” of one performance measure (objective) onto another (subjective). Bol & Smith’s subjects were asked to evaluate subjective performance, and were informed of another dimension of (objective) performance; total sales (which were fictive). Raters were directionally influenced by the sales figures when giving overall performance ratings. This result was even significant when excluding participants who explicitly considered the additional information as valuable to give an overall performance rating. This suggests the sales volume information had an influence on raters even if they did not explicitly consider it to be significant.
employees can be aware of a narrowed scope in an internal benchmarking system that favors only one or a few types of performance measures, and be negatively affected. But employees might be equally sensitive to the aforementioned “spillover effect” of objective performance measures, and be biased instead. Following this latter argument, the following is proposed:
Hypothesis IB: Higher ranking on total sales volume correlates with increased valuation of financial performance
If employees do not consider a benchmarking system unbalanced and unfair, this might be due to them being biased towards the performance measure the appraisal system favors. In such a case, it would appear that HIb would be confirmed.
Rater bias in performance appraisal
Managers with a favorable attitude towards one of the two different performance measures could cause a bias in the appraisal of employees. Because managerial discretion is often allowed within performance reward systems (Murphy & Oyer, 2003), it is important to ensure fairness by exposing how potential biases influence this personal discretion.
Rater bias has been the subject of much research, and behavioral anchors have been suggested to be one such source of bias in performance raters (Murphy & Constans, 1987). Behavioral anchors are generalizations of behavioral phenomena linked to good, average or bad performance. Examples include “High performing sales agents produce more calls” or “Average performing employees do not work over-hours”. Such anchors could be scaled to create a consistent rating system of employee behavior and their performance (Kingstrom & Bass, 1981; Schwab, Heneman, & DeCotiis, 1975).
Because high performers sometimes also display behavior similar to lower performers, and vice versa, raters might be influenced by the recall of these behaviors when appraising, thus giving a biased rating.
If an employee’s performance scale is based on sales figures rather than behavioral cues, managers could be biased towards sales figures as indications of high performance. The “spillover effect” confirms such a bias is possible in managers.
Theory suggests that performance appraisals can influence a person’s CSE. A manager’s rater bias could be transferred similarly during performance appraisal, affecting the views of an employee:
Hypothesis 2A : Higher emphasis on financial performance in performance appraisal is related with a moderation of the valuation of financial performance in employees.
Hypothesis 2B : Higher emphasis on financial performance in performance appraisal is related with a moderation of an employee’s core self evaluations.
This suggests the emphasis managers put on objective performance measures acts as a moderator in the relationship between an employee’s total sales and their own valuation of objective performance measures (and equally, the relationship between total sales and core Self Evaluations).
The emphasis managers put on the particular performance measure that is favored in the appraisal system could increase the effect it has on employees. If managers emphasize sales volume, employees might focus
on this performance measure more when considering their own performance. This would increase the effect a sales ranking list already has on the employee.
Similarly, an employee’s emphasis on objective performance measures could moderate the relationship between total sales and core Self Evaluations. Core Self Evaluations account for feelings of locus of control and generalized self efficacy , which an employee could feel their achieved total sales reflect. If a person derives a sense of self efficacy from their performance, or feels their total sales are completely in their control, performing well (or not) could influence this. A manager emphasizing the value of this performance (versus other areas of performance) could moderate the effect total sales performance has on an employee’s CSE.
Design
The research is designed as a case study of a Dutch recruitment agency based in Amsterdam with an internal benchmark ranking system based on total sales. The company consists of 37 employees, of which around 33 sales agents are divided into teams lead by a team manager. Based on input (e.g., total hours phoned) and output (total sales) team managers periodically appraise sales agents for their performance. Total sales ranking is publicly available in the office space which allows all agents to know how they perform relatively to their peers on financial measures. Input measures such as total calls and hours called are also publicly known. Other subjective measures are left at the discretion of the individual team managers and are not publically known. The agents often have had experience in recruitment at competing agencies before their employment at this company. Best practices are a combination of best practices from several companies in the industry.
This provides a unique case of a company where agents are ranked and divided into teams with their own managers, which allows all hypotheses to be tested.
The variables in the model are quantified to test for a main effect of total sales rank on employee CSE and their emphasis on objective performance measures, and an interaction effect of their team manager’s emphasis on this outcome variable.
Questionnaires will be taken to assess the emphasis employees put on objective and subjective measures of their own performance. Team managers will also be interviewed about the emphasis they put on subjective and objective performance measures when assessing the performance of their subordinates.
Because it is a specific case with a small sample, additional interviews will be necessary to collect qualitative data and to possibly identify potential factors and explanations that are unique to this company.
The interviews will be taken with a few key members of the organization (e.g., a high performer, a low performer and two managers) to provide a more in depth view of this case.
METHOD
Design
The design consisted of a mix between self evaluation questionnaires and a survey. Data was collected from 29 participants, of which 3 were team managers. The surveys were in Dutch and all subjects were fluent speakers.
Measurements
The independent variable “Total Sales Rank” was collected through sales records of the company. Ranks were assigned according to number of respondents, with the highest ranking sales agent given the rank = n, the second = n - 1, etc.
Dependent Variable 1 “Core Self Evaluations” was measured with a 5 point Likert-scale questionnaire developed by Judge et al. (2003). The questionnaire consisted of 12 items, of which the following as an example question: “I am confident I get the success I deserve in life” with 5 point scale answers ranging from 1 = “I strongly disagree” to 5 “I strongly agree”.
Dependent Variable II “employee emphasis on financial Performance” (EEFP) was measured with a self-developed 5 point Likert-scale questionnaire. The questionnaire consisted of 6 items of which 2 items regarded an objective performance measure (Total sales and “Phone stats” (=amount of hours phoned)) and 4 items regarded the subjective performance measure (Customer Satisfaction, Quality of work process, Quality of sales pitches and Team spirit). An example question is: “When considering how well I am doing as a Business Developer, I generally look at how satisfied my customers are” with 5 point scale answers ranging from 1 = “I strongly disagree” to 5 “I strongly agree”.
Moderator Variable “Manager emphasis on financial performance” (MEFP) was measured with a 5 point Likert-scale questionnaire. The questionnaire consisted of 6 items. To avoid response bias (Furnham, 1986) a total of 3 team manager evaluated the emphasis the other 3 managers put on objective and subjective performance measures. An example question is: “Think about your colleague X; when evaluating the performance of his team members, he puts a strong emphasis on total sales” with answers ranging from 1 = “I strongly disagree” to 5 “I strongly agree”.
All surveys were either produced in Dutch or in the case of the Core Self Evaluations Scale, translated to Dutch. The Core Self Evaluations Scale was found on the website of T. Judge, one of its authors
The survey was produced and distributed through a digital surveying service ( www.qualtrics.com). The service provided a link to a single survey that sorted team managers from sales agents through initial questions. The link was distributed throughout the company via internal emails, which requested the targeted participants to complete the questionnaire.
Analyses
Regression analysis is performed to determine the relationship between total sales ranking and employee Core Self Evaluations.
The main model consists of a moderator model, with an independent variable (“Total Sales Rank”) affecting the dependent variable “employee emphasis on financial performance”. This effect is moderated by the variable “Manager emphasis on financial performance”. Moderator analysis is performed in SPSS to look for a main effect between MEFP on EEFP or Total Sales Rank on EEFP, and an interaction effect between MEFP, Total Sales Rank and EEFP. Predictors Total Sales Rank and MEFP were tested to be better predictors of EEFP together, rather than separately.
Research Proposals
Based on theory suggesting core self evaluations in employees are potentially negatively influenced by performance measures they consider unreflective of their actual performances, I propose that a higher total sales ranking positively influences CSE in employees. Someone who performs less well on sales, but better in other areas, might consider the sales ranking misrepresentative of their overall performance. Whereas someone who does perform well might have less to worry about in regards to other areas of performance, if total sales is the dominant measure of performance.
Similarly, I propose that a higher total sales ranking results in a higher employee emphasis on financial performance. Meaning, someone who achieves higher sales is more inclined to consider this performance measure appropriate than someone who achieves a lower amount of sales. A lower ranking in total sales might make a person consider other aspects of their performance significant as well (or even more important) in order to make themselves feel good about their own performance, regardless of their “score”. This could be a way of self motivation.
Furthermore, I propose the emphasis a manager puts on financial performance during performance appraisal has a positive moderating effect between total sales ranking and CSE. This would mean that an employee performing well on total sales experiences a higher Core Self Evaluation if their manager emphasizes this performance measure.
Finally, I propose that the moderating effect of a manager’s emphasis on financial performance is also present in the effect of Total Sales on EEFP. This means that employees do not only consider financial performance more important if they manage to achieve more sales, but this effect is also strengthened by the emphasis a manager puts on it. This would suggest that employees with managers who value financial performance more than other managers also tend to have an increase in their own valuation of financial performance.
A manager could create the aforementioned “spillover effect” in the employee’s self evaluation in regards to their overall performance, and which performance measures they’re biased towards.
RESULTS
Participants
Data were collected at a recruitment office building in Amsterdam consisting of 37 employees. The total amount of respondents was 30, of which 4 were managers. All respondents were male and lived in the Amsterdam area. The average amount of years in service was a little over 2 years (M = 2.05). Most respondents had completed a vocational level of education (72.4%), followed by smaller amount of respondents having completed academic level of education (17.2%) and the remainder mid level of education (10.3%).
Four teams were surveyed, with varying sizes and response rates. Team 1 consisted of one manager and 13 sales agents with a response number of 10 (76.9%). Team 2 consisted of one manager and 6 sales agents with a response rate of 100%. Team 3 consisted of one manager with 6 sales agents with a response rate of 71.4%. Team 4 consisted of one manager and two sales agents, of which the response rate was 100%. Team 5 consisted of one manager and 4 agents with a response rate of 80%. Of the total number of employees (37) the response number was 30 (81.1%).
Correlations
To analyze correlations I used the Spearman Ranks correlation test for nonparametric data. This test was used because the sample size was too small for a parametric test. An advantage of using Spearman’s Rank test is the fact total sales are already considered a ranking.
Figure 2: Means, standard deviations and correlations (Spearman’s Rank) of all variables.
No significant correlation was found between the independent, dependent and moderator variables contrary to expectations. Visual inspection of the data showed an apparent relationship between total sales ranking and employee emphasis on financial performance, as was expected. Although a causal link is cannot be established, it seems that higher performers experience a lower amount of valuation for total sales ranking as a performance measure.
Figure 3: Scatterplot of Total Sales Rank and EEFP shows a negative trend. Regression
As expected, the regression analysis showed a notable, yet not significant (ß = 0.425, ns.), main effect of Total Sales Rank on employee CSE. This would suggest as people achieve higher total sales, their core self evaluations increase: they derive their confidence and self-worth from their achievements in sales ranking. No main effect of manager emphasis on financial Performance on employee CSE was found (ß = -0.113, ns.). The amount of emphasis a manager puts on an employee’s financial performance in relation to their overall performance does not seem to affect how the employee judges their own performance. No interaction effect
was found between Total Sales Rank, MEFP and employee CSE (ß = 0.086, ns.). Both models show a negligible amount of predictive power (main effects: Adj. R2 = -0.083; interaction effect: Adj. R2 = -0.101). A moderate, negative effect was found of total sales ranking on employee emphasis on financial Performance (ß = -0.191, ns.). This would suggest employees might increase their focus on objective performance measures rather than subjective measures as they perform relatively worse financially. There was no main effect of MEFP on EEFP (ß = 0.011, ns.), suggesting a manager’s emphasis on objective performance measures is not “spilled over” to the employee. A moderate, negative interaction effect was shown between the three variables (ß = -0.29, ns.), but was not significant. This suggests that an employee in an office with both a total sales ranking and a manager that emphasizes objective performance measures experiences a “spillover effect” of objective performance measures to a higher degree than an employee, in another office, where only either of the two factors is present. The effective is negative, suggesting the increase in focus on objective performance measures in the employee is increased as their total sales is decreased.
Figure 5: Regression analyses of Hypotheses 1B and 2B Interviews
Interviews were held with four participants, of which one was a team manager. The selection of the employees was based on varying levels of financial performance. No other criteria were used.
The employees were explicitly asked about what they thought about having their financial performance publicly displayed and ranked, and how it affected their sense of performance. They were also asked what they felt was generally the focus in a performance review, and to describe how and when a team manager would give feedback or guide the employee.
With regard to internal benchmarking all participants indicated not to mind having their performances ranked publicly. No one confirmed to feel more or less motivated by their ranking. They gave similar answers regarding the sources of their own confidence; having been selected and feeling equal to their peers regardless of total sales rank was more important to their self confidence than their sales output.
When asked what the focus was of performance appraisals, they confirmed there was little to no performance appraisal “unless sales were bad”. Lower performers were approached to discuss performance, which is solely expressed in total sales (output), and plans are made to improve performance (i.e., increase
sales). Employees are coached into improving their input to achieve higher output. Several forms of input have a target set (e.g., total cold calls, total messages, possible after hours) which are more strictly monitored until sales improve.
Being able to achieve high output autonomously was considered very important. The highest performer in the office confirmed to receive little to no qualitative performance appraisal. This employee did not regard his sales ranking important, but focused his efforts on building solid relationships with his clients which allowed him to achieve greater returns. The employee considered a focus on client satisfaction to be very important in performing well, and expressed there ought to be more focus on this when coaching lower performing peers.
These responses reflect the negative trend in the relationship between total sales ranking and employee emphasis on financial Performance. As financial performance declines, managers increase the amount of performance appraisal on the employee, emphasizing the importance of their financial performance.
Discussion
Summary
With the intention of finding a relationship between total sales ranking and an employee’s core self evaluations (and the effect on their bias towards objective performance measures), and the potential interaction effect of a manager’s bias towards objective performance measures, some notable results were found. Hypothesized was that total sales ranking was a predictor of high employee CSE, which was not supported by these findings. Also was hypothesized that a manager’s emphasis on financial performance was a predictor of high employee CSE, which was not supported either. An interaction effect between the variables was considered, but no such interaction was supported by the results.
In a similar model, it was hypothesized that total sales ranking predicted a higher emphasis on financial performance measures in employees. Although not significant, a negative rather than a positive effect appeared from analysis of the data and consequent interviews with a few employees. I also proposed that a manager’s emphasis on financial performance would be a predictor of a higher emphasis on financial performance measures in employees. Such an effect was not supported by the results. I analyzed the interaction between the manager’s emphasis on objective performance measures, the sales rating of the employees and the employee CSE and employee emphasis on objective performance measures in two different interaction models. This analysis showed that the effect of total sales ranking on an employee’s
emphasis on objective performance measures was greater when moderated by a positive emphasis put on objective performance measures by managers.
Unpredicted Results and Alternative Explanations
Both core self evaluations and employee emphasis on financial performance were considered to be influenced positively by total sales ranking.
Core Self Evaluations appeared to be randomly divided among the participants, and was high on average (M = 4.0654, SD = 0.4069). Theory suggests high performance could be dependent on motivated employees with high core self evaluations, because this generated a belief in the ability to take on challenges (Judge & Hurst, 2007), though no apparent relationship was found. it is possible the core self evaluations of the employees in this sample depended on outside factors, regardless of the fact that total sales ranking was the dominant form of performance measure.
Literature also suggests individuals with high core self evaluations might be more likely to seek out jobs where they can exercise autonomy and see opportunities to excel (Judge & Hurst, 2007). Judge et al concluded that individuals with high CSE are more likely to see opportunities within their socio-economic atmosphere, and capitalize on them.
CSE were also suggested to predict whether employees would consider their performance measures as representative of their job and their actual performance, or not. No significant variance was found between higher or lower performance with regard to this. The interviews did not allow me to determine how “fair” employees considered the total sales ranking; some acknowledged other performances were also important, but this did not make them consider the total sales ranking as misrepresentative.
It is possible that company culture and hiring and selection processes played a role in the average CSE in this case. This may imply that the company culture elevated the overall CSE of the employees, or managers selected candidates based on a tendency towards a high CSE.
In the interviews some respondents confirmed practices during the selection of new employees. Some claimed the company culture was purposefully predominantly male and young (average age 26.5 years), with almost all employees having a history in sales employment. A manager claimed the company preferred to work with people who are self-confident and could handle working hard autonomously.
No effect of a manager’s emphasis on financial performance measures was found on the employees’ CSE. Similar to the relationship with total sales ranking, the CSE of the employees in this case might be dependent on too many external factors to be predicted merely by MEFP.
No interaction effect was found between total sales ranking, MEFP and EEFP. The employees are apparently not more or less biased about objective performance measures when their performance is measured on a total sales ranking list, nor does an effect occur (or strengthen) when there is also a team manager emphasizing their financial performance.
There was an apparent negative trend in how an employee’s emphasis on financial performance measures was dependent on total sales ranking, contrary to earlier predictions. Interviews provided further insight into a possible explanation. Lower ranking employees more frequent and critical feedback than higher ranking employees, and were guided into improving their sales output by increasing their input. This increased their focus on their financial performance and ranking. Higher ranking employees received less feedback and less coaching into improving their performance. A high performing employee claimed that high financial performance resulted in more autonomy, which was a goal within the company.
Thus higher performing employees might be more “immune” to the spillover effect of a manager’s emphasis on financial performance measures because they were granted more autonomy in their work. Theory suggests that employee autonomy contributes to a more positive Leader-Membership Exchange (Sears & Hackett, 2011), in which employees will seek out more opportunities, are more motivated to seek out resources and ultimately perform better. If higher performers are given more autonomy, they might distance themselves from a bias in performance measures.
In the interviews some of the higher performers expressed how their focus shifted from financial performance towards qualitative performances such as growing relationships with clients, to further increase opportunities as their performance (and in turn, autonomy) increased. This mirrors that autonomy might be an explanation of this apparent “immunity”.
Limitations
Whereas this case study provided me with useful data of this company (e.g., having transparent interviews with employees and receiving actual sales data) that would be difficult to obtain in most other situations, the sample size was a limiting factor in this research. The sample size (N = 26) was too small for a parametric test to analyze relationships between the variables. Most of the sampling was done in a period with a lot of vacationing by the employees, which may have been a cause for the limited response rate. Regardless, even a complete sample of the whole company would’ve been too small for parametric testing (37 employees). This also is one of the explanations of the lack of significant results.
Alongside this problem was the fact that the sample was very homogenous. Interviews with employees seemed to confirm the company was a make up of very like minded people, with similar backgrounds,
and most had a similar level of education. The homogenous nature of the company culture makes it difficult to find effects that are representative of sales units in general.
The research model also depended on the assumption that all employees received equal and similar performance appraisals. In the interviews it became clear that higher performing employees received more autonomy in their work, resulting in less appraisal in general (i.e., less coaching). Lower performing employees received more performance appraisal and more coaching, and also had different input targets set (e.g., total cold calls made). Because of this a relationship between a manager’s emphasis on financial performance and an employee’s bias towards financial performance cannot be properly established. In this case it is unknown what the bias of higher performing employees would be if all employees received equal amounts of performance appraisal and equal input targets.
The sizes of the teams also provided a limiting factor. The company was divided into four teams, of which the largest had 13 members and the smallest team had 2 members (excluding team managers). The two other teams did not have equal sizes, either. Not only was it difficult to compare the variances between the groups (or to properly measure it inside a small group), the group size could have an effect on how team managers enacted performance appraisals and considered financial performance in the teams they managed.
A manager of a small team might have more opportunity to actively monitor their employees’ actions, and thus have a more complete view of how their team members conducted their work. This could allow them to consider the qualitative aspects of their input more than a team manager with a large team. A team manager with a large team might have to resort to merely looking at total sales simply because the size of their team limits them to monitoring other qualities.
Finally, the surveys consisted of self-developed scales that were not tested for their validity prior to this research. The scales may not properly measure a bias towards objective performance measures, and thus it cannot be verified how much the managers differed in their bias.
Practical implications and Future research
An important observation of the results is that the internal benchmark system itself did not seem to affect how employees felt about their ability to perform (CSE). This is desirable for companies trying to benefit from such a performance management system. Employees who would otherwise be qualified to perform well at the company could be demotivated or otherwise turned off by the total sales ranking, if they didn’t consider it representative of how they actually performed.
However, due to the sample size it cannot be determined at this time that a total sales ranking does not affect an employee’s CSE. In future research it might be possible to observe such an effect in a controlled setting, with a control group, or an assessment of multiple companies in different countries. It is important to focus on internal benchmark systems that are publically available to all employees.
The fact that the employees in this case received varying degrees of feedback and critique from their managers was unexpected in the setup of this research, which has important implications for current companies and future research. It demonstrates that case studies such as this one may specify or even contradict the general evidence for relationships between variables under certain circumstances and add to our understanding of the mechanisms involved.
It is difficult to determine whether or not the moderation effect of a manager’s emphasis on objective performance measures would occur in situations where all employee received equal levels of feedback and coaching. Taking into account that some of the higher performing employees considered their focus not being biased towards financial performance as they reached the top total sales ranking - allowing them to focus on important business practices such as strengthening business relations and networks - the choice to intensify feedback and critique for lower performing individuals has desirable and undesirable effects and implications.
As new employees start to build their business network, it is hard to point back at a track record of success when trying to establish trust with other businesses. As such past successes haven’t been made yet, a focus on a short-term successes such as quick and many sales allows an employee to build a track record of success that can signal reliable performance to prospects. As employees become more autonomous in their efforts, they are given more freedom to build upon these sales skills by learning to strengthen bonds with their clients.
On the other side it is possible a bias towards financial performance could hinder the creativity of would-be top performers as they reach the middle, but never expand their repertoire. As feedback and coaching declines with increased performance and the employee becomes more autonomous, those who want to find ways to build further upon their basic skills have to depend on their own resourcefulness. There didn’t appear to be a formal next step in feedback and coaching from quick sales skills to more in depth business-to-business relationship building. This has the risk that some employees might end up “stuck in the middle” of the sales ranking: skilled enough to perform autonomously, but not guided beyond to narrow the performance gap.
A company like this could benefit from the experiences and successes of top performers in order to narrow the gap between middle and top performers. These skills could be reviewed in performance appraisals and then incorporated into feedback and guidance for junior employees or those who struggle to perform better
in the long-term. This takes advantage of the internal benchmarking system, as suggested by Bielecki et al (2012).
Future research could determine if in a similar scenario the same moderation pattern appeared when equalling the intensity of the feedback across all the employees. If a similar result appeared - that higher performers seem to be “immune” to a bias for objective performance measures - it could imply a few possible developments the employee goes through.
Possibly the employee learns to “think for themselves” as they grow in their role, and start to see the bigger picture of different types of business strategies. It is also possible that only those who “learn to think for themselves” reach the top, if a causal link can be determined between a lack of bias towards objective performance measures and a high total sales ranking. This would imply the biased feedback of managers can hold some employees back from performing better, whereas other individuals are not affected by the “spillover effect”.
A future study would have to include more psychological factors of the employees to determine if personality that has a mediating role in the susceptibility to the suggested “spillover effect”. Big 5 Personality Traits could be predictors of how employees could be affected by either internal benchmark systems or the a bias towards (a) performance measure(s).
As this was an exploratory research in a small sampled case, the scales that were used to measure biases in the employees and managers were improvised and were used for the first time. The build upon the observations in this research, and to acquire more reliable results, I suggest testing these scales so a reliable scale can be developed. The scales that were used in this case study are available in the appendix of this thesis, and can be used to determine which items should be added or retracted, rephrased or what other type of Likert-scale should be used.
Future researchers might also benefit from using different types of design, such as a time series design at a large salesforce, or a collection of sales divisions from different companies, to determine if a change in total sales ranking does affect employees somehow. A hurdle could be the fact that such sales figures are not easy to obtain due to their sensitive nature. Substituting actual sales figures with assigned rankings could function as an alternative to protect companies who do not wish to disclose actual data. Hopefully the implications and potential benefits from researching these factors as described in this case study will help convince managers to investigate this topic further, and allow researchers access to representations of actual sales figures.
Conclusion
As companies desire to manage and improve the performance of their employees, it is important to remain vigilant to the potential side effects of a one-sided approach to performance measures. Being able to observe what effect benchmarking has on employees, whether beneficial or undesirable, could further improve performance management systems and ultimately narrow the performance gap between top performers and middle performers.
I have proposed that total sales ranking could affect how employees feel about their own performance, in addition to the moderating effects of a manager’s bias towards objective performance. Obviously, due to the nature of the sample it would be difficult to generalize any conclusions based on observations in this case. However, it is not unreasonable to assume that the approach to performance management within this company has different effects on different employees.
At this time it’s not possible to determine how strong and in what direction these effects move. But the observations are convincing that a relationship between a performance management system like internal benchmarking and how employees feel about that system might be present.
Literature
American Accounting Association (1971): Report of the committee on non-financial measures of effectiveness. Accouting Review. Vol 46. Supplement. p165-211
Aguinis, H., Gottfredson, R. K., & Joo, H. (2012): Delivering effective performance feedback: The strengths-based approach. Business Horizons, 55(2), 105—111.
Aguinis, H., Harry Joo, Ryan K. Gottfredson (2011): Why we hate performance management—And why we should love it. Business Horizons (2011) 54, 503—507
Baiman, S. & Rajan, M. (1995): The informational advantages of discretionary bonus schemes. The Accounting Review 70, pp. 557-579.
Baker, G., Jensen, M. & Murphy, K. (1988): Compensation and incentives: practice vs. theory. The Journal of Finance, 43(3), 593-616.
Banker, R., Potter, G. & Srinivasan, D. (2000): An Empirical Investigation of an Incentive Plan That Includes Nonfinancial Performance Measures. The Accounting Review Vol. 75, No. 1 (Jan., 2000), pp. 65-92
Bielecki, A., Albers, S., Mantrala, M. (2012): Salesperson Efficiency Benchmarking Using Sales Response Data: Who is Working Hard and Working Smart?,
Arbeitspapiere des Instituts für
Betriebswirtschaftslehre, CAU Kiel
Bol, J. & Smith, S. (2011): Spillover Effects in Subjective Performance Evaluation: Bias and the Asymmetric Influence of Controllability. The Accounting Review. Vol. 86, No. 4, p1213–1230
Bommer, W., Johnson, J., Rich, G., Podsakoff, P. &
MacKenzie, S. (1995): On the interchangeability of objective and subjective measures of employee performance: a meta-analysis. Issue Personnel Psychology Personnel Psychology Volume 48, Issue 3, pages 587–605
Bond, S., Carlson, K., Meloy, M., Russo, J. & Tanner, R. (2007): Information distortion in the evaluation of a single option. Organizational Behavior and Human Decision Processes 102: 240– 254.
Bono, J. E., & Judge, T. A. (2003): Core self-evaluations: A review of the trait and its role in job satisfaction and job performance. European Journal of Personality, 17(Suppl1), S5-S18.
Cascio, W. F. (2006). Global performance management systems. In I. Bjorkman & G. Stahl (Eds.), Handbook of research in international human resources management (pp. 176—196). London: Edward Elgar Ltd.
Crosby, L., Evans, K & Cowles, D. (1990): Relationship quality in Services Selling: An Interpersonal Influence Perspective. Journal of Marketing 5,4 (July)6, 8-81 Cross, K. & Lynch, R. (1992): Food for measure. CMA
Magazine. p20-23
Erez, A. & Judge, T. (2001): Relationship of core
self-evaluations to goal setting, motivation, and performance. Journal of Applied Psychology 86 (6): 1270–1279
Furnham, A (1986): Response bias, social desirability and dissimulation. Personality and individual differences 7. 385-400
Hauser, J., Simester, D. & Wernerfelt, B. (1994): Customer Satisfaction Incentives. Marketing Science. Fall. p327-350
Ittner, C., Larcker, D. & Meyer, M. (2003): Subjectivity and the weighting of performance measures: Evidence from a balanced scorecard. The Accounting Review, 78 (3), 725-759.
Johns, G. (2006): The essential impact of context on
organizational behavior. Academy of Management Review, 31, 386–408.
Johnson, H. & Kaplan, R. (1987): Relevance Lost: The Rise and Fall of Management Accounting. Harvard Business School Press, Chapter 11.
Johnston, M. & Marshall, G. (2010): Salesperson Performance: Behavior, Role, Perceptions, and Satisfaction, in: Churchill, G. A., N. M. Ford, and O. C. Walker (Eds.):
Sales Force Management, 10th Ed.,
McGraw-Hill/Irwin, Chicago, p184-213. Judge, T., & Bono, J. (2001): Relationship of core
self-evaluations traits—self-esteem, generalized self-efficacy, locus of control, and emotional stability—with job satisfaction and job performance: A meta-analysis. Journal of Applied Psychology, 86, 80-92.
Judge, T., Erez, A. & Bono, J. (1998): The power of being positive: The relation between positive self-concept and job performance. Human Performance 11 (2-3): 167–187.
Judge, T. & Hurst, C. (2007): Capitalizing on one's advantages: Role of core self-evaluations. Journal of Applied Psychology 92 (5): 1212–1227
Judge, T., Locke, E., Durham, C. & Kluger, A. (1998):
Dispositional effects on job and life satisfaction: The role of core evaluations. Journal of Applied Psychology, Vol 83(1), Feb 1998, 17-34.
Kacmar, K., Harris, K., Collins, B., Judge, T. (2009): Core self-evaluations and job performance: the role of the perceived work environment. Journal of Applied Psychology 94 (6): 1572–1580
Kaplan, R. & Norton, D. (1990): The Balanced Scorecard: Measures that drive performance. Harvard Business Revew. Jan/Feb. 1992. pp. 71-79
Kingstrom & Bass. (1981): A critical analysis of studies comparing behaviorally anchored ratings scales (BARS) and other rating formats. Personnel Psychology, 34, 263–89.
Kren, L. & Tyson, T. (2009): Trade-Offs in Objective and Subjective Performance Evaluation: A Case Study Examining the Validity of Agency Theory Predictions. Management Accounting Quarterly. Vol. 10, No. 2
Ledingham, D., Kovac, M. & Simon, H. (2006): The New Science of Sales Force Productivity, Harvard Business Review, 9, 124-133.
Landy, F. & Fair, J. (1983): The Measurement of Work Performance Methods, Theories and Application. NewYork: Academic Press.
Liu, Y., Combs, J., Ketchen, D. & Ireland, R. (2007): The value of human resource management for organizational performance. Business Horizons, 50(6), 503—511.
Moers, F. (2005): Discretion and bias in performance
evaluation: the impact of diversity and subjectivity, Accounting, Organizations and Society, 30, 67-80. Murphy, K. & Constans, J. (1987): Behavioral anchors as a
source of bias in rating. Journal of Applied Psychology, Vol 72(4), 573-577.
Murphy, K., & Oyer, P. (2003): Discretion in executive incentive contracts: Theory and evidence. Working paper, University of Southern California and Stanford University.
Nisbett, R., Zukier, H. & Lemley, R. (1981): The dilution effect: Nondiagnostic information weakens the implications of diagnostic information. Cognitive Psychology 13: 248–277.
Parsons, L. (2004): Measuring performance using stochastic frontier analysis: An industrial salesforce illustration. ISBM Report, 6-2004, Institute for the Study of Business Markets, Pennsylvania State University. Pla s, K, & Sobo ́tka, M. (2010): When the uncountable
counts: An alternative to monitoring employee performance. Business Horizons, 53(4), 349—357. Rackham, N. & DeVincentis, J. (1999): Rethinking the Sales
Force: Redefining Sell- ing to Create and Capture Customer, McGraw-Hill, New York.
Rajan, M. & Reichelstein, S. (2006): Subjective performance indicators and discretionary bonus pools. Journal of Accounting Research, 44 (3), pp. 585-618.
Rajan, M. & Reichelstein, S. (2009): Objective versus subjective indicators of managerial performance. The Accounting Review 84(1): 209-237.
Rigby, D. & Bilodeau, B. (2007): Selecting Management Tools Wisely, Harvard Business Review, 85 (12), 20-22. Saari, L., & Judge, T. (2004): Employee attitudes and job
satisfaction. Human Resource Management, 43, 395-407.
Sears, G. & Hackett, R. (2011): The influence of role definition and affect in LMX: A process perspective on the personality - LMX relationship. Journal of Occupational and Organizational Psychology 84 (3): 544–564.
Singleton-Green, B. (1993): If it matters, measure it. Accountancy. May, 1993. pp. 52-53
Schwab, Heneman, & DeCotiis. (1975): Behaviourally anchored rating scales: A review of the literature. Personnel Psychology, 28, 549–62.
Van der Stede, W., Chow, C. & Lin, T. (2006): Strategy, choice of performance measures, and performance. Behavioral Research in Accounting 18, pp. 185-205. Zhao, X. & Yu, Z. (2007): Value of subjective performance
evaluation: A theoretical perspective. Journal of Modern Accounting and Auditing 3(4), pp. 46-52.
Appendix
List of figures
Figure 1: A conceptual model of this research
Figure 2: Means, standard deviations and correlations (Spearman’s Rank) of all variables Figure 3: Scatterplot of Total Sales Rank and EEFP
Figure 4: Regression analysis of Hypotheses 1A and 2A Figure 5: Regression analyses of Hypotheses 1B and 2B
……… 11 ……… 16 ……… 16 ……… 17 ……… 18
