1
An Empirical Test of the Matthew Effect Among The
Elites in a Profession
Master Thesis – Msc. Business Administration
Author:
Jelle Eitjes
Supervisor:
Nathan Betancourt
Submission Date:
23 June 2017
Institution:
University of Amsterdam
2 Abstract
A status effect occurs when the difference in status between two actors within a hierarchy causes them to be rewarded differently for the same output. Empirical studies have demonstrated that a shift in status affects reward outcomes of actors when this effect happens early in their careers and when of moderately high skill in their profession. This paper tests the resilience of this effect comparing outputs of actors in the domain of science at the very top in their profession after one group experiences a major status shifting event - winning the Nobel Prize. I find that even among elites in the relatively objective domain of physics, appraisers do react to status shifts, but the effect size is very weak. The contribution to the literature of status and uncertainty is a robust test of the Matthew Effect among elites of a status ordering using a realistic counterfactual. The findings suggest quality perceptions decouple very little from actual quality in appraisals of actors among the highest ranks of a status ordering in an objective domain.
Keywords: Status Hierarchies, Matthew Effect, Social Cues, Uncertainty, Elites
3 1- Introduction
Status hierarchies are a prevalent feature in any social system and a well-studied phenomena as they decide how fame and fortune are distributed. Formally a status ordering may be defined as an ordered differentiation among individuals, groups, or entities with regard to one or multiple valued outcome(s). In a hunter gatherer setting this may entail the ordering of the males on their perceived ability to provide food, where in an organizational setting the ordering could regard employee-ability to lead and contribute to the company’s success, or it may even concern the ordering of toothpaste brands on how well they are perceived to prevent cavities. Being on the top ranks of the ladder comes with its perks. At a given point in time, those actors among higher ranks of an order have been found to receive more credit for their inputs than lower ranked actors keeping their efforts equal. High status actors are the preferred choice in exchange relationships over low status actors (Thye, 2000), are able to negate better prices in market contexts (Podolny, 1993) and – in science- are provided with better resources and opportunity ((Merton, 1968). To define generally, those among the higher ranks are perceived of more favorably because they are believed to possess some desired characteristics which in turn rewards them with access to resources at more favourable terms (Bothner, Godart, & Lee, 2009). From a meritocratic perspective, it is reasonable to demand that these status orderings accurately reflect an actor's or object’s ability to perform on the desired dimension. It is however the perceptions of quality from the audience which drive the ordering and these perceptions do not necessarily reflect the actual quality distribution. Instead they are found to divert from actual quality, generally when there’s some degree of uncertainty about how to judge quality by the audience (Gould, 2002; Merton, 1968; Podolny, 1993) Such misordering, would likely not have received as much scholarly attention when corrected over time, but because those higher up the hierarchy are bestowed with better resources and opportunity, initial differences tend exaggerate over time, perpetuating inequality which is referred to as the Matthew Effect (Merton, 1968) The empirical work so far has studied the effect among chip producing companies (Podolny, Stuart, & Hannan, 1996), academics (Simcoe & Waguespack, 2011)between companies in a market setting (Podolny, 1993), and wine producers (Benjamin
4 & Podolny, 1999). All actors here receive more favorable appraisals for the products keeping constant quality. The understanding of how status drives inequality is of tremendous importance to the administration of various systems. The Matthew Effect mechanism suggests that actors at the top ranks of a status ladder access resources at favorable terms disproportionate to their ability or quality. Such misallocation of resources due to status judgments affects companies making hiring and incentive decisions, universities appointing positions, or policy makers creating welfare programs to name a few. Isolating causal effects is particularly difficult in the domain of social sciences (Manski, 1993) and it’s an issue which poises the empirical work on the Matthew Effect as well. To gage the effect size in a natural setting, one must devise an experiment where a sample group of actors receives a treatment of “extra status” observing a net difference in their reward compared to the counterfactual at a later point in time. To isolate the pure status effect, into consideration must be taken as well that that those who received a status treatment, over time, gain access to extra resources which come accompanied with the extra status, a benefit which increases the actual quality of the outputs they are rewarded for (Merton, 1968) In observing differences, one can’t confidently untangle whether the treated group reaps an additional reward because of the status effect, or because they upped the quality of their output which they’re rewarded for. The contribution of this paper is to offer an empirical design which addresses these difficulties quite well. Through the analysis of data on Nobel Prize winners and nominees in physics in combination with the bilbiometric data of their academic outputs. It is assumed that a scientist when winning the Nobel Prize, experiences an upward shift in status at a discrete point in time (the “status treatment”). The counterfactual - the control group of actors who did not receive this status treatment - is constructed from the nominees for the same price who were unfortunate enough not to win. To observe an effect of status, we can thus look at the differences in reward outcomes between these two groups, where reward outcomes are measured by the citations that each group receives to selected publications. In science, actors find reward for their outputs through the citations their publications receive from fellow scientists who in their work pay homage to their ideas (Sher & Garfield, 1965). To isolate a pure status effect, I measure outcomes at the paper level excluding papers published by winners after they have won from the analysis because these may be affected by status-derived-resources
5 in their citation outcomes. Similar research designs have been used to test a Matthew Effect in science before, but the strength of this design is the use of a real counterfactual. Additionally, this design tests for an effect among the elites of a profession, a sub range in the status hierarchy where one may expect a diminished effect size, because both the treatment and control group are of considerable status already. Another contribution of this design contribution relates to the literature on the conditions which are necessary for divergence between actual quality and status. One accepted truth here is that uncertainty is a necessary contextual condition for divergence between quality and status. Under such condition audiences have a hard time ordering according to actual quality and thus rely on social cues such as an actor's status. This paper operationalizes a differentiation of uncertainty between the different disciplines in science not used in Matthew Effect research designs before. By testing in the field of physics, which compared to other domains of science, is extremely objective in its standards for quality judgments with arguably little uncertainty the resilience of the Matthew Effect is tested. I find that there is an effect size, but that the effect is much weaker than what has previously been found in a comparable study in a more uncertain domain (medicine). This suggests that among elites status effects occur at diminished strength or objective domains with less uncertainty bias are less prone to status effects.
2 - Literature Review
Literature on status orderings
Status orderings are an almost universal phenomenon in sociology demonstrated in both modern societies and more primitive ones. They are observed among children in the schoolyard, hunter-gatherers, employees and academics to name a few. Actors in a ranking may be individuals, (legal) entities, or objects. This work will build on the conceptualizations of (Erickson, 1988; Gould, 2002; Merton, 1968) that status hierarchies emerge from member’s quality perceptions about those in the ordering but that these perceptions aren’t necessarily aligned with the real distribution of quality, or quickly divert from it over time. Before reviewing literature about why and when decoupling happens it is important to note that these orderings tend not to correct themselves over time, but instead are found to exaggerate and diverge leading to cumulative
6 advantages for those in more favorable positions in the rankings early on. This idea was first observed by (Merton, 1968) when studying the sociology of science. He observed two consequences in his account of the 41st chair of the Royal French Academy. A prestigious club of scientists where membership was in fixed supply at 40 seats.
“The French Academy, it will be remembered, decided early that only a cohort of 40 could qualify as members and so emerge as immortals. This limitation of numbers made inevitable, of course, the exclusion through the centuries of many talented individuals.”(Merton, 1968 p.56 )
The scientists who were elected into the Royal French Academy would be perceived of more favorably and were lauded more by appraisers than non-members of equal ability or record. Their status also resulted in greater access to resources for future studies, talented graduate students, while having fewer lecturing responsibilities (and thus more time to research). This privilege would in turn increase the actual quality over time of while those outside the Academy had fewer opportunities to improve. The observation of this cumulative effect by Merton is qualitative in nature, but the effect has since been studied using robust empirical designs. To name a few, these authors find that slightly higher growth rates early in an career can be used to predict career success and longevity many years down the line (Allison & Stewart, 1974; Petersen, Jung, Yang, & Stanley, 2011).
Status orderings and Uncertainty
Because these initial differences exaggerate over time, the subject of how these orderings arise are well studied. One important insight is that the orderings of actors on status and actual quality tend to uncouple when there is a degree of uncertainty about making quality judgments of those in the rankings (Erickson, 1988; Gould, 2002; Merton, 1968). When this is the case, the audience increasingly relies on social cues in ranking actors even when ‘actual’ quality may be ordered differently. Certain types but not all types of uncertainty have empirically been identified as contextual factors causing audiences to rely on social cues
7 (Lynn, Podolny, & Tao, 2009). When industries or markets are new, organizations prefer transacting with reputable partners (Podolny, 1993) startup companies - whose track record is generally unproven - are more likely to reach public initial offerings and reap higher market capitalizations when connected to affluent investors or strategic alliances (Stuart, Hoang, & Hybels, 1999), goods with technology or artistry of uncertain value, are increasingly judged with reliance on status of the producer (Bradley & Lang, 1994; Podolny et al., 1996; Posner & Petersen, 1990) just as in the arts and foods affiliations matter as cues (Greenfeld, 2006). When uncertainty is at play major individual level cues that are relied on include gender, race, educational background, or parental status (Castilla, 2008; Correll, Benard, & Paik, 2007; Goldin & Rouse, 1997). In the domain of science major social cues used to infer quality from actors in ranking - scientists - under uncertainty conditions include the prestige of the university issuing one’s diploma (Blau, 1955) the university a focal department is affiliated with (Keith & Babchuk, 1998), or designations such as prizes and appointments an individual scientist receives for his work ((Azoulay, Stuart, & Wang, 2013; Merton, 1968). Prize designations as a cue prove a particularly fruitful avenue for studying the effects of status on reward in academia as will be explained one section down, but first should be noted that uncertainty can be differentiated on various dimensions. For example,(Azoulay et al., 2013) modelled uncertainty, making a distinction between upcoming and established scientists where it was considered that the latter had more information available about them in the form of public records, proof of work etc than upcoming ones. It was found here that - in line with the theory on uncertainty - audiences relied increasingly on the prize cue in judging the upcoming scientists compared to the established ones. In Academia we may also differentiate uncertainty on a different dimension. Consider the different disciplines of science, broadly categorized in natural and social. The domain of social science has a difficulty making causal inferences because endogenous effects are hard to estimate in this domain (Manski, 1993) and empirical proof standards - perhaps as a consequence - are low (DeRose et al., 2005). Appraisers I argue thus increasingly rely on cues to make quality judgments. In comparison, the domain of physics relative to other domains of science is one of the most objective domains in which performance can be assessed objectively for three reasons. First, its theories are defined in the language of mathematics which leaves little room for ambiguity
8 and misinterpretation unlike the natural languages in which social sciences are described. Second, the standard of empirical proof for the major journals in physics is set at five sigma which is considerably more rigorous than the two sigma standard in social sciences. Third physics makes predictions about natural phenomena which can objectively be tested with little room for feelings or opinions about a theory. Theories of uncertainty predict that in absence of uncertainty, social cues should have a minimal impact on appraisals and through differentiating uncertainty along the dimension of the scientific disciplines, this paper creates insights about how the domain related uncertainty affects to what extend the Matthew Effect plays a role in orderings.
Nobel Prize as a social cue
With a new meaningful differentiation on uncertainty to cover ground in the empirics of the Matthew Effect, we may now turn to the cue serving as the treatment effect. Where cues such as diplomas and affiliated universities are difficult to incorporate in empirical models it is designations such as the Nobel Prize which are particularly interesting when aiming to isolate the effects of a status shift on reward. The winner-takes-all nature of prizes provides us with an observable sudden increase in status for those who win at a discrete point in time while also offering a realistic counterfactual group of nominees who remained untreated. The Nobel Prize is one of such designations, awarded once a year to one or more scientists with the most important contribution to their domain (although in some years the organizing committee refrains from awarding a prize, because no worthy contribution has been made that year (Nobel Media AB). These domains include physics, chemistry, medicine, literature, peace and economics. A panel of experts decides who receives a nomination and who ultimately wins the price. In nature the Nobel Prize is similar to the selection into the Royal French Academy in the sense that their designations are in scarce supply. The difference between nomination and winning is very similar to what (Merton, 1968) hypothesized as the difference between the 40th and fictional 41st chair. Generally considered a magnificent achievement, becoming a Nobel Prize laureate can be considered a large social cue for assessing performance.
9 To determine the impact of such a status shift we need a gage for reward. The citation score index which tracks the citations between publications of scientists is a rich source of quantitative data on the performance of scientist’s their products - publications. This data allows us to quantify the impact of a status shift on reward. Although the debate about what citations mean is ongoing (see discussion section) The assumption in this work is made that the predominant reason for citing is to pay homage to another scientist’s work (Garfield, 1987; Sher & Garfield, 1965; Zuckerman, 1987). Citation scores are central to the reward system in science. Universities look at citation metrics for hiring decisions (Hargens & Schuman, 1990) and scientific journals - the outlets for scientist’s their products - compute their impact factors by the citation which their outputs generate. Where reward in market settings may entail a financial measure, in academia actors care about citations. Citations are available in relative abundance and designatable by anyone that is actively publishing. It is thus a more decentralized gage of performance than other award designations and sum totals among papers follow a natural lognormal distribution when plotted for frequency (Redner, 1998)
Status effects among elites
An additional benefit from this empirical model is that it allows to study the Matthew Effect among those in the highest rank of their profession which has not been done previously. Studies to date have focused on quantifying the effect of initial differences between actors around the mean of a quality distribution at t=0 where the effects have been shown substantial later on. (Allison & Stewart, 1974; Petersen et al., 2011). The Novelty in testing is that Nobel Prize winners and nominees are both of exceptional caliber already in their profession which raises the question if the effect of a major status shift still matters when comparing the number 1’s with the number 2’s in a profession.
10 3 - Method
To ensure the isolation of a relatively pure status effect it is important to mind Merton’s (1968) distinctive pathways through which status can increase performance 1) an increase in status increases the perceptions of performance on an output of a given quality and 2) the increase in status provides the scientist with better resources through which he can increase his subsequent outputs. Publications which were published before winning the Nobel Prize are unconnected to any of the resources that may be provided to the scientist through the second pathway because they have already been published. It is on these publications that we can observe appraisal differences due to the Matthew Effect and thus isolate a relatively pure status effect. (Azoulay et al., 2013)
Because papers published by winners after winning may benefit from additional resources which come accompanied by winning the Nobel Prize, these papers are removed from the data. It will be assumed that papers which were published before winning remain constant in quality. The empirical test is organized as such that the status shift is defined as a given author winning a Nobel Prize where perceptions of performance are defined as changes to citation outcomes. To isolate the effect of a status shift on perceptions of performance in this design, there are two main considerations to make: 1) to what extend do citation outcomes reflect perceptions of performance? 2) What other factors influence citation outcomes, and how can these be controlled for?
The number of citations a given article receives may depend on the journal it’s published in, the time it was published, its co-authors, the branch of science, etc. Although many of these factors can be controlled for in bilbiometric data, it is quite assumptuous to state this list of confounding factors is exhaustive. Because of this inherent difficulty to control for all hidden variables, the empirical design in this work will use a difference in difference approach which is a suitable technique for estimating effect sizes in non-controlled experiment settings where selection of treatment is not random. The difference-in-difference design deals with pre-existing differences between the treatment and control group, but it does rely on a parallel path
11 assumption for its validity. The parallel path assumption in the context of this study means that when the treatment group of papers – those papers whose author won a Nobel Prize – were to remain untreated, it exhibits no significant difference in the outcome variable compared to the control group of untreated papers as time progresses.
In order to create a treatment and control group where we can confidently make this parallel path assumption, the following provisions on selection into the sample are made. On the author-level only authors are included who themselves were at least nominated for the Nobel Prize. The assumption is made here that all these authors are of a very high caliber although some were fortunate enough to win a prize in very limited supply by arbitrary decision. To deal with heterogeneity on the paper level this design makes use of a Coarsened Exact Matching algorithm (Iacus, King, Porro, & Katz, 2012) to match a treatment and control group on the pair level before time of treatment in order to ensure this assumption is valid. By matching articles in pairs on the covariates 1) publication date 2) number of co-authors 3) author position on the author list 4) number of nominee and sum of citations up until treatment. The idea here is to create keep equal as many confounding variables as possible, in order to observe a pure effect of status.
The rationale for not choosing propensity score matching, is that in this design citation outcomes are measured at the paper-level while treatment happened at the author level. When balancing the treatment and control group of articles on probability of selection into treatment, the parallel path assumption on the article level may be violated (Azoulay et al., 2013)
With the variables explained by the literature and the steps laid out as in the method, the hypothesis may now be formulated as follows:
12 Publications from authors who win a Nobel Prize (a shift in status) experience an increase in
Perceived performance as measured by citation scores after treatment compared to a matched control group of publications from Nobel Prize nominees.
4 - Results
This section will describe the steps taken to clean the data, how the sample was constructed and the results from the analysis. Starting with the dataset of dataset with data on publications from nominees and winners of the Nobel Prize between the years of 1901 and 1950. The data includes 1655 articles from 54 winners and 19649 papers from 372 nominated authors with an average of approximately 40.5 lifetime collected over a 100 year time frame between 1875 and 1975. There is an overlap between the winner and nominee group. Some authors were nominated prior to winning the Nobel Prize in later years.
These overlapping instances were removed to ensure that if an author had ever won there was no way his articles could be in the treatment group. There were 2 instances were a winner collaborated with a nominee (who never ended up winning) on the same paper, meaning this author-paper combination had to be excluded from the candidate control group. Any papers published by winners after they won were removed. There were no instances were the same author won a Nobel Prize (treatment) twice. Before conducting the matching procedure the dataset is described as follows.
13 Papers Winners
1901 first np year 1950 last np year 40 winners
704 papers from winners 17 papers per author
Papers Nominees
331 nominees
17800 papers from nominees 53 papers per author
The following covariates for matching are considered:
(1) Publication year; (2) specific journal (3) number of authors; (4) focal-scientist position on the authorship list; and (5) the total number of citations received before the award was given
With 704 papers against a 17800 potential matches the coarsening was first attempted at an exact match on publication date. There were difficulties in obtaining the position on the authorship covariate, so this was excluded from the matching procedure.
It wasn’t possible to compute any baseline balance statistic because the covariate 5) citations to award is a conditional value for the control group because they never received treatment, meaning that if a control paper published in 1901 is a possible match to a treatment paper published in 1901 with its author winning a prize 1920, citations for those 19 years had to be calculated for the control paper, but if it was matched instead with a paper published in 1901 where treatment happened in 1925 the citations over that interval had to be computed.
To solve for this each paper in the treatment group was assigned its own stratum with papers from the control group assigned to the strata with duplication as long as they had an exact match on publication date. This did mean that some control papers occurred one in more than one strata when they were a feasible match to more than one treatment paper. This duplication is taken care of at the end.
14 The CEM algorithm was fed a dataset of 122354 papers of which 704 unique winning papers with an average duplicate rate 6.8 papers. After running the CEM algorithm three times in a row dropping values that were left unmatched after each iteration. When a L1 balance of < 0.15 was achieved, extra control group members were dropped achieving a 1:1 match. Because control group data was duplicated earlier some control group papers occurred more than once over the strata. It was decided to drop these which led to a drop from 558 matched pairs to 504. There might have been a possibility to drop the duplicates more efficiently, but no duplication function between strata function was provided with the CEM algorithm. 504 very balanced matched pairs are large enough a sample to move on, especially with longitudinal data. The following balance and descriptive statistics can be provided of the sample:
Table 2 - Matching Summary CEM procedure:
Number of strata: 7924
Number of matched strata: 558 (504 remaining after removing duplicates)
Nominees Winners All 111242 696 Matched 558 558 Unmatched 110684 138
Multivariate L1 distance: 0.13799283
15
Group Annual citations between publication & Nobel Prize
Publication Year Year Nobel Prize Number of authors to the paper Number of nominated co-authors Nominees 0.894 1907 1920 1.496 0.0 Winners 0.950 1907 1920 1.494 0.0 n=2x504 ; μ
Note that For annual citations between publication & Nobel Prize the value for the paper of the nominee is computed using the year of the winner it was matched with
Table 4 - Coarsened Exact Matched Sample Covariate standard deviation -
σ
Group Annual citations between publication & Nobel Prize
Publication Year Year Nobel Prize Number of authors to the paper Number of nominated co-authors Nominees 2.338 17.19 13.88 0.754 0.0 Winners 2.415 17.19 13.88 0.727 0.0 n=2x504 ; σ
The L1 balance statistic as reported by the CEM algorithm is approx .14 which reflects that the papers between groups were matched well. The descriptive statistics show that the sample was matched relatively
16 well. The results indicate that papers between the nominee and winner group were on similar trajectory before treatment.
Because nominal variables can not be included as covariates in a matching procedure, they may pose a source of exogenous heterogeneity as well. These variables are added as fixed-effects in the regression, but the variance is visualised below. The boxplot in graph 1 shows that winning authors – over a lifetime – are cited more numerously than nominees
Graph 1 - Boxplot of total citations on author level (left) and paper level (right) by nominee and winner
The boxplot on a paper level indicates a balance between papers from the treatment and control group, although on the author level winners score higher on lifetime citations than the nominee group.
17 Graph 2 - Citation trend over time matched sample
In graph 2 over time, a trend is visible in citations. One plausible explanation for the dip in 1915 to 1920 and 1940 to 1945 are World War I and World War II, for the purposes of this analysis, it should just be noted that there a considerate amount of between year variance in citations which should be controlled for. When plotting the matched sample citation data for winners against nominees, there is no immediate effect visible.
18 It should be noted that at T=0 the mean citation value between the treatment and control group in the matched sample can be different, because the sum of citations before treatment was used as a covariate, and not citations in the year of treatment. Because citation activity seems quite volatile between years, a moving average graph is constructed for a visual inspection of trends.
19 Graph 4 - Average citations per year before and after treatment by winners and nominees moving average
On visual inspection it does seem that there is a noticeable effect in the first 5-10 years after winning, to test the hypothesis a difference-in-difference regression is used with dummy variables for time (time) and treatment (winner). The interaction variable is defined as time * winner and this measures the effect of interest. Additionally the model is using fixed effects at the year level to absorb any variance that might occur because of citation differences between the years. It was attempted to include author-level fixed effects but these had to be excluded due to collinearity. The R-squared of the model did not increase with clustering the standard error of the authors and made the interaction variable insignificant at P <0.10. The winner variable (treatment) was dropped by the model because it was highly correlated with the interaction variable. This makes sense because between groups differences before treatment were minimized due to the matching procedure. The results suggest that winning a Nobel Prize leads to a .0974416 increase in
20 annual citations after winning compared to a matched control group of similar papers from nominees. However, the overall variance explained by this model is very low.
Table 5 – Regression Output
Regression Output
Dependent Variable: Annual Citations To Paper
Method Fixed-effects (within) regression
Number of observations 69,574 Number of groups 1008 (n = 2x 504) R-squared: within 0.00328 between 0.0002 overall 0.0225 F(104,68462) 22.3 Prob > F 0 corr(u_i, Xb) -0.0201
Variable Coef. Std. Err. t P>t [95% Conf. Interval]
winner 0 (omitted)
time 0.1092702 0.015372 7.11 0.000 0.079142 0.139399
interaction (winner * time) 0.0974416 0.016537 5.89 0.000 0.065029 0.129854
Year of citation (fixed effects)
1874 0.0043983 0.237739 0.02 0.985 -0.46157 0.470366 1875 0.0010111 0.23236 0 0.997 -0.45441 0.456437 1876 0.0017168 0.224307 0.01 0.994 -0.43792 0.441358 1877 0.000456 0.219836 0 0.998 -0.43042 0.431335 1878 0.0094761 0.211656 0.04 0.964 -0.40537 0.424321
21 5 – Discussion
The observed effect size suggests that when testing for a Matthew Effect between a treated groups against a realistic counterfactual while also controlling for preexisting differences on the article-level, the effect size is relatively weak. This finding is in line with other recent empirical work on the Matthew Effect and suggests that when uncertainty is low and when considering only the elites of a ranking, the Matthew Effect plays an even smaller role in deciding reward outcomes. The findings are in line with the theory of status and uncertainty, which predicts that when there is no ambiguity, orderings will rank according to quality. The findings also suggest that there might be a diminishing effect of status at the higher ranks of the ladder. It is reasonable to posit that those around the mean of a status distribution stand more to gain from an increase in status than those at the top of a ranking who already possess significant amount of it. The contribution of this work has been to establish the boundaries of where status effects occur (or conditions under which the effect size is much weaker than in other empirical work) and its findings raise an interesting question whether it is elitism or domain uncertainty preventing a stronger Matthew Effect from occurring, or even a conjunction of the two. Is it the domain of physics - which I argued is very objective in its standards of appraisals - that kept the effect size weak? Is it because nominees and winners are both of considerable prestige already putting a cap on the effect of status? Or is there more information available about those at the top of a status ordering which in itself decreases uncertainty? A limitation of this study is that it did not untangle the effects of domain uncertainty and preexisting status of the subjects of study. Future studies ought to test the effects of uncertainty arising from the nature of the scientific disciplines through keeping constant the subjects - winners and nominees of the Nobel Prize - comparing effects between domains, i.e. physics, chemistry, economics. Additionally, future studies may also keep constant the domain and study prizes targeting different strata of the quality distribution within that domain. The Nobel Prize is awarded to the very best of a profession, but there are plenty of other prizes awarded which can be incorporated into a study design as well. The contribution of this paper has been to create the differentiation of uncertainty between different domains of science (at least in the study of the Matthew Effect).
22 As for the practical implications, there should be some caution. The major practical implication of literature on the Matthew Effect and cumulative advantage is twofold. Normatively, it may be argued that because of cumulative advantages arising from differences in initial conditions, those at lower ranks of the status ordering in a given field should be provided with offset resources and opportunity. Practically, practitioners in academics and organizations may be concerned with awarding opportunity and resources to actors whose status exaggerates their actual quality or ability. This work studied a very specific context of the Matthew Effect which quietly suggests that in the domain of physics there might be little status bias to worry about in decisions or that status effects do not advantage elites with high levels of existing status that much. The implications of subsequent research untangling the effects of domain uncertainty and elitism which will serve more profoundly.
In this section, it is also useful to reconsider what citation outcomes really mean. I followed the normative interpretation of citations (Sher & Garfield, 1965) where actors are believed to cite paying homage to other actors. An alternative explanation from the social constructivist view on citations suggest that citations take place not necessarily for this reason. In this account citation happens because citing a famous author’s work helps garner recognition (Case & Higgins, 2000; Latour, 1987) or because in the process of getting published certain citations are expected to be included even though they are not relevant to the citers’ content. (Serenko & Dumay, 2015) It is this view which may explain why under conditions of relative certainty we still observe a post-win citation boost among winners. If there is no uncertainty and if citation scores reflect performance, the theory of status and uncertainty would not predict an increase in citation scores after a status shock. The abrupt winner-takes-all ordering system of prize designations should not influence that of gradual citation outcomes. The social constructivist view of citations would state that actors aren’t necessarily uninformed about who belongs where in the ordering and thus need to rely on social cues in their citation behavior, but merely cite those with the best recognition for opportunistic reasons of their own. It is really hard to untangle what the results mean and they are unable to reject either explanation. Through using advanced analytical techniques from social network analysis, the origins of
23 post Nobel Citations may be traced and provide details about what papers from what scientists react to a status shift in their citing behavior. These findings may help researchers better understand the driving behaviors behind citations which are important for the empirical work on status effects because of the assumption that citations reflect appraisal by fellow scientists.
Conclusion
This paper has provided a robust empirical test of the Matthew Effect in the domain of physics where the effect of a status increase – winning the Nobel Prize – was studied on the reward outcomes of those actors believed to be at the top of a status ordering. Through the use of a realistic counterfactual and adapting article level controls, selection effects problematic to causality studies are to a large degree taken care of. By considering only articles published by the winner group published before winning, any resource flows connected to their prize are removed and a relatively pure Matthew Effect is netted out. The effect size observed by the study is relatively small on the reward outcome. Contrasting the findings with previous studies which did not control for product level differences and synthetic counterfactuals, this suggest the Matthew Effect may be much weaker than previously found. However – in line with theories of status and uncertainty – the reduced effect size can also be attributed to the relative absence of uncertainty in the domain of physics which influences to what degree status decouples from quality. A major contribution of this work has been to introduce a differentiation of uncertainty on the dimension of the different academic disciplines where it is argued that some domains are much less uncertain than others which affects the extent to which the Matthew Effect is expected to be present. The groundwork put down by this paper, paves the way for new empirical models to test for differences in uncertainty originating from particular domains of science. The Matthew Effect among elites of a profession is found to be very weak, suggesting there are diminishing returns to the effect of a status increase, once an actor is already well reputed in a field.
24
References
Allison, P. D., & Stewart, J. A. (1974). Productivity differences among scientists: Evidence for
accumulative advantage. American Sociological Review, , 596-606.
Azoulay, P., Stuart, T., & Wang, Y. (2013). Matthew: Effect or fable? Management Science,
60(1), 92-109.
Benjamin, B. A., & Podolny, J. M. (1999). Status, quality, and social order in the california wine
industry. Administrative Science Quarterly, 44(3), 563-589.
Blau, P. M. (1955). The dynamics of bureaucracy: A study of interpersonal relations in two
government agencies Chicago, U. P.
Bothner, M. S., Godart, F. C., & Lee, W. (2009). What is Social Status? Comparisons and
Contrasts with Cognate Concepts,
Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The self-assessment manikin and the
semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1),
49-59.
Case, D. O., & Higgins, G. M. (2000). How can we investigate citation behavior? A study of
reasons for citing literature in communication. Journal of the Association for Information
Science and Technology, 51(7), 635-645.
Castilla, E. J. (2008). Gender, race, and meritocracy in organizational careers 1. American
25
Correll, S. J., Benard, S., & Paik, I. (2007). Getting a job: Is there a motherhood penalty? 1.
American Journal of Sociology, 112(5), 1297-1338.
DeRose, J. J., Toumpoulis, I. K., Balaram, S. K., Ioannidis, J. P., Belsley, S., Ashton, R. C., . . .
Anagnostopoulos, C. E. (2005). Preoperative prediction of long-term survival after coronary
artery bypass grafting in patients with low left ventricular ejection fraction. The Journal of
Thoracic and Cardiovascular Surgery, 129(2), 314-321.
Erickson, B. H. (1988). The relational basis of attitudes. Social Structures: A Network Approach,
99, 121.
Garfield, E. (1987). Citation Data is Subtle Stuff-a Primer on Evaluating a Scientists
Performance,
Goldin, C., & Rouse, C. (1997). Orchestrating Impartiality: The Impact of" Blind" Auditions on
Female Musicians,
Gould, R. V. (2002). The origins of status hierarchies: A formal theory and empirical test 1.
American Journal of Sociology, 107(5), 1143-1178.
Greenfeld, L. (2006). Different worlds: A sociological study of taste, choice and success in art
Cambridge University Press.
Hargens, L. L., & Schuman, H. (1990). Citation counts and social comparisons: Scientists' use
and evaluation of citation index data. Social Science Research, 19(3), 205-221.
26
Iacus, S. M., King, G., Porro, G., & Katz, J. N. (2012). Causal inference without balance
checking: Coarsened exact matching. Political Analysis, , 1-24.
Keith, B., & Babchuk, N. (1998). The quest for institutional recognition: A longitudinal analysis
of scholarly productivity and academic prestige among sociology departments. Social
Forces, 76(4), 1495-1533.
Latour, B. (1987). Science in action: How to follow scientists and engineers through society
Harvard university press.
Lynn, F. B., Podolny, J. M., & Tao, L. (2009). A sociological (de) construction of the
relationship between status and quality 1. American Journal of Sociology, 115(3), 755-804.
Manski, C. F. (1993). Identification of endogenous social effects: The reflection problem. The
Review of Economic Studies, 60(3), 531-542.
Merton, R. K. (1968). The matthew effect in science. Science, 159(3810), 56-63.
Petersen, A. M., Jung, W. S., Yang, J. S., & Stanley, H. E. (2011). Quantitative and empirical
demonstration of the matthew effect in a study of career longevity. Proceedings of the
National Academy of Sciences of the United States of America, 108(1), 18-23.
doi:10.1073/pnas.1016733108 [doi]
Podolny, J. M. (1993). A status-based model of market competition. American Journal of
27
Podolny, J. M., Stuart, T. E., & Hannan, M. T. (1996). Networks, knowledge, and niches:
Competition in the worldwide semiconductor industry, 1984-1991. American Journal of
Sociology, 102(3), 659-689.
Posner, M. I., & Petersen, S. E. (1990). The attention system of the human brain. Annual Review
of Neuroscience, 13(1), 25-42.
Redner, S. (1998). How popular is your paper? an empirical study of the citation distribution.
The European Physical Journal B-Condensed Matter and Complex Systems, 4(2), 131-134.
Serenko, A., & Dumay, J. (2015). Citation classics published in knowledge management
journals. part II: Studying research trends and discovering the google scholar effect. Journal
of Knowledge Management, 19(6), 1335-1355.
Sher, I. H., & Garfield, E. (1965). New tools for improving and evaluating the effectiveness of
research. Research Program Effectiveness, Proceedings of the Conference Sponsored by the
Office of Naval Research, Washington, DC, 135-146.
Simcoe, T. S., & Waguespack, D. M. (2011). Status, quality, and attention: What's in a (missing)
name? Management Science, 57(2), 274-290.
Stuart, T. E., Hoang, H., & Hybels, R. C. (1999). Interorganizational endorsements and the
performance of entrepreneurial ventures. Administrative Science Quarterly, 44(2), 315-349.
Thye, S. R. (2000). A status value theory of power in exchange relations. American Sociological
28
