What makes an academic paper more cited? An empirical investigation on a top management journal

(1)

Master thesis

What makes an academic paper more cited?

An empirical investigation on a top management journal

MSc. BA Strategic Innovation Management

University of Groningen, Faculty of Economics and Business

June 2018

by

Eleni Grigoropoulou (s3436683)

Supervisor: Dr. Pedro de Faria

Co-assessor: Prof. Dr. Dries Faems

(2)

2

ABSTRACT

Previous research has given attention to the role of citations in the academic field, yet little is known about the characteristics that increase the likelihood of an article to get cited. This study focuses on the understudied field of management. All 1864 original articles from the Journal of Management Studies, from its first year of publication, 1964, until 2017, were analyzed. Articles’ characteristics were categorized into three perspectives: universalism, constructivism and presentation. Results showed that the title, the size of the article, the number of authors, their affiliation and the type of study, all affect citations. This study showed that literature review papers have stronger correlation with citations than any other type of study. Moreover, studies conducted from both men and women and studies whose authors have different affiliation were those with the biggest evolution over time. Nowadays, more than 40% of the articles have both men and women contribution and almost half of the papers of the journal that I examined are written from authors with different affiliation. The insights from this research show that both subjective and objective characteristics of the paper matter for the reader in order to cite a paper. Therefore, authors should pay attention to all article characteristics, from the type of study to the length of the title and the size of the paper. In this way, they can better position their papers or increase the chances their paper to get cited and editors can give support to the authors, in terms of the presentation aspects of a paper.

Keywords: citations, research methods, citation content analysis, web indicators, Google

(3)

3

1. Introduction

In the late 19th century and especially after World War II, university research has been controlled by specific and strict rules in order to give emphasis to the knowledge production and to the importance of collegial recognition (Benner and Sandstrom, 2000). Scientific articles are the result of academic research, which adds new information to the body of knowledge on a specific topic. Over the past 50 years an exceptional growth in research and education in the management field was observed. The intellectual development of the field worldwide led also to the rapid growth of journals (Clark et al., 2014). Articles that are published in journals are available to be read by other scientists, researchers and readers worldwide. Therefore, they are also available for public examination and judgment. During this examination, if the readers believe that the paper provides new information and has scientific influence, then they decide to cite it and use it as a reference to their studies (Stewart, 1983).

Since the introduction of article impact or citations in the early 1960s, they are used by fellow researchers, academic committees and readers in general to evaluate how influent and recognized an article is (Bergh et al., 2006). Through the use of citations, the results of a study become more persuasive, since they find support in previous published works. Previous scientific works indicate that articles with many citations reflect also better quality research (Antonakis et al., 2014; Bergh et al., 2006). The relationship between citations and quality emerges from the assumption that the authors select the articles they want to cite based, first of all, on the quality (Seglen, 1997).

Moreover, citations constitute a means of academic recognition and determine in a notable extent the position of the authors and researchers in the scientific field. According to Baldi (1998), the citation counts can be perceived as an indicator of the ‘’quality’’ of the author’s work. It is generally believed that authors with many citations are also excellent academics and have contributed a lot with their studies in the scientific community. Furthermore, universities take into consideration the citation counts about hiring and promotion decisions (Baldi, 1998; Posner, 1999). Also, authors who cite another paper and use it as a reference for their study, show a reliable writer ethos (Hyland, 1999).

(5)

5

1995). In addition, Meho (2007) provided evidence that 90% of papers that have been published in journals are never cited and 50% of the articles are never read by anyone else other than their authors and editors of the journals that they are published. Research conducted in marketing, engineering, medical and sociological field examined the citations and quality of the academic projects (Yitzhaki, 2002; Stremersch et al., 2007; Baumgartner & Pieters, 2003; Peters & van Raan, 1994; Baldi, 1998; Reed et al., 2007). Existing literature looks at different dimensions that can determine the article’s popularity such as the length of the article, title length, type of research conducted, author’s characteristics, abstract and keywords.

For this research, in order to explain the characteristics of an article and their relationship with the citations that it will get, three theories will be used: the universalism, constructivism and the presentation theory, as they are presented in the work of Stremersch et al. (2007).

In order to understand what differentiates the highly cited articles from the less or no cited at all articles, I will try to answer the following question:

Which characteristics make an article successful (in terms of citations)?

Because journal market has grown significantly the last decades, a thorough insight into the influence factors of management-related journals, and especially their articles, seems necessary (Bergh et al., 2006; Baumgartner & Pieters, 2003). Nevertheless, due to the literature gap on the management field on this subject, I will examine the characteristics of the business and management articles that affect the citations. Because academic articles call for time, effort and money, results from this study can help management researchers manage their time better and evaluate their research plans more properly. Understanding the characteristics of highly cited papers might help them to conduct their research with the objective of increasing its visibility and impact. In general, everyone involved in the management field, from the editors to the managers, can benefit from a clearer view of what attracts the readers.

(6)

6

prestigious journals in the management field and is highly ranked in the Management category of the ISI Journal Citation Reports (ranked 17/121 in business field and 23/194 in management field in 2016)1. I will look at different characteristics such as type of study, number of authors, countries of affiliation, gender, title and article length, funding, awards and citations and link them to existing literature.

The rest of the thesis is organized as follows: in the next section I review the literature that provided the theoretical framework for my research. The third section analyses the methodology and presents the data collection, and the fourth section presents the results from my study of the Journal of Management Studies. In the concluding section, I summarize the main findings and discuss the implications and limitations of this study.

2. Literature Review

In this section I will provide a structured overview on the findings from previous literature. First, I will give an explanation of the meaning of citations and their role. Following, I will analyze the three perspectives that underlie my research, based on evidence from existing literature.

A citation is defined as a reference to a previously published work; it is based on the assumption that it is of high quality and relevant to the argument the author wants to make (Baird & Oppenheim, 1994). An author chooses to cite an article either to give proper credit to the work or ideas of others or to criticize their findings. When authors cite another work they indicate that the discussed subject has been examined before and therefore they want to draw the reader’s attention (Baird & Oppenheim, 1994). Given the great importance of citations, previous literature indicates that not all citations are based on the same criteria and some do not even reflect on high quality.

Existing literature looks at different dimensions and theories to explain why a paper is cited. Stremersch et al. (2007) based their research about article’s characteristics and citations considering all three perspectives (universalism, constructivism and presentation perspective). According to the first perspective, universalism supports the idea of equal opportunities and creates expectations that same criteria will be used for judgment (Cullen et al., 2004). In particular, for my theoretical background, according to Stremersch et al. (2007) as well as Baldi (1998), universalism theory states that the citations an article receives

1

(7)

7

are result of what one says and not who one is. In this theory emotion neutrality prevails. Author’s characteristics do not define the quality of the article and the decision of the author to cite a paper. Importance is given to core characteristics of the article itself, such as type of research or awards won.

Many characteristics of the articles, such as the type of research, year of publication and awards have been identified as factors of the universalism theory. Many authors have paid attention especially to the type of research. There might be divergence in citations across the different types; for example a quantitative research might result in more citations because it contains more numerical evidence and thus makes it easier for the reader to understand (Sawyer et al., 2008). According to Antonakis et al. (2014) and Peritz (1983), articles with higher methodological diligence might lead to more valid results and more citations. Articles that review the literature tend to receive more citations than any other type of research (Walter et al., 2003, Monastersky, 2005). The combination of integrative knowledge of a specific field of study and how it should evolve, as well as the clarification and synthesis that such articles offer, explain the high number of citations that they get (Antonakis et al., 2014). The qualitative type of research is used generally less often than the quantitative (Bergh et al., 2006). Antonakis et al. (2014) mention that in qualitative studies there is more uncertainty about the results than in the quantitative studies; the idiosyncratic methods used in this type of study might reflect on the validity of the findings and therefore affect the citations.

(8)

8

that will be the most highly cited, but that it can help differentiate them from those that are unlikely to ever be cited.” Awards help readers distinguish a good paper from an average one, but cannot help them distinguish the best from the good ones (Lee et al., 2003).

Apart from the universalism theory, Stremersch et al. (2007) studied also the article’s characteristics according to the constructivism theory. This theory indicates that the factors that affect the number of citations in an article are connected with the authors and personal characteristics of them such as their affiliation, country of affiliation and their gender rather than with the actual content of the article. In more general terms, what is said is of less importance than who says it (Laband, 1986). Baldi (1998) in his work stated that ‘’ citations are allocated largely on the basis of an author’s position within the stratification structure of science’’. Often scientists cite articles based on the affiliation of the article’s author rather than on the intellectual content of the article itself in order to show their support to the authors and the validity of their results. Therefore, Baldi (1998) considers the number of citations as reflection of authors’ prejudices.

Rousseau (2001) and Stremersch (2007) paid attention to the number of coauthors as another constructivism determinant of citations. Rousseau (2001) stated that the more co-authors an article has, the more citations it might get because it is likely another author to know them. Moreover, the phenomenon of self-citation is very often and therefore in papers with many co-authors the total number of citations increases (Van Raan, 1998). Therefore, multi-authored articles have more chances to receive citations than single-authored ones. However, he concluded that the saying ‘’the more co-authors, the more citations’’ is not always precise. Other factors should be considered as well, such as the international scientific collaboration. Internationally co-authored papers tend to be cited more than those from a single country (Van Raan, 1998; Persson, 2010). Indeed, international collaboration opens the doors to more readers, creating a bigger network. However, previous literature gives indications that international collaboration does not result in high citation impact (Van Leeuwen, 2008).

(9)

9

submit so often papers for publication. Second, Nock (2001) noted that women are more likely to write books than academic articles in journals. Third, women often focus on scientific areas that are not widely ‘’famous’’ and fourth, they do not collaborate very often with others (Wold & Wenneras, 2010). Nevertheless, a later study conducted by Swygart (2004) showed that female authors are not underrepresented in academic articles. Wold & Wenneras (2010) although they acknowledge that women score lower than men in terms of citations, they also pointed out that this instance can be caused from other factors such as affiliation of the women authors and not just because of gender biases. Therefore, it is still not clear if there is a connection between gender and citations.

Existing literature studies also the presentation theory. This theory focuses on how the authors present their study and findings (Stremersch et al., 2007). They say that the way a paper looks influences the output. The title length, the length of the article and the number of keywords are identified as characteristics of this theory. They can be characterized as attention grabbers, since they affect people's perceptions of the content of the article. The title is one of the most important elements of an article. It can be characterized as the most comprehensive statement of the article’s content and it constitutes the most characteristic aspect of it, since it appears in bibliographies, online databases and reference lists. By constituting one of the most important attention grabbers, it leads to the initial selection or rejection of the article by the reader (Yitzhaki, 2002). Stremersch et al. (2007) argue about the impact of title length on citations; from the one side, articles with longer titles provide more useful information and thus they can be more effective, but from the other side they can be more complex and thus discourage further reading. Regarding the keywords, Stremersch et al. (2007) state that they are of great significance since they synthesize the main points of the article. The number of keywords might have a positive impact on the citations because they increase the likelihood of an article to appear in electronic databases searches.

(10)

10

However, Judge et al. (2007) showed that longer articles were cited more times than shorter ones. The length can affect the readability and therefore the citations.

Very often scientific projects get external grants from funding agencies and this indicates their good quality. Funding can be perceived as a sign of recognition, which enables the reproduction of the research project (Benner and Sandstrom, 2000). Previous studies focused on the funding’s contribution to the citations and the overall quality of the article. However, it is still not clear. Some consider it as positive, as an evidence that a high quality research rewarded by gaining funds from an institution. Others consider it as negative, thinking that the results might have been biased (Auranen & Nieminen, 2010). Nevertheless, Rigby (2011) mentions that funding and funding information in general in a paper, influences its impact.

(11)

11

3. Methodology

In this section I will explain how the data was collected. Following, more information about the gathered data will be provided. In addition, I will present the dependent and independent variables that were used for the statistical analysis. Their definitions will be mentioned as well as the way they were codified or calculated in the database.

To assess the predictors of citations, it was needed to examine a representative number of articles. All the articles that were published in the Journal of Management Studies, one of the oldest general management journals, were analyzed. Since its launch 54 years ago, it is known for publishing high quality research and has a long established history of excellence in the management research2. Since 1964 it has a total number of more than 250.000 citations and an impact factor of 3.9623, indicating its very good quality.

3.1. Data Collection

(12)

12

Nevertheless, because of the wide range of academic articles that Google Scholar offers and because the citations retrieved from there are also used to evaluate the ‘’quality’’ of authors, it is necessary to consider them. Therefore, in order to gain a concrete picture of the total number of citations, both online academic search engines were used. All the articles have retrieved from the official website of the Journal of Management Studies via the access of the University of Groningen.

(13)

13

3.2. Measurements

Variable Definition Description

Dependent

SmartCat citations Number of citations the article received obtained from SmartCat

Google Scholar citations Number of citations the article received obtained from Google Scholar

Independent

Universalism theory

Type of research Type of research that is conducted in the article (Quantitative, Qualitative, Literature review,

Conceptual, Book review)

(0 = does not belong to method type;

1 = belongs to method type). Some articles may cover more than one method types Prize/ Award If the article won a prize or

award for best paper

0= the article hasn’t won a prize or award;

1= the article won a prize or award

Age Age of the article Calculated as 2017 minus the

year of the article’s publication

Issue Issue of the journal that the

article is published

Special Issue If the article is published in a special issue of the journal

0= the article is not published in a special issue of the journal;

1= the article is published in a special issue of the journal Constructivism Theory

Cross-country If the article has two or more authors and they have different countries of affiliation

0= the article has one author or two or more authors who have all the same country of affiliation;

1= the article has two or more authors with different countries of affiliation Number of Countries Number of the countries of

authors’ affiliations

Count the number of countries of the article’s authors

Cross-gender If the article has two or more authors and they have different gender

0= the article has one author or two or more with same gender;

1= the article has two or more authors with different gender

Number of Authors Number of authors of the article

(14)

14 Presentation Theory

Title length Length of the article’s title Count the words of the article’s title

Length Length of the article Count the total number of

pages of the article (including the references pages)

Number of keywords The number of keywords in the article

Count the number of keywords of the article

Funding If the article received

funding from an educational institution, public or private company

0= the article didn’t receive funding;

1= the article received funding

4. Results

In this section I present the results of the statistical analysis. First, the key findings of the descriptive statistics are discussed. Next, the outcomes of the regression analysis will be explained.

4.1. Descriptive Statistics

(15)

15 Table 1

Means, Standard Deviations and Correlations of the Study Variables

(16)

16

The types of articles published in JMS were distributed as follows: Quantitative= 37.87%, Qualitative= 36.64%, Literature Review= 16.85%, Conceptual= 8.79% and Book Review= 0.80%.

As can be seen from Graph 1, trends in the types of study slightly changed over time. Quantitative and qualitative studies were the dominant ones through the 53-year period. Book reviews were not present the first decades and only reached a total number of six in the 90s. The majority of the papers are quantitative or qualitative and in recent years there is a higher percentage of quantitative papers (51,65%) than qualitative (30,11%). It is remarkable that in the current decade a little bit more than half of the papers published were quantitative (51,65%). Literature reviews showed an upward trend in the examined period, starting from 10% in the 60s and reaching 24,11% in the 90s. Finally, conceptual papers fluctuated in low percentages although the last years showed an upward trend.

Graph 1

Graph 2 presents the evolution of the average length of the articles over time. We can observe that articles were on average a lot shorter than nowadays. Specifically, in the 60s and 70s the average number of pages of the articles was 14, whereas in the 10s we see that this number has increased to 27 pages. This can be explained by the fact that over the years the papers that reviewed the literature increased. Such papers are longer papers and call for more space since they have more references than the other types of studies (Peters and Van Raan, 1994). Especially during the 90s and 00s where the literature reviews showed a significant increase, we can also observe that these are the decades where the average length of the articles increased the most.

(17)

17 Graph 2

Graph 3 shows the evolution of studies that had both male and female contribution in time. The results are in line with the evidence collected for the number of authors and the author’s gender; in the 60s and 70s there was lack of information about the author’s gender and therefore the data was not reliable. Throughout the years, the cross-gender studies had only upward trend. Especially during the 90s the number of studies that conducted from both men and women increased significantly (15,34%). The next decade it increased even more and as of today, because the decade is not yet completed, we cannot have a concrete view of the total number of cross-gender studies. However, until today 41,71% of the papers have both gender contributions and it is expected this number to increase until 2020.

Graph 3 0 10 20 30 40 50 60 70 80 90 100 60s 70s 80s 90s 00s 10s A ve rag e Le n gth Decades

Average length/ Decade

Average length 2,75% 6,21% 9,02% 15,34% 28,52% 41,71% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 60s 70s 80s 90s 00s 10s N u m b e r o f c ro ss -g e n d e r st u d ie s Decades

Cross-gender studies/ Decade

(18)

18

Graph 4 provides information about the articles whose authors had different affiliation. For the first two and a half decades (60s until mid 80s) the articles did not provide us with any information about the affiliation of the authors. Thus, we have no evidence of the international collaboration during that period. However, after mid 80s we can see that international collaboration not only existed, but increased significantly also. More specifically, we can see in the 90s, 8,49% of the total number of articles that published during that period, were cross-country studies. That percentage tripled during the 00s and until today we ended up having almost half of the papers (48,34%) categorized as cross-country studies.

Graph 4

Table 2 shows the average number of authors per decade. As we can see, there is not only a difference in the number of authors but also in their distribution. The table shows that over time, authors are working more together and the teams are growing. For the first four decades (60s, 70s, 80s and 90s) we see that most of the articles published were written by only one author. In the next years the difference between one and two authors’ papers became smaller and the number of solo authors decreased consistently. The collaboration of more than two authors in a paper started increasing. For the last two decades results showed that most papers are not the individual written ones but those with two authors instead (40,14% in the 00s and 34,53% in the 10s). Moreover, we can see that the percentages of more authors have been increased and there are even papers written by eight and thirteen authors. Collaborative studies have increased a lot from the 90s until today thank to the evolution of technology. The World Wide Web enables the joint creation of content by many authors at the same time. Authors have the opportunity to add, remove or change the content of their paper without being all together in the same place (Kaplan & Haenlein, 2010). 5,26% 8,49% 25,70% 48,34% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 60s 70s 80s 90s 00s 10s N u m b e r o f c ro ss -c o u n tr y st u d ie s Decades

Cross-country studies/ Decade

(19)

19

Average number of authors/Decade

1 2 3 4 5 6 7 8 13 60s 81,65% 16,51% 1,83% 70s 57,51% 37,30% 4,66% 0,51% 80s 57,51% 32,33% 7,14% 1,50% 1,50% 90s 44,66% 41,37% 12,33% 0,82% 0,82% 00s 29,23% 40,14% 24,65% 4,40% 1,41% 0,18% 10s 13,81% 34,53% 30,39% 17,68% 2,49% 0,27% 0,27% 0,27% 0,27% Table 2

The average number of total citations was 44.95 for SmartCat citations and 153.25 for Google Scholar citations.

(20)

20 Graph 5

Graph 6

Descriptive statistics provided us with some remarkable findings. We can see that studies that have both male and female contributions or their authors have different affiliations were those with the biggest evolution over time. Nowadays, almost half of the papers have been written from both men and women. The percentage of international collaboration in the 10s is almost nine times bigger than in the 80s. Moreover, a very interesting finding is that over the years, authors started working more together and the teams are growing. Regarding the type of study, most articles were quantitative and qualitative studies and they were also the ones with the most citations on average.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 60s 70s 80s 90s 00s 10s

Average citations SmartCat

Quantitative Qualitative Literature Review Conceptual Book Review 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 60s 70s 80s 90s 00s 10s

Average citations Google Scholar

(21)

21

4.2. Correlation among the variables

The correlation among the study’s variables can explain the regression analysis results and possible multicollinearity problems. To check for potential multicollinearity problems, I considered the variance inflation factors (VIFs). With a mean VIF value 4.46 and a highest VIF value 17.96, which is above the critical value of 10 (Mason & Perreault, 1991), multicollinearity problem exists (Table 3). To deal with this problem, I chose to conduct the regression analysis through three models, including only a limited number of dummy variables every time. More specifically, for Model 1, the analysis included only a dummy variable that identified the quantitative studies as 1 and all the other types of studies as 0. Indeed decreasing the number of dummy variables proved to solve the multicollinearity issue and new VIF test showed that the mean VIF value is 1.42 and the maximum VIF value is 2.45, well below the threshold value of 10 (Table 4).

Table 3

Variance Inflation Factor

(including all types of studies variables)

(22)

22 Table 4

New Variance Inflation Factor (including only one dummy variable)

Variable VIF Tolerance 1/VIF Number of countries 2.45 0.408 Cross-country 2.10 0.475 Age 1.85 0.541 Number of authors 1.61 0.619 Length 1.40 0.715 Cross-gender 1.17 0.853 Title length 1.13 0.884 Special Issue 1.13 0.884 Quantitative 1.10 0.910 Issue 1.07 0.932 Funding 1.05 0.953 Prize/ Award 1.02 0.982 Mean VIF 1.42

4.3. Multivariable Analysis

Table 5 presents the results from the regression analysis. The regression analysis was conducted with four simple ordinary least squares (OLS) regression models. More specifically, Table 5 shows the regression analysis with four models, including the SmartCat citations as the dependent variable. Model 1 includes only a dummy variable that identifies the quantitative studies with 1 and all the other types of studies as 0. The same method was used for Model 2 and 3 but the qualitative studies in the former and the literature reviews in the latter, identified as 1 and the other types as 0. Model 4 includes the variables from all types of studies. Considering Model 4, although there is multicollinearity problem, it was included for consistency reasons. Table 6 presents the regression analysis results, using the same models, but this time considering the Google Scholar citations as the dependent variable.

The models were estimated with Stata. The regression analysis conducted in 1512 articles out of the total sample of 1864 articles because of missing information on gender and affiliation of the authors in the 60s and 70s.

(23)

23

positive predictor of citations, it doesn’t mean necessary that the more different countries of affiliations in a paper, the more citations it will get. Noteworthy is however that the variable of the quantitative study in this model that was identified with 1, was not significant.

Four variables from Model 1 were also significant in Models 2, 3 and 4. More specifically, title length, cross-country, length and age of the article were also significant but with different values. There are however some differences among the four models. First, funding was significant only in Model 1. Second, in Model 2, qualitative study was significant for citations (p< .01). Third, for Models 2 and 3, number of authors and literature review were significant.

Concerning the results from Table 6, again the same independent variables as in Table 5 were used, but this time the dependent variable was the Google Scholar citations. This second analysis seemed necessary. Since I was already aware of the differences in the citations of the two online bibliometric indicators in theoretical level, I wanted to ascertain this difference in practice through a regression analysis. Model 1 of Table 6 shows that five variables were significant: title length (p< .01), quantitative study (p< .1), funding (p< .01), length (p< .01) and age of the article (p< .01). We can observe that Model 2 and 3 have the same significant variables as Model 1, again with different values, but the main difference is that instead of the quantitative study being significant, in Model 2 is the qualitative study (p< .01) and in Model 3 is the literature review (p< .01). Model 4 which includes all types of studies, has the title length, literature review, length and age of the article as significant variables.

Concerning the impact of awards and funding on citations, all four models from both SmartCat and Google Scholar, showed that they related negatively. Noteworthy is that awards decrease citations by 12.8% and funding by 8.6% in Model 1 of SmartCat table while awards decrease citations by 46% and funding by 40% in Model 1 of Google Scholar table. This negative relationship between funding and citations can be explained by the study of Auranen & Nieminen (2010) where they explained that papers that have received funding might be considered as biased and therefore readers do not cite them.

Another interesting finding is the significant relationship between the age of the article and citations for all models (p< .01) in both SmartCat and Google Scholar model. Older articles may have gotten more citations either because of historical reasons or because they are still actively used or simply because they had more time to receive many citations (Crossan & Apaydin, 2010). As for the relationship between cross-gender studies and citations, both models showed that it is not significant.

(24)

24

(25)

25 Table 5

Regression results (SmartCat)

Variables (1) (2) (3) (4) Title length -1.660*** -1.799*** -1.478*** -1.458*** (0.515) (0.508) (0.505) (0.511) Issue -1.056 -1.054 -1.049 -1.050 (0.887) (0.884) (0.876) (0.876) Special Issue -2.466 0.323 -1.831 -2.004 (6.104) (6.048) (5.977) (6.038) Cross-country 10.391* 10.140* 11.537** 11.556** (5.471) (5.448) (5.402) (5.406) Number of countries -3.768 -4.408 -5.049 -5.820 (5.041) (5.028) (4.981) (5.000) Cross-gender 4.601 5.080 4.787 5.068 (3.919) (3.907) (3.869) (3.871) Number of authors 3.529 2.909 3.906* 4.029* (2.277) (2.252) (2.234) (2.254) Quantitative -6.307 5.962 (4.013) (16.024) Qualitative -14.094*** 1.102 (3.964) (15.874) Book Review -10.283 (28.766) Conceptual 15.065 (16.816) Literature Review 31.493*** 36.424** (4.906) (16.573) Prize/ Award -12.743 -16.182 -12.132 -12.916 (18.923) (18.853) (18.669) (18.685) Funding -8.643** -6.926 -5.883 -4.878 (4.348) (4.364) (4.316) (4.356) Length 1.063*** 1.224*** 1.116*** 1.272*** (0.359) (0.361) (0.354) (0.367) Age 0.788*** 0.925*** 0.791*** 0.868*** (0.263) (0.264) (0.259) (0.263) Constant 37.320** 36.927** 25.712* 16.252 (14.739) (14.688) (14.655) (21.202) Observations 1,512 1,512 1,512 1,512 R-squared 0.028 0.035 0.053 0.056

(26)

26 Table 6

Regression results (Google Scholar)

Variables (1) (2) (3) (4) Title length -6.622*** -7.189*** -6.107*** -5.932*** (1.811) (1.787) (1.780) (1.798) Issue -3.759 -3.748 -3.730 -3.692 (3.121) (3.111) (3.086) (3.081) Special Issue -16.103 -5.429 -12.940 -15.779 (21.480) (21.290) (21.067) (21.244) Cross-country 27.119 25.993 30.650 31.175 (19.255) (19.178) (19.043) (19.020) Number of countries -11.474 -13.575 -15.557 -20.082 (17.741) (17.700) (17.556) (17.591) Cross-gender 14.973 16.703 15.638 17.034 (13.791) (13.755) (13.639) (13.622) Number of authors 4.832 2.345 5.717 7.134 (8.013) (7.927) (7.875) (7.932) Quantitative -26.409* 52.843 (14.123) (56.381) Qualitative -49.886*** 37.631 (13.954) (55.854) Book Review 19.378 (101.214) Conceptual 114.398* (59.167) Literature review 105.529*** 159.996*** (17.294) (58.314) Prize/ Award -45.552 -58.532 -44.548 -46.389 (66.595) (66.368) (65.810) (65.743) Funding -39.994*** -34.015** -30.872** -25.132 (15.303) (15.364) (15.214) (15.328) Length 3.669*** 4.225*** 3.830*** 4.717*** (1.263) (1.271) (1.249) (1.291) Age 3.416*** 3.912*** 3.442*** 3.818*** (0.926) (0.928) (0.914) (0.924) Constant 145.153*** 143.501*** 105.918** 24.867 (51.872) (51.705) (51.659) (74.600) Observations 1,512 1,512 1,512 1,512 R-squared 0.039 0.044 0.060 0.066

(27)

27

5. Discussion

In this section, I will discuss more in depth the results from the regression analysis and the descriptive statistics and I will provide possible explanations about them. Then, I will discuss the managerial implications of my study. Lastly, I will mention the limitations of this thesis and the possibilities for potential future research.

This study demonstrates new insights into citation analysis. My central research question for this study was: which characteristics make an article successful (in terms of citations). By documenting many different characteristics of the articles from the Journal of Management Studies, objective and subjective ones, from 1964 until 2017, a clear picture emerged about their relationship with citations. My results indicate that dimensions from all three perspectives discussed, play significant role for the citations (literature review, qualitative study and age of the article from the universal perspective, cross-country and number of authors from the constructivism perspective and title length and length of the article from the presentation perspective). Results from the regression analysis indicated that literature reviews have stronger correlation with citations than the other types of studies. Another very important result is the significant effect of international collaboration on the citations for both Google Scholar and SmartCat citations. Supporting the findings of Van Raan (1998) and Persson (2010) international collaboration creates a bigger network and makes it easier for the reader to locate a paper. Moreover, because results showed that title length and length of the article are significant for citations, editors are recommended to publish longer papers with not very concise titles.

Interesting findings emerged also for the funding and the awards. According to my research results, awards do not have any significant relationship with citations neither for SmartCat nor for Google Scholar. On the contrary, results showed that funding has significant relationship with citations in some models of the SmartCat and Google Scholar model. My results indicate that articles that have received funding from government/ institution or have won an award proved to relate negatively with citations. As Auranen & Nieminen (2010) mention, the probability that the results might be biased deters some readers from citing an article. Noteworthy is the difference between the coefficient values of funding and awards in the two bibliometric indicators. In Google Scholar, where the citations from students, non-academic users etc. are included, we can see that the coefficient values are much higher than those of SmartCat. That means that readers, who do not have any previous experience with publishing an article or do not know the insights of the publishing procedure, consider funding and awards as indicators of bias and do not trust the results. That does not seem to apply in the same extend with researchers.

(28)

28

articles that were not included at all either to one online bibliometric indicator or the other. Therefore, university administrators, government agencies, researchers and readers should consider the citations from both online libraries when examining a paper in order to have a concrete picture (Amara & Landry, 2012).

It is a fact that because of the importance of citations and their use in job evaluation, salaries and reputation, authors feel pressured to write papers of high quality in order to be published in top journals and get many citations (Monastersky, 2005). Due to this pressure to publish their work, receive grants or awards from institutions in order to support their research, they often focus more on particularistic aspects of their paper such as collaborating with many other authors, from both genders and from different countries. Nevertheless, this study showed that not only such particularistic aspects are important in order for a paper to be cited. The length of the title, the size of the article, the number of authors, their affiliation and the type of study, affect also the citations and they should all be considered when writing an article.

There are however some variables that are not significant in neither models of the regression analysis such as the cross-gender, issue, special issue and awards. One reason for this instance might be the fact that this study focused only on one journal, whereas all previous studies examined more than one (Baumgartner & Pieters, 2003; Reed at al., 2007; Judge et al., 2007; Yitzhaki, 2002). Also, previous studies that indicated the significance of these variables conducted the research in other fields (marketing, engineering, and medical) so probably my results indicate that these variables do not count that much for management authors when considering citing an article.

(29)

29

It should be mentioned that most previous studies conducted in other fields such as marketing and medical field. Therefore, my results might be slightly different than similar researches from other fields. A similar research in the marketing field may have found stronger indications about the significance of presentation aspects of the papers, because how a paper looks might be very important for marketing researchers. Also, Reed et al. (2007) found that specifically for the medical field, the quality of the article is highly connected with the funding. Grants or awards are considered as indicators of high quality research. Therefore, they would be very significant variables for the citations in the medical field.

5.1. Managerial Implications

The contribution from this research is addressed to management researchers, authors, editors and students. Authors who aim to publish papers in management journals, first of all, should carefully consider the type of study that they will conduct. They can increase the probability their article to be cited if they conduct a literature review, quantitative or qualitative study. They should also consider collaborating not just with other authors, but most importantly with foreign authors, since international collaboration proved to be a very important predictor of citations. According to my findings, how an article is presented proved to be very important. Both authors and editors should allocate their attention to the presentation characteristics of the paper before publishing. Very short papers with very concise titles proved not the best attention grabbers. Therefore, everyone involved in the writing and publishing procedure of academic papers should have in mind that every single detail of the paper counts for the reader when considering citing a paper.

Although citations should not be the only factor on which decisions about ‘’quality’’ of the papers and the authors are based, it is an important indicator that the paper contains valuable information that can contribute to the field of study. In this research certain characteristics of papers identified that could help research to be visible and impactful. In addition, because academic articles call for time, effort and money, taking into consideration this study will help managers to control their time better and evaluate their research plans more properly.

5.2. Limitations and future research

(30)

30

time, it has been used in previous studies and it has been observed that authors’ affiliation from higher ranking universities predict more citations (Antonakis et al., 2014; Bergh et al., 2006; Stremersch et al., 2007).

Furthermore, it should be mentioned that I focused on the number of citations the articles got, without considering self-citations. Self-citations are explanation of the highest impact of articles that have multi-authorship (Van Raan, 1998). Therefore, because they might bias the results of citations, it would be interesting for future research to consider them. Although previous literature has discussed keywords, in the journal that I analyzed their use started only after 2011, so their variable was soon excluded from the analysis, since it altered the results. Nevertheless, keywords is an interesting characteristic that future research can consider, since they increase the likelihood of an article to appear in electronic databases searches (Stremersch et al., 2007).

In addition, because my research conducted only in one journal, it didn’t seem necessary to consider the journal’s impact factor as a predictor of citations. However, future researchers who plan to conduct a study on more than one journal, is advisable to consider the journal impact factor as a citation predictor, since previous literature indicated that high impact journals receive better papers for publication and it is also a positive predictor of citations (Judge et al., 2007). It would be also interesting for future researchers to study journals with different impact factors, top journals and non-top journals, to see if the same kind of variables are used and what the differences in their significance are. Lastly, my study contained data only from the management and business field. Therefore, the results have limited generalizability.

6. Conclusion

(31)

31

References

Amara, N., & Landry, R. (2012). Counting citations in the field of business and management: Why use Google Scholar rather than the Web of Science. Scientometrics, 93(3), 553-581. Antonakis, J., Bastardoz, N., Liu, Y., & Schriesheim, C. A. (2014). What makes articles highly cited?. The Leadership Quarterly, 25(1), 152-179.

Auranen, O., & Nieminen, M. (2010). University research funding and publication performance—An international comparison. Research Policy, 39(6), 822-834.

Baird, L. M., & Oppenheim, C. (1994). Do citations matter?. Journal of Information Science, 20(1), 2-15.

Baldi, S. (1998). Normative versus social constructivist processes in the allocation of citations: A network-analytic model. American Sociological Review, 829-846

Baumgartner, H., & Pieters, R. (2003). The structural influence of marketing journals: A citation analysis of the discipline and its subareas over time. Journal of marketing, 67(2), 123-139.

Benner, M., & Sandström, U. (2000). Institutionalizing the triple helix: research funding and norms in the academic system. Research policy, 29(2), 291-301.

Bergh, D. D., & Perry, J. (2006). Some predictors of SMJ article impact. Strategic Management Journal, 27(1), 81-100.

Clark, T., Floyd, S. W., & Wright, M. (2006). On the review process and journal development. Journal of Management Studies, 43(3), 655-664.

Clark, T., Wright, M., Iskoujina, Z., & Garnett, P. (2014). JMS at 50: Trends over Time. Journal of Management Studies, 51(1), 19-37.

Crossan, M. M., & Apaydin, M. (2010). A multi‐dimensional framework of organizational innovation: A systematic review of the literature. Journal of management studies, 47(6), 1154-1191.

Cullen, J. B., Parboteeah, K. P., & Hoegl, M. (2004). Cross-national differences in managers' willingness to justify ethically suspect behaviors: A test of institutional anomie theory. Academy of management Journal, 47(3), 411-421.

Davenport, E., & Snyder, H. (1995). Who cites women? Whom do women cite?: an

exploration of gender and scholarly citation in sociology. Journal of Documentation, 51(4), 404-410.

(32)

32

Hyland, K. (1999). Academic attribution: Citation and the construction of disciplinary knowledge. Applied linguistics, 20(3), 341-367.

Judge, T. A., Cable, D. M., Colbert, A. E., & Rynes, S. L. (2007). What causes a management article to be cited—article, author, or journal?. Academy of Management Journal, 50(3), 491-506.

Kaplan, A. M., & Haenlein, M. (2010). Users of the world, unite! The challenges and opportunities of Social Media. Business horizons, 53(1), 59-68.

Laband, D. N. (1986). Article popularity. Economic Inquiry, 24(1), 173-180.

Lee, J. D., Vicente, K. J., Cassano, A., & Shearer, A. (2003). Can scientific impact be judged prospectively? A bibliometric test of Simonton" s model of creative productivity. Scientometrics, 56(2), 223-232.

MacRae Jr, D. (1969). Growth and decay curves in scientific citations. American Sociological Review, 631-635.

Mason, C. H., & Perreault Jr, W. D. (1991). Collinearity, power, and interpretation of multiple regression analysis. Journal of marketing research, 268-280.

Meho, L. I. (2007). The rise and rise of citation analysis. Physics World, 20(1), 32.

Monastersky, R. (2005). The number that’s devouring science. Chronicle of Higher Education, 52(8), 14.

Nock, D. A. (2001). Careers in print: Canadian sociological books and their wider impact, 1975-1992. Canadian Journal of Sociology/Cahiers canadiens de sociologie, 469-485.

Peritz, B. (1983). Are methodological papers more cited than theoretical or empirical ones? The case of sociology. Scientometrics, 5(4), 211-218.

Persson, O. (2010). Are highly cited papers more international?. Scientometrics, 83(2), 397-401.

Peters, H. P. F., & van Raan, A. F. (1994). On determinants of citation scores: A case study in chemical engineering. Journal of the American Society for Information Science, 45(1), 39. Posner, R. (1999). The Theory and Practice of Citations Analysis, with Special Reference to Law and Economics.

Reed, D. A., Cook, D. A., Beckman, T. J., Levine, R. B., Kern, D. E., & Wright, S. M. (2007). Association between funding and quality of published medical education research. Jama, 298(9), 1002-1009.

Rigby, J. (2011). Systematic grant and funding body acknowledgement data for publications: new dimensions and new controversies for research policy and evaluation. Research

(33)

33

Rousseau, R. (2001). Are multi-authored articles cited more than single-authored ones? Are collaborations with authors from other countries more cited than collaborations within the country? A case study. Berlin: Gesellschaft für Wissenschaftsforschung.

Sawyer, A. G., Laran, J., & Xu, J. (2008). The readability of marketing journals: Are award-winning articles better written?. Journal of Marketing, 72(1), 108-117.

Seglen, P. O. (1997). Citations and journal impact factors: questionable indicators of research quality. Allergy, 52(11), 1050-1056.

Shadish, W. R., Tolliver, D., Gray, M., & Sen Gupta, S. K. (1995). Author judgements about works they cite: three studies from psychology journals. Social studies of Science, 25(3), 477-498.

Stewart, J. A. (1983). Achievement and ascriptive processes in the recognition of scientific articles. Social Forces, 62(1), 166-189.

Stremersch, S., Verniers, I., & Verhoef, P. C. (2007). The quest for citations: Drivers of article impact. Journal of Marketing, 71(3), 171-19

Swygart-Hobaugh, A. J. (2004). A citation analysis of the quantitative/qualitative methods debate's reflection in sociology research: Implications for library collection development. Library Collections, Acquisitions, and Technical Services, 28(2), 180-195. Thelwall, M., & Kousha, K. (2015). WEB INDICATORS FOR RESEARCH EVALUATION. PART 1: CITATIONS AND LINKS TO ACADEMIC ARTICLES FROM THE WEB. El profesional de la información, 24(5).

Van Leeuwen, T. (2008). Strength and weakness of national science systems: A bibliometric analysis through cooperation patterns. Scientometrics, 79(2), 389-408.

Van Raan, A. (1998). The influence of international collaboration on the impact of research results: Some simple mathematical considerations concerning the role of self-citations. Scientometrics, 42(3), 423-428.

Walter, G., Bloch, S., Hunt, G., & Fisher, K. (2003). Counting on citations: a flawed way to measure quality. Medical Journal of Australia, 178(6), 280-281.

Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. MIS quarterly, xiii-xxiii.

Wold, A., & WENNERÁS, C. (2010). Nepotism and sexism in peer-review. In Women, science, and technology (pp. 64-70). Routledge.

(34)

34

Appendix

Table 7

Top-10 published articles in the Journal of Management Studies (according to citations from SmartCat)

Table 8

Top-10 published articles in the Journal of Management Studies (according to citations from Google Scholar)

What makes an academic paper more cited? An empirical investigation on a top management journal

Master thesis