Eight clusters : a dynamic perspective and structural analysis for the evaluation of institutional research performance
Thijs, B.
Citation
Thijs, B. (2010, January 27). Eight clusters : a dynamic perspective and structural analysis
for the evaluation of institutional research performance. Retrieved fromhttps://hdl.handle.net/1887/14617
Version: Not Applicable (or Unknown)
License: Licence agreement concerning inclusion of doctoral thesis in the Institutional Repository of the University of Leiden
Downloaded from: https://hdl.handle.net/1887/14617
Note: To cite this publication please use the final published version (if applicable).
4 OTHER A
PPLICATIONS
4.1 Bibliometric indicators
In the previous section the application of the classi cation model in studies on institutional research performance was presented. The model proved to be helpful in benchmarking and evaluative studies. The classi cation of research institutions has also other applications in which grouping of institutes on the basis of eld activity is fruitful.
When discussing the third principle, the use of multiple indicators, I stressed the importance of fully understanding the scope and validity of these indicators.
It is in the light of this requirement that the application described here can be seen. The classi cation helps in the constructing, validating and understanding of bibliometric indicators.
Actually, the idea for clustering institutes originated in the research on self-citations. In this study (see Part II, chapter 1) we wanted to know in how far the in uence of author self-citations on bibliometrics indicators at the meso- level deviates from that at the macro level and to what extend national reference standards could be used in bibliometrics meso analyses.
As most bibliometric indicators are strongly eld dependent we felt the need for a grouping of institutes based on the elds in which they are active. The clustering and classi cation of institutions seemed the right solution. Although the rst version of the classi cation had six cluster it proved to be very helpful.
The paper concluded that there are differences in self-citation patterns between institutions active in different els.
Two other papers on citation indicators were published after the nalization of the model (Glänzel, et al 2008; Glänzel, et al, 2009). The classi cation of institutes helped in the examination of which level of hierarchical subject classi cation should be used to build subject normalized citation indicators. Three hierarchical levels of subject classi cation of journals are available. First, there is the subject categories system used in the Web of Science (Wos). On top of that we have the hierarchical system jointly developed in Leuven and Budapest (Glänzel and Schubert, 2003). It comprised sixty sub elds and sixteen major elds.
From the multidisciplinary cluster, 676 institutes were taken in order to investigate the relation between citation indicators normalized each at different level of subject classi cation. In order to avoid too much fractionation, weighting and averaging over too many WoS elds we concluded that the sub eld (60 elds) normalized standards are most preferred.
4.2 Temporal analysis of research profi les dynamics
In this second application the classi cations of one institute at different points in time are compared. This makes it possible to detect pro le changes within an institute. Or if a country is investigated, a pattern in institutional pro le change could be detected and could reveal the effect of policy changes.
4.2.1 Brazil
This application was used in the paper on Brazilian institutes (Leta, Glanzel, Thijs, 2006; not included in this thesis). The pro les of Brazilian research institutes are quite different from the European institutes. This resulted in a different cluster solution with only ve clusters. In the paper this difference is explained: ‘Such differences are maybe due to the consolidate pro le of European scienti c institutes which are very specialized, especially in engineering &
exacts science and medicine. The high costs of equipment and logistic as well as the scarce quali ed manpower are certainly factors that reduce the chances of Brazilian institutes to carry on research projects in Medicine and Engineering’.
These are the groups:
• Cluster A: Phys/Chem/Eng – natural and technical sciences
• Cluster B: BioSci –biosciences
• Cluster C: Chem – chemistry
• Cluster D: Agri/Bio – agricultural sciences
• Cluster E: GenMed – medical research
1991-1994 1995-1998 2000-2003
Cluster A: (Phys/Chem/Eng.) 58 64 64
Cluster B: (BioSci) 19 22 17
Cluster C: (Chem) 5 7 12
Cluster D: (Agri/Bio) 22 17 18
Cluster E: (Genmed) 31 25 24
Total of institutes 135 135 135
Table 6. Number of Brazilian institutes within the 5 clusters in three different periods
The temporal analysis indicates that 78 institutes had a stable pro le over the three periods while 57 shifted from one cluster to another. Over the time, clusters D and E have “lost” some institutes while clusters C and A have gained institutes. Among the shifts, we highlight the following ones:
• From Cluster B to Cluster A: Federal University of Para and Federal Rural University of Rio de Janeiro. Both universities are well-known as their research projects in biological sciences. Our data, however, show a new tendency towards a research in natural and technical sciences
• From Cluster D to Cluster A: State University of Norte Fluminense.
Founded in the countryside of Rio de Janeiro state, this university was designed to be dedicated to agriculture projects mainly. Almost fteen years after its foundation, the new PhD programs changed its prior characteristics.
• From Cluster E to Cluster B: Ludwig Institute and National Institute for Cancer Research. Both institutes are devoted to cancer research and they are renowned by the strong association between clinical assistance and research. However, our data suggest that their current research is more related to biology issues (which includes for instance Biotechnology &
Applied Microbiology, Microbiology) rather than to medicine.
4.2.2 Europe
This application of the classi cation model on European research institutes has not yet been published. Some preliminary tests are done so far. We compared the classi cation of 1.176 institutes based their research pro le in 1994-1996 and in 2003-2005. The selected institutes all had at least 20 publications in both time frames; 85.3% of all institutes have a stable classi cation.
Classi cation based on 1994-1996 publications
Classi cation based on 2003-2005 publications
BIO AGR MDS GSS TNS CHE GRM SPM
BIO 67 3 4 2 2
AGR 9 34 1 1
MDS 4 2 359 1 12 6 2 10
GSS 39 1 1
TNS 16 1 140 3
CHE 3 2 28
GRM 4 59 16
SPM 4 1 15 47 277
Table 7. Comparison of classi cation of institutes based on research pro les in 1994-1996 and 2003-2005
When applied to all European institutes 2 major changes were found:
• A shift from a specialized cluster to a closely related cluster e.g. shifts between BIO and AGR or between GSS, TNS and CHE or between the medical clusters. (83 cases)
• Shifts from and to the multidisciplinary cluster. (80 cases).
Ten stranger cases of shift are observed. Five institutes shift from BIO/
AGR to the specialized medicine cluster and institutes in GRM/SPM move to the Biology cluster. One institute with a Geo classi cation changed to an Agriculture classi cation. A more in-depth analysis of these changes will subject of future studies.
4.3 Collaboration
A last application of the classi cation of institutions is in the paper about collaboration (see Part II, Chapter 5). Like publication and citation behavior also the collaboration between researchers or between institutes is strongly eld dependent. (Bordons et al. 1996; Gomez, et al., 1999). In this paper we used the classi cation of institutes in groups of likewise institutes to gain a better understanding of the role of elds on collaborative behavior and on the choice of partners of co-authored papers. Seven research questions were formulated, all of them looking at differences between groups.
• Does the share of collaborative papers in an institute differ among groups?
• Do citation indicators on sets of collaborative papers differ among groups?
• Are citation indicators of sets of co-authored papers higher than those on papers with no collaboration?
• Is there a difference between groups in improvement of citation based indicators?
• Do pairs of institutes that collaborate have a more similar research pro le than those pairs that do not collaborate?
• Do institutes that are more similar in research pro le collaborate more?
• Is there a difference in most preferred partners between the group?
In the rst part of the paper we found that there is indeed a difference between groups in choice of type of collaboration. Institutes in Geo and Space sciences have about 67% of all their papers jointly with a partner from abroad while for medical institutes this share is less than half. However, these medical institutes prefer much more extramural domestic collaboration in their research.
In a next section the in uence of collaboration on citation indicators was studied. Visibility of extramural co-authored research is higher than for papers with only intramural collaboration. Also on these indicators we found a deviating behaviour for the medical groups.
In the last section we tried to identify the research pro le of the most preferred partners. Collaborating institutes have indeed a pro le that is more alike, however the correlation between their pro le similarity and collaboration strength is rather weak. When looking at combinations of groups, it is clear that the institutes of the multidisciplinary cluster are the most preferred partners for all specialised groups. Also institutions of the same group or the closely related group are likely partners.
References
Bordons, M., Gomez, I., Fernandez, M.T., Zulueta, M.A., Mendez, A. (1996), Local, domestic and international scienti c collaboration in biomedical research, Scientometrics, 37 (2), 279-295.
Glänzel, W., Schubert, A.(2003), A new classi cation scheme of science
elds and sub elds designed for scientometric evaluation purposes, Scientometrics, 56 (3), 357–367.
Glänzel, W., Schubert, A., Thijs, B., Debackere, K. (2008), A new generation of relational charts for comparative assessment of citation impact, Archivum Immunologiae et Therapiae Experperimentalis, 86 (6), 373-379.
Glänzel, W., Thijs, B., Schubert, A., Debackere, K. (2009), Sub eld-speci c normalized relative indicators and a new generation of relational charts:
Methodological foundations illustrated on the assessment of institutional research performance, Scientometrics, 78 (1), 165-188.
Gomez, I., Fernandez, M.T., Sebastian, J. (1999), Analysis of the structure of international scienti c cooperation networks through bibliometric indicators, Scientometrics, 44 (3), 441–457.
Leta, J., Glänzel, W., Thijs, B. (2006), Science in Brazil. Part 2: Sectoral and institutional research pro les. Scientometrics, 67 (1), 87-105.
Appendix I 16 fi elds used for journal classifi cation
X. MULTIDISCIPLINARY SCIENCES X0 multidisciplinary sciences
A. AGRICULTURE & ENVIRONMENT A1 agricultural science & technology A2 plant & soil science & technology A3 environmental science & technology A4 food & animal science & technology
Z. BIOLOGY (ORGANISMIC & SUPRAORGANISMIC LEVEL) Z1 animal sciences
Z2 aquatic sciences Z3 microbiology Z4 plant sciences
Z5 pure & applied ecology Z6 veterinary sciences
B. BIOSCIENCES (GENERAL, CELLULAR & SUBCELLULAR BIOLOGY;
GENETICS)
B0 multidisciplinary biology
B1 biochemistry/biophysics/molecular biology B2 cell biology
B3 genetics & developmental biology
R. BIOMEDICAL RESEARCH R1 anatomy & pathology
R2 biomaterials & bioengineering R3 experimental/laboratory medicine R4 pharmacology & toxicology R5 physiology
I. CLINICAL AND EXPERIMENTAL MEDICINE I (GENERAL & INTERNAL MEDICINE)
I1 cardiovascular & respiratory medicine I2 endocrinology & metabolism
I3 general & internal medicine I4 hematology & oncology I5 immunology
M. CLINICAL AND EXPERIMENTAL MEDICINE II (NON-INTERNAL MEDICINE SPECIALTIES)
M1 age & gender related medicine M2 dentistry
M3 dermatology/urogenital system M4 ophthalmology/otolaryngology M5 paramedicine
M6 psychiatry & neurology M7 radiology & nuclear medicine M8 rheumatology/orthopedics
N. NEUROSCIENCE & BEHAVIOR N1 neurosciences & psychopharmacology N2 psychology & behavioral sciences
C. CHEMISTRY
C0 multidisciplinary chemistry
C1 analytical, inorganic & nuclear chemistry C2 applied chemistry & chemical engineering C3 organic & medicinal chemistry
C4 physical chemistry C5 polymer science C6 materials science
P. PHYSICS
P0 multidisciplinary physics P1 applied physics
P2 atomic, molecular & chemical physics P3 classical physics
P4 mathematical & theoretical physics P5 particle & nuclear physics
P6 physics of solids, uids and plasmas
G. GEOSCIENCES & SPACE SCIENCES G1 astronomy & astrophysics
G2 geosciences & technology G3 hydrology/oceanography
G4 meteorology/atmospheric & aerospace science & technology
E. ENGINEERING
E1 computer science/information technology E2 electrical & electronic engineering E3 energy & fuels
E4 general & traditional engineering
H. MATHEMATICS H1 applied mathematics H2 pure mathematics
S. SOCIAL SCIENCES I (GENERAL, REGIONAL & COMMUNITY ISSUES) S1 education & information
S2 general, regional & community issues
O. SOCIAL SCIENCES II (ECONOMICAL & POLITICAL ISSUES) O1 economics, business & management
O2 history, politics & law
U. ARTS & HUMANITIES U1 arts & literature U2 language & culture
