Mapping Research Fields Using Co-Nomination Analysis: the case of Heavy Flavour Physics

(1)

Proceedings of the 23rd International Conference on Science and Technology Indicators

All papers published in this conference proceedings have been peer reviewed through a peer review process administered by the proceedings Editors. Reviews were conducted by expert referees to the professional and scientific standards expected of a conference proceedings.

Chair of the Conference Paul Wouters

Scientific Editors Rodrigo Costas Thomas Franssen Alfredo Yegros-Yegros

Layout

Andrea Reyes Elizondo Suze van der Luijt-Jansen

The articles of this collection can be accessed at https://hdl.handle.net/1887/64521 ISBN: 978-90-9031204-0

This ARTICLE is licensed under a Creative Commons Atribution-NonCommercial-NonDetivates 4.0 International Licensed

(2)

Heavy Flavour Physics

Maria Karaulova^*, Maria Nedeva^** and Duncan Thomas^**

*maria.karaulova@manchester.ac.uk

All at Manchester Institute of Innovation research, The University of Manchester, United Kingdom M13 9SS

** maria.nedeva@manchester.ac.uk; Duncan.Thomas@manchester.ac.uk

Introduction

In this paper we present, and critically assess, the use of co-nomination analysis to delineate and map the intellectual, and collaboration, structures of the Heavy Flavour Physics (HFP) research field. Identifying and mapping research fields is not a trivial task and is currently approached primarily by using bibliometrics based techniques (Porter & Rafols, 2009; Lee, 2008). Here we posit that using a reputation-based approach building on snowball sampling, namely co-nomination analysis (Georghiou et al. 1988; Nedeva et al. 1996), can be successfully used to identify and map research fields. Furthermore, we argue that, using co- nomination analysis, while displaying a few methodological and resource related issues, overcomes some of the problems traditionally associated with bibliometrics.

Ever since advances in modern computing made large scale bibliometric analyses of research outputs possible, research policy, and the related fields of sociology of science and the study of science dynamics, have become increasingly reliant on metrics related to journal article metadata. Citation-based indicators, for instance, have become more integrated into formal research assessments, research funding allocation decisions, and individual level appointment and tenure decisions for academics. Citations are normally assumed to be a reliable proxy of research quality and impact (Tahamtan et al., 2016).

Bibliometrics based techniques, such as bibliometric coupling and co-citation analysis, are also used in the study of the dynamics of contemporary science to delineate research fields and map their intellectual, collaboration and communication structures (Porter and Rafols, 2009; Van Den Besselaar and Heimeriks, 2006; Klavans and Boyack, 2001). Blanket use of bibliometrics based techniques and methodologies may not be appropriate in fields with specific (understand deviating from the norm) authorship conventions and citing behaviours.

These incorrect assumptions in turn, produces spillover effects across research assessment methodologies. For example, Manganote et al. (2016) highlight how output and citation conventions in high energy physics (HEP) may significantly affect university ranking tables.

The limitations of citation-based methods are especially evident in research fields with unconventional publishing practices. High energy physics experimentalists, for instance, work

1 This work was supported by The Centre for Research Quality and Policy Impact Studies (R-QUEST) funded by

(3)

in very large collaborations at multinational research sites with large-scale infrastructure, like CERN. They frequently publish results in convention-based author collectives numbering hundreds or thousands. Some articles collect high number of citations but credit for them is not specifically attributable to individual authors, groups of authors, or even to host organisations. In this field, individual attribution of scientific credit is opaque to most popular citation-based techniques.

In this paper, we use co-nomination analysis, a reputation-based approach combining snowball sampling and social network analysis (SNA), to delineate the field of heavy flavour physics (HFP), a research field within high energy physics, and examine its intellectual influence and collaboration structures. This method, we posit, can substitute (and complement) citation-based metrics in mapping research fields, ascertaining sources, and distribution, of intellectual influence and capturing the cognitive and social dynamics of research fields.

This paper, we believe, makes three main contributions. First, we use co-nomination analysis to identify a research field by accessing it empirically through mapping, in real time, its knowledge network rather than inferring networks by the analysis of science artefacts (publications) that embody the past. Second, we use co-nomination to disambiguate the sources of intellectual influence in HFP and map the intellectual structure of the field.

Complementing data on large formal collaborations, we highlight whether formal leadership in these collaborations also exerts intellectual influence. Third, this co-nomination approach can supplement citation-based approaches to judge intellectual influence in fields where citing behaviour does reflect cognitive links between researchers. Limitations of citations are well known (Hicks, 1987; MacRoberts and MacRoberts, 1989). One of them is that only a certain type of intellectual link can be cited, and only the contribution of those community members who partake in writing research can be acknowledged. The benefit of co-nomination is that it provides insights into networks of intellectual influence in multiple stakeholder networks, or research networks where academic publications are not the main means of sharing research results.

Background: studying knowledge communities

The primary interest of this paper is in studying the structure and the dynamics of research fields as knowledge communities. Knowledge communities are a form of social organisation in science characterised by shared rules and standards. Knowledge communities are organised around coherent bodies of knowledge and operate, and persist through institutional mechanisms such as conferences, journals, facilities, datasets, methodologies, training arrangements etc (Nedeva, 2013). These can include members with different background to the extent to which they take part in the knowledge production processes (in some capacity) and subscribe under the established (but continuously negotiated) rules of the community.

Knowledge is a ‘crystallising agent’ around which epistemic communities are organised (Luukkonen and Nedeva, 2010). Quantitative analyses of codified text data, such as citations in publications, capture certain aspects of knowledge networks, but omit influences by non- publishing actors (e.g. practitioners, policymakers, engineers and technicians that design experimental equipment, citizens, research participants) whose co-creation may be recognised as a marker of quality.

Intellectually, the type of knowledge produced and exchanged within knowledge communities varies, but so does the social organisation of research. Crane (1969) wrote that scientific

(4)

networks in different research fields display different structural characteristics. We conceive of ‘scientific communities’ as a type of ‘knowledge communities’, with a broader membership, existing as a network of professionals characterised by shared set of normative beliefs (Haas, 1992), with resources exchanged internally, and organisational mechanisms to legitimise the field. Unlike networks, knowledge communities are developed and sustained around diverse exchanges, including information, resources and reputation.

Overall despite the prevalence and ubiquitous institutionalisation of quantitative methods of mapping science, we argue that in all cases citation data captures only part of a knowledge network: its intellectual organisation and, somewhat, social organisation. Additionally, knowledge communities with opaque traditions of intellectual attribution, like HFP, are difficult to map using these methods. Snowball sampling methods work better to capture the social organisation of the knowledge community and, in part, its intellectual organisation, thus complementing bibliometric techniques.

High Energy Physics and Heavy Flavour Physics

HEP aims to find proof for physics beyond the Standard Model and answers about fundamental questions about our Universe (Heuer, 2012). HEP is arguably the most expensive branch of public science in the world, with gigantic trans-national facilities, such as the Large Hadron Collider LHC) a necessary requisite for HEP experiments. HEP’s grand goals, resource intensity, and societal relevance (nuclear weapons were developed as a result of early HEP research) established it as the most ‘prestigious’ branch of science (also regarded as such by HEP physicists themselves, see Traweek, 1988). HEP as a discipline has been at times perceived as a ‘model’ scientific field to be replicated by other disciplines (Nelson, 2016). HEP is also internally hierarchical in terms of the size and prestige of its multiple experimental facilities around the world (e.g. CERN, DESY, SLAC, Fermilab).

Socially, HEP is divided into theoretical and experimental branches with distinct publication and citation patterns (Lehmann et al., 2003); the HFP research field echoes this pattern.

Theory groups in HEP are small, are not long-lived and are globally dispersed. Experimental groups are very large (up to many thousand researchers) and remain stable collectives over extended periods of time. Due to the scale of HEP research, the experimental branch has developed a formal system of decision-making, resource allocation and research process organisation (Canals et al., 2017; Engelen, 2012). In particular, large formal collaborations produce research articles with hundreds or thousands of authors, typically listed in a set, alphabetical order, making it is nearly impossible to discern intellectual contribution of individual authors or small groups (Birnholtz, 2006). Publication rates are high, where each formal collaboration can easily publish one article per week.

HFP is a research field within the broad domain of HEP. HFP shares many organisational and resource characteristics with other fields in HEP and is mainly distinguished by its intellectual specialisations (Gershon and Needham, 2015; Lambert, 2011). HFP concerns itself with detection and measurement of matter-antimatter symmetries in multiple types of fundamental particles, known as quarks. HFP does not have strict boundaries and its members may have secondary interests elsewhere in HEP.

Experimental HFP mainly concentrates within the LHCb (‘beauty’) collaboration in CERN, with 700 scientists from 66 organisations (CERN, 2017) and other experiments in CERN or elsewhere, whilst theory HFP is dispersed. However, HFP is a distinctive research field socially from HEP, as it has a separate ‘communication system’, including pre-print sharing,

(5)

specific workshops and identity (after Whitley, 2000). HFP’s publication conventions influence authors’ perceptions of: what it means to contribute to an article; how new researchers establish themselves in HFP; how recruitment and promotion decisions are made in organisations; how authority in the field accrues; and, most importantly, how researchers acquire and exert intellectual influence, e.g. in research resource allocation decisions, and in strategy and planning for future HFP experiments and equipment development. These conventions naturally preclude the use of bibliometric research tools to identify the structure of HFP and its sources of intellectual influence.

Research methodology: Co-nomination to map influence in HFP

Co-nomination sampling has been used to map knowledge networks for foresight and evaluation programmes to identify expert networks around certain topics (Nedeva et al., 1996). More recently, Fagerberg and Verspagen (2009) have used snowball sampling to study the emergence and development of the field of ‘innovation research’. Co-nomination has not typically been used to map research fields, at least since Libby and Zaltman mapped theoretical physics in 1967 (cited in van Raan, 2013, p.112), although co-nomination networks have previously been found to be plausible when compared with citation-based scores (Giusti and Georghiou, 1988).

Co-nomination is a snowball sampling method. Study of a knowledge network starts with selection of a small number of members of a field and asking them to nominate (‘cite’) other members who intellectually influence their research. An intellectually proximate relationship is assumed between nominees (see Figure 1).

Figure 1 Assumptions in Co-Nomination Sampling Method

In the subsequent round, nominees become nominators. Co-nomination sampling is iterative, with multiple rounds conducted until nominations converge. The frequency with which persons are cited is the assumed strength of their intellectual influence; persons who are co- nominated are assumed to be semantically linked. This enables the identification of clusters of intellectual influence, and links to external research fields. The resulting network of linkages is not dissimilar to the network resulting from author co-citation analysis, where author pairs are counted as being semantically linked if they are cited together in an article, regardless of the actual works that are cited (White and Griffith, 1981).

Co-nomination can be run as a short online questionnaire, and its results can be analysed quantitatively. Formulating co-nomination questions are key methodological decisions to map any research field. In this exploratory work, we test three types of questions to map HFP (see

(6)

Table 1). First we question direct intellectual influence, whereby the nominator nominates anyone intellectually influential without restriction. The second is a more focused ‘cognitive link’ question where the nominator nominates people socially proximate to them, whose opinions matter for the future development of the field (here, HFP). The first question reflects

‘citing’ behaviour; the second question explores future clusters of growth and the field’s social organisation. The third question asks for nominations of people influencing the technical shaping of research projects. These cognitive links are not often reflected in codified texts through citing behaviour but are important in research fields with complex infrastructure, like HFP/HEP.

Table 1 Co-Nomination Questions Asked in the Study

Co-Nomination Question Type of Link Studied

Q1 Can you please name up to five people that influence you the most intellectually at the moment?

Current direct intellectual influence

Q2 Can you please name up to five people with whom you most recently discussed a research problem you want to study?

Cognitive link, social organisation of the field

Q3 Can you please name up to five people with whom you recently discussed the technical design of your research project?

(think about one specific project)

Contributions to experiment design; the involvement of non- academics and non-researchers We also ask multiple ‘passport’ questions about the nominator and about the nominees, including current position, and year and place of PhD completion (if applicable).

The co-nomination sampling data collection process is depicted in Figure 2. Before the first round of co-nomination, the research started with an ex-ante characterisation of the HFP research field, validated through a scoping interview with a HFP researcher. We sampled the initial dataset by accessing participants lists from the two most recent sessions of three regular HFP-related research workshops (assumed to be attended by HFP field members, where research results are presented):

 CHARM workshop, on charm quark physics;

 BEAUTY workshop, on beauty quark physics;

 CKM workshop, the largest, on a broader theme of weak interactions in Cabibbo- Kobayashi-Maskawa (CKM) unitarity triangles.

E-mail addresses were accessed from publicly available sources.

Figure 2 Co-Nomination Process

The results obtained through co-nomination data collection have been compared with the results of author co-citation analysis of Heavy Flavour Physics.

(7)

Data Analysis

The co-nomination data was collected from October 2017 until March 2018 (data collection results can be accessed in Table 2). The network has been drawn based on the Force Atlas 2 layout: it pushes linked nodes closer to each other and draws unrelated nodes far apart (Cherven, 2013). Additionally, Johnson’s hierarchical cluster analysis based on the similarities of weighted average distances between nodes. We conducted two interviews with members of the research field to interpret the results of the co-nomination.

Table 2 Results of the Data Collection (co-nomination survey) The number of…

Invitations 1,479 Survey Responses 291

Response rate 19.68%

Nominations 2,823 Co-nomination pairs 3891

Author citation data was collected from the Web of Science on the 06 April 2018. A keyword- based Boolean search query was used to identify key research articles in HFP. The aim of the exercise was to maximise the precision of the retrieval and not recall.

A total of 2,439 items published in 2013-2018 (type ‘article’, language ‘English’, refined by the web of Science Category “Physics, Particles & fields”) were analysed. The data was cleaned and first author co-citation map was built for authors who were cited more than 10 times. The network has 1978 nodes (17% of all author) and 389,956 edges.

The analysis of co-nomination data was performed in the UCINET software. The visualisation of both networks and the analysis of the bibliometric network was performed in Gephi 0.9.2.

Results

Figure 3 and Figure 4 present the author co-citation network and the co-nomination network.

Figure 3 visualises co-nominations in response to Question 1 of the survey, as it is the most similar to author co-citation.

(8)

Figure 3 Author Co-Citation Network of Heavy Flavour Physics

(9)

Figure 4 Co-Nomination Network of heavy Flavour Physics (nominations for Question 1 of the survey; only 2 main components are visualised)

Author co-citation analysis highlights the prevalence of large collaboration in the intellectual landscape of heavy flavour physics research. However, it does not revel intellectual influence.

Rather, the network map shows general intellectual foundations of particle physics research in general: the biggest individual reference is to the developer of a software package used to analyse collision results; also some general references to major works in particle physics are among the most cited articles. None of the authors are actually heavy flavour physicists.

In contrast, the co-nomination network reveals the actual structure of intellectual influence in heavy flavour physics. The vast majority of nominated scientists, and all of the nominated scientists at the core of the network, work in heavy flavour physics. One theory physicist (in the pink cluster) emerged as the most intellectually influential figure in the field: this is because the propositions that this physicist made are being actively discussed and tested in experiments, providing the path in which the field may be developing in the future. Other major nodes (in the green cluster) are experimental physicists who occupy administrative positions of influence within their experiments: these are spokespeople and coordinators of the LHCb experiment at CERN.

The structure of the network reveals the role of theorists (the bottom half) and the experimentalists (the top half) in heavy flavour physics. The 10-cluster solution, which was endorsed in follow-up interviews, revealed groupings associated with the various formal CERN experiments among the experimentalists, and groupings based on intellectual orientations among the theorists. Theorists and experimentalists remain intellectually segregated.

(10)

The intellectual influence network of HFP shows that perhaps that pathways to intellectual influence differ in the experimental and in the theoretical parts of the same field; which may have implications for the incentives and strategies of scientists working in this field. The network also reveals related research fields in particle physics that are influential over the heavy flavour physics, such as Higgs physics, neutrino physics and the lattice gauge theory.

None of these fields, however, comes close to the core of the field, which is an indicator of a solid field identity.

Conclusions

Our approach re-invents co-nomination sampling as an innovative method to map research fields. It can serve as an alternative tool to citations to measure intellectual influence where the traditional methods demonstrate their shortcomings. It can also map intellectual influence in hybrid fields that do not consist just of academics publishing in peer-review journals.

Potentially it can map fields where books are the main outputs, and fields where research relies on grey literature and reports. The technique can also map intellectual influence around issues that go beyond research, for example, policy processes requiring input from experts and stakeholders, as well as research scientists.

The shortcomings of the technique involve the sample and the response bias, as with all survey-based methodologies. The results are representative of the network structure, yet the peripheral nodes are inaccurate. The popularity of certain scientists may be inflated by the hypes and fashions in the field (for example, about the future direction of the field) and therefore not durable in time. Finally, the method is expensive and takes a significant amount of time. However, these limitations reflect the broader methodological limitations of chain- referral sampling. We conclude that using co-nomination illuminates aspects of intellectual influence in research fields that is otherwise undiscoverable by other available methodologies.

References

Birnholtz, J.P., 2006. What does it mean to be an author? The intersection of credit, contribution, and collaboration in science. J. Am. Soc. Inf. Sci. Technol. 57, 1758–1770.

Canals, A., Ortoll, E., Nordberg, M., 2017. Collaboration Netwoks in Big Science: The Atlas Experiment at Cern. Prof. Inf. 26, 961–971.

CERN, 2017. LHCb Collaboration URL https://home.cern/about/experiments/lhcb (accessed 2.12.18).

Crane, D., 1969. Social Structure in a Group of Scientists - Test of Invisible College Hypothesis. Am. Sociol. Rev. 34, 335–352.

Delfanti, A., 2016. Beams of particles and papers: How digital preprint archives shape authorship and credit. Soc. Stud. Sci. 46, 629–645.

Engelen, J., 2012. The Large Hadron Collider project: organizational and financial matters (of physics at the terascale). Philos. Trans. R. Soc. -Math. Phys. Eng. Sci. 370, 978–985.

Fagerberg, J., Verspagen, B., 2009. Innovation studies—The emerging structure of a new scientific field. Res. Policy 38, 218–233.

(11)

Gershon, T., Needham, M., 2015. Heavy Flavour Physics at the LHC. Comptes Rendus Phys.

16, 435–447.

Giusti, W., Georghiou, L., 1988. The use of co-nomination analysis in real-time evaluation of an R&D programme. Scientometrics 14, 265–281. https://doi.org/10.1007/BF02020079 Haas, P.M., 1992. Introduction: epistemic communities and international policy coordination.

Int. Organ. 46, 1–35.

Heuer, R.-D., 2012. The future of the Large Hadron Collider and CERN. Philos. Trans. R.

Soc. -Math. Phys. Eng. Sci. 370, 986–994.

Hicks, D., 1987. Limitations of Co-Citation Analysis as a Tool for Science Policy. Soc. Stud.

Sci. 17, 295–316.

Lambert, R.W., 2011. Heavy Flavour in a Nutshell. ArXiv11051061 Hep-Ex Physicshep-Ph.

Lehmann, S., Lautrup, B., Jackson, A.D., 2003. Citation networks in high energy physics.

Phys. Rev. E 68, 026113.

Luukkonen, T., Nedeva, M., 2010. Towards understanding integration in research and research policy. Res. Policy, Special Section on Government as Entrepreneur 39, 674–686.

MacRoberts, M.H., MacRoberts, B.R., 1989. Problems of Citation Analysis: A Critical Review. J. Am. Soc. Inf. Sci. N. Y. N 40, 342–349.

Manganote, E.J.T., Schulz, P.A., de Brito Cruz, C.H., 2016. Effect of high energy physics large collaborations on higher education institutions citations and rankings. Scientometrics 109, 813–826.

Mele, S., Dallman, D., Vigen, J., Yeomans, J., 2006. Quantitative analysis of the publishing landscape in high-energy physics. J. High Energy Phys. 2006, S01.

Nedeva, M., 2013. Between the global and the national: Organising European science. Res.

Policy 42, 220–230.

Nedeva, M., Georghiou, L., Loveridge, D., Cameron, H., 1996. The use of co-nomination to identify expert participants for Technology Foresight. RD Manag. 26, 155–168.

Nelson, R.R., 2016. The sciences are different and the differences matter. Res. Policy 45, 1692–1701.

Nilsen, V., Anelli, G., 2016. Knowledge transfer at CERN. Technol. Forecast. Soc. Change 112, 113–120.

Seidel, R.W., 2008. From Factory to Farm: Dissemination of Computing in High-Energy Physics. Hist. Stud. Nat. Sci. 38, 479–507.

Tahamtan, I., Safipour Afshar, A., Ahamdzadeh, K., 2016. Factors affecting number of citations: a comprehensive review of the literature. Scientometrics 107, 1195–1225.

Traweek, S., 1988. Beamtimes and Lifetimes. Harvard University Press.

(12)

van Raan, A.F.J. (Ed.), 2013. Handbook of Quantitative Studies of Science and Technology.

Elsevier.

White, H.D., Griffith, B.C., 1981. Author cocitation: A literature measure of intellectual structure. J. Am. Soc. Inf. Sci. 32, 163–171.

Whitley, R., 2000. The Intellectual and Social Organization of the Sciences. Oxford University Press.