Representing Gender in Conceptual Spaces

(1)

Representing Gender

in Conceptual Spaces

(2)

Layout: typeset by the author using LA_TEX.

(3)

Representing Gender in Conceptual

Spaces

An investigation of the construction and possible gender bias in

conceptual spaces

Iris Luden 11330872

Bachelor thesis Credits: 18 EC

Bachelor Kunstmatige Intelligentie

University of Amsterdam Faculty of Science Science Park 904 1098 XH Amsterdam Supervisor dr.M.A.F Lewis

Institute for Logic, Language and Computation Faculty of Science

University of Amsterdam Science Park 907 1098 XG Amsterdam

(4)

Abstract

Conceptual spaces is a theory about modelling concepts geometrically, which can be applied to natural languages. It provides an alternative to other natural language processing methods such as word embedding which have proven to contain biases. This thesis focuses specifically on gender bias in natural language representation. Conceptual spaces is a relatively explainable way of representing concepts of natural language and may therefore gain more insight into the biases present in text cor-pora. This research uses the method of Derrac & Schockaert of constructing a conceptual space from a corpus of Wikipedia pages of females and males. Three conceptual spaces have been constructed from this corpus; one of females, one of males and one of both combined. Using multi-dimensional scaling, a support vector machine and clustering methods, the salient directions of both the male and female data set have been determined. The properties of the three constructed conceptual spaces were analysed and compared with gender biases of the implicit association test. The female data set showed more salient terms related to arts, and contained more explicit naming of gender. This indicates possi-ble adaptions of gender bias from the corpus. Future research needs to be done about the influence of imbalanced data sets on these differences.

(5)

Introduction

Natural language processing (NLP) is a field in artificial intelligence focused on interpreting human lan-guage to allow computers and humans to interact. A common approach in NLP is to represent words as vectors, commonly called word embeddings. Influential algorithms for building these word embed-dings include GloVe and Word2Vec (Pennington, Socher, & Manning, 2014; Mikolov, Sutskever, Chen, Corrado, & Dean, 2013). These methods map texts onto vectors of real numbers, and make use of co-occurrence matrices, dimensionality reduction and/or neural networks to extract meaning from texts. However, these methods are mainly unsupervised and turn out to contain biases in their representation of natural language, especially in gender related concepts (Bolukbasi, Chang, Zou, Saligrama, & Kalai, 2016; Ní Loideáin & Adams, 2018; Leavy, 2018; Caliskan, Bryson, & Narayanan, 2017).

For example, when an algorithm based on a Word2Vec representation of language is asked to fill in the missing word for a sentence such as “Father is to doctor as mother is to (...).”, the algorithm will fill in ‘nurse’ (Bolukbasi et al., 2016). Even though the answer ’nurse’ is logically possible, it is not necessarily the right answer. A mother can be a doctor just as a father can be a nurse . The reason for this bias is that machine learning algorithms learn through training data. If the training data contains stereotypi-cal concepts of gender, algorithms will perpetuate this bias (Leavy, 2018). When algorithms reproduce normative assumptions about roles of men and women in society, this can have a wider social effect of perpetuating gender bias (Ní Loideáin & Adams, 2018). Solutions proposed to avoid or remove these biases from word embedding algorithms include altering algorithms with brute force by removing or replacing representations of gender related words (Bolukbasi et al., 2016).

Another problem with word embeddings is that they lack explainability. The meaning of these lan-guage representations are not explicit and the validation of these representations is not clear. This makes it difficult to explain how biases are embedded in these representations of language. This is problematic because this makes it impossible to know how algorithms make the choices that they make. Because al-gorithms can perpetuate biases, it is important that they are transparent and explainable as to how they produce these biases. When there is transparancy, it is easier to prevent these biases. Therefore this re-search will focus on a more explainable of representing concepts of natural language, namely conceptual spaces.

Conceptual spaces theory was introduced by Peter Gärdenfors in his book Conceptual Spaces: The ge-ometry of thought to provide a more explainable and meaningful representation of concepts of natural language. The idea of conceptual spaces is to define entities using meaningful dimensions so that en-tities can be compared along these meaningful dimensions (Gärdenfors, 2004). A conceptual space is a metric space which can be used to decode the meaning of natural language concepts and properties. These metric spaces are high-dimensional, but the number of dimensions is relatively low in comparison to other NLP methods. Each dimension corresponds to a ‘primitive cognitive feature’; these dimensions are called quality dimensions. Specific entities correspond to points in the conceptual space, while con-cepts and properties are represented by convex regions within this space (Gärdenfors, 2004).

This research will explore to what extent biases are reflected in a conceptual spaces approach for rep-resenting concepts of natural language, specifically for gender-related terms. As we will discuss, concep-tual spaces is a more explainable approach than conventional NLP, and might therefore gain more insight into how the biases are embedded in conceptual spaces. Throughout this thesis gender concepts will be discussed. The concept of ‘gender’ will be approached in a linguistic matter, and not in sociological or biological matters. This research does not wish to claim that this concept is in any way a definitive or adequate idea. The term ‘concept’ is used within the framework of the conceptual spaces theory.

(7)

The exact implementation of Gärdenfors’s conceptual spaces theory in AI is still undetermined; there exists no conventional method for realizing a conceptual space. This leaves much room for different kinds of implementations. Therefore it is extremely relevant in the field of conceptual spaces to exper-iment with implementing the theory. The dimensions of a conceptual space and the positioning of en-tities in it need to be defined. Theoretically, this could be done qualitatively by deciding on the most important features and properties of a concept. This would however be very labour-intensive and ineffi-cient. A more evident method would be to empirically or quantitatively define the important features of a concept. Derrac & Schockaert introduce such a method in the paper Inducing semantic relations from conceptual spaces: A data-driven approach to plausible reasoning (Derrac & Schockaert, 2015). The pa-per illustrates how conceptual spaces can be induced from large text corpora, and how semantic relations can be derived from these conceptual spaces. The benefit of this approach is that it allows extracting se-mantic relations from large corpora in a relatively unsupervised manner. Moreover, this is one of the first substantial attempts to realize conceptual spaces. However, since the method makes use of a corpus of textual data, the algorithm might still be trained by biased data. This raises the question to what extent these biases are reflected in the constructed conceptual space. Since the distinction between a difference and a bias is sometimes difficult to make, we will compare the results to other previously determined bi-ases by the implicit association test (IAT) (Caliskan et al., 2017; White & White, 2006). This thesis uses a binary conception of gender, that is to say of male and female. But undoubtedly, gender can be seen as a spectrum rather than a dichotomy. The reason for using a binary approach of gender is that a clear distinction between the two is made in the English language.

In this research, we use the method of Derrac & Schockaert to build conceptual spaces for three data sets: one data set of men, one of women, and one of men and women combined. For this purpose we col-lected and cleaned a corpus of Wikipedia pages of Dutch people. To inspect whether biases are refcol-lected using conceptual spaces theory, this research will compare the characteristics of the three conceptual spaces. The focus lies on the construction of a conceptual space for females and males, and the differ-ences and similarities that these conceptual spaces show. This yields the following research question of this thesis:

Using the approach of Derrac & Schockaert to construct a conceptual space of males and fe-males, what are the key differences between the two conceptual spaces?

Answering this question is relevant for several reasons. First of all, research on conceptual spaces is still in its infancy. The possibilities for representing normative concepts such as gender in conceptual spaces have yet to be explored and evaluated. However, it promises to be a valuable method because of its explainability. Since conceptual spaces is more explainable than current methods for representing language in AI, this method might gain more insight into how bias are being taught to the algorithm. Moreover, practically constructing a conceptual space and experimenting with implementations is valu-able to this relatively young field. This way this research contributes to this specific field of conceptual spaces.

The first section will discuss the theoretical foundations for this research. Gärdenfors’s conceptual spaces theory, and how this theory is more explainable than other natural language processors will be explained. Then we will describe how this theory has been put into practice by Derrac & Schockaert, and discuss the potential, capabilities and drawbacks of this approach. Next we will address the issue of biases in natural language processors. Section 2 will explain how the data set of Dutch Wikipedia pages was collected and cleaned. The method, section 3, will denote the steps that were taken to construct the three conceptual spaces. Finally, we will analyse the differences between the female and male conceptual spaces with regard to possible bias.

(8)

1 Literature Review

1.1 Gärdenfors’ Conceptual Spaces

Conceptual spaces is a theory on how concepts of human language can be represented, modelled and how different concepts can be combined. The theory is a framework based on cognitive science, phi-losophy, linguistics and and artificial intelligence developed by Peter Gärdenfors in his book Conceptual Spaces, a geometry of thought (Gärdenfors, 2004). A conceptual space is used to encode the meaning of natural language concepts into a model which could be used for computation and analysis. The main idea is to represent concepts and entities in a metric space to allow comparisons of similarity and differ-ence between entities or between concepts.

First we will explain briefly what the approaches are used so far in artificial intelligence, what their limitations are and why conceptual spaces might resolve some of these constrains.

1.1.1 Symbolic and connectionist approaches

Historically, approaches of aritficial intellicence (AI) can be divided into two categories: a symbolic ap-proach and a connectionist apap-proach. Symbolic apap-proaches are built on the assumption that operations can be done by the manipulation of symbols. Symbolic models are based on a set of predicates with a denotation. Operations can be done on these predicates, based on rules of logic and syntax. The mod-els can formulate an output by executing logical operations on the input symbols. The model has been prescribed what operations to make for every possible input. The main advantage of this approach are that it is explainable; the all computational operation of the model could be explained and described. However, when it comes to unfamiliar input or problems, symbolic models are not prepared. In other words, symbolic models are bad at learning.

Connectionist models are trained on large data sets and use statistics to make conclusions about them. Neural networks are an example of this; they are presented with some input and learn based on statistical theory. These methods are good at learning and categorisation tasks, but since they do not follow clear logic, they are much less explainable. Such models do not grant an explanation on what basis two concepts differ. Instead they merely return similarity values between two entities (Derrac & Schockaert, 2015; Gärdenfors, 2004).

Focusing in on the field of computational linguistics in particular, a similar distinction can be seen between modelling word meanings logically such as montague semantics or using distributional or neu-ral models of meaning (Muskens, 1996; Gärdenfors, 2004, 2014). The aim of the conceptual spaces ap-proach is to make the representation and interpretation of concepts more explainable than connection-ist approaches, but more extensible than symbolic approaches (Gärdenfors, 2004). Conceptual spaces prescribes a structure and rules for interpreting concepts, and at the same time it presents methods for learning and categorizing new concepts. Therefore constructing spaces are relevant for representing normative concepts.

1.1.2 What is a conceptual space?

A conceptual space is a metric space consisting of quality dimensions which are organised into domains. These metric spaces are high-dimensional, in which these quality dimensions correspond to ‘primitive cognitive features’. Specific entities correspond to points in the conceptual space, while concepts and properties are represented by convex regions within this space (Gärdenfors, 2004; Derrac & Schockaert,

(9)

2015). The distance between any two points in the metric space should be defined. Examples of possi-ble distance metrics are the Euclidean distance or the taxicab distance. Defining a meaningful distance metric allows to constitute a similarity measurement between objects (Gärdenfors, 2004).

Figure 1: An example of conceptual space (Derrac & Schockaert, 2015, p. 67)

Consider figure 1. This is a schematic example of a 2-dimensional conceptual space for vehicles. Recall that each entity is represented as a point in the geometric space, denoted by a vector with a value for each dimension. Representing entities as vectors with meaningful dimensions allows comparisons between two points. Particularly, when computing the similarity between two point using a distance metric, the outcome is substantial because we can argue why the two points are similar. For example, when two points in space have a high similarity value along one dimension, say technological advance, we can argue that these two points are similar because they are both at the same level of "technologically advancedness". This is different from vector-based models such as Word2Vec, for which we can say that vectors are similar along particular dimensions, but we don’t know what those dimensions mean. The conceptual spaces representation on the other hand, is capable of explaining on what basis two entities differ from each other.

Depicting entities as regions also enables the notions of overlap and mutual exclusiveness, member-ship, prototypes and betweenness. The location of a point within a region allows the notion of mem-bership; a conceptual space could be designed in such a way that the points located within a specified region all member of a certain concept. Two points could be compared based on whether they are lo-cated in the same region. Moreover, an entity lolo-cated in the middle of a region (the centroïd) could be interpreted as the prototype of this region. In order for a region to represent a natural property, a region should be convex. A convex region is a region which does not have any interior angles greater than 180°. This means that when two objects are located in the same region representing a property or concept, all other objects located between those two objects should be located in the same region as well. Suppose object x and y are both located in region A, and object z is located between x and y, then z should also be located in region A (Gärdenfors, 2004). This allows the notion of betweenness, since we can argue with certainty that if a car and a bike are both vehicles, then a motorbike must also be a vehicle given that it is located between a bike and a car. Similarly, we can explain why a motorbike should be located between

(10)

a bike and a car: "A motorbike is something between a bicycle and a car" can be explained by the mean-ingful dimensions, as a motorbike is more technologically advanced than bicycles, and somewhat more environmentally friendly than most cars.

1.1.3 Implications of conceptual spaces

Representing entities in a conceptual space allows us to explain and compare entities along meaning-ful dimensions. Conceptual spaces are also capable of portraying the characteristics and structures of a concept. For instance, Gärdenfors’s conceptual spaces approach has the advantage of being able to distinguish borderline instances of a concept from more prototypical instances. Moreover, represent-ing properties and concept as regions enables the notion of the possibility of two overlapprepresent-ing concepts, and concepts which are mutually exclusive. These distinctions can not be made by word-embeddings which represent natural language terms as points or vectors with arbitrary dimensions. These methods allow similarity measurements between terms, for example by calculating the cosine similarity between two vectors, but they are not capable of explaining the exact nature of the (dis)similarity of two terms (Derrac & Schockaert, 2015). This makes conceptual spaces a more rich and an explainable method than state-of-the-art word embeddings.

Whereas conceptual spaces offer more explainability, it is important to notice that this is not the same as an objective representation of language. This contraint goes for all NLP methods because the mean-ing of words an concepts is neither definite nor neutral. Therefore it is difficult to determine whether the representation of language is true, objective and legitimate. These questions belong to the domain of philosophy of language. This is relevant for this research, because gender concepts are inherently normative. This research does not have the pretense to treat them in a non-normative, objective man-ner. The construction of conceptual spaces is particularly relevant with regard to the normative aspect because it may elicit how these norms are constructed. The next section will explain the method that Derrac & Schockaert (2015) propose to construct a conceptual space. The same method will be used in this research.

1.2 The method of Derrac & Schockaert

The article Inducing semantic relations from conceptual spaces: a data-driven approach to plausible rea-soning demonstrates a method for constructing a conceptual space and determining its quality dimen-sions (Derrac & Schockaert, 2015). The paper illustrates how conceptual spaces can be induced from large text corpora, by representing documents as vectors. These vectors are constructed by approximat-ing the significance of the terms which occur in each document. A metric space can then be induced by applying multi-dimensional scaling to these vector representations. Next they propose an unsupervised method to derive the salient directions of the space to explain the semantic relations between objects located in this space, using a support vector machine (SVM) (Derrac & Schockaert, 2015).

This method will be used in this thesis so we will summarize the method in detail in this section. First we will discuss the main steps proposed by Derrac & Schockaert:

• Composing a vector representation for each document

• Dimensionality reduction using multi-dimensional scaling (MDS) • Separating entities using a support vector machine (SVM)

(11)

• Determining the salient directions using clustering methods.

Lastly we will review the explainability and address the implications and drawbacks of this approach. 1.2.1 Creating a vector representation for each document

Since the aim is to create a conceptual space in which the documents are represented as points, the first step is to compute and create a vector representation of each document. Given that each entity corresponds to a text document, Derrac & Schockaert (2015) first calculate a bag-of-words vector repre-sentation of each document. The bag-of-word method assumes that the meaning of a document can be expressed by the terms which occur in its text, and discards the word order of the text. Suppose the entire corpus consists of T unique terms, then each document vector is of length T. Each entry of the document vector is the count of a unique term tjin a specific document di: c(di,tj). The bag-of-words vector~vBOW_di

for any document di is denoted as:

~vBOW_di =¡c(di,t1), c(di,t2), c(di,t3), ...c(di,tT)

¢

Solely taking the count of terms in a text document does not yet provide a meaningful representation of the document. Suppose that a term has a high frequency in a document: this seems to imply that the term is also relevant to the document. But when this term has a relatively high frequency in every document of the corpus, the descriptive strength is lower than when the term rarely occurs in the rest of the corpus. Similarly, when a term has a relatively low occurrence in a specific document, but the term does not occur at all throughout the rest of the corpus, this implies that the term is rather characteristic for the document concerned. Moreover, some documents contain more terms than others, which makes comparisons between document vectors unbalanced. To address this, they compute the positive point-wise mutual information measure (PPMI) for each pair of terms and documents.

The PPMI value measures how strongly a term is associated with a document. The PPMI measure is a probability measure of association between two random variables. It assumes independence be-tween both random variables and estimates the degree of correlation based on their individual and joint distributions. The PPMI value for term t and document d is given by:

PPMI(t , d ) = max µ 0, log pd t p_{d ∗}× p∗t ¶ (1) where pd tis the probability of t term t co-occuring with document d, pd ∗is the probability of document

d and p_∗t is the probability of term t, all assuming independence:

pd t= c(d , t ) P d0Pt0c(d0, t0) (2) p_{d ∗}₌X t0 pd t0 (3) p_∗t₌X d0 pd0t (4)

The final vector representation for a document is contains all PPMI values for each unique term t ∈ T : ~vP P M I_di=¡PP M I(di,t1), P P M I(di,t2), P P M I(di,t3), ...P P M I(di,tT)

(12)

1.2.2 Multi-dimensional scaling

Derrac & Schockaert use multi-dimensional scaling to reduce the number of dimensions of the vector representations. There are two reasons for mapping the document vectors onto fewer dimensions. The first is that a vector consisting solely of word counts and weights is very large; the vector length of each document vector will have the length of the total amount of terms occurring in the corpus. In this re-search, with about two thousand documents with an average length of 573 words, this number comes down to about 50,000. Many of the entries of a document vector will equal zero, since not all terms in the corpus occur in each document.

Secondly, just because a term does not occur in a document, does not necessarily mean that the term is not relevant to the document. To illustrate this, consider a document about Annie M.G. Schmidt, a Dutch writer of children’s books. In the corresponding document, the term ‘book’ has not been men-tioned, but only the terms ‘booklet’, ‘volume’ and ‘fiction’ are in the text. This does not necessarily imply that the term ‘book’ is not relevant to the document. Hence the PPMI vectors the documents do not cover all the relevant information.

Reducing dimensionality smooths representations so that documents containing ‘book’ become sim-ilar to documents containing ‘booklet’, ‘volume’ or ‘fiction’. To reduce dimensionality, Derrac & Schock-aert (2015) use multi-dimensional scaling (MDS). This method can be used to map high-dimensional data onto lower dimensional data while preserving the distances and dissimilarities between the objects of the high-dimensional space (Groenen & van de Velden, 2005). Since the distance between two points should be a meaningful metric in conceptual spaces, MDS was preferred over other methods of dimen-sionality reduction such as SVD and PCA because MDS generates an n-dimensional Euclidean space in which the distances and dissimilarities are preserved. In this space, each document can be associated with a point in such a way that the distance between two points also approximates the dissimilarity be-tween two points (Derrac & Schockaert, 2015, p. 72). This way, the data that is being scaled down keeps similar properties as the high-dimensional data from which it is extracted (Kruskal, 1978; Groenen & van de Velden, 2005). MDS takes a dissimilarity matrix as input. A dissimilarity matrix contains all the dissimilarity values between each combination of two documents. Derrac & Schockaert (2015) use the normalized angular distance measure to calculate the dissimilarity between two document vectors. Note that this (dis)similarity measure calculates how strongly two documents are associated, whereas the pre-viously used PPMI values were used to calculate how strongly a single term is associated with a single document. Derrac & Schockaert (2015) use MDS to scale down to 20, 50 and 100 dimensions.

(13)

1.2.3 Finding directions using support vector machines

As Derrac & Schockaert state, ‘Interpretable directions should correspond to natural language terms’ (Derrac & Schockaert, 2015, p. 74). Therefore they collect all nouns, adjectives, noun phrases and adjec-tive phrases from the corpus. These terms and phrases are then used to separate the entities in space using a linear support vector machine.

Figure 2: A 2-dimensional example of a sepa-rating hyperplane using SVM. The dashed lines are the support vectors. The color of each point represents its class. The blue arrow orthogo-nal to the hyperplane represents the direction of the term concerned.

A support vector machine (SVM) is a supervised learning model which can be used for classification tasks. Given a set of entities which are represented as points in a geometrical space, the SVM estimates a hyperplane which separates these points into two classes (Derrac & Schockaert, 2015; Croft, Metzler, & Strohman, 2010; Joachims, 1998; Cortes & Vapnik, 1995). Given a term phrase, the entities can be split into two classes: a positive class, (documents that contain the term or phrase), and a negative class (documents that do not). The entities are classified according to what side of the hyperplane they are located in the ge-ometrical space.

The hyperplane is defined by maximising the mar-gins of the separating hyperplane. This means that the distance of the closest negative point to the hyperplane plus the distance of the closest positive point to the hy-perplane is maximized (Croft et al., 2010). The margins are depicted as dashed lines in figure 2. Once a separat-ing hyperplane is found, Derrac & Schockaert (2015) in-terpret the vector orthogonal to this hyperplane as the direction of the term within the metric space. The di-rection of the term is depicted as a blue arrow in the example in figure 2.

1.2.4 Selecting salient directions

The performance of the SVM is measured using Cohen’s kappa score, because this measure takes into account the class imbalance. Cohen’s kappa value describes how representative the observed accuracy value is of the performance compared to the expected accuracy, where the expected accuracy is the per-formance supposing that the classifier labels completely randomly (Viera, Garrett, et al., 2005). Particu-larly for an unbalanced division of classes, this random chance is of great influence. For example, given a class imbalance of 90% to 10%, an accuracy value of 90% could be achieved by simply labeling all points with the larger class. By taking into account the random chance in unbalanced classes, Cohen’s kappa value is more representative performance metric than the accuracy value.

The terms or phrases for which the Cohen kappa scores were at least 0.5 were considered as salient directions. Let this be the set T0.5, and let TSbe the set of selected salient terms. The salient terms were selected with the following steps:

step 1: Select the term with the highest Cohen’s kappa score as salient direction. Call this term t1, and its

corresponding vectorv~t1. Now t1is the only item in the set T

S

(14)

step 2 Iteratively select the next salient term as follows: For each ti ∈ T0.5, and each tj ∈ TS, calculate

the cosine similarity betweenv~tiandv~tj. Then select the term tiwhich minimises the maximum

value of cosine similarity with any of the already selected salient terms as the next salient term. So the next salient term tiis the term t minimizing maxj <icos(v~j,v~i). In other words, select the term

which is least similar to all other previously chosen salient terms.

Derrac & Schockaert (2015) decided to select twice as many salient terms as directions of the metric space, because some of the directions might be linearly dependent.

Lastly, all terms with a Cohen’s kappa score of at least 0.1, call this set T0,1were divided into clusters based on the cosine similarity of the direction of each term. Each term was divided into the cluster of the salient terms to which it was most similar. Then the directions for the salient terms were updated by defining their direction as the centroïd of all terms in their cluster.

In the follow-up research by Schockaert ‘Learning Conceptual Spaces with Disentangled Facets’ (2019), they address how the features found by Derrac & Schockaert (2015) can be grouped into semantically meaningful facets. The notion is made that the clustering method used leads to semantically thematic clusters. For example, the terms with a similar theme such as ‘zombie’, ‘scary’, ‘horror’, are expected to be grouped together (Alshaikh, Bouraoui, & Schockaert, 2019). In this research we expect to obtain clusters of terms with semantic themes as well.

1.2.5 Implications of Derrac & Schockaerts’ method

Derrac & Schockaert create a spatial representation of concepts, in which they find meaningful direc-tions along which they can compare entities. It is one of the first extensive attempts to make the abstract theory of conceptual spaces concrete. Still, the salient directions and semantic relations are derived from large text corpora. This implies two important issues. First of all, the method quantitatively calculates the document vectors, which can be seen as a form of word embedding. The approach also includes unsupervised techniques such as MDS. This raises the question to what extent this specific implemen-tation of Derrac & Schockaert remains explainable. Secondly, using a large corpus to train an algorithm has consequences for the objectivity of the representation of language. This implies that this approach of representing terms in a conceptual space is still not necessarily free of bias. The meaning of texts can be subjective and fluent over time. Moreover, the meaning or definition of a concept can be normative as well. Derrac & Schockaert do not address the problem of normativity of concepts, nor the problem of data-subjectivity in their research. This is why this research aims to inspect how bias is reflecting using the approach of Derrac & Schockaert for constructing conceptual spaces.

To emphasize the problems that come from neglecting normativity and subjectivity of language, the next section will delve deeper into the biases within the field of natural language processing.

1.3 Gender bias in machine learning

This section will explore the existing biases in current implementations of natural language processing in AI. We will first set out which biases are currently recognised to be intertwined in machine learning representations of language. Then we will discuss some possible techniques to detect those biases in representations of natural language in AI. We will do this by focusing on two papers about detecting gender bias in AI. Then we will discuss the origin of the reflection of gender bias in AI. At last we will hypothesize on the possibilities to bypass such biases in representing language.

(15)

1.3.1 Acknowledged bias in machine learning

Caliskan, Bryson & Narayanan (2017) observed that the GloVe vector representations of words contain stereotypical representations of concepts in the English language. To confirm this, they looked at the stereotypical topics previously found by the Implicit Association Test (IAT) to inspect the presence of bias in the GloVe representation of language.

In the IAT, test subjects are asked to respond to the words they are presented with on a screen by pressing on of two keys on their keyboard. For example, one of the keys is intended for females, and the other for males. The IAT measures the difference in response time of their test subjects for combinations of topics. This test is based on the assumption that when people strongly associate two topics with each other, their response time is shorter than when they are proposed with two topics that do not have an association. For example, if women are expected to be more associated with the word ‘teacher’ than men, the response time for categorizing words related to ‘teacher’ with female names is expected to be lower for female names than for male names (White & White, 2006; Caliskan et al., 2017).

The stereotypes found by the IAT concern strong associations between what Caliskan, Bryson, & Narayanan call ‘morally neutral’ topics such as ‘flowers’ and ‘instruments’ as well as ‘more problematic’ topics such as ‘gender’ and ‘race’ (2017). Caliskan, Bryson, & Narayanan (2017) used statistics to measure association between the vector representations of words in order to see whether these stereotypes are re-flected in the algorithm (Caliskan et al., 2017). First the ‘morally neutral’ topics "flowers v.s. insects" and "weapons v.s. musical instruments" were revised. The GloVe representation of language proved to as-sociate ‘flowers’ and ‘instruments’ with ‘pleasant’, while ‘insects’ and ‘weapons’ with ‘unpleasant’. Then they use the same technique to demonstrate that machine learning representations of language absorb stereotyped biases for what they call ‘more problematic’ topics such as gender and race. Similarly, Eu-ropean names proved to be significantly more associated with pleasant terms while African American names are more associated with unpleasant terms. Caliskan, Bryson & Narayanan (2017) demonstrated that in the GloVe representation of words female names are more associated with ‘family’ and ‘art’ than male names, while male names are more associated with the terms ‘career’, ‘science’ and ‘mathematics’ (Caliskan et al., 2017).

Caliskan, Bryson, & Narayanan (2017) point out that the word embeddings "know" these properties of flowers, insects, musical instruments, and weapons without having any direct experience of the world, and with no other representation of semantics than the (implicit) metrics the co-occurrence statistics of words. From this, they conclude that their results of strong associations between the stereotypical topics imply that machine learning technologies may perpetuate cultural stereotypes (Caliskan et al., 2017, 9. 3). Likewise, Bolukbasi et al. (2016) found that the Word2Vec algorithm contains strong biases when it comes to male and female related terms. Bolukbasi et al. (2016) propose a method to recognise bias in word embeddings, specifically for Word2Vec. They do this by calculating the distances between the vec-tor representations for male and female terms with an analogous meaning. With this approach they have found that the Word2Vec algorithm contains many logically incorrect associations between semantically gender-neutral words and a specific gender. For example, the term ‘queen’ is semantically correctly asso-ciated with the term ‘woman’, but the association of ‘nurse’ or ‘receptionist’ with ‘woman’ is logically and semantically incorrect. They also observed implicit sexism in the vector representations by examining ‘she/he analogies’ between gender associated words. For example, the Word2Vec model comprehends that a man is related to a computer programmer the same way as a woman is related to a home maker. They find these analogies using a vector calculation of the form:

−−−→

(16)

where ‘−−−−→wordi’ is the vector representation of word i.

Examples of ‘she/he analogies’ Bolukbasi et al. (2016) encountered were: nurse-surgeon, cosmetics-pharmaceuticals, sewing-carpentry, lovely-brilliant and cupcakes-pizzas (Bolukbasi et al., 2016). It was discovered that gender neutral words are linearly separable from gender definition words in the word embedding vector space. They use this knowledge to adjust the vector space in order to remove these biases from the word embeddings (Bolukbasi et al., 2016). They call this ’hard-debiasing’; they remove certain properties of the vector space after it has been constructed.

1.3.2 How gender bias is reflected in text corpora

There are several reasons why machine learning algorithms echo cultural biases. First of all, most algo-rithms are trained on real data. Any data set is collected and composed by humans and therefore in-evitably contains properties which are results of decisions which have implicitly or explicitly been made in the process of creating the data set. Secondly, differences, stereotypes and biases are entwined in written and spoken natural language documents (Leavy, 2018). Leavy (2018) explains how algorithms trained on (English) text corpora are prone to adapt gender biases. Five of the mentioned causes for biases in English text corpora which are relevant to this thesis are enlisted below.

1. The under-representation of women in corpora

Mentions of individual men, as distinct from mentions of men as a general category, occur signif-icantly more often than mentions of individual women. The amount of times a reference is being made to a woman in a text is smaller than the amount of times a male is referred to in a text (Leavy, 2018).

2. Explicitly naming of properties differ for men and women: Bias can be recognised in language when certain properties are explicitly named for one gender, while for the other gender it is not even required to name these properties because they are given. For example, ‘family man’ is a commonly used phrase while ‘family woman’ is equivalently used. Similarly, phrases such as ‘sin-gle mum’, ‘working mother’ and ‘career woman’ which are commonly used in the media reveal social preconceptions of women, since the terms ’single dad’, ‘working father’ and ‘career man’ are generally not expressed (Leavy, 2018).

3. Naming gender in occupations: When using occupation-words for which one gender is naturally more associated, a gender specification is often made. For example: ‘female lawyer’, ‘woman judge’, and ‘male nurse’. This identifies their existence as counter to the societal expectations (Leavy, 2018).

4. Men are more frequently described in terms of their behavior while women were described in terms of their appearance and sexuality (Leavy, 2018). Moreover, some adjectives are exclusively used to describe one of the two genders and there are many adjectives which are strongly associ-ated with one of the two genders. This was concluded from an investigation on a large corpus of the BNC (British National Corpus) (Pearce, 2008). These adjectives can be seen in figure 3.

5. Women are described as ‘girls’ more often than men are described as ‘boys’. The term ‘girl’ is 3 times more likely than the term ‘boy’ to refer to an adult and that women were described as girls in in order to characterize them as immature, innocent, of youthful appearance, subordinate status, emotionally weak or financially dependent (Leavy, 2018).

(17)

Lastly, Leavy (2018) argues that artificial intelligence will reflect the values of its creators. An issue Leavy points out is that developers of AI are ’overwhelmingly male’ (Leavy, 2018, p. 14). At the same time, Leavy stresses that "leading thinkers in the emerging field addressing bias in artificial intelligence are primarily female, suggesting that those who are potentially affected by bias are more likely to see, understand and attempt to resolve it" (Leavy, 2018, p. 14). Leavy concludes from this that diversity in the area of machine learning, both in training sets as well as in developing teams, is essential to avoid gender bias. Taking into account the origin of these biases in applications of AI, Leavy argues that rather than producing biased algorithms and removing them after they have been programmed and trained, the creators of the algorithms should focus on preventing generating biases in algorithms in the first place. Therefore Leavy proposes a simple quota system for the gender balance in training data because this way the biases are prevented bottom-up (Leavy, 2018).

This research focuses on constructing a conceptual space using the method of Derrac & Schockaert in the first place. Secondly, this research will evaluate which differences and which possible biases between men and women are reflected using conceptual spaces. For this, the five points of Leavy summarized above and the scientifically proven biases by the Implicit Association Test (IAT) will be used as tools to examine possible biases in our conceptual spaces.

Figure 3: Adjectives used to describe males and females retrieved from the BNC (Pearce, 2008, p. 14) (a) Adjectives used exclusively to describe either males or females

Female Bossy, chattering, gossiping, submissive, bitchy, hysterical, weeping

Male Gregarious, cautious, affable, amiable-looking, avuncular, funniest, good-natured, jovial, likeable, mild-mannered, personable, cruel, dour, indifferent, insensitive, insuffer-able, braver, humane, law-worthy, patient, sincere, toler-ant, trusted, trustworthy, truthful, upstanding, anxious, in-sane, astute, scholarly, self-educated, ignorant

(b) Adjectives with a strong association with males or females Female promiscuous, spirited, vivacious, glad, grateful,

dissatis-fied, distraught, mad, neurotic, silly, satisdissatis-fied, resourceful, strong-minded, daft, dependent, dumb

Male Eminent, influential, powerful, humble, charming, consid-erate, funny, happy, arrogant, cruel, dangerous, evil, hate-ful, violent, earnest, faithhate-ful, loyal, generous, good, reason-able, thoughtful, mad, scared, sensitive, upset, sane, bril-liant, clever, gifted, learned, rational, retarded

(18)

2 Building the data set

A data set of Dutch Wikipedia pages was used to train the algorithm proposed by Derrac & Schockaert for composing a conceptual space. This data set was chosen because the approach requires descriptive texts on specified topics, and Wikipedia provides this. Furthermore, Wikipedia is a platform with recent data which is regularly updated. Wikipedia pages about Dutch people have been collected and cleaned, giving 4229 candidate texts. Pages of people who were born before 1900 were excluded. Pages of people of whom the gender or birth year was unknown have been removed, and pages with too short a description, i.e. of fewer than 45 words, have been excluded as well. This leaves us with 2058 relevant Wikipedia pages for this research.

2.1 Collecting the data

The Wikipedia pages of Dutch people were imported from the Wikipedia article "List of Dutch people". This is a dynamic list which, according to Wikipedia, ’may never be fully complete, or may require con-stant updates to remain current’. Wikipedia is an open platform to which every user is allowed to make adjustments and additions, and for this reason not all information in this list might be accurate. This page also redirects to other lists of Dutch people which are more specified, such as "List of Dutch women artists", " List of Dutch Sportspeople" and "List of Dutch Frisians". The URLs from a total of 18 lists have been imported and stripped. This has been programmed using the ’Beautiful Soup 4’ library for pulling data out of html and xml files (Richardson, 2007). This resulted in a data set of roughly 5000 URLs. Most pages could be easily identified by reviewing the extracted birth years of the documents, assuming that these pages are less likely to contain a birth year in the description. Finally, a last manual check has been made to verify that the data set consisted of solely pages of people. The distribution of the Wikipedia pages by birth year and gender can be reviewed in figure 4 below.

(19)

2.2 Cleaning the data

For extracting the texts from the html files of the Wikipedia pages, the Beautiful Soup library1was used (Richardson, 2007).

Before removing all punctuation from the text, two specific elements have been examined in order to preserve the original words in the text. This was the case for words with hyphens or quotations. For example, the word "don’t" is one for which we wish it to remain unchanged, while in a sentence such as: "The term ‘football’ was used since 1899", we want the apostrophes to be removed. The same accounts for hyphens. In other words, quotations and hyphens positioned at the beginning and end of words were removed, and the ones in the middle of a words were preserved. The next step was to split the text into sentences. Sentences in brackets have been saved as a separate additional sentence in order to preserve the original sentence structures. After this, all remaining punctuation has been removed and all capital letters have been replaced with lower cases.

In cleaning the texts, some complications were encountered. Abbreviations such as M.Arch or B.Arch (short for Master of Architecture and Bachelor of Architecture), are being transformed into the words ‘march’ and ‘barch’. These words are therefore transformed into words with a different meaning. A solu-tion for this has been explored, but an appropriate alternative was not found. Nonetheless these issues did not cause any complications since the amount of times such a transformation occurs is negligibly small.

2.3 Selecting relevant documents

The usage of pronouns such as ‘he’, ‘she’, ‘him’, ‘his’ and ‘her’ in the first two paragraphs of the Wikipedia descriptions have been examined in order to extract each person’s gender. This resulted in a gender classification for 3949 out of the 4229 documents. Because of the size of the data set, the accuracy of this method of classification has not been fully reviewed. Instead, a random sample of 50 documents has been selected to determine whether this method resulted in a correct classification of the gender for each document. Out of the 50 inspected documents, all were correctly classified. Since we are interested in examining the differences in gender, only the documents for which the gender was known have been included.

Secondly, the data set has been restricted to people who were born after 1900. The reason for this is that we are interested in examining how males and females are being depicted the 20th and 21st century. Conveniently, out of the 1733 pages of people who were born before 1900, only 191 were female. Remov-ing this group of people born before 1900 also helps reduce the gender imbalance in the data set. The Wikipedia pages of people for which the birth year was unknown have been excluded from the data set as well. The last restriction was to discard all documents with too limited a description. All documents with a description of less than forty-five words were discarded. This is because texts with such a short description seldom involve more information than the birth year, birth place and profession. The final data set consists of 2060 documents of which 619 were classified as female and 1439 as male.

(20)

Figure 5: Amount of words and sentences of each document

(21)

3 Method

3.1 Corpus analysis

For this section and the results sections we use the following terminology:

t term (word)

d document (set of words)

Cf the set of female documents

Cm the set of male documents

Nf Number of documents in the female corpus

Nm Number of documents in the male corpus

The mixed data set will simply be referred to as C . The method has been applied to all three data sets (Cf, Cmand C ) in exactly the same manner.

3.1.1 Creating a dictionary of the corpus

A dictionary of all terms in each corpus has been created. The dictionary excludes terms which contain numbers and terms of only one character. Stop words, retrieved from the NLTK library (Loper & Bird, 2002)2, were excluded as well since these are words which occur in all documents and therefore do not contribute to the meaning of a document. The dictionary for the female corpus (Cf) will be named Df,

and for the male corpus (Cm) will be named Dm.

3.1.2 Tf-idf measure

In order to gain more insight in the significant terms of the corpus, the ifidf measure (Term Frequency -Inverse Document Frequency) was calculated for each term in view of the relative relevance of each term. The tf-idf measure is a method to measure how important a term is for a given document. Given our terminology, the tf-idf measure can be calculated as follows:

tf-idf = t f × i d f t f (t , d ) = c(d, t)

length of d

i d f (t ) = N

Number of d with t

tf is the count of a term in a single document. Every tf value needs to be normalized to compensate for the fact that some documents are larger than others. This can be done by dividing each count by the document length. The idf should be low for terms with high occurrence throughout all documents of the corpus.

(22)

3.1.3 Collecting phrases

Not only the single words of the corpus have been analysed, but also the noun and adjective phrases which occur in the corpus have been collected. This was done using a Regex Parser and a part-of speech tagger from the NLTK library (Loper & Bird, 2002)3. The grammar used to tag the sentence structures of noun and adjective phrases are the following:

AD J : < RB > ∗ < J J > +

N P : < AD J >?(< N N S > | < N N >)+

Figure 7: Example of a parse tree

3.2 Computing a vector representation for each document

This section explains how the vector representation of each document was constructed. First, a bag-of words representation was computed for each document. Then the PPMI-measure was used to extract the strength of association between each term and document. Then a dissimilarity matrix was created using the cosine dissimilarity measure, calculating the strength of association between each combination of two documents. This dissimilarity matrix was created in order to do a dimensionality reduction using multi-dimensional scaling.

3.2.1 Bag-of-words vector

Recall that each entry of the bag-of-words vector represents one term in the corpus:

~vBOW_d1 =¡c(d1,t1) c(d1,t2) . . . c(d1,tT)¢, where c(dj,ti)denotes the count of term i in document j. For

each document, this vector was constructed by counting the number of times each term occurred in the document.

3.2.2 PPMI-measure

As explained in section 1.2.1, the bag-of-words vectors with word counts on each entry, need to be trans-formed into vectors consisting of P P M I values. The PPMI vectors of a document di contains the PPMI

values for each term tj∈ T : ~vP P M I_di =¡PP M I(d1,t1) P P M I(d1,t2) . . . P P M I(d1,tT)¢.

For computational purposes, the bag-of-words vectors have been placed on rows to form a matrix of shape (N x T), where N is the number of documents of the corpus, and T is the number of unique terms

(23)

of the corpus. Call this matrix BOW . This bag-of-words matrix can be transformed into a PPMI matrix (P P M X ) with some basic operations.

BOW =       c(d1,t1) . . . c(d1,tT) c(d2,t1) . . . c(d2,tT) .. . . .. ... c(dN,t1) . . . c(dN,tT)       =       ~vBOW_d1 ~vBOW_d2 .. . ~vBOW_dN       7−→ P P M X =       ~vP P M I_d1 ~vP P M I_d2 .. . ~vP P M I_dN       (5)

This was efficiently computed using the equation (6) below.

P P M X = max¡0,log(BOW ® E)¢ (6)

where BOW is the bag-of-words matrix containing the observed values denoted in equation (5), and E is the matrix containing the expected values. The operator ® is defined as the Hadamard division, which is the element-wise division of the two matrices of the same shape. Matrix E is calculated by taking the outer product of the total

E =~r ⊗~c Σ (7) Σ = N X i =1 T X j =1 c(di,tj) (8)

whereΣ is the sum of all entries of matrix BOW , and~r, ~c are the vectors containing the sums of the rows and columns of BOW respectively:

~r =       PT j =1c(d1,tj) PT j =1c(d2,tj) .. . PT j =1c(dN,tj)       (9) _{~c = ¡P}N i =1c(di,t1) PN i =1c(di,t2) . . . PN i =1c(di,tT) ¢ (10)

Note that~r is simply an array containing the document lengths, and ~c an array containing the total term frequencies. Vector~r is of shape (N × 1) and vector ~c is of shape 1 × T , which makes matrix E indeed of shape (N × T ), the same as BOW . Note that each row of both the BOW and the PP M I matrices is the vector representation of a specific document. The number of unique terms T of the corpus defines the length of each bag-of-words vector representation for each document. The vector lengths of were therefore 22826, 40275 and 49145 for the female, male and mixed data sets respectively.

3.2.3 Multi-dimensional scaling

To do a multi-dimensional scaling on all document vectors, we fist needed to calculate a dissimilarity matrix. In this research the normalized angular distance was used to define the dissimilarity of two doc-ument vectors. For any two vectors vi, vj, the angle can be calculated using:

∠(~vi,~vj) = 2 π× arccos µ ~_v i· ~vj ||~vi|| · ||~vj|| ¶ (11) In our case, the angular distances have be calculated between the PPMI vector representations of~vP P M I

all documents. ∠(~vP P M I_di,~vP P M I_{d j}) = 2 π× arccos Ã ~v P P M I_di·~vP P M I_{d j} ||~vP P M I_di|| · ||~vP P M I_{d j}|| ! (12)

(24)

The dissimilarity matrix d M used as input for MDS was constructed as follows. For readability purposes the document vectors~vP P M I_di are abbreviated as simply vdi.

d M =            ∠(~vd1,~vd1) ∠(~vd1,v~d2) . . . ∠(~vd1,~vdN) ∠(~vd2,~vd1) ∠(~vd2,v~d2) . . . ∠(~vd2,~vdN) ∠(~vd3,~vd1) ∠(~vd3,v~d2) . . . ∠(~vd3,~vdN) .. . . .. ... .. . . .. ... ∠(~vdN,~vd1) . . . ∠(~vdN,~vdN)            (13)

Since the angle of a document vector to itself is always equal to zero, the diagonal of this dissimilarity matrix consists of zeros. Moreover, since_∠(~vdi,~vdj) =∠(~vdj,~vdi), the dissimilarity matrix is symmetric.

The dissimilarity matrices used for MDS were of shape N × N , where N is the number of documents, which is (619 × 619), 1439 × 1439 and 2058 × 2058 for the female, male and mixed data sets respectively. Multi-dimensional scaling was done using the function from the sklearn module4using the standard values for all parameters. In this research the dissimilarity matrix has been reduced to 20 components. We will name the matrix of reduced dimensionality M X _20. Note that each row still represents one spe-cific document. The rows of M X _20 are thus the vector representations of the documents. To illustrate this, consider figure 8 below where each row consists of 20 entries and represents a single document.

M X _20 =       20 × N       =         ~vd1 ~vd2 ~vd3 .. . ~vdN        

Figure 8: Schematic depiction of the output of MDS. Each document vector~vdiis now of length 20

4_{Scikit learn software package (Pedregosa et al., 2011).}

(25)

3.3 Determining salient directions using SVM

A support vector machine (SVM) with a linear kernel has been used to discover the meaningful directions of the created metric space. For this the standard parameters of the skl ear n.svm package were used (Pedregosa et al., 2011). The linear SVM has been used to separate entities with the term from the entities without the term. A SVM takes the document vectors and a class vector as input. The class vectors for a term is constructed as follows: cl ass(ti) = [I (ti, d1), I (ti, d2), . . . , I (t1, dN)], where:

I (t , d ) := (

1 if t in d

0 otherwise (14)

The SVM returns a hyperplane which separates the entities with the term from the entities without the term. The direction of the term in the metric space corresponds to the direction of the vector perpen-dicular to this hyperplane. Below in figure 9 is a 2-dimensional example of how the SVM separates the entities in space using a hyperplane. These figures were constructed from a sample of 250 documents from the female data set, where the dimensionality was reduced to two dimensions using MDS. In the simplified version of 2 dimensions, the hyperplane is denoted by a line, and the support vectors are de-noted by the dashed lines in the figure below. The blue arrow represents the direction in the space of the salient term. The brown points are the positive cases and the blue points the negative cases.

Figure 9: Using SVM to find a separating hyperplane (a) Politician Performance Cohen’s Kappa: 0.7975054 Accuracy: 0.9022556 Precision: 0.8173913 Recall: 0.9494949 (b) Olympics Performance Cohen’s Kappa: 0.6284000 Accuracy: 0.8721805 Precision: 0.5479452 Recall: 0.9756098

For finding salient directions, only nouns, adjectives noun phrases and adjective phrases were con-sidered. The most frequently occurring terms and phrases are listed the appendix. Derrac & Schockaert

(26)

(2015) use a threshold of a document frequency of at least 100 documents. This means that they use SVM to find the directions for only the nouns, adjectives, noun phrases and adjective phrases that occur in at least 100 documents. However, the threshold for the minimum document frequency of the consid-ered terms was chosen so that the amount of terms was approximately 23000, and therefore arbitrary. For this thesis all terms and phrases that occur in at least 1, 5% of the documents and in at most 60% of the documents have been collected. The first reason for this is that the intention is to find terms which can be interpreted as meaningful directions. Consider a data set of only Dutch people; the term ‘dutch’ will likely occur in most documents, but the fact that the term is relevant to most documents makes the term ‘dutch’ not relevant (and therefore meaningless) in this space. Secondly, when we try to separate points with the SVM using terms which occur in nearly all documents, the hyperplane may obtain the best results by returning all documents, which results in a hyperplane that does not separate the entities in space at all.

The performance of the SVM was measured using Cohen’s kappa score, precision, accuracy, recall and the F-measure. The salient terms and their directions in the conceptual space were selected from the set of terms with the highest Cohen’s kappa score. For the female and mixed data set the threshold was a minimum of a Cohen’s kappa score of 0.47, and for the male a minimum of 0.5. The threshold was lowered with 0.03 for both the female and mixed data set because the threshold of 0.5 resulted in an insufficient amount of salient candidates for the female data set5.

3.3.1 Selecting salient terms and corresponding clustering

The salient terms and their clusters have been elected using the iterative method described in section 1.2.4. The pseudo-code for the procedure of selecting salient terms is as follows: Suppose s1, ..., s2Dare

the selected salient terms. For each salient term, we associate a cluster, respectively namedS1, ...,S2D.

All terms with a Cohen’s kappa score of at least 0.1 were divided into the selected salient directions, let this set of terms be called T0.1. Each cluster contains all terms t ∈ T0.1which are most similar to the corresponding salient direction according to cosine similarity. Figure 10 is an illustrates how the terms are being assigned to a salient directions. The vector of the salient direction was then redefined as the centroïd of the vectors of the terms in their cluster:~v∗_t

i=

1 |Ci|

P

t ∈Ci~vt.

Figure 10: How term are being assigned salient directions. The blue and red lines are the salient direc-tions, and the pink and light blue lines the directions of the terms of T0.1

5_{For the female data set, the threshold of at least a Cohen’s kappa score of 0.5 resulted in 39 candidate terms, while the threshold}

0.47 resulted in 54 candidate terms. For the mixed data set the threshold of 0.5 resulted in 35 candidate terms, while the threshold of 0.47 resulted in 47 candidate terms.

(27)

Listing 1: Pseudocode Selecting salient directions 1 def s e l e c t _ s a l i e n t (T0.5, D ) : 2 ’ ’ ’T05 _{is s o r t e d f r o m h i g h e s t to l o w e s t K a p p a s c o r e ’ ’ ’} 4 c a n d i d a t e _ t e r m s = t e r m s of T0.5 5 f i r s t _ s a l i e n t = t e r m of T0.5 w i t h h i g h e s t K a p p a s c o r e 7 add f i r s t s a l i e n t to s e l e c t e d s a l i e n t 8 r e m o v e f i r s t s a l i e n t f r o m c a n d i d a t e _ t e r m s 10 # s e l e c t t w i c e as m a n y s a l i e n t t e r m s as d i m e n s i o n s 11 for i in r a n g e 2* D : 13 m a x _ s i m i l a r i t i e s = [] # i n i t i a l i z e l i s t of s c o r e s 15 # c a l c u l a t e h i g h e s t s i m i l a r i t y s c o r e for c a n d i d a t e s 16 for ( c a n d i d a t e in c a n d i d a t e _ t e r m s ) : 18 s i m i l a r i t i e s = [] # i n i t i a t e l i s t of s i m i l a r i t i e s 20 # S i m i l a r i t y b e t w e e n c a n d i d a t e and all s a l i e n t t e r m s 21 for s e l e c t e d in s e l e c t e d _ s a l i e n t : 23 add c o s s i m( c a n d i d a t e , s e l e c t e d ) to s i m i l a r i t i e s 25 # S a v e m a x i m u m s i m i l a r i t y 26 add max( s i m i l a r i t i e s ) to m a x _ s i m i l a r i t i e s 28 ’ ’ ’ s e l e c t c a n d i d a t e w i t h l o w e s t max s i m i l a r i t y 29 to all p r e v i o u s l y s e l e c t e d s a l i e n t t e r m s ’ ’ ’ 30 n e w _ s a l i e n t = c a n d i d a t e of min( m a x _ s i m i l a r i t y ) 31 add n e w _ s a l i e n t to s e l e c t e d _ s a l i e n t 33 r e t u r n s e l e c t e d _ s a l i e n t

Listing 2: Pseudocode clustering salient directions

1 def c r e a t e _ c l u s t e r s ( s a li en t , s a l i e n t _ v e c t o r s _ 0 1 ) : 2 3 for c in T0.1: 4 5 for s in s a l i e n t : 6 c a l c u l a t e c o s s i m( c , s ) 7 8 a s s i g n c to m o s t s i m i l a r s a l i e n t v e c t o r 9 10 for e a c h c l u s t e r : 11 s a l i e n t v e c t o r = c e n t r o i d of a s s i g n e d c l u s t e r m e m b e r s 12 13 r e t u r n c l u s t e r s 14

(28)

3.4 Detecting differences and gender bias in conceptual spaces

3.4.1 IAT target words

We inspected the differences between the salient directions of the male and female conceptual spaces. The previously acknowledged gender biases in natural language representations explicated in section 1.3.2 were used as a tool to investigate the differences between the male and female conceptual space.

We speculated whether these differences could be interpreted as biases by comparing them to the results of the IAT. Table 1 shows the target words used in previous IAT tests. We inspected the occurrence of the topics and their target words in the male and female conceptual spaces, and observed whether there terms occurred as salient terms or in the clusters. If the words of one topic occur in one of the conceptual spaces, and not in the other, this may indicate that the conceptual spaces have adapted biases from the corpus. If the target words do not occur more often in one of the conceptual spaces than the other, this may indicate that bias has not evident in the conceptual spaces, or at least not noticeable using this approach.

Topic Associated words

Family family, home, parents, children, cousins, marriage, wedding, rela-tives, garden, kitchen, marriage, laundry

Career career, office, manager, salary, job, briefcase, profession, employees, executive, management, professional, corporation, business

Mathematics math, algebra, geometry, calculus, equations, computation, num-bers, addition

Arts poetry, art, dance, literature, novel, symphony, drama, sculptures, Shakespeare

Science science, technology, physics, chemistry, Einstein, NASA, experiment, astronomy, math, biology, geology, engineering

Liberal Arts history, arts, humanities, english, philosophy, music, literature Male terms male, man, boy, brother, he, him, his, son, father, uncle, grandfather,

husband

Female terms female, woman, wife, girl, sister, she, her, hers, daughter, mother, aunt, grandmother

(29)

3.4.2 Unequally sized data sets

Addressing point 1 mentioned section 1.3.2, the under representation of women in corpora, we inspected the influence of the under representation of females in the mixed conceptual space. Since there were more than twice as many male documents, a check was done by reducing the male data set to the num-ber of female documents. This was done by randomly selecting 619 documents out of the 1439 male documents. A new mixed conceptual space was also constructed out of the two equally sized data set. The salient directions were defined in the same manner as described in the methods, for both the re-duced male data set and of the mixed data set (now consisting of 1238 documents). Since the results differ based on what random subset of the male data set was selected, we ran this process 15 times. Out of the 15 runs, the 40 salient terms of the male and and mixed conceptual space were calculated.

This was done to compare which salient directions the mixed conceptual space adapted from the female and male documents, when using an equal number of male and female documents opposed to using an unequal number of male and female documents. The salient terms and the overlap between the three conceptual spaces are depicted in figure 12 in the results section. Results of the performance of the SVM can be reviewed in the Appendix.

3.4.3 Other differences between males an females.

Furthermore, we inspected whether there is a sign of the other biases explicated in section 1.3.2 by Leavy (2018).

Point 2 was that properties that are generally associated with one of the two genders, are made explicit when they are about the opposite gender, e.g. "family dad", "working mother". Point 3 was that gender indications are explicitly made when naming someone’s occupation, when this persons gender is not associated with the occupation, e.g. "female lawyer", "male nurse". These two points will be examined by focusing in on the usage of gendered-terms. For this we used the gender specific target words used in the IAT to indicate the male and the female gender, which can also be seen in table 1.

To address point 4, about the difference in adjectives used to describe men and women, we inspected the prevalence of the terms of figure 3 in the documents and in the conceptual spaces. This figure showed which adjectives were more strongly associated with females and males according to the research of (Pearce, 2008). Lastly, point 5 says that women are 3 times more described as girls than men are as boys. We inspected whether this was the case in our corpus, and whether this was reflected in the conceptual spaces.