An Intensional Approach in Understanding the Latent
Variable Model
University of Amsterdam
Riet van Bork
August 14, 2014
Abstract
This paper examines the implicit assumptions underlying latent variable models. To do so, this paper introduces a distinction between the intension and extension of variables. The intension of a variable is a function while the extension consists of the outcomes of this function applied to individuals. A conceptual framework is built upon this distinction to obtain a better understanding of the latent variable model. An observed variable is defined as the data pattern with its corresponding intension. The observed variable measures the latent variable that caused this data pattern. Based on the framework two psychometric questions are discussed. First, it is discussed what it entails for two variables to be identical. Variables are identical when they have identical intensions. Second, the question of validity is defined as whether the intended variable is identical to the variable that explains the shared variance of the multiple observed variables, theta. When the variable theta corresponding to test X, and the intended variable have the same intension, we may conclude that test X is valid with respect to the intended variable. The distinction between the intension and extension of variables has led to a fruitful framework that may be used in future research to answer key psychometric questions.
Introduction
In Psychology, constructs (e.g., intelligence) are studied through the observation of behaviour (e.g., administering an IQ test). Empirics indicate that multiple measures of the same construct (e.g., multiple IQ tests) positively intercorrelate. This phenomenon is known as the positive manifold (Van Der Maas et al., 2006). To explain the correlation among these observed variables, it is assumed that these variables are caused by one and the same underlying construct (e.g., scores on different IQ tests are all caused by the underlying construct intelligence). Latent variable analyses are analyses in which a latent (i.e., unobserved) construct is introduced to explain the correlation between multiple manifest (i.e., observed) variables. The conceptual framework of latent variable analysis originates in the work of Spearman (1904), who abstracted the ‘general intelligence’ factor (g-factor) from different branches of intellectual activity.
𝜀1
Latent variable
Manifest variable 1 Manifest variable 2 Manifest variable 3 Manifest variable 4
𝜀2 𝜀3 𝜀4
Figure 1: Latent variable model.
“all branches of intellectual activity have in common one fundamental function (or group of functions), whereas the remaining or specific elements of the activity seem in every case to be wholly different from all that in all the others.” - Spearman, 1904, p. 284
So according to Spearman, there is one fundamental function that explains the shared variance in all branches of intellectual activity. By ‘remaining or specific elements’, Spearman is suggesting that the variance in the observed variables that is not explained by the common factor does not correlate. This idea is fundamental to latent variable models: conditioning on the latent variable will render the observed variables statistically independent (Borsboom et al., 2003).
In latent variable analyses, the latent variable is assumed to cause all shared variance among the manifest variables. Put differently, the latent variable is seen as the ‘common cause’ of these manifest variables, see Figure 1. Currently, finding a relation between measures of various abilities is seen as a well-established empirical method to provide evidence for a common causal factor, underlying these measured abilities (Jensen, 1998).
The use of latent variable analysis is widespread in psychological research. The ‘Big-Five factor structure’ of personality, for example, adopts five latent factors to explain correlations in 80 observed traits (McCrae & Costa, 1987). These five dimensions (Neuroticism, Extraversion, Openness, Agreeableness and Conscientiousness) are universally used to describe personality. The idea behind such a theory is that, for example, the observed traits ‘spontaneous’ and ‘pas-sionate’ are understood as being caused by the underlying factor ‘extraversion’, while the observed traits ‘nervous’ and ‘vulnerable’ are understood as being caused by the factor ‘neuroticism’.
Psychopathology is dominated by latent variable models as well. Indeed, most theories con-sider psychiatric disorders as latent variables that cause the observed symptoms. For example, the latent variable ‘depression’ is said to cause the observed symptoms ‘insomnia’ and ‘fatigue’. But even at this level correlations can be found between the disorders. In the extreme, Caspi et al. (2014) abstract an even more general factor from these different disorders, named the p-factor. This general psychopathology dimension is supposed to underlie all mental disorders, just like the g-factor (i.e., general intelligence, described above) is supposed to underlie all branches of
intellectual activity.
As a result of the frequent use of latent variable analysis in psychology, many psychologi-cal theories consist of unobservable theoretipsychologi-cal entities (Borsboom et al., 2003). The question is, however, what is implied by seeing observed behaviour as caused by a stratified model of underlying factors. The consequences of this approach differ across research fields.
In cognitive psychology, for example, the latent variable model of intelligence (Spearman, 1904) resulted in an extensive research tradition that focuses on finding a neural basis for general intelligence (e.g., Colom, Jung, & Haier, 2006; Duncan et al., 2000; Gray, Chabris, & Braver, 2003; Haier, Jung, Yeo, Head, & Alkire, 2004). Consequently, group differences in intelligence test results will be understood as a physical difference between such groups.
To provide another example, in clinical psychology latent variables describe psychological disorders (e.g., depression) as latent variables that cause the symptoms (e.g., insomnia and fatigue). Consequently, research focuses on the aetiology of psychiatric (latent) disorders instead of on the observed symptomatology (Cramer, Borsboom, Aggen, & Kendler, 2012). The impact of the latent variable perspective within clinical psychology is not limited to research but influences treatment as well. The common cause approach implies that a patient will not fully recover when only symptoms are tackled, but that one should rather try to tackle the ‘essence’ of a disorder.
In the aforementioned examples, factor analysis is used as a statistical analysis to abstract a latent factor. Factor models are a subset of latent variable models, in which both the manifest and latent variables are believed to be continuous variables (Borsboom, 2008). Other latent variable models are (1) Item Response Theory (IRT) models, in which a continuous latent variable is measured by dichotomous manifest variables; (2) Latent Class Models, in which a categorical latent variable is measured by categorical manifest variables; and (3) Mixture Models in which a categorical latent variable is measured by continuous manifest variables.
The abundant use of latent variable analyses in psychology asks for a thorough understanding of what is implied by underlying common factors. It is important to note that the latent variable approach makes assumptions that are left implicit and raises questions that are left unanswered. This paper scrutinizes some of those assumptions, and provides answers to some of those questions. The most basic assumption that the latent variable model makes is the assumption that latent variables are not observed, while manifest variables are. So, the latent variable ‘depression’ is not observed, while the variables ‘insomnia’ and ‘fatigue’ are considered observed . However, it is not quite clear what observing actually entails, nor is it clear who the observer actually is. The researcher himself probably did not see the lack of sleep. The question arises as to how it is best to explicate this distinction between manifest and latent variables.
In addition, latent variable models implicitly assume that the shared variance of multiple manifest variables are caused by identical latent variables. After all, the variable that causes shared variance in the manifest variables, is the same for each of the manifest variables separately, and latent variables only constitute a single factor if they are identical to each other. But what does it actually entail for two variables to be identical?
Another controversy surrounds the notion of validity; under what conditions do we conclude that the observed variables are a valid measure of the variable that is believed to be the common cause of these observed variables? Borsboom et al. (2004) comment on how the validity literature has developed in the past 50 years: “The concept that validity theorists are concerned with seems strangely divorced from the concept that working researchers have in mind when posing the question of validity”. According to Borsboom et al. (2004) the question of validity has become more difficult over this time and seems to involve all important test-related issues instead of remaining the simple question whether a test measures what it purports to measure.
The controversies mentioned above, stress the need for a conceptual framework that reveals the implications and assumptions of latent variable models. This paper proposes such a
frame-work. This framework distinguishes between the intension and extension of variables. The intension of a variable is a function. The extension consists of the values that result from apply-ing this function to a domain of individuals. The framework directly responds to the issues raised above. However, more importantly, it provides a general understanding of the latent variable model. As such, the framework can be deployed to respond to other psychometric controversies, related to latent variable analysis, that this paper does not address directly.
The outline of this paper is as follows. First, I give a short description of the proposed conceptual framework. Second, I elaborate on the concepts that this framework is based on. I explain the distinction between intension and extension in variables and define observed variables and measurement. Third, I discuss what conditions are necessary and what conditions are sufficient for variables to be identical. I argue that experimental intervention could serve as a method to determine whether a supposedly single factor consists of multiple factors with different intensions. Finally, I build upon the framework to define validity.
A conceptual framework: Intension and extension
This sections spells out a conceptual framework that reveals some of the implicit assumptions of the latent variable model. Within this framework I define what measurement is, and what con-stitutes observed variables. The section starts with a brief and general outline of the framework. Subsequently, the section elaborates on the different ingredients of the framework separately. The conceptual framework proposed in this paper is based on the distinction between the in-tension and exin-tension of variables, and on the distinction between observed variables and latent variables. The intension of a variable is a function. For example the intension of the variable ‘height’ is the function that determines someone’s height. The extension of a variable is this function applied to a domain. For example, the function ‘height’ applied to the domain: John, Mary, Lisa and Peter, results in an extension that consists of four values; the height of John, Mary, Lisa and Peter. So the intension of a variable determines the extension of the variable, within a given domain. Figure 2 is a visual representation of this framework for the variable ‘height’.
The framework proposed in this paper defines observed variables as data patterns with their corresponding intensions. Observation, according to van Fraassen (2001, p. 154), is “perception, and perception is something that is possible for us without instruments” (i.e., with the naked eye). According to this definition, the only ‘observed variables’ in most psychological research are values in a dataset. The intension of an observed variable is the function that resulted in the exact values that are observed in the dataset. In turn, the extension of the observed variable is the total set of values in the dataset. Observed variables measure latent variables that cause the data patterns.
For example, the intension of the observed variable that measures the latent variable ‘height’ is the function ‘what someone writes down as a result of the question “how tall are you?”’.1 In
turn, the extension of the observed variable is the total set of values that this function determines within a given domain. The relation between the intension and extension of an observed variable is, by definition, without error. In contrast, the relation between the extension of the latent variable and the extension of the observed variable(s) that measure(s) that latent variable is full or error. A research respondent could write down the wrong value for his/her height for all sorts of reasons. This error results in a distinction between the extension of the observed variable and the latent variable (i.e., the extensions of the observed and latent variable do not perfectly
1This observed variable measures height, given that height indeed causes these values that form the extension
Extension Intension Observed/Manifest Unobserved/Lat ent … height … … 1.78 … … 1.82 … … 1.73 … … 1.69 … ‘What someone writes down as a result of the
question: “How tall are you?”’ Human height Causes 4 2 Determines Determines 3 1 ‘The length of a human body from top to toe’ Data Variable
Figure 2: The variable ‘height’, when intension is taken into account.
correlate). The conceptual framework is summarized in the Figure 2.
In sum, the difference between measuring a variable and observing a variable is that we observe the data but we do not observe the extension of the variable that caused these data. We observe the values that John, Mary, Lisa and Peter wrote down, but we measure their actual heights because we believe that these values we observe are caused by the heights of John, Mary, Lisa and Peter respectively (i.e., by the extension of the latent variable ‘height’ applied to John, Mary, Lisa and Peter).
The remainder of this section elaborates on the different aspects of the conceptual framework separately. In the subsequent sections I use the proposed framework to offer an answer to two psychometric questions. In the section ‘Validity’ I discuss the question of what validity is. In the section ‘Identity between variables’ I discuss under what conditions variables are identical.
Intension and Extension. Suppose you are at a party with Mary and her boyfriend Peter. At the party, you run into Mary’s old friend Lisa who has not met Peter yet. Now, if you tell Lisa that Peter is Mary’s boyfriend, then you would provide Lisa with new information and she might be very interested. However, if you tell Lisa that ‘Mary’s boyfriend’ is ‘Mary’s boyfriend’, Lisa might feel like you told her nothing new. So, although ‘Peter’ and ‘Mary’s boyfriend’ designate the same person, both expressions cannot be substituted for each other because they bear a dif-ferent meaning. Meaning and designatum (redif-ferent)2are not the same thing. This distinction is
nicely captured in the ‘morning star-evening star’ paradox, introduced by Frege (Gamut, 1991). In this paradox Frege shows that the following statements have different meanings:
(1) The morning star is the morning star.
2By designatum I mean ‘that what is designated by an expression’. This is synonymous to referent since this
is ‘that what is referred to by an expression’. Note that designatum differs from designator, since the latter is the expression that designates, instead of that what is designated by the expression.
(2) The morning star is the evening star.
Both the ‘morning star’ and the ‘evening star’ designate the planet Venus. Yet, they bear different meanings: the morning star is the star that is observable in the morning, while the evening star is the star that is observable in the evening. Meaning and designatum are not the same.3 If they were, it would be possible to substitute ‘morning star’ with ‘evening star’, and the two statements above show that this is not the case. Statement (1) is a tautology, whereas statement (2) is a contingent truth (for a long time (2) was considered untrue).
The expression ‘Mary’s boyfriend’ designates Peter as a person, but the meaning of this expression is something like ‘the person who is in a relationship with Mary and is male’ (Figure 3). The meaning of an expression is the conceptual content of that expression. For Lisa, ‘Mary’s boyfriend’ still has meaning even though she does not know to whom this expression refers. Even if Mary did not have a boyfriend, the expression ‘Mary’s boyfriend’ would still be meaningful. For example, in the sentence: ‘Mary’s boyfriend has to be someone who loves adventures’, the expression ‘Mary’s boyfriend’ does have a meaning but no designatum.
The meaning and designatum of an expression relate to each other such that the meaning of the expression determines the designatum (Gamut, 1991; see Figure 3). Note that ‘to determine’
‘Mary’s boyfriend’
‘The person who is in a relationship with Mary at this moment, and is male’
Has as its meaning Designates indirectly
(via the meaning)
Determines
Meaning
Designatum
Figure 3: An expression with its meaning and designatum.
does not mean ‘to cause’. Clearly, the meaning of an expression does not cause the state of affairs in our world. However, the meaning of an expression does determine to whom, or to what that expression refers in our world. The expression ‘Mary’s boyfriend’ means the person who is in a relationship with Mary and is male. In turn, the expression designates Peter, because currently
3In his paper ‘ ¨Uber sinn und bedeutung’, Frege concludes that sense and reference are two distinct components
of an expression. The sense of an expression contains the mode of presentation (‘Art des Gegebenseins’ in Frege (1892)) of that expression, whereas the reference (i.e., what I call designatum) is that what is designated by the expression. Sense can be understood as the meaning of an expression(Gamut, 1991). For a more extensive understanding of the distinction between sense and reference, read “Sense and reference” (Frege, 1948); an English translation of ‘ ¨Uber sinn und bedeutung’ (1892).
Peter is the person who is in a relationship with Mary and is male. Clearly, Peter did not get involved in a relationship with Mary as a consequence of the meaning of ‘Mary’s boyfriend’. When Mary and Peter break up and Mary starts to date John, the expression ‘Mary’s boyfriend’ designates John, instead of Peter, even though the meaning of the expression ‘Mary’s boyfriend’ does not change.
When two expressions bear the same meaning, they designate the same person or thing. For example, if ‘Mary’s boyfriend’ and ‘Mary’s male partner’ bear the same meaning, they both designate first Peter and later John. Expressions with the same meaning always designate the same designatum (Figure 4). Obviously, the expression ‘Mary’s boyfriend’ does not cause the relationship between Mary and Peter (or John). However, the meaning of the expression ‘Mary’s boyfriend’ does determine that the expression designates Peter (or John).
‘Mary’s boyfriend’
‘The person who is in a relationship with Mary at this moment, and is male’
Means
Designates indirectly (via the meaning) ‘The male partner of Mary’
Means
Designates indirectly (via the meaning)
Determines
Figure 4: Two expressions with the same meaning.
The distinction between meaning and reference corresponds to the distinction between inten-sion and exteninten-sion (Figure 3). “The inteninten-sion of an expresinten-sion is something like its conceptual content, while its extension comprises all that exemplifies that conceptual content.” (Gamut, 1991, p.14). Peter is the person that exemplifies the conceptual content (i.e., boy in relationship with Mary) of ‘Mary’s boyfriend’. Hence, Peter is the extension of this expression. The extension of an expression is not always a single entity like Peter, it is often a set of entities. For example, the extension of the expression ‘man’ is the set of all men in the world, since they all exemplify the conceptual content of the expression. Likewise, the extension of the expression ‘digit’, is
the set: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}, while the intension of the expression ‘digit’ is: ‘single symbol referring to a whole number’ (Gamut, 1991). The distinction between intension and extension is at the core of the argument in this paper.
Above I explained the distinction between intension and extension. Also I showed how the extension can change over time while the intension stays the same (e.g., the extension of Mary’s boyfriend first consisting of Peter and later John). This property, that expressions can have different extensions across contexts while their intension remains the same, serves as a good tool to better understand this distinction and extend this distinction to variables in the remainder of this paper. This property is nicely captured by the notion of possible worlds. In the next section I will explain the notion of possible worlds. In the sections that follow, I will use this to extend the distinction between intension and extension to variables and subsequently to answer the question of identity between variables and the question of validity.
Possible worlds. Above I demonstrated that an expression may have different extensions in different contexts. The extension of an expression can differ over time (e.g., ‘Mary’s boyfriend’ first designates Peter and later John) or over place (e.g., in France ‘the president’ most likely designates the French president, while in the US it most likely designates the US president). However, even for some expressions that have the same extension over place and time, things could have been different. For example, the expression ‘the author of the book Harry Potter’ designates J.K. Rowling, regardless of time and place in our world. However it is not necessary that J.K. Rowling wrote Harry Potter. One could easily imagine a different context in which J.K. Rowling did not become a writer, or wrote another book or titled the book differently; one can think of numerous contexts in which J.K. Rowling would not be the author of Harry Potter. This idea is nicely captured in the notion of possible worlds.
The concept ‘possible world’ has been part of the philosophical lexicon since at least the work of Leibniz4, however possible world semantics can be most clearly traced back to the work
of Carnap (1947, in Fitting, 2014). Today, the notion features prominently in the philosophy of language and modal logic.5 In possible world semantics the actual world is our reality, ‘the
way it is’; the world in which J.K. Rowling wrote the book Harry Potter. Possible worlds are hypothetical worlds in which things could have been different. These worlds are used to show that even an expression like ‘the author of Harry Potter’ could have a different extension in another context; namely in another possible world.For example, worlds in which the expression ‘the author of the book Harry Potter’ does not designate J.K. Rowling. Lewis (1973) writes on possible worlds:
“I believe that things could have been different in countless ways; I believe permissible paraphrases of what I believe; taking the paraphrase at its face value, I therefore believe in the existence of entities that might be called ‘ways things could have been’. I prefer to call them ‘possible worlds’.” – Lewis (1973, p.84)
According to Carnap (1988, p.9), possible worlds are similar to what he himself calls ‘state descriptions’ in his book ‘meaning and necessity’ and to what Wittgenstein called ‘possible states
4Most of what we know about the work of Leibniz is based on notes and other bits of pieces that were not
intended for publishing (Mates, 1968). For an overview of Leibniz’s work on modal logic, I recommend the section ‘Leibniz’s Modal Metaphysics’ in the Stanford Encyclopedia of Philosophy (Look, 2013), or ‘Leibniz on possible worlds’ (Mates, 1968).
5Modal logic is one example in which intensional contexts are studied formally. Other examples could be found
in Gamut (1991). However, for the sake of clarity, I will keep to the notion of possible worlds as a tool to illustrate my argument, without explaining other forms of intensional logic. For the same reason I will not go to deep into the technical matter of intensional logic.
of affairs’ in his book ‘the tractatus’. The ‘possible worlds’ concept returns at several points in this paper. The concept is especially relevant to the discussion on identity between variables and the discussion of validity.
The above indicates that the distinction between the intension and extension of variables originates in the philosophy of language and modal logic. Nevertheless, the distinction bears much significance for the analysis of latent variable models. The next section demonstrates why this is the case. In the next section I extend the distinction between the intension and extension of expressions to the intension and extension of variables. In this way I construe the base of the framework proposed in this paper.
Intension and extension in variables. This section uses the distinction between intension and extension of expressions to distinguish between the intension and extension of psychological variables. I take variables to be functions; they can be applied to entities (for psychological variables these entities are individuals), resulting in the corresponding value or attribute for that entity on that variable. For example height applied to John, gives John’s height as an outcome. Note that whether or not these outcomes can be measured is a separate issue. Usually, these attributes are labelled with numbers. For example sex applied to John gives the attribute of being male as its outcome, which is usually labelled with a number (e.g., 1 for men and 0 for women). Also one can make these variables as specific as one wants. Take for example the variable: the reaction time someone shows at recognizing a word after being primed with a picture of a tree. The crux is that variables adopt varying values when applied to different entities (e.g., in psychology over different individuals).
Carnap (1988) distinguishes between the intension and extension of predicates. Predicates are very similar to variables as defined above, and hence, it is valid to use the insights of Carnap to the analysis of psychological variables. Predicates are characteristic functions that give truth values as output. For example the predicate ‘being human’, applied to J.K. Rowling, generates the truth value 1 since J.K. Rowling is a human being. By contrast, the same predicate applied to Mickey Mouse generates the truth value 0, since Mickey Mouse is not a human being. So, predicates are characteristic functions, that give some kind of ‘no, that is not true’ (i.e., truth-value: 0) or ‘yes, that is true’ (i.e., truth-value: 1) as output, when applied to an entity. Psychological variables are very similar to predicates. Predicates applied to an entity generate a truth value (i.e., 1 or 0). Likewise, psychological variables generate an attribute when applied to an entity. Dichotomous variables generate two attributes, categorical variables generate multiple attributes, and continuous variables generate an infinite amount of attributes. For example, the dichotomous variable ‘sex’ applied to an entity either generates the attribute ‘man’, or the attribute ‘woman’. All attributes that together form the set of possible outcomes of a variable are at the same time predicates, when they are taken separately. For example, ‘being exactly 1.80 meters’ is both a possible outcome of the continuous variable length applied to a specific entity, as well as a predicate that can be applied to an entity or set of entities. According to Carnap (1988), the intension of a predicate is a property, and the extension of a predicate is the class that exemplifies that property.
“With respect to a predicator, say ‘H’ in S, we have distinguished between its exten-sion, the class Human, and its intenexten-sion, the property Human.” – Carnap (1988, p.44)6
Thus, predicates have as their intension a property and as their extension a class of entities that exemplify this property. The property determines of which entities the class consists. Building
6Carnap specifies the intension and extension of a predicator on p.19 of his book ‘Meaning and Necessity’. H
on the work of Carnap, I define the intension of a variable as a function while the extension is the set of values that this function generates when applied to a set of entities. For example, the extension of the predicate ‘male’ is the class of all men, and the extension of the predicate ‘female’ is the class of all women. These classes are the sets of all entities that have truth-value 1 when this predicate is applied to them (i.e., {x|P (x)}, in which P is the predicate and x is an entity). Correspondingly, the intension of the dichotomous variable sex is the function that determines whether someone is a man or a woman (e.g., having XX- or XY-chromosomes7). In turn, the extension of the variable sex is the class of all men and the class of all women. In practice, the entities that form a class are labelled. For example, the class of all men is referred to with the value 1, and the class of all women is referred to with the value 0. Likewise, the intension of the continuous variable ‘height’ is the function that determines the height of something or someone. The extension of the variable ‘height’ then consists of all the values that are generated by that function when applied to a given population in time and place. 8
The exact same intension could in another possible world determine a different extension, since this world can have a different population. For example, it can be imagined that in another possible world Michael Jackson would still be alive and therefore be an element of the class of men that exemplify ‘man’ in that possible world. But even with other people in these classes (different extension) it would still be the case that the one class would consist of only individuals with XY-chromosomes and the other class of only individuals with XX-XY-chromosomes (same intension).9
Again, I would like to highlight that the function of sex determines the classes, but does not cause who has XX-chromosomes and who has XY-chromosomes. The function returns a value from the input of the variable (the entity to which this function is applied, x). The way this function evaluates the input is the intension of the variable, f (..). The outcomes of this function, f (x), is the extension of this variable.
The role of the input shows that the extension depends on the domain to which it is applied. For psychological variables this domain is the human population. This dependence on the domain has as a consequence that another possible world with a different domain (within Psychology this could be another possible human population) will give a different extension with the exact same function. Just like a function: f (x) = 2x, will give different results applied to the domain {1, 2, 5} and {2, 4, 6} (resp. {2, 4, 10} and {4, 8, 12}). This accounts for the idea that on another possible world, where Michael Jackson is still alive, the extension of the variable sex would be different, since Michael Jackson would be part of the input.
The above spells out the distinction between the intension and extension of psychological variables.10 In the following sections I deploy that distinction to discuss the psychometric
con-7Note that this is only a (hypothetical) definition of sex that simplifies the reality and does not do justice to
those who do not fit into this simplified dichotomy. For an extensive theory on gender read Butler (2011).
8Usually these attributes or values that result from a function are labelled, and then one could just as well
speak of the extension as the set of digits (e.g., the sets of zeros and ones when women are labelled 1 and men are labelled 0) but keep in mind that these are often arbitrarily chosen digits. Also note that labelling is not the same as measuring. When we label all men with zero and all women with one, I do not mean that the extension of sex becomes the ones and zeros we write down after measuring the variable sex. In contrast, I mean that the attributes that result from applying a variable to a domain, can be labelled just like entities get a truth-value as a result of applying a predicate to them (true is labelled 1, untrue is labelled 0). There is no error in this step because it is no measurement.
9Some might argue that this having XY- or XX-chromosomes is an example of a natural kind instead of an
intension, I elaborate a little on that in footnote 10.
10For those who are familiar with the work of Kripke, one could argue that variables like ‘depression’ do not
have an intension, but rather rigidly designate the thing that is named depression. Variables can be viewed as natural kinds. According to Kripke (1980), natural kinds do not designate via the meaning but directly refer to a designatum that is equal over all possible worlds. Natural kinds always refer to the kind that we have named as such. Kripke (1980) gives the example of gold which refers to goldness in every possible world. Kripke (1980) elaborates extensively on rigid designation. However, for my purpose here it is important to note that rigid
troversies, mentioned before. Before that, I discuss the distinction between observed (manifest) and unobserved (latent) variables.
Observed variables and measurement. Above, I briefly mentioned the definition of obser-vation provided by van Fraassen (2001, p. 154): “Obserobser-vation is perception, and perception is something that is possible for us without instruments” (i.e., with the naked eye). This defini-tion of observadefini-tion is quite strict. However, for my purposes here (i.e., to distinguish between observed and unobserved variables) the strict definition provided by Van Fraassen is very useful, as I demonstrate below.
Variables consist of an intension (i.e., a function) and an extension (i.e., the outcomes of the function). When we speak of observing a variable, all that we observe is the extension of a vari-able. After all, we observe outcomes of individuals on a variable, we do not observe the function itself. It seems clear which variables are observed and which variables are latent. Consider a question like “how frequent do you attend parties?”, with which someone intends to measure extraversion. Extraversion is believed to be latent while the frequency with which someone at-tends parties is observed. However, what is really observed? All that we as a researcher observe is the data that resulted from the question “how frequent do you attend parties?”. The data pattern that we observe is the extension of a variable, and this variable I will call the observed variable. However as aforementioned, variables also have an intension, namely the function that resulted in this extension. For this reason I define observed variables as data patterns with their corresponding intension.11 The intension of the data pattern that is obtained with the question about attending parties, is something like: ‘the answer someone writes down as a result of the question “how frequent do you attend parties?”’. This function differs from the function ‘how frequent someone attends parties’ because there is no error in the relation between the intension and extension of a variable. After all the intension is the meaning of the data, not the cause of the data. If Lisa writes down a 4 while she actually attended 6 parties (e.g., she lied or mis-counted), this 4 is still ‘what Lisa writes down as a result of the question ‘how frequent do you attend parties?” (i.e., the function “what someone writes down..” applied to Lisa). In contrast the function ‘how frequent someone attends parties’ applied to Lisa would result in 6 since Lisa actually attended 6 parties.
This view of observed variables in which observed variables are data patterns with their intension, possesses some important advantages over perspectives in which observed variables are variables out there in the world that we observe by means of the data. Borsboom (2008) argues that a variable ‘out there’ is either observed or unobserved with respect to the data pattern in a dataset. According to Borsboom (2008) a variable is called observed when the researchers
designation is not inconsistent with the framework proposed in this paper. One could replace what I call intension, for what Kripke calls the designatum of a rigid designator (both are equal over possible worlds). What I call the extension, is what some who support the idea of natural kinds, call the metaphysical extension; “Following Salmon (1981, p. 46), I call the class of a kind’s members or instances its “metaphysical extension”. The same class that serves as the metaphysical extension of a kind serves as the semantic extension of a general term for that kind.”(LaPorte, 2006, note 4, p.334). Do not mistake me, the intension and Kripke’s designatum are not the same, but the designatum of a natural kind could fulfil the same role as the intension within the conceptual framework proposed in this paper. The distinction between the designatum of a natural kind (e.g., goldness or depressionness) and the (metaphysical) extension (the lumps of gold that exemplify gold or the set of depression outcomes), implies that natural kinds are not necessarily incompatible with the framework proposed in this paper.
11Note that because of the intension of the observed variables, two variables that cause the same data pattern
do not result in the the same observed variable. After all, one can distinguish between the two observed variables by their different intensions. For example suppose that in a certain domain of individuals, the variable sex and the variable handedness result in the same data pattern, then the observed variables differ since one of the two has as its intension ‘what someone answers to the question “what is your sex”’ and the other has as its intension: ‘what someone would answer to the question “are you right or left handed?”’. In this way we can distinguish between observed variables that happen to have the same data pattern (i.e., same extension).
assume that the relation between a variable and the data pattern it causes, is epistemically certain (i.e., when full accessibility to the variable exists). In this view, the variable ‘attending parties’ is observed (rather than the data pattern that this variable causes) when the relation between the data and this variable is epistemically certain. Apart from the fact that Borsboom (2008) does not explain why some variables in the world are fully accessible and others are not, his view in which variables out there are observed, is problematic in multiple ways. Most notably, a view in which variables out there can be observed means that continuous variables (e.g., height, reaction time) can never be observed, since we cannot observe infinity. In the way I define observed variables this is not a problem since continuous variables can cause people to answer in a certain way, but their answer or the data in the dataset will always be finite. So the variable ‘height’ in the world, applied to John would be a value with infinite decimals behind the comma, but ‘what someone would answer when he/she is asked for his/her height’, applied to John, results in the number that John writes down, which will be finite. So, one advantage of defining observed variables as data patterns with a corresponding intension, is that it implies that the observed extension consists of numerical values that are all finite, even when the variable that one intends to measure is continuous.
Moreover, the definition of an observed variable proposed in this paper avoids the dichotomy between variables that are fully accessible and variables that are not, like in the view of Borsboom (2008). Of course, some variables are more accessible than others, but I believe this is better portrayed as a scale of accessibility instead of a rigid dichotomy. The accessibility of a variable decreases along the causal chain that precedes the data pattern. For example, the data pattern obtained with the question: “How frequent do you attend parties?” has a different relation to the latent variable ‘liking to go to parties’ than to the latent variable ‘extraversion’, in which the latter is less accessible with respect to the data. However neither variable is fully accessible. The further down along the causal chain, the more error between the data pattern and the variable it intends to measure. Accessibility is a property of the measurement relation and not a criterion that generates a dichotomy between variables in the world (see Figure 5).
Now that I have described the relation between the intension and extension of the observed variable as a non causal relation, we can distinguish ‘observing a variable’ from ‘measuring a variable’. The observed variable is a data pattern with its corresponding intension, whereas a variable is measured by an observed variable when this variable causes the data pattern. Variables in the world can be measured because we can observe a data pattern that we presume to be caused by the variable in the world. Measurement is thus the causal relation between a latent variable and one or multiple observed variable(s). The observed variable is a measure of the latent variable that caused this observed variable. The above is summarized in Figure 2 for the variable ‘height’.
As an example, consider the variable ‘sex’ (Figure 6). This variable in the world is measured with a question like ‘what is your sex?’. Mary and Lisa circle the ‘f’ of female (which is labelled with an 1), John and Peter circle the ‘m’ of male (which is labelled with a 0). We expect that the class to which Mary and Lisa belong (the class of women) leads Mary and Lisa to circle the ‘f’, while the class to which Peter and John belong (the class of men) leads Peter and John to circle the ‘m’ (i.e., box 3 causes box 1). However this relation might include error; e.g., Lisa could by accident circle the ‘m’. When box 3 does not cause box 1, then the observed variable is not a valid measure of the latent variable. I elaborate on this in the section ‘validity’.
This conceptual framework can be applied to different sorts of variables. In Figure 2, the framework is applied to the continuous variable ‘height’. In this figure the extension of the latent variable (box 3), consists of values with infinite decimals behind the comma, while the extension of the observed variable (box 1), consists of finite numbers. In Figure 7 the framework is applied to the variable ‘extraversion’, of which the extension in the world is unknown, as is usually the
0 1 0 1 data world
Highly accessible but not observed
Observed Fully accessible and therefore observed
Observed
1 2
Figure 5: Two different views applied to the variable sex. (1) represents the view of Borsboom (2008), (2) represents the view proposed in this paper.
case for psychological variables. In Figure 8 the framework is applied to the variable ‘amount of phlogiston’. The theory of phlogiston postulated a substance that is contained in flammable materials and is released during heating, in the form of fire (Borsboom & Markus, 2013). In the 18th century it was believed that the amount of phlogiston contained in a material, could be measured by the difference in weight before burning and after burning. This theory is obsolete and presently we no longer believe the theory that phlogiston is the cause of the difference in weight. Hence, phlogiston has no extension in our world, and box 3 is left empty. For this reason the causal arrow from box 3 to box 1 is also left out. After all, the extension of phlogiston does not exist and therefore cannot be the cause of the reduced weight after burning a material.12 In
this case the function fails to designate an extension in the world. Nevertheless, the intension of the observed variable does determine an extension. This is because the intension of the observed variable is a function that is only based on operationalizations. The initial weight before burning and the weight after burning, do exist and thus this function can generate numbers which serve as the extension of the observed variable.
In this section, I provided a conceptual framework that reveals the implicit assumptions of the latent variable model. In the next two sections, ‘Identity between variables’ and ‘Validity’, I demonstrate that the framework is also suited to respond to key psychometric controversies.
12In the other figures one might also question the causal arrow from box 3 to box 1. Whether there is really
a causal relation between the extensions of the latent and the observed variable, is captured by the notion of validity. For this reason I discuss this causal arrow in the section ‘Validity’.
Extension Intension Obs erv ed/ Mani fest Unobs erv ed/ La te nt ID sex … Mary 1 … Lisa 1 … John 0 … Peter 0 …
„The number someone writes down as a
result of the question
“what is your sex?”‟
„Being male or female, having XY- or XX-chromosomes‟ Causes 4 2 Determines Determines 3 1 Variable Function: f(…) f(Mary) = 1 f(Lisa) = 1 etc. Data Sex f(John) = 0 f(Peter) = 0 etc.
Figure 6: The variable ‘sex’, when intension is taken into account.
Identity between variables
According to Geach (1973, in Noonan & Curtis, 2014), “x and y are to be properly counted as one just in case they are numerically identical”. This numerical identity “requires absolute, or total, qualitative identity and can only hold between a thing and itself” (Noonan & Curtis, 2014). So, being identical is much more than just sharing certain properties. According to this definition of identical, latent variable models imply that shared variance of observed variables is caused by identical latent variables.
The claim. Suppose a common factor model in which a common factor F , proclaims to cause the shared variance v∗ in four observed variables O1, O2, O3 and O4.13 In this model, v∗ is the
intersection of the variance in O1 (v1), the variance in O2 (v2), etc. This means that v∗ is fully
part of v1, v2, v3and v4. If F causes v∗, and v∗is part of v1to v4, then v∗in O1is caused by the
same variable that causes v∗in O2, v∗in O3and v∗in O4. This variable is the common factor F .
So, the assumption that one single factor causes the shared variance in several observed variables implies that the latent variables that cause v∗ in the observed variables are identical (Figure 9).
14 After all, being identical would be the only legitimate reason to conclude that the variables
13The reason why my example takes four manifest variables is because Mulaik & Millsap (2000) explain that at
least four manifest variables are needed to identify a latent variable. Therefore four is the minimum of manifest variables with which we can identify that there is actually one single factor that underlies these indicators. However the example I describe can be extended to larger numbers of manifest variables.
14Note that this idea does not exclude the possibility of other variables also playing a role in causing the
manifest variables, that do not have to be equal over the different manifest variables. Actually, it will almost always be the case that other variables (other than the common factor and unsystematic error) explain some of the variance in the manifest variable as well. I will not go any deeper into this matter, but it is important to understand that in every model where a common cause structure is assumed, at least two identical variables are assumed, independently of other variables playing a role in causing variance in the manifest variables (Figure 10).
Extension Intension Obs erv ed/ Mani fest Unobs erv ed/ La te nt … Extr. … … 65 … … 72 … … 43 … … 55 …
‘The sum score on an extraversion test’ Extraversion Causes 4 2 Determines Determines 3 1
?
Variable Data ‘Enjoying human interactions. Taking pleasure in activities that involve large social gatherings, such as parties.’Figure 7: The variable ‘extraversion’, when intension is taken into account.
are one and the same common factor. But for what reason is it assumed that the variable that causes shared variance in the one observed variable is identical to the variable that causes this variance in the other observed variable? Below, I deploy the conceptual framework, laid out in the section above, to analyse what it means for variables to be identical. More specifically, I discuss the conditions under which variables are, and are not identical.
Perfectly correlating data patterns. One approach to the identity of variables is to posit that variables are identical when the data patterns that they cause are identical (i.e., correlate perfectly). According to this approach, F1 and F2in Figure 11a are identical, when O1 and O2
correlate perfectly. The advantage of this approach is that it is easy to test, because data patterns are observed. However, the approach is problematic as well. Data patterns are not only caused by the variable that one intends to measure with this data pattern, but also by unsystematic and systematic error. Suppose two IQ tests in which general intelligence is the common factor that causes v∗. Now, even though IQ test 1 and IQ test 2 both measure the same latent variable, it is very unlikely that the two observed data patterns of the two tests correlate perfectly (i.e., that all variance is shared variance). Instead, it is very likely that random error and/or additional factors influence the outcomes of the two tests differently. Hence, perfect correlation of the data patterns is not a necessary15condition for the latent variables to be identical. Below, I demonstrate that
perfect correlation of the data patterns is not a sufficient16 condition either.
In the unlikely event that data patterns correlate perfectly, there are two possibilities. First, it could be that one factor is measured twice. In this case, the two factors are indeed identical. Second, it could be that two different factors have caused the same data pattern. This either means that two intensionally different factors correlate perfectly at that particular point in time and place, or it means that two different factors in combination with systematic and unsystematic
15A is a necessary condition for B when it is true that whenever A is not the case, B is not the case. 16A is a sufficient condition for B when it is true that whenever A is the case, B is the case.
Extension Intension Obs erv ed/ Mani fest Unobs erv ed/ La te nt ‘What a scientist in the 17th century writes down after burning a piece of wood and subtracting the left weight from the initial weight’
‘A fire-like element, contained within combustible bodies, that is released during combustion’ 4 2 Determines Determines 3 1 4mg
Empty set, since nothing meets the intension of phlogiston.
Variable
Data
Amount of ‘phlogiston’
Figure 8: The variable ‘amount of phlogiston’, when intension is taken into account.
error somehow caused the same data pattern. In the latter case, the observed variable that is the result of F1and ε1correlates perfectly with the observed variable that is the result of F2 and ε2
(see Figure 11a). So, perfect correlation in data patterns is neither a sufficient nor a necessary condition for variables to be identical.
Perfectly correlating factors. Perfect correlation between F1and F2is a necessary condition
for F1 and F2 to be identical. If F1 and F2 do not correlate perfectly, then it is possible to
distinguish between the two, i.e., they are not identical. However, perfect correlation between factors is not a sufficient condition for variables to be identical, as I will demonstrate below.
Common cause models implicitly assume that perfect correlation between latent factors is sufficient reason to take them to be one and the same factor (which implies that they are iden-tical). Common cause models assume conditional independence between the observed variables that are caused by the common factor. As Borsboom (2008) notes: “Common cause models are characterized by the fact that the common cause “screens off ” covariation between its effects: If X is the common cause of Y and Z, then Y and Z must be conditionally independent given X.” But to condition on a latent variable, is also to condition on all variables that correlate perfectly with that latent variable.
Consider the hypothetical case in which left/right handedness correlates perfectly with sex: All women are right handed and all men are left handed. Now take two tests with items that are related to sex but not to handedness (e.g., ‘hair length’ and ‘frequency of dress wearing’). Obviously the two tests correlate since they both measure sex. However, when controlling for handedness, the correlation between the two tests vanishes.17 This conditional independence
leads to the conclusion that handedness is the common factor that caused v∗ the items, despite the fact that sex, not handedness, caused v∗. Now suppose that there is one test that measures
17For this example I do not take into account feminine behaviour. Of course it could be the case that women
Identical Latent variable x 𝜀1 Implies Manifest variable 1 Manifest variable 2 Manifest variable 3 Manifest variable 4 Manifest variable 1 Manifest variable 2 Manifest variable 3 Manifest variable 4 𝜀2 𝜀3 𝜀4 𝜀1 𝜀2 𝜀3 𝜀4 Latent variable x Latent variable x Latent variable x Latent variable x
Figure 9: A common cause implies that variables are identical.
sex, and one test that measures handedness. When all items of the two tests are combined in one data set, a factor analysis would generate one factor that underlies both tests, despite the fact that the two factors are not the same. In this hypothetical example it is obvious that controlling for handedness is the same as controlling for sex. It is, however, easy to imagine cases in which the spurious relation is less obvious. Some researchers are more explicit about the assumption of perfect correlation as a sufficient condition for identical variables. For example Schmidt & Hunter (1999) write: “Suppose that two constructs correlate 1.00 at the true score level (i.e., ˆ
rxtyt = 1.00). That is, suppose they really are the same construct under two different labels.”
18
So, perfect correlation between F1and F2is a necessary condition for F1and F2to be
identi-cal, but it is not a sufficient condition as the example of perfect correlation between handedness and sex shows. In the following, I introduce the distinction between intension and extension of variables and the notion of possible worlds into the discussion on identical variables.
Intension to define Identity. The approaches that define identical as perfect correlation (between latent variables or between observed data patterns) only take into account the extension of variables. In the section ‘Intension in variables’, I explained that variables that have a different intension (i.e., a different function) could still have the same extension (i.e., the outcomes of this
18Note that there is a difference between true scores and latent factors, which means that the condition of
perfectly correlating true scores is a stronger condition than perfectly correlating factors F1and F2. For example
in the right model in figure 11, if systematic error would be added to this model (e.g., suppose an extra factor G only causing O1 but not O2), this would have as a consequence that the true scores will no longer perfectly
correlate, while in the model of figure 11, this factor G would become part of 1 and F1 and F2 would still
correlate perfectly (i.e., the model would look the same but the true scores of O1 and O2 would not correlate
perfectly). In other words, for true scores of O1 and O2 to correlate perfectly, one needs more than just one
factor for O1to perfectly correlate with one factor for O2, instead all factors underlying the variance of O1should
together perfectly correlate with all factors underlying the variance in O2 (i.e., all systematic variance in O1
𝑂1 𝐹2 𝜀1 𝜀2 Identical 𝐹1 𝐹1 𝐹3 𝑂2
Figure 10: The implication of variables being identical when a common cause is assumed, is independent of (multiple) other latent causes.
function when applied to individuals). Handedness and sex are not the same variables, because they have a different intension. So, even if their extensions perfectly correlate, the two are not the same. In another possible world they could correlate less than 1 or not correlate at all.
So, it is insufficient to look only at the extension of variables to identify identical variables. As Carnap describes: “If two designators are equivalent, we say also that they have the same exten-sion. If they are, moreover, L-equivalent, we say that they have also the same intenexten-sion.”(Carnap, 1988, p.1). L-equivalent means equivalent in all state descriptions. In possible world semantics this corresponds to being equivalent over all possible worlds. Later on, Carnap explains that two designators are only identical when they are L-equivalent:
“It is customary to regard two classes, say those corresponding to the predicators ‘P’ and ‘Q’, as identical if they have the same elements, in other words, if ‘P’ and ‘Q’ are equivalent. We regard the two properties P and Q as identical if ‘P’ and ‘Q’ are, moreover, L-equivalent. By the intension of the predicator ‘P’ we mean the property P; by its extension we mean the corresponding class. It follows that two predicators have the same extension if they are equivalent, and the same intension if they are L-equivalent.” – Carnap (1988, p.16)
In possible world semantics this corresponds to the idea that classes are identical when they are equivalent in that world. So, when the extensions of two predicates are identical then the predicates are equivalent. However the intensions of two predicates are only identical if the predicates are equivalent in every possible world (i.e., their extensions are identical in19
every possible world). This idea can be extended to variables. Suppose that two variables (i.e., functions) determine the same output (i.e., they correlate perfectly) when applied to the same
19It is very important to note the difference between ‘in every possible world’ and ‘over possible worlds’. Two
variables are identical in every possible world when at every possible world these two variables are identical to each other. So they are not identical over possible worlds.
𝐹1 𝑂1 𝐹2 𝑂2 r = 1.00 𝐹1 𝑂1 𝐹2 𝑂2 r = 1.00 𝜀1 𝜀2 𝜀2 𝜀1
A: Perfectly correlating data patterns B: Perfectly correlating factors
Figure 11: Perfect correlation as the condition for latent factors F1and F2 to be identical.
population in this actual world. In this case, the two variables are equivalent because their extensions are the same. For example, in the case that the variable sex is perfectly correlated with the variable handedness, the variable sex would generate classes that are identical to the classes generated by the variable handedness. That is, the class of right handed people contains the same members as the class of women, and the class of left handed people contains the same members as the class of men, in our population. However, the two variables are only identical if their extensions are identical in every possible world. So, if the function that corresponds to the variable sex determines the same extension as the function that corresponds to the variable handedness, in every possible world, then the functions are identical. Following the insights of Carnap, I define variables as identical when they have identical intensions, and hence identical extensions in every possible world.
Taking intension into account. Identical variables correlate perfectly (i.e., are equivalent) in every possible world, regardless of the circumstances. They will never cease to correlate perfectly. When perfectly correlating variables cease to correlate perfectly, they are not identical and they have never been. They were equivalent, but they are not identical.
If F1 that explains v∗ in O1 stops to correlate perfectly with F2 that explains v∗ in O2,
then we know that F1and F2 are not identical, and hence they are not a single common factor
that underlies O1 and O2. When two factors correlate perfectly, it is almost impossible to
determine whether they are identical or ‘just’ equivalent. One way to determine whether factors are identical, is to take theory into account. Theory tells us that handedness and sex are not identical variables, because their meaning (or intension) is different. When we conduct a factor analysis we do not only conclude whether the shared variance is caused by one single factor just by looking at the statistical results of this analysis, but also by interpreting the items. More generally, theory enables one to distinguish between shared variance that is actually caused by
a common factor and shared variance that is not.
However, in practice, solid theories to interpret F1 and F2 (and to distinguish between the
two) are often unavailable. Put differently, it is not always possible to use interpretation alone to uncover that the common factor found with factor analysis, actually consists of highly correlating factors that differ in intension. In these cases, experimental intervention could be a solution. Experimental intervention introduces a variable that is believed to influence the common factor. Suppose again two tests, one with items that correlate with handedness and one with items that correlate with sex. Now suppose that a factor analysis has only identified handedness as an underlying factor that explains the shared variance. To determine whether or not handedness really is the common factor that explains the shared variance, one could teach a random sample of the population to write with the other hand. This random sample would include both men and women, since it is unknown that there actually is a second underlying factor (i.e., sex). If there had been only one common factor underlying the data patterns, the factor loadings would remain the same. However, in this particular example, the factor loadings on handedness will decrease, and a second factor will be identified.
Variables that happen to correlate highly due to unknown circumstances (and therefore are concluded to be one factor) can start to correlate less when we intervene experimentally. In this case factor loadings found before intervention will differ from the factor loadings found after the intervention took place. If the common factor really is one single factor, this would not happen.
Validity
In this section I explore what the distinction between the intension and extension of variables means for the notion of validity. My inquiry centers around two distinct approaches to validity: the ‘data approach’ and the ‘real-cause approach’. Within both approaches, validity is either defined as (1) the similarity of the extensions of variables, or as (2) the similarity of the intensions of variables.
Suppose a researcher wants to measure general intelligence. The researcher naturally starts by filling in box 4 of Figure 12, with the concept of the variable he intends to measure. Second, the researcher chooses a test that is believed to generate a data pattern that is caused by the intended variable (general intelligence). So, the researcher administers an IQ test and finds that the item scores (box 1 in Figure 12) share some variance.20 Now, according to the latent variable
model, this shared variance is caused by the common factor ‘general intelligence’. When this is indeed the case, then the researcher should conclude that the IQ test is a valid measure with respect to the intended variable ‘general intelligence’. Validity is not a property of a test itself; the IQ test mentioned in the above example is not a valid test with respect to the latent variable ‘depression’, or ‘extraversion’. After all, neither depression nor extraversion caused the IQ data. So, if validity is not a property of a test itself, then what is it? Below, I define validity within the parameters of the analytical framework proposed earlier in this paper.
Within the analytical framework proposed in this paper, the question of validity comes down to the question of whether or not the causal arrow between box 3 and box 1 in Figure 12 is present. After all, the observed variable (box 1/2) measures the latent variable (box 3/4) if and only if the extension of the latent variable causes the extension of the observed variable (see paragraph ‘Observed variables and measurement’). This section examines what this causal arrow entails within the framework proposed in this paper. Throughout this section, I refer to Figure 12 which applies the analytical framework to the latent variable ‘general intelligence’.
20One can only detect shared variance in multiple observed variables. For example multiple items that together
Extension Intension Obs erv ed/ Mani fest Unobs erv ed/ La te nt … IQ … … 94 … … 102 … … 111 … Numbers arising upon administering an IQ test. Intelligence Causes 4 2 Determines Determines 3 1
?
Variable IQ scoresThe degree to which an individual can solve cognitive problems. Function: F(…) F(x)= y
Figure 12: Conceptual framework of when a researcher intends to measure general intelligence with an IQ test.
A single observed variable measures all latent variables which cause that observed variable. For example, the observed variable generated by the question “how often do you attend to par-ties?” measures the latent variables: ‘going to parties’, ‘liking to go to parties’, ‘extraversion’, etc. However, psychological research usually involves more than one observed variable, for ex-ample, multiple items on a test. In this case, the different observed variables are preceded by different causal chains. A test with multiple items does not measure all these different causal chains. Instead, it measures the latent variable that explains the shared variance of these mul-tiple observed variables (i.e., the common cause). As a result, for a test with mulmul-tiple items, one variable can be determined that is closest to the observed variables in the causal chain, and explains the shared variance of all observed variables. I will elaborate on this in the subsection ‘what is theta?’.
In this section I discuss validity in the light of the common cause model. Hence, I only define validity for tests that intend to measure a latent variable by means of multiple observed variables. This is because, as aforementioned, multiple observed variables can point out one latent variable that is being measured by the test, instead of the whole causal chain that precedes a single observed variable.
I discuss two distinct approaches to validity: the data approach and the real-cause approach. The data approach is in line with common intuition on validity since it looks for validity in the relation between the observed data and the variable that one intends to measure. It defines validity either as (1) the similarity between the extensions of the observed variable and the intended variable, or (2) as the similarity between the intensions of the observed variable and the intended variable. The ‘data approach’ comes with inherent problems. The ‘real-cause approach’ intends to overcome those problems by introducing ‘theta’, the variable that actually caused the observed data pattern. The ‘real-cause approach’ defines validity either as (1) the similarity between the extensions of theta and the intended variable, or (2) as the similarity
between the intensions of theta and the intended variable. I will demonstrate that error does influence validity in the data approach (the less reliable a test is the less valid it is), but does not influence validity in the real-cause approach of validity.
Data approach to validity. The data approach defines validity as a property of the relation between the observed variable and the intended variable. So the IQ test is a valid measure of general intelligence when the observed variable obtained with this test (box 1/2) is similar to the latent variable this researcher intends to measure with this test (general intelligence, box 3/4). The distinction between intension and extension of variables induces a comparison between (1) the extensions of the observed and the intended variable or (2) the intensions of the observed and the intended variable.
To compare the extensions of the observed and the intended variable, is to examine the correlation between the observed data pattern and the extension of general intelligence (i.e., evaluate the similarity between box 1 and box 3). So, in this approach, validity is defined as the degree of similarity between the extension of the observed variable (e.g., the data pattern of IQ scores) and the extension of the intended latent variable (e.g., the extension of general intelligence). This definition of validity has some implications. First, it implies that random error decreases the validity of a test with respect to the intended variable. For example, if we measure sex with a question like ‘are you male or female?’, the correlation between the ones and zeros in the data pattern and the ones and zeros as a result of the sex variable applied to the same group of individuals, will be smaller if more people per accident circle the wrong answer. Like random error, systematic error decreases the validity of the test as well. Consider the case in which a factor, independent of general intelligence, influences the performance on math questions, but not on other questions of the IQ test. In this case, the factor decreases the correlation between the IQ scores and the outcomes of the general intelligence function applied to the same group of people. In sum, when validity is defined as the degree of similarity between the extensions of the observed and intended variable, then the reliability of a test influences the validity of that test with respect to the intended variable. A less reliable test will be evaluated as less valid as well.
Second, when validity is defined as the degree of similarity between the extensions of the observed and intended variable, then a test is as valid with respect to the intended variable as it is valid with respect to all the variables that correlate perfectly with the intended variable. For example, suppose that general intelligence correlates perfectly with age. In this case, an IQ test would be just as valid with respect to the variable ‘general intelligence’ as to the variable ‘age’. After all, the correlation between IQ scores and general intelligence would be equal to the correlation between these IQ scores and age, as long as age and general intelligence correlate perfectly. Moreover, the observed variable is a valid measure for all variables that correlate with the observed variable itself. This is problematic since validity is supposed to be a property of measurement, and measurement, as I defined it, implies a causal relation between the intended variable and the observed data pattern. So, an IQ test is only valid with respect to general intelligence, when the observed data pattern is caused by the latent variable ‘general intelligence’. Correlation between the extensions of the observed and intended variable is not sufficient.
The second way to define validity within the data approach is to define it as the degree of similarity between the intensions of the observed and intended variable (box 2 and box 4 of Figure 12). So, in this definition, a test is valid when the function of the intended variable equals the function of the observed variable. Yet, as I explained earlier, the function of the observed variable almost always differs from the function of the intended variable, and it is almost impossible to identify the degree of similarity between the two. Moreover, when the intension of the observed variable needs to be similar to the intension of the intended variable,
