CAN LATENT VARIABLES IN INTERINDIVIDUAL DESIGNS
BE A WITHIN SUBJECT’S CAUSE?
Assignment: Bachelor’s thesis Name: Leonie Poelstra Student number: 5870518 Major discipline: Psychology Mentor: Lisa Wijsen Words: 7603
ABSTRACT
During the past decades many researchers have addressed the importance of the study of time- dependent variation within subjects. Current measurement models and methods fail to give results for which the causal effect of a latent variable can be interpreted on the intraindividual level. In these models the score of the individual is a constant. However, there must be variation on the intraindividual level to give a within- subject causal interpretation of latent variables. A solution for this problem would be to give a counterfactual account. For this to be legitimate, the processes on the individual level would have to be equivalent with the processes on the interindividual level. To analyse if the condition of local homogeneity in negative and positive affect is met, the results of a R- technique factor solution (N= 52) and P- technique factor solution for three subjects (N= 92, N= 88 and N = 103) are reported. The analyses and results show a variety in latent variable structures and variability at the individual level.
INTRODUCTION
For many years the individual research differences paradigm has dominated psychology (Lamiell, 1981). Since the introduction of the common factor model by Spearman in 1904, psychological testing has focused mainly on the analysis between subjects. In factor analysis, researchers investigate the structure and effect of latent variables through the analysis of individual differences data (Borsboom, Mellenbergh & van Heerden, 2003). The goal of these models is to establish the number and nature of factors that account for the (co)variation among a set of indicators. In these models factors are unobservable variables that influence more than one observed measure and account for the correlations among these indicators (Brown & Moore, 2012). However, in recent years many researchers have addressed the importance of the study of time- dependent variation within subjects (Molenaar & Campell, 2009; Lamiell, 1981; Borsboom et al., 2003). In standard statistical methods the individuality of each of the subjects in the sample is deemed immaterial. This means that persons do not qualitatively differ in structure for psychological processes. It is assumed that subjects are homogeneous in all aspects relevant to the analysis. This is one of the most important assumptions in individual differences research where the study of variation on the latent variable between- subjects is central (Molenaar, 2006). Meanwhile the acceptance of homogeneity in the population has been widely criticized, since the individual characteristics
of persons are not fully investigated. A human being, as Molenaar & Campell (2009) argue in their article, is a dynamic system of behavioural, emotional, cognitive and other psychological processes evolving over time and place. People could have different processes on the individual level, but in our models we assume that subjects are exchangeable and thus we ignore their individuality. With intraindividual research, investigators want to “tap” these processes across different time points. Until recently researchers have used interindividual designs to draw conclusions about an individual’s scores that might not be valid from a methodological point of view. These critics have led to a new research paradigm, in which the study of the intraindividual processes are central; “Bringing the person back into scientific psychology, this time forever”, Molenaar (2004). In contrast, during the past centuries both practitioners and researchers have interpreted latent variables based on interindividual designs as within- subjects causes. Since it is not clear if these interpretations are allowed or which conditions should be met before addressing these latent variables as within- subject causes it is necessary to give both a theoretical overview and empirical evidence to the main question of this thesis; Can latent variables based on interindindivual designs be a within subject’s cause? First I will discuss the problem with causal inference in psychological research based upon individual differences designs. Then the conditions for ergodicity in psychological processes stated by Molenaar (2004) will be explained. Finally, empirical data has been analysed with factor analysis to study the compatibility of latent variable structures based on inter- and intraindividual designs.
THE PROBLEM WITH CAUSAL INFERENCE
In their article “The Theoretical Status of Latent Variables”, Borsboom et al. (2003) stated that standard psychometric models allow a causal interpretation only in a between- subjects sense: “Individual differences in the latent variable may cause individual differences in test scores in the population, but the latent variable has no causal relevance at the level of the individual”. The authors illustrate that in latent variable models there are two types of causal statements that one can make. The first statement holds that the researcher could say that differences in position of the population on the latent variable cause population differences in the expectation of the item responses. The second interpretation corresponds to the stochastic subject interpretation and poses probabilities at the level of the individual (Borsboom et al., 2003). Before both types are further discussed it is necessary to take a closer look at the
causality. In the light of factor analysis, causality can be thus described when meeting the following conditions; (a) there is a covariation between variability on the latent variable X and the variability in the item responses Y; (b) the difference in position on the latent variable X precedes the difference in expected item responses Y; (c) if there is no variability in the latent variable X, there is no variability in the item responses Y (while keeping other variables constant) (Borsboom et al., 2003).
The first causal statement addresses the influence of a latent variable on the level of the population. For example, population differences in position on the latent variable extraversion (X) cause population differences in the expectation on extraversion’s indicators (Y). This statement corresponds with between- subject designs, namely the (co)variation on the latent variable between- subjects explains the scores on the extraversion indicators in the population. The second statement on the within subject level however can not be made with standard between- subject factor analysis. This is a problem since in many psychological fields (HRM, clinical psychology, intelligence measures, etcetera) people are actually more interested into answering this question. In the case of extraversion, the conclusion then could be as follows; “Leonie talks a lot because she is extraverted”. Looking at our first condition for causality (a), it is concluded that X and Y do not covary on the individual level. In standard measurement models the position of an individual on a latent variable is assumed to be a constant (Borsboom et al., 2003). And when a variable is a constant, there is an absence of variation on the latent variable which could explain the differences on the item’ scores. For a latent variable to have a causal effect, there must be variation in the latent variable otherwise it would be impossible to influence (and thus cause) an individual’s score. In common measurement models, the variation between subjects is the source of the covariation. To give a within- subject causal interpretation of latent variables as in the example above, there must be variation in the within- subject latent variable extraversion (if this variable then exists on my intraindividual level). Otherwise this latent variable could not have a causally effect and therefore the statement is false.
One way to ostensibly get around this problem is to call in a counterfactual account of causation (Borsboom et al., 2003). In the case of my extraversion scores, one could say that if I had a different position (e.g. the position of Oprah Winfrey) on extraversion, I would have produced different item scores (presumably higher). But again this is a not a within- subject formulation of causality since Oprah Winfrey and I are not the same person. The difference in score is again based upon interindividual variability and does not provide information about the influence of my extraversion on the scores. Thus, knowledge based upon individual
research differences is then erroneously transferred to the intraindividual level. The problem with respect to this interpretation is the fact that intraindividual processes are ignored, and latent variables based upon interindividual designs are said to have causal power within persons in absence of justification.
According to Molenaar (2004), the inference that because a latent variable varies among individuals in a population it also varies within persons is a fallacious argument (Weinberger, 2015). Weinberger responded to this statement by arguing that if a latent variable is a cause in the population, it is indispensable also a cause in (at least one of) its members. In his view, a latent variable based upon interindividual designs can be interpreted as a cause within- subjects. In the literature it has been found that many psychological processes are theorized to have different structures on the individual level. It is possible however, that a cause on the between- subject level is not necessarily present as a cause within- subjects. To illustrate this problem, suppose we have three subject’ scores on an extraversion item i (𝑋𝑋11 = 1, 𝑋𝑋12 = 2 and 𝑋𝑋13 = 3). If we then would measure extraversion over time for these three subjects, it could be possible that each subject shows a constant score of respectively 1, 2 or 3. In this case there is variation between subjects but not within subjects. Thus variation among the individuals in a population at a time t will not necessarily reflect the variation within any of its members over time.
An interesting question that rises based on the aforementioned problem is whether there are psychological processes for which the structure based upon variation between- subjects is similar to the structure based upon within- subjects. For these psychological processes, it could be possible to interpret results of interindividual variation on the level of the individual (Molenaar, 2009). If the processes on the individual level are equivalent with the processes on the interindividual level, subjects are interchangeable and therefore we could give a within interpretation. Our reasoning then would be as follows; If I would have an extraversion level of Oprah Winfrey (assuming that Oprah is very extraverted), then I would have had higher scores on extraversion. Normally such an interpretation cannot be made since we do not know if Oprah and I are comparable on the individual level and share the same structure for extraversion. However, for some psychological processes this could be the case, if they obey to stringent conditions based upon ergodic theory which will be further discussed in the next paragraph (Molenaar, 2009). So although in between- subject analysis models are quiet regarding the homogeneity of individuals, for ergodic processes we know they are in fact homogenous for the concerned psychological process. If we then return to our
the idea of using a counterfactual. The importance here lies in the fact that for many psychological processes based upon interindividual designs, we have been assuming that they explain behaviour/ scores on the level of the individual. As stated above, only under very stringent conditions this interpretation is allowed.
CONDITIONS FOR ERGODICITY
In his article, Molenaar (2004) gives two conditions for a process to be ergodic, and thus for which it is possible to transfer the latent variable from the inter- to the intraindividual level. Ergodicity addresses the following question: “Given a set of selected variables, under which conditions will an analysis of interindividual variation yield the same results as an analysis of intraindividual variation” (Molenaar & Campbell, 2009). The first condition of ergodicity states that the same statistical model should apply to the data of all subjects in the population, this refers to the homogeneity of the population. For this condition in factor analysis to be met, both the number of factors as the factor loadings patterns in factor analysis should be invariant across subjects. These factor loadings are the regressions weights that indicate the strength of the relationship between the observed variables and the factor(s). Secondly, for the next condition to be met, the data must be stationary. This posits that the data must have invariant statistical characteristics across time (e.g. having a constant mean and variance). Some examples wherein this condition is violated are developmental processes and learning processes. Molenaar & Campell (2009) give an example in the case of intelligence; the factor loadings explaining the strengths of associations between the factor intelligence and observed variables (e.g. verbal and math tests) change during cognitive development. However, the violation of the first condition of local homogeneity seems less intuitive for latent variables that seem quite constant in their characteristics (e.g. anxiety or mood). For these latent variables the same intraindividual processes might explain the scores on manifest variables. Then the statistical model in the population could fit on individual subjects, since people differ in the same manner at both the within and between level. If a psychological process in concerned to be nonergodic, then one of the two conditions is not met wherefore the structure of interindividual variation differs from the intraindividual structure.
In this study the condition of local homogeneity will be further examined. It would be interesting to test if for latent variables that seem qualitatively identical across persons the same structure is found in individuals. Previous research shows that in many cases local homogeneity is in fact violated. Borkenau and Ostendorf (1998) fitted a five factor model of
personality (Big Five) on subjects and showed that for many individuals a different factor structure has been found with a different intraindividual time series design. Some individuals had a one- factor structure explaining the results, while others had two, three or even more factors that could explain de item correlations. Borkenau & Ostendorf (1998) discussed how these results could be explained. They state that perhaps the Big Five model which is used to describe individual differences in personality traits is not useful to measure states in personality. So the model can be used for enduring personality traits, but not for the measurement of intraindividual fluctuations. Furthermore, they observed that the manner in which the factors related to the items were different for each individual (expressed by the factor loadings). In another study, Shifren, Hooker, Wood and Nesselroade (1997) found a rich variety of structures and variability at the level of the individual. They conducted a study among chronically ill participants to study mood variations. Although the sample was homogeneous regarding disease type, functional abilities and cognitive impairment, the results showed heterogeneous structures and variations of mood. These studies show that the risks of nomethetic generalizations concerning mood structures both within and across groups should not be downplayed. Since little research has been done in this manner, in this study important to see if for latent variables that are assumed qualitatively identical (e.g. mood and anxiety) the structures differ for each individual.
R- TECHNIQUE AND P- TECHNIQUE
In order to assess intraindividual structures Cattell (1947) developed a method based upon R- technique (the R refers to the R- correlation matrix in between- subject designs) called P- technique. The difference between both techniques is visualized en presented in Figure 2 and 3.
Figure 2. Cattel’s three dimensional box; P- technique analysis (Oxford Handbook of Positive Psychology, 2009).
Figure 3. Cattell’s three dimensional box; R- technique factor analysis (Oxford Handbook of Positive Psychology, 2009)
In essence, P- technique is a longitudinal factor analysis, whereby the common factor model is applied to the variance- covariance matrix of a subject’s multivariate time series data. The technique examines the correlations between occasions, in other words how strongly an individual’s position on an independent variable is related to their position on the dependent variable across occasions (Lee & Little, 2012).
In their article, Lee & Little (2012) argue two limitations with respect to classical P- technique. The first shortcoming is the fact that the common factor model does not account for the time- dependent nature of factors. Thus, the model assumes that a construct at time t has an influence on the observed variable only at that time t. This is a problem if we expect that the state of an individual at time t is dependent on the previous state(s) within a subject. However, research has been conducted by Molenaar & Nesselroade (2009) and they found that classical P- technique is in some cases also able to recover parameters of dynamic processes. Moreover, for some latent variables it can be theorized that stationary is present since it is not expected that the statistical characteristics change over time. In this study therefore the condition of local homogeneity is central.
The second limitation in P- technique models is that they implicitly specify that a construct has a constant nature by assuming that factor loadings are invariant across occasions (Lee & Little, 2012). In the case of intelligence, we would expect the g factor to influence the test scores in the same way and degree on every occasion. This actually does
not have to be true, since the factor structure might change over time. If the structure changes, it would then be conceptually difficult to perform analysis since we would have a different factor model for each individual on a different time t. On the contrary, it is arguable to keep the model as simplistic as possible if we want to use factor analysis to make abstract inferences and the model fits. Also, it is arguable that for many psychological constructs the nature of the construct could be constant, and the variation in the model is due to the position of a subject on a stochastic variable.
To analyse if local homogeneity in a psychological process can be met, a latent variable with qualitatively identical features for individuals would have to be studied. Latent variables as mood or anxiety could have the same structure on the within- subject level, since the dimension on which persons differ from themselves might correspond to the dimension on which persons differ from each other (Borsboom et al., 2003). A two- factor interindividual structure of mood (positive and negative affect) has been found with young, middle and older adult samples (Kleban et al., 1992; Watson, 1988, cited in Shiffren et al., 1997). Other earlier studies on mood have found similar results of R- technique and P- technique analyses (Russell, 1980,; Watsom & Tellegen, 1985; Zevon & Tellegen, 1982, cited in Shiffren et al., 1997). Zevon & Tellegen (1982) identified the factors positive and negative affect in the self- reports of 21 participants in the sample. The similarity between each subject’s factors, the R- factors and P- composite factors (the latter were based on the median loadings on factors across participants) was evaluated by coefficients (factor loadings) of congruence. When they averaged the factor loadings from their P- factor analyses across their 23 participants, the average loadings correlated .97 for positive affect and .94 for negative affect. On the contrary, Feldman (1995) found systematic differences in the P- correlations between the participants in mood structures. This could indicate that congruence between R- factor and P- factor solutions is obtained only if the P- factor loadings (or correlations) are averaged across participants. Thus, the question if the same structure is found on the interindividual and intraindividual level remains unclear. To see if the condition of local homogeneity holds for the structure of mood we conduct a factor analysis on the two- factor structure of positive and negative affect. In the analysis below first a factor model with R- technique has been fit on the between- subject data, next a P- technique factor analysis has been conducted for three randomly chosen participants to study structure for the within- subjects.
METHOD Participants
Researchers at the University of Amsterdam have collected data of participants in mood over repeated measurements in time. Each participant had to fill in a questionnaire on depression items at seven time points per day (next to this, participants had to answer a question about arousal). Participants were gathered trough the website https://www.lab.uva.nl/lab, social media and mouth to mouth. There were 60 participants attending whereof 52 have been used to run this analysis due to random missing data in participants. If the participant was a student, they were able to gather 6 so called “participation points” for their curriculum. Overall most of the participants were students and lived in Amsterdam and surroundings. The inclusion criteria were age between 18 and 65 years and the possession of an Iphone or iPod touch. Data on gender were collected in the original study, 14 participants were female. In the present study data on gender is disregarded in the analysis.
Measures
The data was collected with the use of the app Qumi, build by Bas Oppenheim (2016). Installation of the app was possible trough Testflight in the apple store and both the pre- measurement questionnaire as the repeated measurements were collected trough this device. The pre- measurement existed of 56 questions and the repeated measurement questionnaire existed of 13 questions. In this study the items for negative (five items) and positive affect (four items) have been used for analysis. Participants had to fill out the questionnaire at seven time moments per day over a time period of two weeks. The app collected the data anonymously and collected how many times each participant had filled out the questionnaire. Each time the app had send an alert, participants had to fill out the questionnaire within twenty minutes. After twenty minutes the time had expired and for this time moment the data would be missing. These alerts where spread out over each day and secured the time interval between measurements (Bas Oppenheim, 2016).
The data that has been used is part of a larger dataset; “Depression networks bachelorproject UvA 2016”. The items represent symptoms of depression based on the DSM IV whereof four items are said to show negative affect: Stress, Mood, Tiredness,
Worthlessness and Guilt. The other items Interest, Appetite, Energy and Focus are indicators of positive affect. Participants were able to answer the items on a 1 to 5 Likert- scale (1. very little, 2. a little 3. moderate, 4. much and 5. very much).
TABLE 1
Items depression in Qumi app build by Bas Oppenheim (2016)
Item 1: Stress I just had a stressful event
Item 2: Mood I feel down
Item 3: Interest I am interested
Item 4: Appetite I have an appetite
Item 5: Energetic I feel energetic
Item 6: Tiredness I feel tired
Item 7: Worthlessness I feel worthless
Item 8 : Guilt I feel guilty
Item 9 : Focus I feel focused
ANALYSIS
Between- subject analysis
To examine if the positive and negative affect- (two factor) structure in the between- subject’s data is present the R- technique has been used. Since the data is ordinal (Likert- scale 1 – 5), instead of continuous, the polychoric correlations instead of the Pearson correlations have been used in the analysis. The polychoric correlation estimates the linear relationship between two unobserved continuous variables when only observed ordinal data is given. Thus, the observed discrete values are theorized to be due to an unobserved underlying continuous distribution (Flora & Curran, 2004). The correlations are presented in Table 2 below, most of the positive and negative affect measures show a reasonable correlation with the other indicators of the same factor. The item Energy shows however also a strong negative correlation with the indicators of negative affect, which is plausible since Tiredness is an indicator of negative affect. The item Stress shows an unexpectedly low
correlation with Worthlessness and Guilt, the same is true for the item Focus with the other indicators of positive affect.
TABLE 2
Polychoric correlations measures R- technique
Measure 1. 2. 3. 4. 5. 6. 7. 8. 9. 1. Stress - 2. Mood .39 - 3. Tiredness .37 .54 - 4. Worthlessness .07 .72 .28 - 5. Guilt .17 .77 .47 .81 - 6. Appetite -.07 -.18 -.01 -.1 -.07 - 7. Energy -.39 -.41 -.59 -.19 -.21 .35 - 8. Interest .05 -.08 -.15 -.42 -.13 .32 .33 - 9. Focus .07 -.3 .3 .17 -.11 .14 .43 .25 -
Note. Significant correlations are in boldface.
To estimate the model parameters in many cases the maximum likelihood (ML) method is used. However, since the factor analysis’ assumptions of (a) a large sample size (N = 52), (b) indicators that are measured on a continuous scale and (c) the distribution of the indicators is multivariate normal are violated, the weighted least squares (WLS) method has been used to estimate the parameters (Brown, 2015).
To determine the number of factors a parallel analysis is performed to compare the scree of the observed data with the scree of a random data matrix of the same size. Although the negative and positive affect structure is confirmed earlier in theory, the questionnaire in Qumi is new and has not been checked for reliability. Thus in this analysis the factor- item structure is studied first. In parallel analysis the rationale is that the factor should account for more variance than is expected by chance (a random data matrix). A scree plot of the factor numbers in the data is presented in Figure 4.
Figure 4. Parallel analysis between- subject data.
According to the parallel analysis there are two factors that explain more of the variance in the model than by chance. The eigenvalues of factors are 2.48 for the first and 0.6 for the second factor. These represent the variance in the indicators explained by the successive factors (Brown, 2015). Since the second factor has an eigenvalue below 1, according to the Kaiser Guttman- (eigenvalue >= 1) rule this would be too low and there would be only one factor in the model (Field, 2009). Lastly, if we look at the point of inflexion in the scree plot it is theorized there are three factors, but the third factor would not explain much of the variance in the model. Since our theory states there are two factors (positive and negative affect) this seems the most likely solution.
Next to foster interpretability the two extracted factors are rotated to achieve a simple structure; indicators load highly on one factor and each indicator (ideally) that has a high loading on one factor has a trivial or close to zero loading on the other factor. Since positive and negative affect are presumably correlated we use an oblique rotation. This method allows the factors to correlate since the factor axis orientations can be less (or greater) than 90°. In orthogonal rotation the factors are oriented at 90° angles in the two- dimensional space. Since cos (90) = 0 the factors in this case would be uncorrelated (Brown, 2015). Next an exploratory, two- factor model was fitted to the (9,9)- dimensional polychoric correlation
of variance explained by the factors for the indicators) we can see that for the items “Stress”, “Interest”, “Appetite” and “Focus” the communalities are below .3. So the model does not explain much of the variance for these indicators. Furthermore, the item “Interest” has been found not to have expected loadings on either the first or second factor above .3 or under -.3. This item is therefore removed in further analysis.
To perform the R- technique analysis a confirmatory two- factor model was defined and fitted to the (8,8)- dimensional correlation matrix. The items “Tiredness” and “Energy” had residual correlations in the model, just as the items “Focus” and “Tiredness”, since these items possibly share variance that is not explained by the factor Negative Affect but other factors (e.g. sleep or exercise).
Both the WLS and robust weighted least squares (RWLS) solution were given in the output. Since the robust solution has the best performance in estimation for ordinal variables these outcomes are reported (Midi, Rana & Imon, 2009). The absolute chi- square test reported a good fit with 𝜒𝜒2(17, N = 52) = 19.02, p > .05. Since the fit index the root mean square error of approximation (RMSEA) is relatively insensitive to sample size, this index is also given; RMSEA = .05, which shows a good fit of the model (Brown, 2015). The salient standardized loadings are presented in Table 3.
TABLE 3
R- Technique standardized Factor Loadings Factor
Items Negative affect Positive affect
Stress .40 .00 Mood .90 .00 Tiredness .55 .00 Worthlessness .82 .00 Guilt .89 .00 Appetite .00 .34 Energy .00 .95 Focus .00 .46
Most of the loading coefficients seem to resemble patterns found in previous studies, which implies a two- factor structure with high loadings of the items on the explanatory variables. In the next section the P- technique factor models of within subject data will be further examined to be compared to the between subject R- technique analysis.
Analysis first participant
To examine if the positive and negative affect- (two factor) structure for the first participant holds, first a check of difference in scores on items has been performed. This check was conducted to analyse if the subject’s scores differ enough for each item over occasions. The means and standard deviations of the negative and positive affect items of the first participant are presented in Table 4.
TABLE 4
Means and standard deviations first participant
M SD Negative affect Stress 1.24 0.79 Mood 1.68 0.82 Tiredness 2.20 0.63 Worthlessness 1.84 0.68 Guilt 2.11 0.61 Positive affect Appetite 1.80 0.71 Energetic 2.32 0.67 Interest 2.19 0.69 Focus 1.94 0.81
The item Stress showed in 90% of the cases too stable responses (response = 1). Therefore, this item has been removed in further analysis since there is no variability for this participant’s item on occasions.
Since the two- factor structure of positive and negative affect is based upon between- subject designs, first an exploratory factor analysis has been conducted to explore the within
factor structures. Again a parallel analysis is performed to compare the scree of the observed data with the scree of a random data matrix of the same size (Figure 5).
Figure 5. Parallel analysis first participant.
As visualized in the plot there are two factors that explain more of the variance in the model than by chance. The eigenvalue of the first factor was 1.8 and of the second factor 0.89, which indicates a two- factor structure for the first participant.
The exploratory two- factor model with oblique rotation was fitted to the (8,8)- dimensional polychoric correlation matrix by the method of weighted least squares (number of occasions = 88). Looking at the communalities it has been found that for the items Appetite and Guilt the communalities are below .2. Furthermore, the item Appetite did not have an expected loading on either the first or second factor above .3 or under -.3. This item is therefore also not specified in the confirmatory model. Next to this, the item Tiredness showed a high loading on both the first and the second factor. A cross loading has been added in the model for this item on the second factor. The modification indices showed an improvement in the model when the residuals of Tiredness and Energy, and Tiredness and Mood are allowed to correlate. For the latter residual correlation, it could be that if this participant is tired, the scores on mood are also affected because of irritability or grumpiness. This correlation is not explained by the model per se, since overall negative affect would not
explain this variance. When we fit the model to the (7, 7) polychoric correlation matrix the absolute fit showed 𝜒𝜒2 (10, N = 88) =12.87, p > 0.05. Second, the RMSEA indicates an acceptable fit of 0.06.
In Table 5 the factor loadings of the first participant are presented. Although the number of factors for this participant matched with the two- factor structure in R- technique, still the structure differs significantly. First, the item Stress and Appetite are no indicators in the model for this participant. Next, especially the items Mood and Guilt load less strong on the negative affect factor than in the R- technique solution (for the latter respectively .90 and .89). The same is to true for the positive affect item Energetic (R- technique solution = .95). Furthermore, the item Interest has both a high loading and reasonably high communality in this participant’s model structure, while in the R- solution this item did not explain any of the variance in the model at all. Lastly, the item Focus has a much higher factor loading than was found in the R- technique solution (.46).
TABLE 5
P- Technique standardized Factor Loadings first participant Factor
Items Negative affect Positive affect
Mood .55 .00 Tiredness .24 .00 Worthlessness 1.3 .00 Guilt .39 .00 Tiredness .35 .00 Interest .00 .94 Energetic .00 .51 Focus .00 .84
Note. Salient loadings are in boldface.
Analysis second participant
The means and standard deviations of the second participant on each item are presented in Table 6. The participant scored “1” on the Likert- scale for the items Mood and Guilt for 96%
of the time, and even for the item Worthlessness in 98% of the cases. If we remove these three items which are all indicators of negative affect, this factor would only be identified by the items Stress (which also shows almost no variability; score 1 in 88% of cases) and Tiredness. Since the factor of negative affect would be under identified in the model, the model would probably have serious misspecification. Therefore, the two- factor model is rejected and we fit a one- factor model on this participant’s data over 103 occasions only.
TABLE 6
Means and standard deviations second participant
M SD Negative affect Stress 1.20 0.63 Mood 1.04 0.19 Tiredness 1.56 0.95 Worthlessness 1.04 0.31 Guilt 1.08 0.44 Positive affect Appetite 1.49 0.77 Energetic 3.58 0.76 Interest 3.79 0.57 Focus 3.55 0.72
Figure 6. Parallel analysis second participant.
The parallel analysis of the factor numbers in the data is presented in Figure 6. Indeed, the plot shows one factor that explains more of the variance in the model than by chance. This factor has an eigenvalue of 2.22.
An exploratory, one- factor model with orthogonal rotation was fitted to the (6,6)- dimensional polychoric correlation matrix by the method of weighted least squares. The communalities of the items Stress and Appetite are close to zero. Also, both items have been found not to have expected loadings above .3 or under -.3. These items have thus been removed in further analysis, which leaves a one factor structure with four indicators (Focus, Energy, Interest and Tiredness) and a residual correlation between Tiredness and Energy.
The confirmatory analysis on the (4,4) correlation matrix showed an absolute good fit with 𝜒𝜒2 (1, N = 103) = 0.764, p > 0.05. The RMSEA is zero. However, df = 1 so this model is not parsimonious at all. Since WLS estimation method has been used instead of ML estimation the Bayesian Information Criterion (BIC) could not be calculated to correct for parsimony.
TABLE 7
P- Technique standardized Factor Loadings second participant Factor
Items Negative affect Positive affect
Stress .00 .00 Mood .00 .00 Interest .00 .73 Energy .00 .96 Tiredness .00 -.78 Worthlessness .00 .00 Guilt .00 .00 Focus .00 .90
Note. Salient loadings are in boldface.
Both the number of factors as the factor loadings pattern fail congruence with the R- factor solution. The negative affect factor is evidently absent in the solution, while the loadings for positive affect are much higher than for the between subject’s data.
Analysis third participant
The means and standard deviations of the third participant on each item are presented in Table 8.
TABLE 8 Means and standard deviations third participant
M SD Negative affect Stress 1.25 0.52 Mood 1.44 0.58 Tiredness 2.05 0.77 Worthlessness 1.15 0.39 Guilt 1.23 0.51 Positive affect Appetite 2.57 0.85 Energetic 2.54 0.64 Interest 2.99 0.70 Focus 2.53 0.58
None of the items showed too little variability in the scores, so all items are further conducted for analysis. The parallel analysis of the factor numbers in the data is presented in Figure 7.
Figure 7. Parallel analysis third participant.
According to the plot there are two factors that explain more of the variance in the model than by chance. The first eigenvalue is 1.77 and the second eigenvalue is 0.88 which are both sufficient and above 0.7 (Field, 2009).
The exploratory, two- factor model with oblique rotation was fitted to the (9,9)- dimensional polychoric correlation matrix by the method of weighted least squares (number of occasions = 93). If we look at the communalities we can see that “Appetite” explains almost nothing in the model and does not have a sufficient loading on one or two factors. Thus this item is removed in further analysis. Next we fit the confirmatory model to the (8,8) dimensional matrix and a cross loading from the positive affect factor to Tiredness has been added. Again the residual correlation between Tiredness and Energy is specified. A chi- square statistic has been found of 𝜒𝜒2 (17, N = 93) = 24.68, p > 0.05. The RMSEA shows an acceptable fit of 0.07.
The analysis showed that for the items Stress and Tiredness the pattern loadings are not significant (Table 9). However, for the items Guilt, Worthlessness and Mood the loadings somewhat resemble the structure found in R- technique. Interestingly for this participant the positive affect factor loadings are again higher than in the R- solution (and present in the case of the item Interest) except for the item Energetic. Although two factors are present, the regression coefficients show a different structure for this participant’s data than has been found in the R- solution.
TABLE 9
P- Technique standardized Factor Loadings third participant Factor
Items Negative affect Positive affect
Stress .09 .00 Tiredness .11 -.47 Guilt .55 .00 Worthlessness .88 .00 Mood .61 .00 Interest .00 .85 Energetic .00 .60
Focus .00 .69 Note. Salient loadings are in boldface.
DISCUSSION
The analyses and results show a clear variety in latent variable structures and variability at the individual level. Both the number of factors, the factor loadings as the strength of the regression weights differed for each subject, which means that the local homogeneity condition has not been met and thus ergodicity is violated in this study. Since only for ergodic processes it would be possible to transfer results from the interindividual to the within- subject level, it seems that for the latent variable mood this is not legitimate. Thus in this case, the latent variable based upon an interindividual design can not be interpreted as a within subject cause.
Furthermore, some participants did not show variability on items at all. To explain the latter result, Borkenau & Ostendorf (1998) argued the possibility that subjects had interpreted the items as enduring traits instead of states. This explanation seems nevertheless not exhaustive, since it does not explain why for some items in participants there is variability and for others not. Even more, the absence or prevalence of variability differed over subjects and items. This might be evidence for the theory that factor structures within- subjects actually do differ, and that latent variables based upon interindividual designs have a different structure than for the within subject’ structures. The idea that a latent variable based upon interindividual designs must be a cause in at least one of its members seems now harder to defend (Weinberger, 2015). Somehow researchers expect people to show within variability on items that are based upon between- subject studies. This idea could be outdated, and it might be the case that some factors that are based upon these designs have no causal influence on the level of the individual since they are a constant on the intraindividual level.
Of course, these conclusions can not be taken without further discussion. One reason to interpret these results carefully is related to the (un)reliability of the items. Although they were based on DSM IV criteria and previous studies and literature, some items might not have captured the explanatory factor adequately. The item Stress was posed as; “I just had a stressful event”, which is not an indicator of negative affect per se. Persons without negative affect could also have had a stressful event, or persons with negative affect could not have had a stressful event but could still feel stressed. Also, the item Interest was formulated as; “I feel interested”, only this item is quite ambiguous regarding the object of interest.
Participants could have had trouble in interpreting the question, an alternative item could be loss of interest which is often related to depressed mood. Lastly, there was no variation in the presentation of the items, which could affect the answering of participants due to boredom and routine. If the questions would have been addressed in different ways, there might have been more variability in the scores.
A second observation was that especially the items of negative affect showed almost no variability in some subjects. This could be due to the fact that the sample existed of mostly students, which are not a true random sample of the general population with respect to education, wealth, SES etcetera. It could be theorized these students feel less worthlessness, guilt or depressed mood in general. On the other hand, lack of variability in negative affect factors have been found in earlier studies as well (Shiffren et al., 1997). An explanation would then be that for mentally healthy participants, there simply is not much variation on this factor due to absence of symptoms.
Another explanation for the lack of variability is the fact that all the measurements were taken place within two weeks. Although participants had been measured at seven time occasions each day, it might be more informative to collect data over a longer period of time at for example one occasion per day. Persons might nog fluctuate enough within one day to capture within- subject variability well.
A methodological issue was that for the between- subject analysis the number of participants was quite low. Rules of thumb suggest at least ten subjects for each item, which would correspond to 90 participants at least. On the contrary, in the P- technique analysis the number of measurements for each participant was sufficient. Since the solutions for participants still differed on the within- subject level, this flaw may not effect the final conclusion.
Finally, in this analysis only three participants were used to analyse the time- series within subject data. To fully explore the possible structures in the data, it could be interesting to see if the structures of individuals have corresponding features that can be clustered together. For clustered individuals with equal structures, it might be possible that the latent variables in the model could have causal power for subjects belonging to the corresponding group. With for example a multiple group approach each cluster in the sample is considered as a group and the condition of local homogeneity could be tested. If the initial test is significant (the model does not fit for all individuals), follow- up tests should be implemented to identify which individuals differ in structures (Lee & Little, 2012). This technique might
be of use to study groups of patterns in order to find models that explain behaviour for individuals.
ACKNOWLEDGEMENTS
I would like to thank Claudia van Borkulo and Milou Wattel for sharing and providing their data used in the analysis.
REFERENCES
Borkenau, P., & Ostendorf, F. (1998). The Big Five as states: How useful is the five-factor model to describe intraindividual variations over time?. Journal of Research in Personality, 32(2), 202-221.
Borsboom, D. (2015). What is causal about individual differences?: A comment on Weinberger. Theory & Psychology, 25(3), 362-368.
Borsboom, D., Mellenbergh, G. J., & Van Heerden, J. (2003). The theoretical status of latent variables. Psychological review, 110(2), 203.
Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford Publications.
Brown, T. A., & Moore, M. T. (2012). Confirmatory factor analysis. In Handbook of structural equation modeling. (pp. 361 – 379). New York, NY: Guilford Press.
Field, A. (2009). Discovering statistics using SPSS. New York, NY: Sage Publications.
Cattell, R. B., Cattell, A. K. S., & Rhymer, R. M. (1947). P-technique demonstrated in determining psychophysiological source traits in a normal individual.
Psychometrika, 12(4), 267-288.
Lee, I. A. & Little, I. A. (2012). P- technique factor analysis. In Handbook of Developmental Research Methods. New York, NY: Guilford Press.
Midi, H., Rana, M. S., & Imon, A. R. (2009). The performance of robust weighted least squares in the presence of outliers and heteroscedastic errors. WSEAS Transactions on Mathematics, 8(7), 351-361.
Molenaar, P. C. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever.Measurement, 2(4), 201-218.
Molenaar, P. C. (1985). A dynamic factor model for the analysis of multivariate time series. Psychometrika, 50(2), 181-202.
Molenaar, P. C., & Campbell, C. G. (2009). The new person-specific paradigm in psychology. Current directions in psychological science, 18(2), 112-117.
Molenaar, P. C., & Nesselroade, J. R. (2009). The recoverability of P-technique factor analysis. Multivariate Behavioral Research, 44(1), 130-141.
Shifren, K., Hooker, K., Wood, P., & Nesselroade, J. R. (1997). Structure and variation of mood in individuals with Parkinson's disease: a dynamic factor analysis. Psychology and aging, 12(2), 328.
Watson, D., & Tellegen, A. (1985). Toward a consensual structure of mood.Psychological bulletin, 98(2), 219.
Weinberger, N. (2015). If intelligence is a cause, it is a within-subjects cause.Theory & Psychology, 0959354315569832.
Zevon, M. A., & Tellegen, A. (1982). The structure of mood change: An
idiographic/nomothetic analysis. Journal of Personality and Social Psychology, 43(1), 111.
