Research Master Thesis Project
Investigating the Big Five Personality Structure in Psychotic Patients and the General Population through Exploratory Structural Equation Models
Adela-Maria Isvoranu University of Amsterdam
Word count: 7,436 Abstract: 231
Abstract
The Big Five personality structure is, to date, among the most accepted research findings. In spite of its general acceptance, confirmatory factor analysis (CFA) rarely supports this structure and the overuse of exploratory factor analysis (EFA) leads to the problematic
assumption that measurement invariance (MI) holds (i.e., the assumption that the structure of a construct – e.g., neuroticism – is the same across different groups or times) (Marsh et al., 2010). Here, we examined potential differences in the underlying factor structure of
personality in two groups: a group of patients diagnosed with a psychotic disorder and healthy controls. We used a novel statistical technique – Exploratory Structural Equation Modeling (ESEM; Asparouhov & Muthen, 2009) – which integrates the CFA and EFA approaches, allowing for invariance testing. A relationship between personality and psychosis expression is often identified and this line of research has so far naturally assumed that the structure of personality is invariant across levels of psychosis. Thus, it is imperative to establish that differences between personality dimensions are real differences in traits, and not merely a reflection of MI violations. In addition – in order to aid clinicians in understanding
discrepancies in the personality structure of psychotic patients that move beyond underlying dimensions – we carried out an exploratory network analysis (Borsboom & Cramer, 2013) to highlight differences at the item level. Finally, we discussed the application of our results within the current clinical research framework.
Keywords: personality, big5, ESEM, psychosis, measurement invariance, network analysis
Investigating the Big Five Personality Structure in Psychotic Patients and the General Population through Exploratory Structural Equation Models
The Five-Factor Model (FFM) personality traits (i.e., the Big Five; Goldberg, 1993) comprised of Openness (e.g., originality, creativity), Conscientiousness (e.g., self-control, task orientation), Extraversion (e.g., sociability, gregariousness), Agreeableness (e.g., altruistic behaviour, kindness), and Neuroticism (e.g., distress, anxiety) (further detailed in Appendix A) is often associated with clinical phenomena such as depression, substance abuse, and suicidal behavior (Boyette et al., 2013). As a result, the study of personality as a risk and maintenance factor for mental disorders is currently of primary interest. Notably, in recent years, there has been an increased interest in the relation between personality and psychosis expression. This line of research often compares the Big Five structure between healthy and clinical populations, with the aim of uncovering the links and mechanism leading to a disorder. However, several problems with current methodological approaches are often overlooked. Here we detail on a central issue and argue in favor of an alternative novel approach that can overcome limitations in the field.
Exploratory factor analysis (EFA) – a statistical method used to uncover the underlying structure of a set of variables – steadily identifies the Big Five structure of personality and extensive research argues in favor of its validity and stability (e.g., McCrae & Costa, 1997; Marsh et al., 2010). As a result, many instruments measuring the Big Five have been developed (e.g., Donnellan, Oswald, Baird, & Lucas, 2006; Gosling, Rentfrow, & Swann, 2003; Rammstedt & John, 2007); among these, the most widely used have been the Neuroticism Extraversion Openness (NEO) instruments, including the NEO-Five Factor Inventory – the shortest and most accessible NEO-instrument (NEO-FFI; Costa & McCrae, 1992).
Confirmatory factor analysis (CFA), however, has been unsuccessful in providing support for the Big Five on the basis of standard measurement inventories such as the NEO-instruments (e.g., Borkenau & Ostendorf, 1990; Marsh et al., 2010; Vassend & Skrondal, 1997). While EFA is used to explore an underlying factor structure by allowing all items to relate to all factors, CFA is used to confirm an already determined structure by constraining items to relate to just one factor. To exemplify, EFA would be used to explore the factor structure of personality without imposing the Big Five preconceived structure, while CFA would be used to verify this five-factor determined structure. As CFA generally does not provide a good model fit for the Big Five, it has been argued that personality items are related to more than one construct and thus CFA was questioned as a suitable approach in personality research (e.g., McCrae, Zonderman, Costa, Bond, & Paunonen, 1996; Vassend & Skrondal, 1997). Therefore, to date, most studies continue to be carried out almost entirely using EFA; this entails several issues. One is the often over-fitting of the data, which results in high overall correlations (i.e., EFA almost always fits data too well; Hurley et al., 1997). Second, on which this paper focuses, is the important issue of measurement invariance (MI).
Factorial MI is the assumption that the structure of a construct – such as neuroticism or the intelligence quotient (IQ) – is the same across different groups or times (e.g., the assumption that American and Swedish people generally have similar scores on IQ batteries. MI can happen on several levels; (i) configural invariance level (i.e., number of factors invariance); (ii) weak invariance level (i.e., factor loadings invariance); (iii) strong invariance level (i.e., intercepts invariance); (iv) strict invariance level (i.e., residual invariance)
(Meredith, 1993). In current research, it is not possible to investigate MI in EFA. This is because, as described above, MI involves imposing constraints across groups and due to factor rotation techniques this is not possible in EFA. To our knowledge, there are only a few studies (e.g., Booth & Hughes, 2014; Marsh et al., 2010) using methodologically appropriate
models to investigate differences in the factor structure of personality. Thus – because the CFA approach does not fit the data and EFA models do not allow for MI testing – generally MI is naturally assumed in the field.
In an attempt to overcome this issue, the present study aimed to make use of a novel statistical framework to study MI in personality research: exploratory structural equation modeling (ESEM). ESEM is a combination of traditional EFA and CFA approaches that identifies exploratory factors by allowing all items to load on all factors, but it does so within a SEM framework (Asparouhov & Muthen, 2009). Therefore within the ESEM framework the only a priori information needed is the number of factors, while all other parameters are freely estimated. Because the loading matrix rotation gives a transformation of both
measurement and structural coefficients, access to typical SEM parameter estimates, standard errors, and goodness-of-fit statistics is granted (Booth & Hughes, 2014).
Clinical Differences in Personality Traits
Costa & McCrae (1992) have argued that the development of NEO instruments not only contributed to furthering our understanding of personality differences, but could also be used to advance clinical research, as it allows for the systematic examination of the relationship between personality and mental disorders. As a result, numerous studies have started
investigating the Big Five in clinical samples, including ADHD, depression, and anxiety (e.g., Matsudaira & Kitamura, 2006; Miller, Reynolds, & Pilkonis, 2004).
Recent years have seen in particular an increased interest in personality trait differences between patients diagnosed with a psychotic disorder and healthy controls. Extensive research found that psychotic patients display higher levels of Neuroticism (e.g., Herrán, Sierra-Biddle, Cuesta, Sandoya, & Vázquez-Barquero, 2006) and lower levels of Extraversion and Conscientiousness (e.g., Berenbaum & Fujita, 1994; Gurrera, Nestor, &
O’Donnell, 2000) than healthy controls. In addition, there have been several studies that identified differences in all Big Five personality traits between the two groups (Beauchamp, Lecomte, Lecomte, Leclerc, & Corbière, 2006; Camisa et al., 2005). Such results raise the important question whether these differences reflect real differences in the same underlying personality traits or whether the traits themselves are in fact not comparable across groups. This means that a statement arguing that – for instance – patients display higher levels of Neuroticism may in fact not be true if there are violations of invariance.
To date, research carried out in this field does not consider this prospect and routinely assumes MI. As emphasized above, one can only interpret differences in personality traits between groups as true differences if MI holds. For instance, the item “I like to go to parties” may differ in its relation to the factor across groups because a patient diagnosed with a psychotic disorder may interpret this item very differently than a control subject (e.g., not enjoying parties because these can trigger paranoid experiences). Thus, this particular item may be a weaker indicator of Extraversion for patients than for controls. Within this line of reasoning, we argue that evaluating invariance in the two populations is imperative.
Current study
The current study had two main research goals and a secondary research goal. The first main goal was to replicate previous results showing that the ESEM approach indeed provides a better fit to personality data than the traditional CFA approach. The second main goal was to detect violations of MI in personality in patients diagnosed with a psychotic disorder and healthy controls; in other words, we were interested in identifying which items may function differently across groups and are more important indicators for patients or controls.
Our main hypothesis was that at least one level of non-invariance would be identified. This hypothesis was based on findings from two influential research studies. First, Marsh and
colleagues (2010) revealed higher intercepts for women on the NEO-FFI ratings when testing gender invariance, suggesting personality may not be as invariant as generally assumed. Second, a recent study conducted by Fried and colleagues (2016), which found severe violations of MI across time in four depression rating scales. Even though the latter study investigated invariance across time and not groups, it substantiates the need to systematically investigate MI in psychopathology research. In addition, if for rather homogeneous groups (i.e., males and females, depressed patients at different time points) factor structures differ, it can be expected that for less homogeneous groups (i.e., psychotic patients and healthy controls) differences would be even more pronounced.
Our secondary goal was to identify differences between the two populations at the item level. To do so, we conducted an additional analysis based on network models, to expand on the ESEM results by identifying differences in interactions between items (e.g., which pairs of items are more strongly connected for each group). The network approach (Borsboom & Cramer, 2013) has been previously used to investigate personality dimensions, as well as relations between personality and psychopathology (Cramer et al., 2012). In
comparison to structural equation modeling studies, network models conceptualize constructs – such as personality dimensions here – as emerging from the interactions between
components (e.g., personality items). This part of the project was exploratory in nature and aimed to add on to the ESEM results – the network analysis has been shown successful in aiding the understanding of pathways that may lead to the development of a disorder (e.g., Cramer et al., 2012; Fried et al., 2015; Isvoranu et al., 2016; Schmittmann et al., 2013). Importantly, if we present an integrative approach to personality that takes into consideration differences both at the factor and item level, clinicians may be able to better identify and treat the issue. For instance, if it is shown that patients routinely score high on neuroticism, and this is not an effect resulting from violations of invariance, network analysis can highlight
which neuroticism items strongly reinforce each other and thus therapy can be directed towards disconnecting the network of these primary items.
Method Participants
Data from the longitudinal observational cohort study “Genetic Risk and Outcome of Psychosis Project” (GROUP; Korver, Quee, Boos, Simons, & de Haan, 2012), database version 3.2 was analyzed in the current study. The GROUP was designed to investigate vulnerability and protective factors for variation in the expression and the course of nonaffective psychotic disorders. Data were collected from patients diagnosed with a
nonaffective psychotic disorder (n = 1120), their siblings (n = 1057), their parents (n = 919), and a healthy control group (n = 590). The patients were recruited from four academic medical centers in the Netherlands (Amsterdam, Maastricht, Groningen, and Utrecht), while the control group was recruited mainly from universities (through advertisements) across the four sites. Here we used a subset of the data from patients (n = 285) and healthy controls (n = 222) who completed a personality measure, as described below.
Measures
Big Five Dimensions
The Dutch version of the 60-item NEO-FFI (Costa & McCrae, 1992; Hoekstra, Ormel, & Fruyt, 1996) was employed as a measure of the Big Five personality factors, both in the patient and control populations (see Appendix A). Under each factor there were 12 items (see Appendix B) selected from the full 240-item NEO-Personality Inventory-Revised (NEO-PI-R), and based on correlations between each NEO-PI-R item and factor scores (Costa &
McCrae, 1992; Costa & McCrae, 1992). Each item was rated on a 5-point scale ranging from 1 (strongly disagree) to 5 (strongly agree).
Statistical Analyses
Correlated Uniqueness (CU)
The original NEO-PI-R questionnaire (Costa & McCrae, 1992; McCrae & Costa, 2004) included – in addition to the five-factor separation – six facets under each one of the Big Five dimensions; each facet had eight items (from a total of 240 items). In the construction of the shorter version of the NEO-PI-R – i.e., the NEO-FFI (Costa & McCrae, 1992) – the Big Five items most highly correlated with the factor score were selected, without reference to the original item facets (Marsh et al., 2010). This resulted in the overrepresentation of some facets, whereas other facets were not present at all (see Appendix B). Marsh and colleagues (2010) posited that the items coming from the same facets would have higher correlations than items coming from different facets of the same factor and argued that these potentially inflated correlations should be modeled separately as correlated uniquenesses (CUs). This means each pair of items from the same facet should be related, in order to avoid
systematically biasing the parameter estimates (e.g., because item correlations would not be explained by the factor, but by facet origins). Thus, it was argued that a set of 57 CUs (see Appendix C) should be included in all factor analyses of the NEO-FFI responses (Marsh et al., 2010); as a result, all analyses in the current study included these a priori CUs. Due to the large number of parameters to estimate here (even in the absence of CUs) – which may have resulted in problems with model identifications – when carrying out the MI analyses the CUs were constrained to be the same across the two groups. Nonetheless, we consider this to not be an issue as Marsh and colleagues (2010) showed there were no substantial differences in
model fit when allowing the CUs to be freely estimated in the two groups and when constrained to be the same.
Main Analysis
1. Confirmatory Factor Analysis (CFA)
Recent studies have argued that ESEM models generally provide better fit than traditional CFA models (e.g., in personality: Marsh et al., 2010; other domains: Guay, Morin, Litalien, Valois, & Vallerand, 2014; Herman, Perry, & van der Kolk, 1989; Maïano, Morin,
Lanfranchi, & Therme, 2013). Nonetheless, since to date there is still a limited number of published studies using the ESEM approach, the first main analysis of our study was to conduct a traditional CFA and assess whether this underperforms – in terms of goodness of fit – the ESEM analysis.
2. Exploratory Structural Equation Modeling (ESEM)
The second main analysis carried out here was an ESEM analysis to the responses on the 60-item NEO-FFI personality instrument. The ESEM approach is a combination between CFA and EFA: in addition to a CFA measurement model, an EFA with factor loadings matrix rotations is carried out to search for a measurement model that best describes the data; the loading matrix rotation gives a transformation of both measurement and structural
coefficients, giving access to typical SEM parameters (Asparouhov & Muthen, 2009). Therefore, in ESEM, the items are allowed to load on multiple factors (see Figure 1) and the least restrictive model that still allows for tests of measurement invariance is provided (Fried et al., 2016).
We tested the four types of MI as described by Meredith (1993): configural, weak, strong, and strict (please see below for an extended description of MI). The robust maximum likelihood estimator (MLR) was used to address the nonnormal nature of the data (see
Appendix D), while the full information maximum likelihood (FIML) was used to account for missing data. This approach was previously shown as suitable when having a measure of 5+ categories (Rhemtulla, Brosseau-Liard, & Savalei, 2012).
Figure 1. Visualization of an ESEM model of the NEO-FFI
Factorial Measurement Invariance (MI)
Here, we were interested in testing MI across two groups: psychotic patients versus healthy controls. We tested the four types of MI as described by Meredith (1993): (1) configural invariance, (2) weak invariance, (3) strong invariance, (4) strict invariance.
Configural invariance aims to assess whether the number of factors and the pattern of factor-indicator relationships is identical across (here) two groups, by allowing all parameter estimates to be freely estimated. Weak invariance aims to assess whether the factors loadings are invariant over the two groups (i.e., constrain loadings to be the same; see Appendix B-a). Strong measurement invariance is reached if, in addition to the factor loadings, the intercepts of the responses to individual items are further invariant across groups (i.e., both loadings and intercepts are constrained to be equal in the two populations; see Appendix B-b). Finally, strict invariance is satisfied if, in addition to factor loadings and item intercepts, item residual variances are also invariant across groups (i.e., factor loadings, item intercepts, item residual variances are constrained to be equal in the two groups; see Appendix C-c). Each model is
compared to the (previous) less restricted model using a nested chi-square (χ2) difference (here a difftest in Mplus designed for the MLR estimator; Muthén & Muthén, 2012) in order to test whether the model fit has significantly worsened. When a certain level of MI does not hold, one or more of the fixed parameters can be freely estimated over groups in order to obtain the desired model fit – this is known as partial invariance. However, it should be noted that factor loadings cannot be individually freed in ESEM due to rotation issues (for futher techical details please refer to Muthén & Muthén, 2012).
Goodness of Fit
When investigating model fit, several fit indices can be examined. Here we considered the root-mean-square error of approximation (RMSEA), the Tucker-Lewis index (TLI), and the comparative fit index (CFI). These are commonly investigated in the literature (e.g., Kline, 2011; McDonald & Ho, 2002); for the TLI and CFI a value > .90 is generally shown to reflect acceptable fit, while for the RMSEA a value of < .08 is argued to reflect reasonable fit to the data (e.g., Marsh, Hau, & Wen, 2004). RMSEA is a measure of absolute fit (i.e., measure which determines how well the a priori model fits the data), while the CFI and TLI are incremental fit measures (i.e., fit indices that compare the chi-square for the hypothesized model to one from a baseline model) (McDonald & Ho, 2002).
When comparing models to test for the different types of invariance, a chi-square difference test (χ2) was used, as described above. However, as the chi-square difference test was argued to posit several problems with large factor structures (Marsh et al., 2010), we have further investigated differences in model fit using the RMSEA and CFI difference tests, where a change in CFI of less than .01 and a change in RMSEA of less than .015 lend support for the more parsimonious model (Chen, 2007).
Network Analysis
A personality network was constructed to investigate how the items of the NEO-FFI relate to one another at an individual level (Borsboom & Cramer, 2013). In the generated network model, each item (e.g., “I laugh easily”) was represented as a node and an association between two items was represented as an edge; a green edge denotes a positive association between the two items, while a red edge denotes a negative association (Costantini et al., 2015). The network structure was based on L1-regularized partial correlations (Friedman, Hastie, & Tibshirani, 2008; Tibshirani, 1994). For estimating associations between items, partial correlations are preferred over zero-order correlations, as zero-order correlations can be spurious (i.e., result from indirect interactions). In addition, L1-regularization guarantees an optimal balance between parsimony and goodness of fit of the network model because it diminishes the overall strength of parameter estimates. L1-regularization encompasses model selection using the Extended Bayesian Information Criterion (EBIC), which uses a
hyperparameter γ (J. Chen & Chen, 2008; Foygel & Drton, 2011). Commonly, the hypermarater γ is set to 0.5 (e.g., Boccaletti, Latora, Moreno, Chavez, & Hwang, 2006; Isvoranu et al., 2016). However, since the present study included a high number of items in relation to the sample size, the hyperparameter γ was set to 0.1 to ensure the network model will not be too sparse (i.e., a network without any connectivity between items). The details of the influence of γ on the network have been published elsewhere (van Borkulo et al., 2016).
Software
The CFA and ESEM analyses were conducted using the Mplus software version 7.4 (Muthén & Muthén, 2015). The network analysis was conducted using the R statistical software version 3.3.0 (R Development Core Team, 2016) and the freely available R package qgraph (Epskamp, Cramer, Waldorp, Schmittmann, & Borsboom, 2012). To visualize the results of
the CFA and ESEM analyses (including MI), the R package semPlot was used (Epskamp, 2015).
Results
Table 1 below presents demographic characteristics for both patients and healthy controls. Individual t-tests showed significant differences between the two groups in terms of gender (p < .001), IQ (p < .001), and region of recruitment (p < .001). The mean age for both
populations was (roughly) 30 years old and most participants were Caucasian. Approximately 0.05% of the personality data was missing.
Table 1. Demographic Characteristics: Means (SD)
Note. * denotes a significant difference between the two groups; all p-values have been corrected for multiple testing
Variable Controls Patients P-value
Age 29.87 (9.144) 30.07 (7.737) .796 Gender 53% Male 47% Female 81% Male 19% Female < .001* WAIS IQ 112.84 (17.06) 99.10 (16.445) < .001* Ethnicity 83% Caucasian 17% Other 84% Caucasian 16% Other .166 Region 41% Amsterdam 59% Utrecht 67% Amsterdam 33% Utrecht < .001*
Main Analyses
1. The Big Five: CFA versus ESEM
First, we investigated whether the ESEM model provides better fit to the data than a traditional CFA model. Table 2 below presents goodness of fit indices for the two group models. Overall incremental fit indices (i.e., CFI and TLI) suggest the CFA solution does not provide acceptable fit to the data (CFI = 0.785, TLI = 0.769, RMSEA = 0.047) while absolute fit indices (i.e., RMSEA) indicate good fit. When compared to the CFA, the ESEM solution provided much better fit than the CFA, with both incremental and absolute fit indices showing (marginally) acceptable model fit (CFI = 0.903, TLI = 0.880, RMSEA = 0.034).
Table 2. Goodness of fit indices for two group models
Model χ2 df CFI TLI FP RMSEA
Total group CFA 3508.129 1643 0.785 0.769 247 0.047
Total group ESEM 2263.448 1423 0.903 0.880 467 0.034 Note. CFI = comparative fit index; TLI = Tucker–Lewis index; FP = number of free
parameters; RMSEA = root-mean-square error of approximation; CFA = confirmatory factor analysis; ESEM = exploratory structural equation modeling; all analyses include CUs.
Figure 2a and 2b below present an easy visualization of the CFA and ESEM models respectively. Here, it can be observed that overall the items in the ESEM model load more strongly on the factors than the items in the CFA model. In addition, with the exception of the Openness and Agreeableness items – where the items of the two factors are strongly
intertwined with each other – most of the items load more strongly on the ESEM factor that each item was assigned to measure, and less strongly on all the other factors. Notably, most
items from the other scales negatively load on the factor Neuroticism; in addition, when not constrained, all items load on all factors.
Figure 2. Visualization of the a. Big Five CFA Model; b. Big Five ESEM Model
2. Invariance over Groups
Second, as described by Meredith (1993), we investigated four types of invariance:
configural, weak, strong, strict. Table 3 below presents goodness of fit indices for all group invariance.
P1 P6P11P16P21P26P31P36P41P46P51P56P2 P7P12P17P22P27P32P37P42P47P52P57 P3 P8 P13P18P23P28P33P38P43P48P53P58P4 P9P14P19P24P29P34P39P44P49P54P59 P5P10P15P20P25P30P35P40P45P50P55P60
N E O A C
Confirmatory Factor Analaysis
P1 P6P11P16P21P26P31P36P41P46P51P56P2 P7P12P17P22P27P32P37P42P47P52P57P3 P8P13P18P23P28P33P38P43P48P53P58P4 P9P14P19P24P29P34P39P44P49P54P59P5P10P15P20P25P30P35P40P45P50P55P60
N E O A C
Exploratory Factor Analysis / Exploratory Structural Equation Modeling
a.
Table 3. Goodness of fit indices for all group invariance
Model χ2 df CFI TLI FP RMSEA χ2 difftest
Configural Invariance 4447.835 2903 0.817 0.777 877 0.046 -
Weak Invariance 4697.410 3178 0.820 0.799 602 0.043 303.695 Strong Invariance 4851.001 3233 0.808 0.790 547 0.044 150.169 *
Strict Invariance 5203.493 3293 0.774 0.757 487 0.048 360.876 * Note. CFI = comparative fit index; TLI = Tucker–Lewis index; FP = number of free
parameters; RMSEA = root-mean-square error of approximation; CFA = confirmatory factor analysis; ESEM = exploratory structural equation modeling; χ2 difftest = chi-square difference test statistics; * denotes a significant chi-square difference test; all analyses include CUs.
Configural measurement invariance tests whether the number of factors and the pattern of factor-indicator relationships is the same for patients diagnosed with a psychotic disorder and healthy controls. Surprisingly, even though the initial ESEM single-group analysis showed good model fit, when looking at configural MI (i.e., the same model run as a two-group model with no constraints across groups), incremental fit indices suggested the model fit was not acceptable (CFI = 0.817, TLI = 0.777). In an attempt to understand whether the number of factors may differ across groups, a parallel analysis – a method used to
determine the optimal number of factors described by the data (Horn, 1965) – was carried out. Appendix F shows and describes the results of this analysis; overall, the number of factors is shown to be five in both groups. In addition, the absolute fit indices showed good model fit for the configural model (RMSEA = 0.046). Figure 3 below presents a visualization of the configural model in each group. The most obvious differences between groups lies in how the neuroticism items negatively load on the Agreeableness and Conscientiousness factors in the
patient group, while they don’t in the control group, and in how the Conscientiousness items load negatively on the Conscientiousness factor in the control group. However, such
differences should not contribute to the poor fit of the configural model, as the loadings are allowed to differ across groups.
Figure 3. Visualization of Configural Model in the a. Control group; b. Patient group
P1 P6P11P16P21P26P31P36P41P46P51P56P2 P7P12P17P22P27P32P37P42P47P52P57P3P8P13P18P23P28P33P38P43P48P53P58P4P9P14P19P24P29P34P39P44P49P54P59P5P10P15P20P25P30P35P40P45P50P55P60 N E O A C Control Group P1 P6 P11P16P21P26P31P36P41P46P51P56 P2 P7 P12P17P22P27P32P37P42P47P52P57 P3 P8 P13P18P23P28P33P38P43P48P53P58 P4 P9 P14P19P24P29P34P39P44P49P54P59 P5 P10P15P20P25P30P35P40P45P50P55P60 N E O A C Patient Group P1P6P11P16P21P26P31P36P41P46P51P56P2 P7P12P17P22P27P32P37P42P47P52P57P3P8P13P18P23P28P33P38P43P48P53P58P4 P9P14P19P24P29P34P39P44P49P54P59P5P10P15P20P25P30P35P40P45P50P55P60 N E O A C Control Group P1 P6 P11P16P21P26P31P36P41P46P51P56 P2 P7 P12P17P22P27P32P37P42P47P52P57 P3 P8 P13P18P23P28P33P38P43P48P53P58 P4 P9 P14P19P24P29P34P39P44P49P54P59 P5 P10P15P20P25P30P35P40P45P50P55P60 N E O A C Patient Group a. b.
Weak measurement invariance tests whether – in addition to the number of factors and the factor-indicator relationships – the factors loadings are the same across the two groups. To test whether weak invariance holds, the Mplus difftest procedure for the MLR estimator was carried out, as described in Muthén & Muthén (2012). This showed that the model fit has not significantly worsened when the factor loadings were constrained to be the same, compared to the configural model χ2 (275) = 303.695, p = 0.11. In addition, fit indices that control for model parsimony supported the invariance of factor loadings over the two populations (CFI = 0.820 vs 0.817, RMSEA = 0.043 vs 0.046). These indicate that weak invariance holds.
Strong Measurement Invariance tests whether – in addition to the number of factors, the factor-indicator relationships, and the factor loadings – the item intercepts are equal across the two groups. In the same manner as the weak invariance testing, a difftest procedure
showed that the model with intercepts constraints significantly worsened compared to the weak invariance model χ2 (55) = 150.169, p < .001. In addition, incremental fit indices that control for model parsimony did not support the invariance of item intercepts over the two populations (CFI = 0.808 vs 0.820). These indicate that strong invariance does not hold and there is differential item functioning between the patients and healthy controls.
Because of the evidence suggesting item intercepts were not invariant across the two groups, we pursued tests of partial invariance (see Table 4). We identified the 10 highest (intercept) modification indices and freed all ten parameters at the same time: Items 18, 21, 22, 25, 30, 33, 42, 47, 50, and 60 (see Appendix C for item descriptions). The results supported partial invariance of the item intercepts χ2 (45) = 58.821, p = 0.08, CFI = 0.820 versus 0.818.
Strict Measurement Invariance tests whether – in addition to the number of factors, the factor-indicator relationships, the factor loadings, and the item intercepts – the item residuals are invariant across the two groups. In the same manner as the weak and strong invariance testing, a difftest procedure showed that the model with residual constraints was significantly worse than the strong (and weak) invariance model χ2 (60) = 360.876, p < .001. In addition, incremental fit indices that control for model parsimony did not support the invariance of item intercepts over the two populations (CFI = 0.774 vs 0.808). This indicates that strict invariance does not hold and the residual variances of observed scores accounted for by the factors are not the same across the two groups.
Because of the strong evidence suggesting that the item residuals were not invariant across the two groups we again pursued tests of partial invariance (see Table 4). We identified the 10 highest (residual) modification indices and freed all ten parameters at the same time: Items 4, 6, 7, 11, 18, 37, 41, 51, 52, and 54 (see Appendix C for item descriptions). However, the results still did not support partial invariance of the item intercepts; χ2 (50) = 274.0615, p < 0.001, CFI = 0.794 versus 0.820.
Table 4. Goodness of fit indices for all group partial invariance
Model χ2 df CFI TLI FP RMSEA
Partial Strong Invariance 4755.348 3223 0.818 0.801 557 0.043
Partial Strict Invariance 5025.596 3283 0.794 0.777 497 0.046 Note. CFI = comparative fit index; TLI = Tucker–Lewis index; FP = number of free
parameters; RMSEA = root-mean-square error of approximation; CFA = confirmatory factor analysis; ESEM = exploratory structural equation modeling; all analyses include CUs.
Secondary Analysis Network Analysis
Figure 4a and b below present the resulting networks for the control and the patient group respectively. Overall, the pattern of symptom connectivity was similar within the two groups, even though the patient group network appeared to be slightly more connected than the control group. The items were generally clustering together within their original factor, but several Openness items were isolated towards the peripheries of the network (e.g., item 8 Once I find the right way to do something, I stick to it; item 18 I believe letting students hear controversial speakers can only confuse and mislead them; item 38 I believe we should look to our religious authorities for decisions on moral issues; and item 3 I don’t like to waste my time daydreaming). Because we were interested in the most prominent differences between the two groups, we further restricted the networks to display edges above the values of 0.1 and 0.2 and we subtracted the control group edge weights matrix from the patient edge weights matrix to make these differences more visible (see Figure 4c and d); by doing so the associations between the items that were below the aforementioned values were not longer plotted, and only the stronger associations were kept within the network.
When restricting the network with a minimum value of 0.1, several differences were still present; generally, the items within each cluster were slightly more strongly connected for patients, but this pattern was more evident within the Neuroticism cluster (e.g., a positive association between the reverse coded item 26 Sometimes I feel completely worthless and item 46 I am seldom sad or depressed; a positive association between the reverse coded item 21 I often feel tense and jittery and item 31 I rarely feel fearful or anxious). In addition, there appeared to be more between-cluster differences for the patient group, especially associations between Openness and Extraversion, Openness and Agreeableness, and Agreeableness and Conscientiousness. When using a minimum value of 0.2, the only remaining positive
association was between reverse coded item 26 I do not often see myself really as a happy and cheerful person and item 46 I am seldom sad or depressed. To test whether these (stronger) differences were significant, we ran a Network Comparison Test (NCT; van Borkulo et al., 2016).
Figure 4. Network visualization of the NEO-FFI of: a. the control group; b. the patient group; c. the difference between the adjacency matrix of the control and the patient population when using a minimum value of 0.1; d. the difference between the adjacency matrix of the control
and the patient population when using a minimum value of 0.2.
Item dimensions are differentiated by colors. Each edge within the network corresponds to a partial correlation between two individual items. The thickness of an edge represents the
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 ● ● ● ● ● Neuroticism Extraversion Openness Agreeableness Conscientiousness ● ● ● ● ● Neuroticism Extraversion Openness Agreeableness Conscientiousness
Patients − Controls (min value .1)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
Patients − Controls (min value .2)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 ● ● ● ● ● Neuroticism Extraversion Openness Agreeableness Conscientiousness ● ● ● ● ● Neuroticism Extraversion Openness Agreeableness Conscientiousness Controls 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Patients
Control Group Patient Group
a. b.
c. d.
absolute magnitude of the correlation (the thicker the edge, the stronger the connection); green edges indicate a positive connection between two items, while red edges indicate a negative connection between two items. Appendix 3 describes the items based on their
assigned label.
Network Comparison Test (NCT)
To test whether the overall network connectivity was the same across the patient and the control populations – also defined as global strength of the two networks – we ran a NCT (van Borkulo et al., 2016). The NCT is a permutation test (here ran with 1000 iterations) in which the difference between the networks of two groups is calculated repeatedly for
randomly regrouped individuals. This observed difference is calculated by the weighted sum of the absolute connections within one network; the difference is significant at a threshold of 0.05. In addition, the test further returns significant differences – at the same threshold – between all individual edges of the two networks.
We ran the NCT using 1000 permutations and repeated this 100 times to ensure accurate results. In 100% of the returned results, no significant difference was identified between the two groups in terms of global strength or edge comparison (all p > 0.05). Thus, even the stronger differences identified in Figure 4c and d did not reach significance.
Discussion Summary
The present study had three broad aims: (i) examine whether the ESEM approach better identifies the Big Five personality structure than a traditional CFA; (ii) investigate MI over patients diagnosed with a psychotic disorder and healthy controls; (iii) expand on the ESEM
results by identifying differences in interactions between items (e.g., which pairs of items are more strongly connected for each group).
First, our study did indeed support previous findings (e.g., Marsh et al., 2010; Perry et al., 2015) that the ESEM solution fits the data much better than the traditional CFA solution. This suggests that personality items are not unidimensional (i.e., not all items should load strongly on one primary factor), but multifactorial. Second, the ESEM analysis allowed for the testing of MI, which could have not been carried out with CFA models that do not support the Big Five factor structure or the commonly used EFA approach. The group invariance analysis showed that overall, factor loadings were the same across the clinical and healthy populations, but violations of invariance were identified at the intercept and at the residual levels. Finally, the network analysis showed there were no significant network connectivity differences between the two groups.
Methodological and Clinical Implications
First, the current study was the first ESEM study to investigate personality MI in psychiatry research. Within the field of personality, MI research is uncommon due to methodological issues: CFA models display poor fit to the data, while the (over)use of EFA does not allow for investigations of violations of invariance (e.g., McCrae, Zonderman, Costa, Bond, &
Paunonen, 1996; Vassend & Skrondal, 1997; Borkenau & Ostendorf, 1990; Marsh et al., 2010). Marsh and colleagues (2010) introduced ESEM in personality research as an
alternative methodological approach, which overcomes the MI limitation by combining CFA and EFA models. The ESEM approach was shown to perform better than CFA and be a more suitable approach for research within the field of personality and – more generally –
psychological sciences (e.g., Marsh et al., 2010; Guay, Morin, Litalien, Valois, & Vallerand, 2014; Herman, Perry, & van der Kolk, 1989; Maïano, Morin, Lanfranchi, & Therme, 2013).
Our findings are in line with these results. Here, we have shown that the Big Five ESEM model fitted the data better than a traditional CFA, also allowing for MI testing. Given the overwhelming evidence that CFA models are not tenable in personality research, it may be that the main underlying assumption of CFA – the assumption of simple structure (i.e., fixed zero-loadings restriction) – is overly restrictive for this line of research. In consonance, previous research has argued that CFA may be inappropriate for many psychological data due to this specific simple structure restriction (e.g., Dolan, Oort, Stoel, & Wicherts, 2009; Fried et al., 2016). Therefore, in line with the recommendations made by Marsh and colleagues (2010), we advise subsequent studies concerned with personality factor structures (either in clinical or non-clinical populations) to consider ESEM as a viable alternative methodological approach.
Second, to our knowledge, this is the first research that has combined a factor
analytical approach and a network approach. Even though these two methodologies can often be seen as competing, we argue here that they can be used to complement each other: While factor analytic approaches can inform about the general factor structure of personality, the network approach can inform about how individual personality items relate to each other. As Cramer et al. (2012) have pointed out, the network approach does not object to latent variable modeling, but the two methods can work in tandem. Here we have shown that there are no significant differences between psychotic patients and healthy controls when it comes to the number of factors and the pattern and strength of factors loadings (i.e., the items were related to the factors in the same way). The network analysis has revealed, in addition, that there were no significant differences in terms of network connectivity (i.e., the overall connection strength of the two networks was the same) in the two populations. Given such findings, on a broad level, it can be concluded that the two approaches gave similar results and showed the groups have clear personality similarities. However, a more in-depth level, which is not
commonly studied in network analysis, revealed the item intercepts and residuals to be different in the ESEM analysis. Overall, we showed that the two analyses can (i) display (broadly) a similar pattern of results (which can further increase or decrease evidence for a hypothesis) and (ii) complement each other by looking at a construct both at a factor level and an item level. Thus, we recommend future research to not exclude the possibility of
employing both methods on the grounds that they are competing approaches – these can be used adjacent to each other.
Finally, it is important to discuss clinical considerations following from our analyses. Primarily, we have identified differences between psychotic patients and healthy controls at the intercept and residual MI levels, but not at the level of the factor loadings. This finding is substantial, as it highlights the need for future research to accommodate studies of MI before investigating personality trait differences in the two populations.
To date, multiple studies have identified differences between patients and healthy controls on all the Big Five personality traits (Beauchamp, Lecomte, Lecomte, Leclerc, & Corbière, 2006; Camisa et al., 2005). More commonly, psychotic patients display higher levels of Neuroticism (e.g., Herrán, Sierra-Biddle, Cuesta, Sandoya, & Vázquez-Barquero, 2006) and lower levels of Extraversion and Conscientiousness (e.g., Berenbaum & Fujita, 1994; Gurrera, Nestor, & O’Donnell, 2000). The results of the present study, however, raise the possibility that such findings may be invalid, as they might not reflect real trait differences between groups, but violations of invariance. First, it is nonetheless important to note that the invariance of factor loadings was found to hold, meaning that overall the items are measuring the same factors in the two populations equally well. For instance, the item “I like to go to parties” is not a weaker indicator for the Neuroticism factor in patients than in controls – it loads equally well on the factor.
Violations of strong MI, however, suggest that that controls score different than patients on almost all the NEO-FFI items, but especially on Conscientiousness and
Extraversion items; furthermore, violations of strict MI indicate that there is more variance unexplained by the factors in the patients’ scores. Patients more often consider they are not cheerful optimists and they don’t like to be where the action is (Extraversion items); in addition, they less often consider they are productive persons who always get the job done, less often strive for excellence, and less often have a clear set of goals and work toward them (Conscientiousness items). If there is a general answer pattern for patients – for instance – lower on the two Extraversion items, the differences in the Extraversion factor between groups may be merely a reflection of the answers on these two specific items. Similarly, this could also be the case for the Conscientiousness scale. Overall, the difference between the two groups does not lie in how the personality items represent the factor: The items are equally strong indicators for the factors in both groups. However, the differences lie in the pattern of responses given by the patients, in comparison to the control group (e.g., always higher on two specific Neuroticism items). These differences should be considered when investigating personality differences between the two populations.
Within a clinical context, it may be advisable to focus on giving patients a life
prospect that they can focus on, feel accomplished, and feel optimistic about. Indeed, previous research showed that therapy motivation had a positive influence on cognitive rehabilitation (Velligan, Kern, & Gold, 2006). In addition, intrinsic motivation was shown to be critical for psychosocial functioning in schizophrenic patients (Nakagami, Xie, Hoe, & Brekke, 2008). It may be possible that overall psychotic patients and healthy controls do not differ in their personality structure, but due to the quality of life and life prospects, their answer pattern on certain personality questions is distinct. As to date most studies looking at personality in schizophrenia are carried out following the onset of the disorder, it is important for future
cohort studies to establish whether lower Extraversion and Conscientiousness and higher Neuroticism precede disorder onset, or whether they are an artifact of the disorder.
Furthermore, even though it is difficult to explain violations of strict invariance, it would be interesting to investigate whether preceding the disorder there is still higher unexplained factor variance in the patient group, or whether such unexplained variance may result from the disorder.
Given that this was the first study to identify these specific items to be different – and therefore it was data-driven (i.e., partial invariance is based on modification indices in our own data) – replication studies are needed before stating that factor differences commonly identified by previous studies are invalid. Nonetheless, the current findings clearly highlight the need for caution when interpreting personality traits differences between patients
diagnosed with psychotic disorders and healthy controls, and call for MI research within the field of personality and psychiatry.
Limitations and Future Research
The results of this study should be interpreted in light of several limitations. First, even though seemingly sufficient after a power analysis, our sample size was rather small for the types of analyses carried out here. This issue led to convergence problems when carrying out the ESEM analysis and the only solution to circumvent these convergence issues was to constrain the CUs to be equal across the two groups. Nonetheless, we do not regard this as a major problem – Marsh and colleagues (2010) have shown in an ESEM study looking at gender invariance that there were no substantial differences in model fit when allowing the CUs to be freely estimated in the two groups compared to when constrained to be the same. Additionally, due to the limited number of participants, it should be noted that a different hyperparameter γ than commonly adopted in network analysis was used (i.e., γ was set to 0.1
rather than 0.5) to prevent the networks from becoming too sparse. The change of the
hyperparameter should however not have affected the robustness of the results, as it has been used as a way to countervail the results becoming too dependent on the sample size (i.e., with a smaller sample, it is preferred to not overly-restrict the network because it may become a network without any edges; van Borkulo et al., 2016).
Second, an important limitation lies in the characteristics of the control group. In comparison to the patient group, the control group was composed mostly of university students and there were significant differences between the groups in terms of region, IQ (patients had lower IQ than controls), and – of high relevance here – in terms of gender (about 80% males in the patient population compared to a balanced sample in the control group). Marsh and colleagues (2010) identified gender invariance in their personality ESEM study, especially at the intercept and residual levels – similar to the differences identified here. Thus, it may be that our results were strongly dependent on the gender differences and were
partially a reflection of gender invariance. Due to the large sample size discrepancy in our data, assessing gender invariance was not possible. We argue that it is imperative for future research to address this limitation and replicate the results with a more balanced sample.
Third, we identified an issue at the configural MI level – even though the group ESEM analysis identified the Big Five structure, when the groups were split up to investigate configural MI, the model fit was no longer acceptable. Kenny (2015) has argued that if the RMSEA for the null baseline model is less than 0.158, an incremental measure of fit may not be informative. This is because incremental fit indices compare the fit of the model to the fit of the null model, under the assumption that the null model does not fit. However, if the null model fits relatively well, the comparison no longer makes sense. This was indeed the case in our study (RMSEA null = 0.06). In light of this and given that our absolute fit indices (i.e., RMSEA) showed good fit, and a parallel analysis has showed that the number of factors was
the same across the two groups, we carried out the rest of the MI analyses under the
assumption that configural invariance holds. Future research will be necessary to verify that, in larger samples, the configural Big Five model can be established.
Finally, we have investigated partial invariance to identify differences at the intercept level within the two populations. Several such differences were identified and partial
invariance was reached; however, as this is the first study investigating personality MI in a clinical and a healthy population, the parameters freed here were entirely data-driven. It will be necessary for future research to replicate these intercept differences before clear statements can be made.
Conclusions
To conclude, the present study was the first study to (i) investigate personality MI in psychiatry research and (ii) jointly use a factor analytical approach and a network approach. We showed here that an ESEM analysis resulted in better fit to personality data than a traditional CFA. Thus, in line with Marsh and colleagues (2010), we argue that the ESEM approach may be more appropriate for Big Five responses to the NEO-FFI. We identified violation of invariance at the intercept and residual levels, and detailed these alongside their clinical implications in the manuscript. Further studies should aim to replicate these results and caution should be taken when interpreting personality trait differences between patients diagnosed with psychotic disorders and healthy control. Finally, we showed that network analysis can be used in combination with factor analytical approaches, in order to detect differences not only in the underlying factor structure, but also at an item level.
References
Asparouhov, T., & Muthen, B. (2009). Exploratory Structural Equation Modeling. Structural Equation Modeling: A Multidisciplinary Journal (Vol. 16).
http://doi.org/10.1080/10705510903008204
Beauchamp, M. C., Lecomte, T., Lecomte, C., Leclerc, C., & Corbière, M. (2006). Do people with a first episode of psychosis differ in personality profiles? Schizophrenia Research, 85(1-3), 162–167. http://doi.org/10.1016/j.schres.2006.03.026
Berenbaum, H., & Fujita, F. (1994). Schizophrenia and personality: exploring the boundaries and connections between vulnerability and outcome. Journal of Abnormal Psychology, 103, 148–158. http://doi.org/10.1037/0021-843X.103.1.148
Boccaletti, S., Latora, V., Moreno, Y., Chavez, M., & Hwang, D.-U. (2006). Complex networks: Structure and dynamics. Physics Reports, 424(4–5), 175–308.
http://doi.org/10.1016/j.physrep.2005.10.009
Booth, T., & Hughes, D. J. (2014). Exploratory Structural Equation Modeling of Personality Data. Assessment, 21(3), 260–271. http://doi.org/10.1177/1073191114528029
Borkenau, P., & Ostendorf, F. (1990). Comparing exploratory and confirmatory factor analysis: A study on the 5-factor model of personality. Personality and Individual Differences, 11(5), 515–524. http://doi.org/10.1016/0191-8869(90)90065-Y
Borsboom, D., & Cramer, A. O. J. (2013). Network analysis: an integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9, 91–121. http://doi.org/10.1146/annurev-clinpsy-050212-185608
Boyette, L.-L., Korver-Nieberg, N., Verweij, K., Meijer, C., Dingemans, P., Cahn, W., & de Haan, L. (2013). Associations between the Five-Factor Model personality traits and psychotic experiences in patients with psychotic disorders, their siblings and controls. Psychiatry Research, 1–7. http://doi.org/10.1016/j.psychres.2013.06.040
Camisa, K. M., Bockbrader, M. A., Lysaker, P., Rae, L. L., Brenner, C. A., & O’Donnell, B. F. (2005). Personality traits in schizophrenia and related personality disorders.
Psychiatry Research, 133(1), 23–33. http://doi.org/10.1016/j.psychres.2004.09.002 Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance.
Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. http://doi.org/10.1080/10705510701301834
Chen, J., & Chen, Z. (2008). Extended Bayesian information criteria for model selection with large model spaces. Biometrika, 95, 759–771. http://doi.org/10.1093/biomet/asn034 Costa, P., & McCrae, R. (1992). Neo Pi-R. The Revised NEO Personality Inventory Odessa
FL Psychological Assessment Resources, 44, 1–60. http://doi.org/10.1026/0012-1924.52.2.95
Costa, P. T., & McCrae, R. R. (1992). Normal personality assessment in clinical practice: The NEO Personality Inventory. Psychological Assessment, 4(October), 5–13.
http://doi.org/10.1037/1040-3590.4.1.5
Costa, P. T., & McCrae, R. R. (1992). Professional manual: revised NEO personality inventory (NEO-PI-R) and NEO five-factor inventory (NEO-FFI). Odessa FL Psychological Assessment Resources, 3, 101-108. http://doi.org/10.1037//1040-3590.4.1.5
Costantini, G., Epskamp, S., Borsboom, D., Perugini, M., Mõttus, R., Waldorp, L. J., & Cramer, A. O. J. (2015). State of the aRt personality research: A tutorial on network analysis of personality data in R. Journal of Research in Personality, 54, 13–29. http://doi.org/10.1016/j.jrp.2014.07.003
Cramer, A. O. J., van der Sluis, S., Noordhof, A., Wichers, M., Geschwind, N., Aggen, S. H., … Borsboom, D. (2012). Dimensions of normal personality as networks in search of equilibrium: You can’t like parties if you don't like people. European Journal of
Personality, 26(4), 414–431. http://doi.org/10.1002/per.1866
Dolan, C. V., Oort, F. J., Stoel, R. D., & Wicherts, J. M. (2009). Testing Measurement Invariance in the Target Rotated Multigroup Exploratory Factor Model. Structural Equation Modeling: A Multidisciplinary Journal, 16(2), 295–314.
http://doi.org/10.1080/10705510902751416
Donnellan, M. B., Oswald, F. L., Baird, B. M., & Lucas, R. E. (2006). The mini-IPIP scales: tiny-yet-effective measures of the Big Five factors of personality. Psychological
Assessment, 18(2), 192–203. http://doi.org/10.1037/1040-3590.18.2.192
Epskamp, S. (2015). semPlot: Unified Visualizations of Structural Equation Models. Structural Equation Modeling: A Multidisciplinary Journal, 22(3), 474–483. http://doi.org/10.1080/10705511.2014.937847
Epskamp, S., Cramer, A. O. J., Waldorp, L. J., Schmittmann, V. D., & Borsboom, D. (2012). qgraph : Network Visualizations of Relationships in Psychometric Data. Journal of Statistical Software, 48(4), 1–18. Retrieved from http://www.jstatsoft.org/v48/i04 Foygel, R., & Drton, M. (2011). Bayesian model choice and information criteria in sparse
generalized linear models. Arxiv Preprint arXiv:1112.5635, 1–37. Retrieved from http://arxiv.org/abs/1112.5635
Fried, E. I., Bockting, C., Arjadi, R., Borsboom, D., Amshoff, M., Cramer, A. O. J., … Stroebe, M. (2015). From Loss to Loneliness: The Relationship Between Bereavement and Depressive Symptoms. Journal of Abnormal Psychology, 124(2), 256–265.
http://doi.org/10.1037/abn0000028
Fried, E. I., van Borkulo, C. D., Epskamp, S., Schoevers, R. A., Tuerlinckx, F., & Borsboom, D. (2016). Measuring Depression Over Time . . . or not? Lack of Unidimensionality and Longitudinal Measurement Invariance in Four Common Rating Scales of Depression. Psychological Assessment. http://doi.org/10.1037/pas0000275
Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3), 432–441.
http://doi.org/10.1093/biostatistics/kxm045
Goldberg, L. R. (1993). The structure of phenotypic personality traits. The American Psychologist. http://doi.org/10.1037/0003-066X.48.12.1302
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003). A very brief measure of the Big-Five personality domains. Journal of Research in Personality, 37(6), 504–528.
http://doi.org/10.1016/S0092-6566(03)00046-1
Guay, F., Morin, A. J. S., Litalien, D., Valois, P., & Vallerand, R. J. (2014). Application of Exploratory Structural Equation Modeling to Evaluate the Academic Motivation Scale. The Journal of Experimental Education, 83(1), 51–82.
http://doi.org/10.1080/00220973.2013.876231
Gurrera, R. J., Nestor, P. G., & O’Donnell, B. F. (2000). Personality traits in schizophrenia: comparison with a community sample. The Journal of Nervous and Mental Disease. http://doi.org/10.1097/00005053-200001000-00006
Herman, J. L., Perry, J. C., & van der Kolk, B. A. (1989). Childhood trauma in borderline personality disorder. The American Journal of Psychiatry, 146(4), 490–495.
Herrán, A., Sierra-Biddle, D., Cuesta, M. J., Sandoya, M., & Vázquez-Barquero, J. L. (2006). Can personality traits help us explain disability in chronic schizophrenia? Psychiatry and Clinical Neurosciences, 60(5), 538–545.
http://doi.org/10.1111/j.1440-1819.2006.01577.x
Hoekstra, H., Ormel, J., & Fruyt, F. De. (1996). NEO persoonlijkheidsvragenlijsten NEO-PI-R en NEO-FFI. Handleiding. Lisse: Swets & Zeitlinger.
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. http://doi.org/10.1007/BF02289447
Hurley, A. E., Scandura, T. a, Schriesheim, C. a, Brannick, M. T., & Et Al. (1997). Exploratory and confirmatory factor analysis: Guidelines, issues, and alternatives. Journal of Organizational Behavior, 18(6), 667–683.
Isvoranu, A. M., van Borkulo, C. D., Boyette, L. L., Wigman, J. T. W., Vinkers, C. H., & Borsboom, D. (2016). A Network Approach to Psychosis: Pathways between Childhood Trauma and Psychotic Symptoms. Schizophrenia Bulletin, (Advance Access), 1–10. http://doi.org/10.1093/schbul/sbw055
Kline, R. B. (2011). Principles and practice of structural equation modeling. Structural Equation Modeling (Vol. 156). http://doi.org/10.1038/156278a0
Korver, N., Quee, P. J., Boos, H. B., Simons, C. J., & de Haan, L. (2012). Genetic Risk and Outcome of Psychosis (GROUP), a multi-site longitudinal cohort study focused on gene-environment interaction: objectives, sample characteristics, recruitment and assessment methods. Int J Methods Psychiatr Res, 21(3), 205–221. http://doi.org/10.1002/mpr.1352 Maïano, C., Morin, A. J. S., Lanfranchi, M. C., & Therme, P. (2013). The eating attitudes
test-26 revisited using exploratory structural equation modeling. Journal of Abnormal Child Psychology, 41(5), 775–788. http://doi.org/10.1007/s10802-013-9718-z
Marsh, H., Hau, K.-T., & Wen, Z. (2004). In Search of Golden Rules: Comment on
Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler’s (1999) Findings. Structural Equation Modeling: A Multidisciplinary Journal, 11(3), 320–341. http://doi.org/10.1207/s15328007sem1103_2 Marsh, H. W., Lüdtke, O., Muthén, B., Asparouhov, T., Morin, A. J. S., Trautwein, U., &
Nagengast, B. (2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22(3), 471–491.
http://doi.org/10.1037/a0019227
anxiety among Japanese students. Journal of Clinical Psychology, 62(1), 97–109. http://doi.org/10.1002/jclp.20215
McCrae, R. R., & Costa, P. T. (1997). Personality trait structure as a human universal. The American Psychologist, 52(5), 509–16. http://doi.org/10.1037/0003-066X.52.5.509 McCrae, R. R., & Costa, P. T. (2004). A contemplated revision of the NEO Five-Factor
Inventory. Personality and Individual Differences, 36(3), 587–596. http://doi.org/10.1016/S0191-8869(03)00118-1
McCrae, R. R., Zonderman, A. B., Costa, P. T., Bond, M. H., & Paunonen, S. V. (1996). Evaluating replicability of factors in the Revised NEO Personality Inventory: Confirmatory factor analysis versus Procrustes rotation. Journal of Personality and Social Psychology, 70(3), 552–566. http://doi.org/10.1037/0022-3514.70.3.552 McDonald, R. P., & Ho, M.-H. R. (2002). Principles and practice in reporting structural
equation analyses. Psychological Methods, 7(1), 64–82. http://doi.org/10.1037/1082-989X.7.1.64
Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543. http://doi.org/10.1007/BF02294825
Muthén, L., & Muthén, B. (2012). Mplus user’s guide (7th ed.). Los Angeles: Author.
Nakagami, E., Xie, B., Hoe, M., & Brekke, J. S. (2008). Intrinsic motivation, neurocognition and psychosocial functioning in schizophrenia: Testing mediator and moderator effects. Schizophrenia Research, 105(1-3), 95–104. http://doi.org/10.1016/j.schres.2008.06.015 Perry, J. L., Nicholls, A. R., Clough, P. J., & Crust, L. (2015). Assessing Model Fit: Caveats
and Recommendations for Confirmatory Factor Analysis and Exploratory Structural Equation Modeling. Measurement in Physical Education and Exercise Science, 19(1), 12–21. http://doi.org/10.1080/1091367X.2014.952370
short version of the Big Five Inventory in English and German. Journal of Research in Personality, 41(1), 203–212. http://doi.org/10.1016/j.jrp.2006.02.001
Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354– 373. http://doi.org/10.1037/a0029315
Schmittmann, V. D., Cramer, A. O. J., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D. (2013). Deconstructing the construct: A network perspective on psychological phenomena. New Ideas in Psychology, 31(1), 43–53.
http://doi.org/10.1016/j.newideapsych.2011.02.007
Team, R. (2015). R Development Core Team. R: A Language and Environment for Statistical Computing, 55, 275–286.
Tibshirani, R. (1994). Regression Selection and Shrinkage via the Lasso. Technical report, Stanford University, Palo Alto, CA, 1994. http://doi.org/10.2307/2346178
van Borkulo, C. D., Boschloo, L., Borsboom, D., Brenda, W. J. H. P., Waldorp, L. J., & Schoevers, R. A. (2016). Association of Symptom Network Structure With the Course of Depression, 72(12), 1219–1226. http://doi.org/10.1001/jamapsychiatry.2015.2079
Vassend, O., & Skrondal, A. (1997). Validation of the NEO Personality Inventory and the five-factor model. Can findings from exploratory and confirmatory factor analysis be reconciled? European Journal of Personality, 11(2), 147–166.
Velligan, D. I., Kern, R. S., & Gold, J. M. (2006). Cognitive rehabilitation for schizophrenia and the putative role of motivation and expectancies. Schizophrenia Bulletin, 32(3), 474-485. http://doi.org/10.1093/schbul/sbj071
Appendix A. Short description of the Big Five personality measures (Hoekstra et al., 1996)
Appendix B. Visualization of the different types of invariance as described by Meredith (1993): a. Weak Invariance; b. Strong Invariance; c. Strict Invariance
aspects as imagination, aesthetics, feelings, ideas, and values.
Openness
aspects such as efficiency, orderliness, reliability, ambition, self-discipline, and thoughtfulness.
aspects such as trust, honesty, altruism, compliance, modesty, and compassion.
aspects such as kindness, sociability, assertiveness, energy, cheerfulness, and excitement seeking
aspects such as anxiety, anger, depression, shame, impulsiveness, and vulnerability.
Conscientiousness Agreeableness Extraversion Neuroticism a. b. c.
Appendix C. Complete list of the NEO-FFI items (Costa & McCrae, 1992), English
translation. (R) denotes reverse code items and square brackets denote the original facet that the item belong to in the NEO-PI-R
Appendix D. A Priori Correlated Uniquenesses Based on the Design of the NEO 1. Q24R & Q29 2. Q04 & Q34 3. Q04 & Q49 4. Q04 & Q14 5. Q04 & Q39 6. Q34 & Q49 7. Q34 & Q14 8. Q34 & Q39 9. Q49 & Q14 10. Q49 & Q39 11. Q14R & Q39 12. Q19 & Q09 13. Q19 & Q54 14. Q19 & Q44 15. Q09R & Q54 16. Q09R & Q44 17. Q54R & Q44 18. Q05 & Q15 19. Q05 & Q55 20. Q15R & Q55 21. Q20 & Q40 22. Q20 & Q45 23. Q40 & Q45 24. Q25 & Q35 25. Q25 & Q60 26. Q35 & Q60 27. Q10 & Q50 28. Q10 & Q30 29. Q50 & Q30 30. Q02 & Q27 31. Q32 & Q47 32. Q32 & Q52 33. Q47 & Q52 34. Q07 & Q37 35. Q07 & Q12 36. Q07 & Q42 37. Q37 & Q12 38. Q37 & Q42 39. Q12R & Q42 40. Q21 & Q01 41. Q21 & Q3 42. Q01R & Q31 43. Q26 & Q16 44. Q26 & Q46 45. Q16R & Q46 46. Q06 & Q56 47. Q11 & Q41 48. Q11 & Q51 49. Q41 & Q51 50. Q13 & Q43 51. Q13 & Q23 52. Q43 & Q23 53. Q28 & Q08 54. Q53 & Q58 55. Q53 & Q48 56. Q58 & Q48 57. Q18R & Q38
