Mapping out the Positive And Negative Syndrome Scale: the 30-item
Version and the 20-item version.
IJsbrand Leertouwer (student number 10154981) Supervisors: Arjen Noordhof & Lindy-Lou Boyette
Abstract
The Positive And Negative Syndrome Scale (PANSS; Kay et al., 1987) is the most widely used instrument for assessing symptom severity of schizophrenia and other non-affective psychotic disorders. Although both exploratory and confirmatory methods have shown that the most replicated factor-structure of the PANSS involves five factors, controversy remains on how these five factors should be defined. A complex model by Van der Gaag et al. (2006b), including a substantial number of cross-loadings, stands in contrast with a simple model proposed by Wallwork et al. (2012), excluding ten of the original items and assuming no cross-loadings. In this study, we investigated the internal structure of both the full 30-item version of the PANSS and the 20-item version inherent to the Wallwork model, by using both traditional factor analytic approaches and novel network modelling techniques. The analyses suggested that it is hard to divide the items of both versions of the PANSS into five distinct categories, and that network techniques might prove to be a valuable addition to sum scores for separate categories, inherent to factor analytic approaches.
Table of contents 1. Introduction 3 2. Methods 6 2.1 Sample 6 2.2 Instruments 6 2.3 Analyses 7 3. Results 9 3.1 Sample 9
3.2 Confirmatory factor analyses 10
3.3 Exploratory factor analyses 10
3.4 30-item network 11
3.4.1 30-item correlation network; general structure in 2-dimensional space 11 3.4.2 30-item correlation network, centrality of items 12
3.4.3 30-item LASSO network 12
3.4.4 30-item correlation network, deleted items in Wallwork model 13
3.5 20-item network 13
3.5.1 20-item correlation network, general structure in 2-dimensional space 13 3.5.2 20-item correlation network, centrality of items 13
3.5.3 20-item LASSO network 13
3.6 Stability of the Networks 14
4. Discussion 14 4.1 Overview 14 4.2 Factor analyses 15 4.3 Network analyses 16 4.3.1 General structure 16 4.3.2 Specific factors 16 4.3.3 Deleted items 17 4.4 Future considerations 18
5. Tables and figures 20
6. Appendices 35
1. Introduction
The construct of psychotic disorder is in constant development. It has proven to be very difficult to outlay the dimensions of a syndrome so diverse in its nature and dynamic in its course. Yet the way we define the symptom constructs of this heterogenic concept may be crucial for monitoring change over time, and thereby for defining the prognosis and therapeutic approaches (Van Os, 2014; Jerrell & Hrisko, 2013).
The struggle to define psychotic symptom constructs is illustrated by the factor-analytic literature on one of the most widely used instruments to assess the symptoms of schizophrenia and other psychotic disorders: the Positive And Negative Syndrome Scale (PANSS; Kay et al., 1987). Originally, the items of the PANSS were thought to represent a positive symptoms scale (7 items), a negative symptoms scale (7 items) and a general pathology scale (16 items). However, both exploratory and confirmatory methods have shown that the most replicated factor-structure of the PANSS involves five factors, redividing the items over a Positive factor, a Negative factor, a Cognitive/Disorganized factor, a Depressed/Emotional distress factor and a Manic/Excited factor (Lehoux et al., 2009; Wallwork et al., 2012). Although in the current literature there seems to be consensus on the existence of these five factors, controversy remains on how they should be defined.
Van der Gaag et al. (2006a) fitted 25 earlier proposed 5-factor models on a sample of 5769 patients, and found that none of the models was satisfying in terms of fit. Using a ten-fold cross-validation on the same dataset (2006b), the authors postulated a model keeping all 30 items in contrast to some influential previous authors (White et al, 1997). The authors stressed that one of the main differences with previous models was that items were allowed to have cross-loadings. The main idea behind this ruling was that a symptom might have multiple causes. Although some authors have argued that the Van der Gaag model thereby does justice to the complexity of psychotic disorders (Lehoux, 2009), others have argued that the model is overly complex, as a more parsimonious model showed similar model-fit to their dataset (Jerrell & Hrisko, 2013). The on-going lack of consensus and the vast range of available models keep limiting the comparability between the PANSS’ factor-scores across different studies.
In an effort to find consensus, a study by Wallwork et al. (2012) ‘counted votes’ within the literature, and proposed a trimmed model, only keeping items that were attributed to the same factor in 24 of the 29 included studies. The resulting model consists of 20 items, and assumes no cross-loadings. Information on how items are distributed over the five factors in both the Van der Gaag model and the Wallwork model can be found in Appendices A1 and A2. The fact that the Wallwork model assumes no cross-loadings could be beneficial when the goal is to measure the progress of the specific dimensions of psychosis. As the symptom constructs are more clearly separated, there would be optimal divergence between constructs. It could, for instance, become clear if treatment x has a strong effect on positive symptoms, but a weaker effect on disorganized/cognitive symptoms. Along this line, recent authors have proposed visual representations in which each dimension would be represented separately (Van Os & Kapur, 2009; De Haan et al., 2012). However, although separating symptom categories could be desirable for reasons of clarity, it is questionable whether this separation reflects clinical reality. For instance, past authors have argued that cognitive symptoms and negative symptoms are strongly related (Harvey et al., 2006), or that negative symptoms and depressive symptoms show a substantial amount of overlap (Goldman et al., 1992; Markou, 1996).
In contrast to the conceptual overlap of these symptoms, one of the main reasons to exclude items throughout the factor analytic literature seems to have been the
fact that they load on multiple factors; these items were seen as ‘diffuse’. Excluding items that have cross-loadings might serve as a tool to find clearly distinguishable symptom constructs. However, it could lead to the deletion of items that represent genuinely important aspects of multiple symptom constructs (Van den Oord et al., 2006). Additionally, the construct of psychotic syndrome suggests the existence of not only very specific features of sub-aspects but also more general features of psychotic phenomena. By removing items with cross-loadings, such general features would be removed rather arbitrarily. At the other end of the spectrum of excluding items on the basis of factor analyses, some authors have deleted items because they did not seem to fit in any of the factors. Walwork et al. (2012) for instance, have reluctantly excluded the item Suspiciousness (p6) in their model because it led to worse fit. By doing so, they have removed what they themselves consider to be one of the core aspects of psychosis.
Furthermore, as was also implicitly mentioned by Wallwork et al. (2012), reducing the number of items might have an impact on the internal reliability of the models’ factors (two of the models’ factors consist of only three items). Previous authors have reported low internal reliability for the Cognitive/Disorganized factor (Lancon et al., 2000) and the Depressive/Emotional distress factor (Emsley, 2003; Van den Oord, 2006), while including more items. Lastly, a replication study has demonstrated that both the Wallwork model and the Van der Gaag model fail to meet criteria for acceptable fit (Langeveld et al., 2013)1.
Taking the above-mentioned considerations into account, a more fundamental question arises: are factor analytic models suited at all to evaluate the grouping of PANSS items? Originating from the latent variable model, factor analytic methods have some very stringent assumptions, one of the most important being the assumption of ‘local independence’ (Holland & Rosenbaum, 1986). This assumption essentially means that similarities in scores on different items originate only from their shared dependence on a latent variable. Thus, the latent variable model does not allow items to be directly related. Although this approach might be suited for medical conditions, in which an underlying disease is usually the cause of a set of observable symptoms, some authors have recently challenged the validity of the latent variable model for mental disorders (Borsboom & Cramer 2013). The textbook example of when the model might be problematic for these disorders is in the case of symptoms like excessive worrying, sleep disturbances and fatigue, all DSM-IV symptoms of general anxiety disorder (GAD). Strictly speaking, the latent variable model would treat those symptoms as consequences of a shared ‘common cause’, namely the GAD. However, excessive worrying is likely to lead to sleep disturbances, which are in turn very likely to lead to fatigue. The network approach acknowledges this idea, and conceptualizes mental disorders as dynamic systems of causally connected symptoms (Borsboom & Cramer, 2013).
In a network model, symptoms like excessive worrying, sleep disturbances and fatigue would thus be elements in a network of even more mutually dependent symptoms, like for instance concentration problems and irritability, both considered symptoms of GAD as well. Symptoms of other mental disorders may also be included in the network; the worrying and irritability might also lead to lower mood, considered to be a symptom of depression and strictly speaking not of GAD. By looking at mutual influences on symptom level, the network approach thus provides an explanation for the high rates of comorbidity as well (Cramer, Waldorp, Van der Maas & Borsboom, 2010).
1 It should be noted that this study and numerous other studies (White et al., 1997; Van der Gaag
et al., 2006; Jerell & Hrisko, 2013) have compared fit-indices of models that included (and thereby excluded) a varying range of items within the PANSS. This comparison is conceptually incorrect when sets of items differ, however models can be compared to what is considered the absolute minimum criterion.
Another key aspect of the network approach is that it allows for feedback loops. In the case of GAD the concentration problems could aggravate the worrying, and this would complete a loop, in this case a downward spiral. In the case of psychotic symptoms, Van Os (2013) has described how stress may lead to insomnia, which could lead to low mood and ultimately paranoia2, which could in turn lead to even lower mood. In order to assess these causal paths however, many measurement points in time are needed (Brigmann et al., 2013; Bakker et al., 2013).
Creating personal causal networks might be a very elegant solution that does justice to the highly idiosyncratic and complex structure of psychosis. However, another more basic application of network modelling, for which only one measurement point is sufficient, can be used to evaluate the internal structure of a questionnaire like the PANSS. So-called correlation networks essentially visualize the correlation matrix of a set of variables. This makes the direct relation between items intuitively visible (i), as well as the relative orientation of groups of items within the questionnaire (ii). In addition, several new indices can be computed that give information about the items’ involvement in the network, and the general structure of the network (iii). A detailed explanation of these three types of information will follow in the method section. Together, these sources of information can highlight where possible relations between items may lie within a population. They could for instance indicate whether deleted items indeed have many connections with other items, or whether they are isolated from the rest, and therefore hard to fit into factors.
In the case of the PANSS, there are several reasons why items might in fact be related. Both hypothetical causal relations and relations in formulation of the items seem to be present. Theoretically for instance, it is not unlikely that someone actively avoids social involvement (Active social withdrawal; g16) because of his Suspiciousness (p6). Or that someone is reluctant to participate in an interview (Uncooperativeness; g8), because he lacks insight in his condition (Lack of judgement and insight; g12). Other items seem to be strongly related because of their formulation: the description of Uncooperativeness (g8) mentions ‘hostility’, which directly links the item to the actual item Hostility (p7). Definitions of Poor rapport (N3) and Lack of spontaneity and flow in conversation (N6) both mention ‘reduced verbal and nonverbal communication’.
In this study we will use correlation networks to map out the structure of relations between items of the PANSS. Together with traditional factor-analytic techniques, these networks will give a complete overview of the internal structure of the PANSS. We hypothesize that confirmatory factor analyses (CFA’s) for both the Van der Gaag model and the Wallwork model will not meet criteria for satisfactory model fit, in line with previous authors who found a lack of fit in replication studies of five-factor models (I) (White et al., 1997; Van der Gaag et al. 2006; Langeveld et al. 2012). We hypothesize that the model-fit can be improved using the results of exploratory factor analyses (EFA’s), but that the resulting models will contain a substantial number of cross loadings (II) which are unlikely to replicate in other samples. Furthermore, we hypothesize that the internal reliability of the Depression/Emotional distress factor and the Cognitive/Disorganized factor will be low as has been found by previous authors (III) (Lancon et al. 2000; Emsley, 2003; Van den Oord, 2006). Extending the line of thought of the previous hypotheses, we raise the question whether factor analytic approaches are suited at all to assess the grouping of items of the PANSS. Network analyses will provide a different view on its internal structure, exploring beyond the assumptions of latent variable models. The shared connection of symptoms might be a vital part of the PANSS, and even the psychopathology of psychosis itself.
2. Methods 2.1 Sample
For this study, data collection took place at the Diagnostic Centre of the department of Early Psychosis, in the Academic Medical Centre (AMC) of Amsterdam. Full access was granted to case-files of patients who were diagnosed for the first time at the AMC between 2007 and 2014. In total, 1106 patient-files were available. The primary inclusion criterion was a diagnosis of a non-affective psychosis based on criteria of the Diagnostic and Statistical manual of Mental disorders (4th ed., text rev.; DSM–IV–TR; American Psychiatric Association, 2000); that is, schizophrenia, schizophreniform disorder, delusional disorder, brief psychotic disorder, schizo-affective disorder or psychosis not otherwise specified. Additionally, patients were required to have a minimum of 50% complete PANSS scores before supplementation of the data, and case-files had to be traceable in order to supplement the data. All original PANSS scores and DSM-IV diagnoses were given by trained professionals of the AMC, on the basis of clinical interviews.
2.2 Instruments
The Positive And Negative Syndrome Scale (PANSS) is a 30-item rating scale, administered in the form of a semi-structured interview. The questionnaire is generally used only when there is good reason to believe that a patient is actually suffering from a psychotic disorder. Scores range from 1 (absent) to 7 (extreme). The inter-rater reliability has been found to be good (Müller & Rossbach, 1998; Peralta & Cuesta, 1994). The model-fit and replicability of five-factor solutions however has been found to be less robust (White, 1997; Van der Gaag et al., 2006a, Langeveld et. al, 2013); this was the main motivation for the current study. As was previously discussed, the internal reliability of some factors has been found to be low (Lancon et al. 2000; Emsley, 2003). The problems with the internal reliability of the Depression/Emotional distress factor could be due to the fact that the assumption of unidimensionality can be seen as questionable. The factor in both models consists of items measuring ‘anxiety’, ‘depression’ and ‘guilt’. Whether these lose, nonspecific items represent a single construct is arguable. The same goes for the Congitive/Disorganized factor in the Van der Gaag model, as it includes items like Mannerisms and posturing (g5), Difficulty in abstract thinking (n5), and Poor rapport (n5). These items seem to be very diverse as they refer to physical manifestations, cognitive dysfunction, and quality of the interaction respectively.
Studies on the external validity of the PANSS find mixed results. A Mexican study seems to be unique in the sense that it included external validation measures of all five PANSS factors (Fresán et al., 2005). The authors of this study have warned that the correlation between their Cognitive/Disorganized factor and the Mini Mental State Examination (MMSE; Folstein et al., 1975) was low (r =.47), as were correlations between their Manic/Excitement factor and the Overt Agression Scale (AOS; Yudofsky et al., 1986; r =.42), and their Depressive/Emotional distress factor and the Calgary Depression Scale for Schizophrenia (CDSS; Addington & Addington, 1993; r =.45).
In the case of the Manic/Excitement factor, a possible explanation could be that the PANSS factor measures a broader construct than the aggressive tendencies measured with the AOS. Another study for instance, found high correlations between the PANSS’ Manic/Excitement factor and the Young Mania Rating Scale (Y-MRS; Young et al., 1978) in three different sub studies (Lindenmayer et al., 2004).
In the case of the Depressive/Emotional distress factor, a possible explanation for the low correlation could again be that the PANSS factor seems to exist out of loose non-specific items. Previous authors (Kontaxakis et al., 1999) have found even lower correlations when comparing the PANSS factor to the CDSS (r = .26) and other depression measures like the Hamilton Depression Rating scale (HDRS; Hamilton, 1960; r = .26), and the Expanded Brief Psychiatric Rating Scale-Depression subscale (EBPRS- D; Lukoff, Nuechterlein, & Ventura. 1986; r=.17). Other authors have found that the PANSS’ Depression/Emotional distress factor correlated with the “psychological depression” subscale of the Montgomery and Åsberg Depression Rating Scale (MADRS; Montgomery & Åsberg, 1979), but did not correlate with the “behavioural depression” subscale of this measure; a finding they did not have a clear answer to (Lee, Harris, Loughland & Williams, 2003).
In the case of the Cognitive/Disorganized factor, an explanation for the low correlation with the MMSE could be that this factor seems to represent the disorganized aspect of psychosis rather than the (neuro-)cognitive aspect. After all, all items in the Cognitive/Disorganized factor except Difficulty in abstraction (n5) are measured observationally, and not through tests of specific neurocognitive functions. Results of studies that compare the Cognitive/Disorganized factor of the PANSS to such measures have been mixed (Rodriguez-Jiminez et al., 2012; Good et al., 2004). Previous authors have therefore concluded that for purposes of measuring cognitive capabilities, more specific cognitive measures would be suited (Rodriguez-Jiminez et al., 2012).
In summary of the external validation studies, the Depressive/Emotional distress factor and the Cognitive/Disorganized factor seem to be less robust than the other factors, albeit that for the Cognitive/Disorganized factor this problem may be resolved when it is regarded as a Disorganized factor rather than a Cognitive factor3.
2.3 Analyses
Data were analyzed using R (R Core Team, 2014). R is an open source programming language and environment for statistical computing and graphics. In the first part of the analyses, both the Van der Gaag model and the Wallwork model were fitted to the dataset and assessed in terms of goodness-of-fit using R’s ‘lavaan’ package, version 0.5-17 (Rosseel, 2012). In addition to the stringent chi-square test of exact fit4, the Comparative Fit Index (CFI) was computed, which compares the models to their worst possible counterpart, and the Root Mean Square Error of Approximation (RMSEA) was computed, which reports the amount of misfit per degree of freedom. A CFI of above .90 and RMSEA of below .06 are generally regarded as satisfactory (Hu & Bentler 1999). Subsequently, the general factor saturation of the separate factors was evaluated using McDonalds hierarchical omega (ωh) (McDonald, 1999). McDonalds omega has been advocated as a method for evaluating internal reliability that is preferable to the commonly used Cronbachs alpha (Revelle & Zinbarg 2009; Zinbarg et al., 2005; Dunn Baguley & Brunsden, 2014). As ωh is based on the sum of squared loadings on a presumed general factor underlying a scale (Revelle & Zinbarg, 2009), it is less stringent than Cronbach’s alpha in both the assumption of unidimensionality and essential
3Because of this conclusion, we will further refer to the Cognitive/Disorganized factor as the Disorganized factor.
4 Although structural models are typically seen as approximations of reality, the chi-square test of exact fit tests the hypothesis that a model fits the population exactly. Consequently, the chi-square test of exact fit will almost certainly be significant with a large number of degrees of freedom (Bentler, 2007) or a large sample size (Tomarken & Wallner, 2003), and will therefore be less informative than the other measures.
equivalence5. Because calculation of ωh is an iterative approach, models with varying numbers of factors need to be fitted. In our analysis we choose to start fitting a model with three factors, as this is the minimum requirement for model identification. If factors consisted of sufficient items, we also fit a model with four factors. When two models were fit to the factors, both values of ωh were reported. Because no formal guidelines exist for a cut-off value for an acceptable ωh, value, a guideline for an acceptable Chronbach’s alpha was used. The most commonly used guidelines for acceptable alpha were given by Nunnally, who recommended a range from .5 to .6 in 1967, but later changed this recommendation to .7 without explanation (Peterson, 1994). For this study, a cut-off value of .6 was used.
In the second part of the analyses, EFA’s were run for both the full 30-item version of the PANSS and the 20-items selected by the Wallwork model, letting every item free to load on every factor. In line with previous authors, we used a forced five-factor solution with varimax rotation (Citrome et al., 2011) to facilitate comparison with the CFA models. The EFA models were then used to improve the original CFA models. Items that showed high loadings on factors in the EFA were added to their respective original factors, and items that did not show loadings on factors in the EFA were deleted out of the original factors. The fit indices were calculated again for the improved models, and the improved factors were analyzed for general factor saturation again using McDonalds omega.
In the final analyses, correlation networks and adaptive LASSO networks were run for both the 30-item version and the 20-item version of the PANSS. As was previously described, correlation networks essentially visualize the correlation matrix of the items. The items are represented by ‘nodes’, which are connected by weighted ‘edges’ representing their association (e.g. correlations). Strong associations are represented by thick edges, while thin edges represent weak associations. Positive correlations are generally represented by green edges, while red edges represent negative associations. This network technique visualizes patterns of correlations on item level that would otherwise be difficult to detect (Cramer et al., 2012). A particularly visually intuitive method to show the clustering of items is to let the strength of the edge weights determine the location of nodes (Epskamp et al., 2012)6. By doing so, the relative distance between items becomes informative, as well as their orientation towards the centre versus the periphery. More central items are considered to also be ‘central’ in the construct that is being portrayed. Items that cluster together represent a ‘factor’ within a construct, as they are closely related to each other, but less so to other items. In addition to this intuitive visual representation, several indices can be computed that give information about the networks’ structure. For assessing the items’ ‘centrality’ in the network, the ‘strength’, ‘closeness’ and ‘betweenness’ were computed. In network analysis, the strength of a node is defined as a summation of the weights of edges that are connected to it. The strength of an item could be described as the ‘general involvement’
5 Cronbach’s alpha can be seen as a special case of ω
h –namely when the essentially
tau-equivalent model holds. The essentially tau-tau-equivalent model assumes that a scale under evaluation measures one construct (is unidimensional), and consists of items with equal variance and co-variance (they are essentially tau-equivalent). When these assumptions hold, alpha is a reliable measure of reliability. However, it has been reported that alpha tends to underestimate ωh
when indicators are not essentially tau-equivalent, and tends to overestimate ωh when a scale has
multidimensional properties (Zinbarg et al. 2005)
6This visualization makes use of the Fruchterman-Reingold algorithm, which forces the multidimensional network to a 2-dimensional space. (Fruchterman & Reingold, 1991, in Epskamp et al., 2012).
of the item in the network (Opsahl, Agneesens & Skvoretz, 2010). The closeness is defined by the inverse sum of shortest distances from a node to all other nodes (Opsahl, Agneesens & Skvoretz, 2010). Items with high closeness values are able to spread activation across large numbers of other items in the network quickly. The betweenness of a node measures the degree to which it lies on the shortest path between other nodes (Freeman, 1978 in Opsahl, Agneesens & Skvoretz, 2010). Nodes that are high in betweenness are assumed to assert control over the ‘flow’ within networks and are influential because they connect different sub parts of the network with each other. Because central items are highly connected to the rest of the network, they can also be viewed as non-specific. Despite the fact that they are clearly part of the measured construct, they could therefore be unwanted when the goal is to create separate sub-networks.
Although correlation networks are highly useful to address the general structure of a dataset, direct relationships between items are more easily detected in partial correlation networks, which visualize the association between any two variables after all other variables have been accounted for. Adaptive LASSO networks are a special type of partial correlation networks that omit small partial correlations (Costantini et al., 2014).
On the basis of the LASSO networks, the so-called “small-worldness” index can be computed, which assesses the general structure of the network7. The ‘small-worldness’ index is conceptualized as the tendency of a network to have a high degree of clustering, and a short average path length (Watts & Strogatz, 1998). If a network shows properties of a small world, this means that activation of one part of the network is likely to lead to a quick activation of other parts in the network. A textbook example of a small world is the Internet, in which information can spread extremely fast. In the case of psychosis, it could mean that when one aspect of the disorder (for instance hallucinations) is triggered, the rest of the symptoms will quickly be activated as well. As this phenomenon has been reported in clinical observations (van Os, 2014), we hypothesize that the symptoms measured by the PANSS will show properties of a small world. For the computation of the small-world index, the network is compared to a random network with the same number of nodes and edges. If the network has a similarly short path length between items, but a higher degree of clustering (clusters are formed if a set of nodes is fully connected), there is evidence of a small world structure. A small-worldness index of 3 or higher is generally regarded evidence of such a structure (Humphries & Gurney, 2008). Correlation networks were run using with R’s ‘qgraph’ package (Epskamp et al., 2012), as were calculations of fit indices. The adaptive LASSO networks were run with R’s ‘parcor’ package (Krämer, 2009).
3. Results 3.1 Sample
In total, 657 patient files were selected. As presented in Table 1, the majority of the sample was male (74.6%), and the mean age was 23.9 (SD = 6.0). According to the DSM-IV diagnoses, large parts of the sample were diagnosed with schizophrenia of the paranoid type (38.5%), and psychosis not otherwise specified (20.7%). Further, the sample mostly existed of patients of non-Dutch origin (53.6%)8. Lastly, it should be
7 When a network is fully connected (as is the case with regular correlation networks), the
small-worldness index cannot be calculated, as there is always a third path between two nodes (i.e. the network is one big cluster).
8This finding is in line with studies that link immigration to increased risk for psychosis (see
noted that the majority of patients (74,6%) was using antipsychotics prior to the interview that was used for the data of this study. Although some authors have documented that medication use did not affect the factor structure of the PANSS in their dataset (Harvey et al., 2006; Lindenmayer et al. 1995a,b), it is possible that the use of medication has an effect on the network structure.
3.2 Confirmatory factor analyses
Both the original Van der Gaag model and the original Wallwork model failed to meet criteria for acceptable fit (see Table 2). Unfortunately, comparison between loadings in the current CFA (Table 3) and loadings in the original study by Van der Gaag (Appendix B1) was impossible, as the original article reported only unstandardized factor loadings. Factor loadings in the current CFA of the Wallwork model (Table 4) showed little dispersion from their counterparts in the original CFA (Appendix B2), although all loadings were lower than in the original.
The general factor saturation was acceptable (ωh between .60 and .79) for all factors in both models (Table 3 and Table 4), except for the Depression/Emotional distress factor in the Van der Gaag model (ωh= .47). In general, the omega values of the Wallwork models’ factors were higher, showing that the extra items with low loadings in the factors of the Van der Gaag model lead to a decrease in general factor saturation. 3.3 Exploratory factor analyses
Within the EFA of the 30-item version of the PANSS, the five putative factors could be identified, albeit with a large number of cross-loadings (Table 5). In the next step, the loadings of the EFA were used to improve the factors of the van der Gaag model. First, loadings that were not found in the EFA were excluded from the model, and loadings higher than .3 were included in the model. In the current EFA, only one loading that was assumed by the Van der Gaag model was not replicated: the item Difficulty in abstract thinking (n5) did not load on the Positive factor. In total, there were five extra items with loadings higher than .3. Items Excitement (p4), Grandiosity (p5), Poor rapport (n3) and Lack of spontaneity (n6) all loaded on the Disorganized factor. The item Lack of judgment & insight (g12) loaded on the Excited/Manic factor9. After these loadings were added and the absent loading was deleted, the improved model still failed to reach acceptable fit in a CFA (CFI = .853, RMSEA = .072). To investigate if acceptable fit could be reached, loadings higher than .25 (6 loadings in total) and later even loadings only higher than .2 (13 loadings in total) were included to the model. However, even when loadings of .2 were included, the model only came close to acceptable fit (see Table 2).
Likewise, the EFA of the 20-item version of the PANSS included a large number of cross-loadings (Table 6). Again, the extra loadings were used to improve the original model -in this case the Wallwork model. In total, there were five loadings higher than .3: items Grandiosity (p5), Unusual thought content (g9) and Poor attention all loaded on the Excited/manic factor; the item Lack of spontaneity (n6) loaded on the Disorganized factor, and the item Conceptual disorganization (p2) loaded on the Positive factor. After adding these loadings to the model, the CFA did not reach acceptable fit (CFI = .855, RMSEA = .088). Again, loadings of .25 (6 in total) and loadings of .2 (5 in total) were added to the
can also possibly be explained by AMC is located in an ethnically diverse neighbourhood, where the baseline of people of non-Dutch origin is high.
9Interestingly, the only strong (unanticipated) negative cross loading was the loading of Lack of judgment & insight (g12) on the Depression/Emotional distress factor. This is in line with the finding that insight in one’s condition may lead to depressed feelings. (Mintz, Dobson & Romney, 2003).
model stepwise, however this did not lead to acceptable fit-indices. The CFI reached acceptable fit when the model included loadings higher than .2 (CFI = .904), however the RMSEA did not (RMSEA = .075; see Table 2 for details).
As for the internal reliability, the inclusion of the extra items in the new factors of both models resulted in a decrease of omega values (Table 5 and Table 6). However, all but the Depression/Emotional distress factors still showed acceptable general factor saturation. Because many items were now included in multiple factors and all but one factor (the Depresion/Emotional distress factor) still showed evidence of a general factor underlying the items, we were curious whether the 30-item version of the PANSS and the 20-item version of the PANSS showed evidence of general factor saturation in their complete form. In search of the strength of this “general psychosis factor”, we therefore ran an explorative analysis in which we computed the ωh for both questionnaires in their totality. The analyses revealed that tor the full 30-item version of the PANSS, omega was just above the threshold for acceptable general factor saturation (ωh = .63), and for the 20-item version, omega was just below this threshold (ωh = 56). 3.4 30-item network
3.4.1 30-item correlation network; general structure in 2-dimensional space
Both versions of the PANSS showed borderline evidence of general factor saturation, suggesting that the concepts measured by the five putative factors show overlap, or at least some form of connection. In the correlation networks, these connections were further analyzed. The abbreviations that were used in the network models can be found in Appendix D1.
As could have been suspected on the basis of the results from the factor analyses, the 30 items of the PANSS did not fall neatly into five clusters at first sight (Figures 1a ad 1b). Especially the items of the Positive and Disorganized factors were intertwined, while at the same time these clusters seemed to have a central position in the network. The items of the Excited/Manic factor were also strongly connected to the Positive and Disorganized factor, although they formed a more clearly distinguishable cluster. Close to the items of the Disorganized factor, the items of the Negative factor seemed to form an especially strong cluster, as the edges between them were very strong, and the connections with other items were clearly weaker. The Negative factor seemed to be positioned alongside the Disorganized factor. In contrast, the Depression factor seemed to be somewhat isolated from the network, and did not seem to form a strong cluster, with weak edges between its items. Its item Somatic concern (soma, g1) did not seem to be strongly related to other items in the network at all. The only strong path between the items of the Depressed factor and the rest of the network seemed to go through items Anxiety (anxi, g2) and Tension (tens, g6). Finally, the Excited factor and the Negative factor seemed to form opposing ends of the network.
Another finding that warrants attention is that a substantial number of items that were attributed to multiple factors in the Van der Gaag model were indeed located between clusters that represented these factors. Disturbance in volition (Voli, g13) and Conceptual disorganization (conc, p2) were located between items of the Negative and Disorganized factor, Active withdrawal (acWd, g16) seemed to be strongly connected to items of all factors except the Disorganized factor. Suspiciousness (susp, p6) was located between the Positive factor and the Emotional distress factor. Poor rapport (rapp, n3) and Uncooperativeness (unco, g8) indeed seemed to form the bridge between the items of the Negative and Excited factor, although Poor rapport was indeed associated with negative symptoms more strongly, and Uncooperativeness was indeed associated with excited symptoms more. Lack of judgment & Insight (judg, g12) and Difficulty in abstraction (abst, n5)
were also in between the items of the Disorganized factor and the Positive factor. However, the position of other items did not match the Van der Gaag model. Grandiosity (gran, p5) was not as closely connected to the items of the Positive factor, and Preoccupation (preo, g15) was more closely related to items of the Positive factor than those of the Depression/Emotional distress factor.
A final noticeable aspect of this network model was that correlations above .3 were exclusively positive. While this might seem strange, as one would expect that items like Excitement (exci, p4) and Motor retardation (moto, g7) would correlate negatively, the absence of a negative correlation could be explained by the fact that both symptoms were frequently absent at the same time.10
3.4.2 30-item correlation network, centrality of items
In the 30-item correlation network, the items Conceptual disorganization (conc, p2), Poor attention (atte, g11), Stereotyped thinking (ster, n7), Lack of judgement and insight (judg, g12) and Unusual thought content (unus, g9) were the five highest scoring items on strength and closeness (the number and strength of connections with other items, and the number of short connections to other items respectively; see Table 7). Of these items, Conceptual disorganization (conc, p2), Poor attention (ate, g11,) and Stereotyped thinking (ster, n7) were also in the top five highest scoring items on betweenness (they were on positions where they connected other parts of the network). Items Poor rapport (rapp, n3) and Passive withdrawal (PaWd, n4) also scored high on betweenness. The highest scoring item on betweenness however, was Tension (tens, g4). This can be explained by the fact that the item seemed to be the only strong connection between the items of the Depression/Emotional distress factor and the rest of the network, through a strong connection with the item Anxiety (anxi, g2), as was previously mentioned. The fact that Anxiety and Depression had mediocre betweenness values can be explained by the fact that they both seem to connect the item Guilt feelings (guil, g3) to the rest of the network. On the bottom end of the strength and closeness measures were the items Anxiety (anxi, g2), Somatic concern (soma, g1), Depression (depr, g6) and Guilt feelings (guil, g3)(see Table 2), as could be expected on the basis of their peripheral orientation in the visual representation of the network.
3.4.4 30-item LASSO network
The LASSO network of the 30-items (Figures 2a and b) revealed eight partial correlations that were higher than .3. There were strong connections between Unusual thought content (unus, g9) and Delusions (delu, p1), and between Delusions (delu, p1) and Suspiciousness (susp, p6). The connection between Unusual thought content (unus, g9) and Delusions (delu, p1) can be explained by similarities in formulation. The main difference between the two items is that the description of the item Delusions mentions a component of interference with social relations or behavior. The connection between Delusions and Suspiciousness can be attributed to the fact that suspiciousness may indicate paranoid delusions, a specific form of delusions. The strong connection between Poor rapport (rapp, n3) and Lack of spontaneity & flow in conversation (spon, n6) can also be explained by their formulation; both mention reduced verbal communication. Likewise, the connection between Tension (tens, g4) and Anxiety (anx, g2) can be explained by the fact that the definition of the item Tension mentions ‘anxiety’. The same goes for Uncooperativeness (unco, g8) and Hostility (host, p7), as was already mentioned in the introduction and Emotional withdrawal (emWi, n2) and Passive withdrawal (paWd, n4); the main difference between the latter two items being the distinction between withdrawal
out of emotional events and life events respectively. Less literally, but still strongly connected were Emotional withdrawal (emWi, n2) and Blunted affect (blAf, n1); the principal difference here being the basis of rating (personal report v. direct observation, respectively). Lastly, the strong connection between Motor retardation (moto, g7) and Blunted affect (blAf, n1) might arise from the fact that both items rely partly on the observation of ‘physical manifestations’ of inhibition. Finally, the network did not show properties of a small world (small-worldness = 1.05).
3.4.3 30-item correlation network, deleted items in Wallwork model
Some of the items that are deleted in the 20-item version of the PANSS seemed to have a central orientation in the 30-item network. The item Stereotyped thinking, (ster, n7) scored high on all centrality measures, and Lack of judgment & insight (judg, g12), and Disturbance of volition (voli, g13) scored high on strength and closeness. Preoccupation (preo, g15) seemed to have a relatively central position in the network, and had relatively high centrality indices. Active withdrawal (acWd, andg16) seemed to be a bit less strongly connected, and this was also apparent in the centrality indices. Suspiciousness (susp, p6) had a relatively peripheral position in the visualization of the network, and mediocre centrality indices. The items’ central position may be caused by its very strong connection with Delusions (delu, p1). Items Disorientation (diso, g1) and Mannerisms & posturing (maPo, g5) had low centrality indices. A possible explanation for this finding is that the symptoms are not frequently experienced, as evidenced by low average scores (Table 7). As was previously discussed, the item Tension (tens, g4) had a very high betweennes score, and the item Somatic concern (soma, g1) seemed to be isolated from the rest of the network.
3.5.1 20-item correlation network, general structure in 2-dimensional space
Compared to the correlation network of the full version of the PANSS, the five factors were more easily distinguishable in the 20-item version of the PANSS (Figures 3a and 3b). However, there were still quite a few strong edges between items of different putative factors. In general, the model showed a basic structure that was comparable to the model of the full version of the PANSS. The strongest cluster again seemed to be formed by items of the Negative factor. The items of the Depression/Emotional distress seemed to be isolated from the rest of the 20-item network even more strongly now that the item Tension was deleted (tens, g4). The items of the Disorganized and the Positive factor still seemed to have a central orientation. In contrast to the 30-item network, there were no longer correlations of above .3 between items of the Positive factor and the Negative factor in this network.
3.5.2 20-item correlation network, centrality of items
The centrality indices of this network also revealed a similar pattern to the full 30-item version, where Conceptual disorganization (conc, p2), Poor rapport (rapp, n3), Passive social withdrawal (paWd, n4) and Poor attention (atte, g11) had a high score on strength and closeness again; Anxiety (anxi, g2) and Depression (g6) had mediocre scores on betweenness and low scores on strength and closeness again, and the lowest scoring items were those of the Depression/Emotional distress factor again, especially the item Guilt feelings (guil, g3).
3.5.3 20-item LASSO network
The LASSO network of the 20-item version (Figure 4) also revealed some similar connections to the 30-item version. The item Delusions (delu, p1) was again connected to items Unusual thought content (unus, g9) and Hallucinations (hall, p3). Items Uncooperativeness
(unco, g8) and Hostility (host, p7) were again connected. In addition, Hostility was now connected to Poor impulse control (impu, g14). This connection can be explained by the fact that the description of Poor impulse control mentions ‘anger’, as does the description of Hostility. The item Emotional withdrawal (emWi, n2) was again connected to Passive withdrawal (paWd, n4) and Blunted affect (blAf, n1), and Blunted affect was again connected to Motor retardation (moto, g7). The item Motor retardation was now additionally connected to Lack of spontaneity and flow in conversation (spon, n6). This connection does not seem to stem from similarity in formulation of the items, and thus seems to be the first fully conceptual connection. Connections that were not found in the 30-item network were formed between Conceptual disorganization (diso, p2) and Difficulty in abstract thinking (abst, Poor attention), and Conceptual disorganization and Poor attention (atte, g11). The connection between Conceptual disorganization and Difficulty in abstract thinking (abst, n5) could be due to the fact that difficulty in abstraction seems to be a specific form of conceptual disorganization; both are rated on the basis of cognitive verbal processes observed during the interview. The connection between Conceptual disorganization and Poor attention does not seem to originate from formulation or shared basis for rating. The disturbed “goal-directed sequencing and loose associations” mentioned under Conceptual disorganization however, do seem to echo problems with attention. A noticeable aspect of the partial correlations higher than .3 is that they were all between items that were attributed to the same factors. Finally, this network again did not show evidence of a small-world structure (small-worldness = 1.02)
3.6 Stability of the Networks
Both the 20-item correlation network and the 30-item correlation network seemed to be stable when the sample was randomly divided in two groups, and networks per group were compared (Appendices B1-B2). When the sample was divided in three groups according to type of diagnosis, the networks of patients with schizophrenia was somewhat different from the networks of patients suffering from schizo-affective disorder, but very similar to the networks of patients with other psychotic manifestations (i.e. brief psychotic disorder, schizophreniform disorder, psychosis not otherwise specified; Appendices D2-D5).
4. Discussion 4.1 Overview
The main goal of this study was to examine the internal structure of the PANSS by using both traditional factor-analytic techniques and novel network techniques. In line with our first hypothesis, we found that both the Van der Gaag model and the Wallwork model showed unsatisfactory fit to our dataset. Very strong evidence was found for our second hypothesis: even when the original models were extended by adding a large number of low cross loadings, they still showed unsatisfactory fit. Concerning the third hypothesis, only the Depressed/Emotional distress factor showed a lack of general factor saturation. Not only did the other factors indeed show evidence of a general factor underlying their items; the full 30-item version as a whole showed evidence for a general factor as well, and the 20-item version came close to the set threshold.
When the questionnaires were analyzed using network techniques, the same general structure appeared in both network models, where the Positive and Disorganized factor were especially intertwined and related to the Excited factor; the Depression/Emotional distress factor was somewhat isolated from the rest of the network, and the Negative factor showed the strongest clustering among its items. Some
of the deleted items had a strong involvement in the 30-item network, and others were more peripheral/specific. In the following sections, we will discuss interpretations and implications of the factor analytic analyses and network analyses separately.
4.2 Factor analyses
The main question regarding the internal structure of the PANSS was whether it is viable to divide the questionnaire into five subdomains. As was discussed earlier, both the Van der Gaag model and the Wallwork model could only come near the thresholds for acceptable fit when a large number of low cross-loadings were added to the original models. Although these results imply that it might be unrealistic to impose a five-factor structure on the PANSS, it should be noted that previous authors have found that existing models showed superior fit in more chronic samples (Drake et al., 2001). An explanation for these findings is that in the longer course of the illness, the symptoms are more profound and thereby clearly distinguishable than in the initial phase. In the earlier stage of the illness, the medication might adequately suppress some of the symptoms. As was mentioned earlier, the majority of patients in our sample were using antipsychotic medication at the time of the interview. Evidence that our sample might indeed suffer from limited variance within symptoms comes from the fact that all factor loadings in the CFA of the Wallwork model were lower than the loadings reported in the original study. Given the sample, our conclusions are hence limited to the population of patients with recent onset psychosis.
Still, the current study is one of many studies that fail to find adequate fit of CFA’s of existing five-factor models. A possible explanation for the fact that five-factor models of the PANSS are hard to replicate is given by the argument that was already made by Van der Gaag et al. (2006). Some items of the PANSS may have multiple, strongly different causes. An item like Poor rapport (rapp, n3; lack of a good relation between patient and interviewer) can be a result of either hostile behavior or apathetic behavior. An item like Tension (tens, g4; a physical manifestation) can stem from either anxiety or excitement.
The solution by Van der Gaag et al. (2006b) is to include these items in both factors. However, in a specific sample the origin of these symptoms might lie in one of the two explanations stronger than the other, and this could still decrease the model fit. A more serious issue might arise when sum scores are given for factors on the basis of models with such cross-loadings. To stay with the first example: when someone shows a profile with strongly negative symptoms and thereby a heightened score on Poor rapport (n3), this person would artificially have a heightened score on excited symptoms as well when Poor rapport is also part of the equation for that domain. In general, including a large number of items with low loadings could lead to biased sum-scores in a comparable manner. This problem has been addressed previously by Mortimer (2007).
The solution by Wallwork et al. (2012) is to delete all items that load on factors inconsistently, load on multiple factors, or do not load on factors strongly. The result seems to be a more robust yet blunt measure, because of the small number of items per factor. However, our analyses revealed that the general factor saturation of the factors of the Wallwork model was acceptable, and even superior to the general factor saturation of those of the Van der Gaag model. Nonetheless, it should be noted that the model also did not fit the data adequately.
A specifically noteworthy finding within the reliability analyses was that ωh was acceptable for the original ‘Depression’ factor in the Wallwork model (consisting of just Depression (g6), Anxiety (g2) and Guilt feelings (g3)), yet started to decline when other items were added. When interpreting this finding, it is important to keep in mind that ωh essentially measures the degree to which items form a general factor. This is one of the
reasons why ωh is less prone to bias coming from large numbers of items than Chronbachs alpha. However, the question whether a scale with just three items forms a reliable measure of a fluid concept like Emotional distress, is another topic. By any means, we believe that a scale consisting of just three items, one of which actually measuring anxiety, should not be labeled a ‘Depressive factor’. The term given by the Wallwork model can be highly biased in this context, and should be abandoned. The finding that the Depression/Emotional distress factor showed questionable general factor saturation when it was extended, is in line with results found by Emsley et al. (2003) and Van den Oord et al., 2006, who also found low internal reliability of their “Depression” factor. Although the three items of the Wallwork factor are loose and unspecific, the factor could not be made more coherent by extending it with other items of the PANSS.
Not only did the other separate factors of both models show acceptable general factor saturation, both the full 30-item version and the 20-item version of the PANSS showed some evidence of a general factor encompassing all items (i.e. a total psychosis scale) as well. For the 30-item version of the PANSS, the omega value was just above the threshold of .6, while for the 20-item version it was just below. This could be seen as evidence that factors of the 20-item version show a slightly stronger divergence, while the items of the 30-item version may be more encompassing with regard to items that indicate psychotic symptomatology generally. As such, the Wallwork model indeed seems to be preferable for the use of sum scores for specific factors, while the Van der Gaag model might be more suitable for providing a total score for the PANSS.
4.3 Network analyses
4.3.1 General network structure
In general, the structures of the 30-item version of the PANSS and the 20-item version of the PANSS were very similar, suggesting that the deletion of the 10 items did not alter the general structure strongly. The presumptive factors were somewhat more clearly separated in the 20-item network, suggesting that the deletion of the items indeed led to better distinguishable categories. However, there were still some strong connections between factors in the 20-item network, possibly explaining for the lack of model fit of a five-factor model. In line with the finding of strong connections between items, both versions of the PANSS showed borderline evidence of a general factor underlying their items. However, the network structures of both versions did not show properties of a small world. This finding indicates that it is not the case that when one symptom is activated, activation is likely to quickly spread to the other symptoms in the network. Although there is a shared component in both models, there is thus also divergence within these networks.
As was mentioned before, one of the supposed benefits of the 20-item model is that it might be more robust than the 30-item network. A relevant research question regarding this claim could be whether the network-structure of the 20-item version of the PANSS is indeed more consistent than the network-structure of the 30-item network. To answer this question, a large number of 20-item networks and a large number of 30-item networks might be created for a very large sample, so that the stability of centrality indices can be analyzed.
4.3.2 Specific factors
The network analysis revealed that many items that were considered to be part of both the Positive and the Disorganized factor in the Van der Gaag model were indeed located between these factors in the 30-item network, and were deleted in the Wallwork model
(Lack of judgement & insight (judg, g12), Disturbance of volition (voli, g13), Preoccupation (preo, g15)). While this might imply that the Wallwork model includes only the more narrow cognitive functions (Difficulty in abstraction (abst, n5), Attention (ate, g11), Conceptual disorganization (conc, p2)), it is important to note that all items in the Disorganized factor except Difficulty in abstraction are measured observationally, and studies that compare a Disorganized factor of the PANSS to such measures have been mixed (Good et al., 2004, Fresán et al., 2005), as was mentioned earlier. We would like to emphasize again that using the Cognitive factor of the PANSS as an outcome measure for cognitive functions in research could bias interpretation. For such purposes, more specific measures are needed, as previous authors have also concluded (Rodriguez-Jiminez et al., 2012).
The finding that the items of the Depression/Emotional distress factor were somewhat separated from the rest of the networks implies that these items measure a construct that is distinct from (the rest of) psychosis. This was unexpected, as previous studies have described the numerous ways in which depressive symptoms and psychotic symptoms can be intertwined (Birchwood, 2003, Garety et al., 2001). However, it is important to keep in mind that the Depression/Emotional distress factor consisted of few, and unspecific items: depression for instance was measured with a single item. It is possible that some more specific features of depression would correlate with some of the items of the PANSS more strongly.11 Nevertheless, the data at least suggest that the clinicians were able to distinguish between negative and depressive symptoms, as these were not strongly related.
4.3.3 Deleted items
The network analysis also revealed which of the items deleted by the Wallwork model were strongly involved in the 30-item network, and which items were less involved. Items that were strongly involved were Tension (tens, g4), Disturbance of volition (voli, g13), Stereotyped thinking (ster, n7) and Lack of judgment & Insight (judg, g12). Tension was mostly involved because it connected the items of the Depression factor to the rest of the network. The item was not so much related to the items of the Manic/Excited factor, and was thus not as non-specific as might have been suspected. Lack of judgment & insight was conceptually a strongly non-specific item, and its centrality can be seen as statistical proof. Although this item might be very hard to fit in one of the factors, it might be regarded as one of the key markers of the patients’ current state (Quee et al., 2010). Disregarding the item altogether would thus be regrettable. Stereotyped thinking and Disturbance in volition seem to conceptually be more specific, although Stereotyped thinking can be associated with both positive and disorganized symptoms, and Disturbance in volition can be associated with negative and disorganized symptoms. Stereotyped thinking and Preoccupation together seem to represent the tendency to show a repetitive thinking pattern, revolving around a specific theme. Deleting these items because they could be seen as both cognitive and positive, might mean that valuable information will be lost. Disturbance of volition, seems to measure the more cognitive variation of the avolition that is also measured by Passive withdrawal (paWd, n4) and Lack of spontaneity (spon, n6). Because this item is measured observationally, it might be hard to separate from these items. However, we do believe that the item might have unique value, as cognitive indecisiveness might play a roll at varying stages of the illness.
Less involved in the network were Active social withdrawal (acWd, g16), and Suspiciousness (susp, p6). Suspiciousness was mostly strongly connected to Delusions (delu, p1). However, the average centrality of the item does imply that the symptom is also
11 Furthermore, the centrality indices of these items in the LASSO network were higher,
related to other constructs. Active withdrawal was regarded the most non-specific item by the Van der Gaag model, as it was considered to load on all factors except the Disorganized factor. When looking at the network structure, it seems as though this might indeed be the case as strong connections are formed between the item and items of all other factors except the Disorganized factor. However, the average centrality of the item points towards a mediocre involvement of the item in the network. This was unexpected, as social isolation is generally regarded as an aggravator of a range of other symptoms (Garety et al., 2001).
Even less involved in the network were Disorientation (diso, g10), Mannerisms and posturing (maPo, g5), and Somatic concern (soma, g1). Both theoretically and statistically these items indeed seem to be more specific. Mannerisms and posturing and Disorientation (diso, g10) seem to be at the extreme end of disorganized symptoms. In line with this idea, these items had low average scores, as was mentioned earlier. Somatic concern can be conceptualized as a very specific form of a delusion. However, the item was not specifically associated with delusions, or with any other symptom. The item did have a higher mean score than the previously mentioned items, indicating that it was not just the exceptionality of the item that lead to a weaker involvement in the network. Somatic concern might thus be considered to be less strongly related to the concept of psychosis, and might justifiably be deleted.
It should be noted that the item Guilt feelings (guil, g3), which was also included in the 20-item version, had the lowest centrality indices of all items in both models, and was thus also very specific. This is another finding that would not have been clear without the network analyses. After all, the item did load on the Depression factor in both models.
It is important to recognize that both specific and non-specific items can still be regarded as important features of psychosis, and network techniques do justice to this idea. Specific items might be highly informative in personal dynamic networks, both as specific causes or consequences of other symptoms. Specific items like Guilt feelings or Somatic concern for instance might be the origin of a range of other symptoms. Non-specific items will be present more often in these dynamic networks, but might just as well be bridge symptoms and cause more specific symptoms.
4.4 Future directions
The network analyses highlighted that when these five dimensions are indeed considered to be the conclusive and specific dimensions of psychosis, the Depressive domain has to be represented more completely, and an effort has to be made to make items more specific to these five domains. The last-mentioned concern was also expressed by Mortimer (2007). A remedy for this problem might lie in the formulations of items. As we have seen, a substantial number of strong partial correlations arise between items that use the same terminology- sometimes even the exact same words. If the items are defined more specifically, conceptual connections might be found to be more pure.
However, a first consideration is whether these five domains are indeed conclusive. The committee of the DSM-V for instance, does not regard the five dimensions as sufficient to cover the relevant symptom constructs of psychosis, and now speaks of eight dimensions (Barch et al., 2013). For these dimensions, positive symptoms are separated further into Delusions and Hallucinations, a decision that is in line with the finding that the two symptoms were not very strongly related. The DSM-V descriptions further separate cognitive symptoms into Impaired cognition and Disorganized speech, a distinction that was not necessarily clear from our network models. Finally, the DSM-V adds an extra dimension for Abnormal psychomotor behavior, a dimension that was not strongly represented in the PANSS. Although the DSM represents classifying
diagnostics, and the PANSS might rather represent descriptive diagnostics, there might still be a future in basing a questionnaire on these eight dimensions. The specificity of the eight dimensions could again be investigated using network techniques. We do believe that for this research, a new questionnaire is warranted that includes a range of items for all dimensions. However, based on our analyses, we question whether such a questionnaire would provide more specific, distinguishable factors. Our data suggest that a substantive number of psychotic symptoms show some connectedness beyond their so-called domain. Assessment with the PANSS might therefore benefit from the addition of networks.
We recognize that summing items of presumptive dimensions has become such common practice, that it is unlikely that it will be altogether abandoned any time soon. Indeed a network-perspective is fully compatible with using sum-scores as indicators of total network activation or activations of parts of the network. The network might in this case include the more non-specific items and very specific items, while the sum scores might include the items selected by the Wallwork model. Together these sources will form a complete overview of the current state of a given sample.
We believe that network techniques can provide a new range of research questions looking beyond the questionable separation of symptom constructs. Network structures can be compared between groups. Networks of high-risk groups that have not yet crossed the border of an actual diagnosis may for instance be compared to networks of groups that have crossed these borders and more chronic groups. Alternatively, the networks of patients diagnosed with schizophrenia might be compared to networks of patients suffering from other forms of psychosis more specifically than was done in this study.
Another future application of network modeling might lie in extending the networks of psychotic symptoms with other relevant symptoms, considered to be part of relevant disorders like depression and anxiety. This might guide the scope to certain bridge symptoms that are important for transgressing from one state to the other (Cramer et al., 2010). Dynamic networks of ultra-high risk groups, or psychotic groups might be analyzed for common paths between symptoms, as has been done with fewer variables before (Wigman et al., 2015).
To summarize, we have highlighted some shortcomings the PANSS itself, and some problems of using factor analytic methods to analyze the PANSS. Shortcomings of the PANSS include that its Depression factor seems to be inadequate, and the Cognitive/Disorganized factor needs to be regarded as a purely Disorganized factor. In addition, the PANSS seems to suffer from an abundance of overlapping formulations. The main problem with factor analytic approaches stems from the fact that the items of the PANSS seem to be highly interrelated. By trying to separate them, as is the goal of factor analytic approaches, some items that could be regarded genuinely relevant aspects of multiple symptom constructs will be disregarded. We can see a future for such items within multiple applications of network techniques. These techniques do justice to the idea that a symptom might have multiple causes, and might lead to a range of new research questions.
5. Tables and figures Table 1: Sample characteristics
N % Total 657 100.0% Demographic data Male 490 74.60%* Female 147 22.40%* Dutch origin ** 250 38.10%* Non-Dutch origin ** 352 53.60%*
Mean age in years (SD) 23.9 (6.0)
Use of anti-psychotics
Yes 502 76.40%
No 118 18.00%
DSM-IV-TR diagnosis
Schizophrenia, paranoid type 253 38.5%
Schizophrenia, undifferentiated type 96 14.6%
Schizophrenia, disorganized type 20 3.0%
Schizophrenia, catatonic type 4 0.6%
Schizophrenia, residual type 1 0.2%
Schizo-affective disorder 67 10.2%
Schizophreniform disorder 66 10.0%
Psychosis, not otherwise specified 136 20.7%
Brief psychotic disorder 14 2.1%
*These percentages do not add up to 100% because some cases were missing demographic information. ** Patients were considered to be of non-Dutch origin when at least one parent was born outside of the Netherlands.
Table 2: Goodness-of-fit indices for the original CFA models, and extended models Goodness of fit
Included
loadings DF Chi-square CFI (90% interval) RMSEA Van der Gaag Original 380 1713.425 .846 .073 (.070 - .080)
EFA above .30 376 1650.305 .853 .072 (.068 - .075) EFA above .25 369 1548.052 .864 .070 (.066 - .073) EFA above .20 356 1323.262 .889 .064 (.061 - .068) Wallwork Original 160 1007.864 .845 .090 (.085 - .095) EFA above .30 155 950.644 .855 .088 (.083 - .094) EFA above .25 148 754.177 .889 .079 (.073 - .085) EFA above .20 143 671.620 .904 .075 (.069 - .081)
Table 3: Standardized factor loadings in the Confirmatory Factor Analysis of the Van der Gaag model, and omega of the models’ factors. X’s mark items that are deleted in the Wallwork model.
Pos. Neg. Dis. Exc. Emo.
p1 Delusions 0.92 p2 Conceptual disorganisation -0.07 0.83 p3 Hallucinations 0.51 p4 Excitement 0.72 p5 Grandiosity 0.25 0.55 p6 Suspiciousness (X) 0.64 0.15 p7 Hostility 0.79 n1 Blunted affect 0.76 n2 Emotional withdrawal 0.83 n3 Poor rapport 0.69 0.24
n4 Passive social withdrawal 0.78
n5 Difficulty in abstract thinking -0.15 0.71
n6 Lack of spontaneity 0.75 n7 Stereotyped thinking (X) 0.72 g1 Somatic concern (X) 0.13 0.26 g2 Anxiety 0.80 g3 Guilt 0.47 g4 Tension (X) 0.31 0.65
g5 Mannerisms & posturing (X) 0.50
g6 Depression 0.49
g7 Motor retardation 0.62
g8 Uncooperativeness 0.18 0.70
g9 Unusual thought content 0.51 0.32
g10 Disorientation (X) 0.58
g11 Poor attention 0.70
g12 Lack of judgement & insight (X) 0.17 0.56
g13 Disturbance of volition (X) 0.24 0.46
g14 Poor impulse control 0.65
g15 Preoccupation (X) 0.54 0.15
g16 Active social withdrawal (X) 0.17 0.26 0.30 0.15
Table 4: Factor loadings of the Confirmatory Factor Analysis of the Wallwork model, and omega of the models’ factors
Pos. Neg. Dis. Exc. Dep.
p1 Delusions 0.77 p2 Conceptual disorganisation 0.84 p3 Hallucinations 0.50 p4 Excitement 0.69 p5 Grandiosity 0.54 p7 Hostility 0.81 n1 Blunted affect 0.76 n2 Emotional withdrawal 0.82 n3 Poor rapport 0.74
n4 Passive social withdrawal 0.77
n5 Difficulty in abstract thinking 0.63
n6 Lack of spontaneity 0.75 g2 Anxiety 0.61 g3 Guilt 0.54 g6 Depression 0.64 g7 Motor retardation 0.62 g8 Uncooperativeness 0.74
g9 Unusual thought content 0.80
g11 Poor attention 0.69
g14 Poor impulse control 0.64
