Item analysis of single-peaked response data : the psychometric evaluation of bipolar measurement scales Polak, M.G.

psychometric evaluation of bipolar measurement scales

Polak, M.G.

Citation

Polak, M. G. (2011, May 26). Item analysis of single-peaked response data :

Rotterdam. Retrieved from https://hdl.handle.net/1887/17697

Version: Not Applicable (or Unknown) License:

Licence agreement concerning inclusion of doctoral thesis in the Institutional Repository of the University of Leiden

Downloaded from:

Note: To cite this publication please use the final published version (if

applicable).

Chapter 5

The Developmental Profile:

Validation of a Theory Driven Instrument for Personality

Assessment ¹

Abstract

The Developmental Profile is an instrument for personality assessment. It covers both maladaptive and adaptive characteristics. In this chapter, we examined its internal consistency and construct validity in a Dutch sam- ple of 763 participants from various clinical and non-clinical settings. The internal consistency reliability estimates were good for the clusters of lev- els (adaptive, neurotic, and primitive), although not for all separate levels.

Confirmatory factor analysis showed an overall good fit, with the excep- tion of the level of primary narcissism. Furthermore, empirical evidence was found for the interpretation of a patient’s Developmental Profile ac- cording to increasing levels of aggregation, with as a highest level a single maladaptivity-adaptivity scale score. This scale significantly distinguished between different patient groups.

5.1 Introduction

Personality pathology is generally regarded as one of the most relevant clinical pre- dictors of the course and outcome of psychotherapy. The Diagnostic and Statisti- cal Manual of Mental Disorders (fourth edition [DSM-IV]; American Psychiatric Association, 1994) Axis II is still the standard to classify personality pathology, although from both a scientific and clinical point of view it is strongly criticized

1This chapter has been published as: Polak, M. G., Van, H. L., Overeem-Seldenrijk, J., Heiser, W. J., & Abraham, R. E. (2010). The Developmental Profile: Validation of a theory-driven instrument for personality assessment. Psychotherapy Research, 20, 259-272.

This study was supported by a grant from the Dutch Psychoanalytic Foundation. This foundation had no involvement with the design and conduct of the study; data collection, management, analysis, or interpretation; and manuscript preparation, review, or approval.

We thank Harry Stroeken, PhD, for his help in obtaining the participation of nonpsychiatric volunteers.

(Widiger, Simonson, Sirovatka, & Regier, 2006). Among other things, the categor- ical classification, the heterogeneity within diagnosis, and the absence of defining normal personality functioning limit its value. Nevertheless, for various person- ality disorders according to DSM IV, psychotherapeutic approaches have been developed (e.g.,Giesen-Bloo et al., 2006; Clarkin et al., 2007; Svatberg, Stiles, &

Seltzer, 2004)

However, at the level of case formulation or planning specific psychotherapeutic strategies, the utility of Axis II diagnosis can be questioned (Verheul, 2005), in part, because it does not operationalize systematically useful clinical concepts such as interpersonal functioning, coping with stressors and self image.

Psychodynamic concepts, although less scientifically elaborated and rather difficult to measure, are considered as more closely related to the way clinicians think about patients, and to theoretical concepts that are frequently applied in the psychotherapeutic practice to understand patients.

A central and distinctive feature of the psychodynamic approach is to adopt a developmental perspective on personality. This implies that personality grows dur- ing life, alongside a line of increasing maturity. It recognizes that the intrapsychic and relational level of functioning in an individual are ultimately determined by the interplay of immature and mature characteristics. Therefore, both patholog- ical and healthy features of personality need to be determined in order to obtain a balanced view on the structure of personality and its role in the therapeutic process.

An example of such a theoretically and clinically driven instrument for per- sonality assessment, which takes a hierarchically arranged view on personality, is the Developmental Profile (DP; Abraham, 1993, 2005; Abraham et al., 2001).

Aspects of the predictive value have been studied, and yielded promising results, for instance, the prediction of drop out during clinical psychotherapy (Ingen- hoven, Duivenvoorden, Passchier, & van den Brink, 2009). Furthermore, Van et al. (2008) and Van, Dekker, Peen, Abraham, and Schoevers (2009) have demon- strated the predictive value for outcome of psychotherapy for depression of some separate developmental lines such as object relations and problem solving behav- ior. In addition, the DP appeared to be useful in various clinical settings for determining psychotherapeutic strategies (Ingenhoven, 2005, 2010; Van Marle &

Abraham, 2005).

The aim of the current chapter is to examine the internal consistency reliability and the construct validity of the DP. In particular, the paper seeks to validate the hierarchical structure of the DP, reflecting its developmental perspective on

5.1. Introduction

personality.

5.1.1 Description of the DP

As a frame of reference the DP uses nine developmental levels and nine devel- opmental lines (see appendix 5.A and 5.B of this chapter). Together the nine developmental levels and the nine developmental lines, form a matrix of eighty- one personality characteristics, which correspond to the items of the DP.

The DP represents an attempt to organize and standardize psychodynamic personality diagnostics. As such, the instrument builds on the work of Anna Freud (1963), Loevinger and Wesseler (1970), Bellak, Hurulch, and Geddiman (1973), Kernberg (1981), and Luborsky and Crits-Cristoph (1990). The concept of the developmental lines and the term developmental profile is derived from the work of Anna Freud (1963). Available descriptions of developmental stages of behavioral domains were integrated in the DP. The developmental lines of norms and cognitions correspond to the stages of moral and cognitive development described by Kohlberg (1981) and Piaget (1962), respectively.

The developmental levels in the profile matrix are ordered according to the degree to which they are associated with the severity of maladaptive psychosocial functioning. The lowest two levels, Lack of Structure and Fragmentation refer to Kernberg’s (1981) psychotic and borderline personality organization. The level of Egocentricity refers to narcissistic problems as elaborated by Kohut (1971). The next three levels Symbiosis, Resistance, and Rivalry, represent the oral, the anal and the phallic character as described by Abraham (1925). The three adaptive levels Individualization, Solidarity, Generativity, are based on Erikson’s (1966) model of adult development.

Clearly these levels reflect a continuum of severity form adaptive to very mal- adaptive levels of personality functioning. The levels are divided into healthy, neurotic and primitive or borderline ranges of personality (Psychodynamic Diag- nostic Manual; PDM, 2006, p. 20ff). Neurotic levels represent conflict pathology.

The primitive level that ranges form the border of neurotic to psychotic condi- tions represents developmental deficits. In the DP this has been operationalized as the primitive cluster (Lack of Structure, Fragmentation, and Egocentricity), the neurotic cluster (Symbiosis, Resistance, and Rivalry), and the adaptive cluster (Individuation, Solidarity, and Generativity).

The developmental levels are not mutually exclusive (Wilson & Gedo, 1993) but indicate a tendency of behavior. Therefore, at the item level maladaptive

behavior in one domain does not exclude adaptive behavior in another domain.

For instance, the behavior of a brilliant manager who scores high on Productivity (61) may also be characterized to a significant degree by behavior on the level Egocentricity. This is in accordance with the prevalence of personality disorder characteristics in non-patient samples (Vaillant & Drake, 1985; Zimmerman &

Coryell, 1989). By registering both types of functioning, the DP makes it possible to chart complex and even contradictory human behavior.

The item definitions were based on clinically relevant behavioral patterns avail- able in the literature. Since these descriptions were rather heterogeneous, the following criteria were used for item selection and definition: Clinical significance;

Existence of a theoretical link with either adaptation or maladaptation of the individual’s functioning, where adaptation refers to age-appropriate capabilities;

Possibility to relate the behavior to a single developmental level. This latter crite- rion excluded general psychopathologic phenomena, such as anxiety or depression.

5.1.2 Hypotheses

The current paper examines the internal consistency reliability and construct va- lidity of the DP. This was operationalized in the following hypotheses.

1. The levels and the clusters of levels have sufficient internal consistency reli- ability.

2. Underlying the items of the Developmental Profile is a correlated nine-factor model, with each factor representing a level and each item having a posi- tive loading on its corresponding level only. Furthermore, the levels can be reduced to a three-factor model, which justifies the aggregation of the nine level scores into three cluster scores.

3. Underlying the levels, a bipolar scale can be formed, ranging from highly maladaptive to highly adaptive. On this scale, the levels are ordered ac- cording to their position in the scoring form. Subsequently, the levels can be grouped into three ordered clusters.

4. Various patient groups can be distinguished according to these scale values, assuming that forensic inpatients form the most maladaptive group, and the normal controls form the most adaptive group. The inpatients and outpatients are positioned in between, with the latter as the more adaptive of the two.

5.2. Method

5.2 Method

5.2.1 Interview and Registration Protocol

The Developmental Profile is scored following a semi-structured interview, which takes two to four hours, covering such domains as relationships, schooling and work, reactions to stressful events, dealing with needs anxiety or anger and inad- equacy.

The DP manual (Abraham, 2005) provides detailed definitions of items and anchor points for scoring. Scoring the DP requires an introductory course that takes about 15 hours. The interviewer indicates the degree to which each of the personality characteristics is present using a four-point scale ranging from 0 (not present) to 3 (very clearly present). Subsequently, for each level the sum of the 9 corresponding items is determined. These level scores make up the patient’s Developmental Profile.

Previous studies by Van et al. (2000) and Van, Polak, Abraham, Overeem- Seldenrijk, and van Keulen (2005) showed sufficient interrater reliability for scor- ing the various levels of the DP with, respectively, quadratically weighted kappa (Cohen, 1968) values ranging from .53 to .84 (M = .70); and ranging from .60 to .79 (M = .67).

In the current paper we investigate the justification of the interpretation a pa- tient’s Developmental Profile according to increasing levels of aggregation. First, the item scores (i.e., the manifest behavior described in the registration protocol) may be summed up into level scores, which indicate the degree of functioning on each specific developmental level. Second, one may combine the level scores into a single score on a bipolar maladaptivity-adaptivity scale ranging from max- imal maladaptive functioning (Lack of Structure) to maximal adaptive function- ing (Generativity), using a data analytic approach. Furthermore, the level scores may be grouped into three successive clusters. The first cluster, referred to as the primitive cluster, consists of Lack of Structure, Fragmentation, and Egocentric- ity. The second cluster, referred to as the neurotic cluster, consists of Symbiosis, Resistance, and Rivalry. The third cluster, referred to as the adaptive cluster, encompasses Individuation, Solidarity, and Generativity.

5.2.2 Participants

For the purpose of this psychometric evaluation data were pooled of patients from various sites in the Netherlands were the DP was administered for use either in

daily clinical practice or in research. The pooled sample consisted of 763 patients:

27 forensic inpatients (3.5 %), 468 psychotherapeutic inpatients and day treatment patients(61.3 %), 166 outpatients (21.8 %), and 102 non-psychiatric controls (13.4

%). The sample consisted of 484 women (63.4 %; mean age, 31 years, SD = 10.5), and 277 men (36.3 %; mean age, 36 years, SD = 10.7). For 2 patients (0.3 %) gender information was missing.

5.2.3 Procedure

The DP was administered following a two-step procedure: a therapist or psy- chologist conducted the DP interview, which was rated afterwards by an external psychologist based on the written text. The raters did not have additional in- formation about the patient (e.g., a clinical diagnosis) in order to prevent any scoring bias. In total, 35 different therapists and psychologists administered the interviews. Each of these practitioners participated in the DP introductory course, which was discussed earlier.

5.2.4 Data Analysis

The internal consistency reliability (Hypothesis 1) was determined using Cron- bach’s α coefficient (Cronbach, 1951). Two analyses were completed to evaluate the construct validity of the DP. First, the item structure (Hypothesis 2) was studied using confirmatory factor analysis (CFA). Since, in the DP, the levels are regarded as cumulative (or unipolar) scales (level scores are determined by sum- ming the corresponding item scores), which are theoretically related, a correlated nine-factor model was estimated. However, CFA is not suited for investigating the ordered structure of the levels (Hypothesis 3), because the scale underlying the various levels is bipolar (or substitutive). Therefore, the level scores are an- alyzed with correspondence analysis (CA). CA is known to represent bipolar or substitutive scales correctly (Heiser, 1981), unlike factor analysis, which is suited for the analysis of unipolar or cumulative scales (Van Schuur & Kiers, 1994).

CFA was conducted using EQS 6.1 software (Bentler, 2003). The usual goodness- of-fit indices are reported, including the root-mean-square residual (RMR), the root-mean-square-error of approximation (RMSEA), the goodness-of-fit Index (GFI), the Bentler-Bonnet normed fit index (NFI), the comparative fit index (CFI), the chi-square, and the chi-square divided by degrees of freedom in the model (chi- square/df ). General guidelines indicate that for good model fit the RMR should be less than 0.10, the RMSEA less than 0.05, the GFI greater than 0.95, the CFI

5.3. Results

and NFI greater than 0.90, that chi-square is non-significant, and chi-square/df less than 2 (Kline, 1998). For the individual factor loadings, we used a cutoff value of ± .30 as the minimum level of practical significance (Kline, 1998), and α

= .01 as the significance level for 2-tailed statistical tests (Stevens, 1996).

CA was performed in SPSS 14.0, and is part of the categories module (Meul- man & Heiser, 2004) of SPSS. CA, also known as dual scaling (Nishisato, 1996), produces an optimally weighted combination of patients’ scores, comparable to principal component analysis (see for example from the field of personality assess- ment, Maraun, Slaney, & Jalava, 2005). Although both techniques have slightly different terminology, for instance in the context of CA, instead of the term vari- ance, the term inertia is rather used.

The aim of CA is to find an optimal graphical representation of patients and developmental levels in as few dimensions as possible, which usually results in a two-dimensional plot.

When data are strongly one-dimensional (Hypothesis 3), a two-dimensional representation will show what is often referred to as the arch-effect, where the levels and patients are ordered along an arch according to their position on the scale (e.g., Greenacre, 1984, p. 227). In that case, only the scores on the first dimension are used as scale scores, because the second dimension is an artifact due to the nonlinearity of the item responses. The scores resulting from CA can be standardized in different ways depending on the focus of the analysis. In the current paper the level scores were standardized to have mean 0 and variance 1, and each patient’ s score is computed as the weighted average of his corresponding level scores. To cross-validate the CA solution, we performed ten stratified random splits with the four patient groups as strata. For each split, the correlation was computed between the CA scale values in both subsamples.

Twenty-seven patients (3.5 %), who were outliers in the solution due to a strongly atypical response pattern, were discarded from both CFA and CA. As an exclusion criterion a cut-off score of 6 on the level Egocentricity (20) was used.

This group will be described separately in the discussion of this paper.

5.3 Results

5.3.1 Internal Consistency Reliability of the DP Levels

Table 5.1 shows the internal consistency reliability.

Table 5.1: Internal consistency reliabilities for the levels and clusters of levels of the Developmental Profile (N = 763).

Developmental level Cronbach’s α Clusters

CCAdaptive 0.79 CCNeurotic 0.69 CCPrimitive 0.78

Levels

CCGenerativity (80) 0.48 CCSolidarity (70) 0.60 CCIndividuation (60) 0.64 CCRivalry (50) 0.53 CCResistance (40) 0.44 CCSymbiosis (30) 0.69 CCEgocentricity (20) 0.81 CCFragmentation (10) 0.60 CCLack of Structure (00) 0.58

Cronbach’s α coefficients for the clusters of levels ranged from .69 to .79 (mdn, .78), indicating sufficient reliability for the cluster scores. For the separate levels α coefficients ranged from .44 to .81 (mdn, .60), indicating good and reasonable reliability for, respectively, levels Egocentricity, Symbiosis, and Individuation, but unsatisfactory reliability for the remaining levels.

5.3.2 Confirmatory Factor Analysis of the DP Item Scores

Construct validity of the DP (Hypothesis 2) was investigated with CFA follow- ing a two-step procedure. First, a nine-factor model was tested on the items scores with each factor representing a level. Each item was allowed to load on its corresponding factor only, and factors were allowed to correlate.

Second, a three-factor model² was tested on the level scores, to test the pre- sumption that level scores may be aggregated into three clusters (primitive, neu- rotic, and adaptive). Table 5.2 shows the fit statistics for these factor models. We will first interpret the nine-factor model, which was fitted on the individual items.

As can be seen in Table 5.2, the nine-factor model meets the criteria of RMR, RMSEA, NFI and CFI. The GFI criterion is not met by the model. The signif-

2An additional CFA was performed to test a three-factor model using item level data, which provided further support for the 3 cluster hypothesis. Interested readers could ask the corre- sponding author for the complete CFA results.

5.3. Results

Table 5.2: Confirmatory Factor Analysis Goodness-of-Fit Indices.

Model χ²(df ) χ²/df RMR RMSEA GFI NFI CFI

Nine-factor 6687.5 (3123) 2.14 .020* .039* .807 .926* .959*

Three-factor 6149.5 (24) 6.23 .055* .084 .955* .999* .995*

Note. RMR, root-mean-square residual; RMSEA, root-mean-square-error of approximation; GFI, goodness-of-fit index; NFI, normed fit index; CFI, comparative fit index (CFI) The symbol “*” indicates good model fit.

icance of the chi-square statistic depends on the sample size and given our large sample size and consequently high power, a significant chi-square is not necessarily a sign of a poor fit. As an alternative the chi-square/df ratio is computed as a relative measure of fit. Here, this ratio indicates acceptable model fit.

Table 5.3 shows the standardized factor loadings for the nine-factor model, where each factor represents a level of the DP.

Sixteen items (19.8 %) had non-significant loadings (α = .01). In Table 5.3 we report the percentage of items with a factor loading > .30 for each factor. For most factors the majority of the items had a loading exceeding .30. However, the factors that represent the levels Egocentricity, Resistance, and Generativity had more poorly fitting items. For the latter two levels, this is in agreement with the lower alpha coefficients. Apparently, the items of these levels had relatively low inter-correlations. The level Egocentricity, showed good reliability, but appeared not to fit the structural model of the DP very well. We elaborate on the deviant position of Egocentricity in the discussion.

Subsequently, the three-factor model that was tested on the level scores, which showed an acceptable (according to the chi-square to df ratio, and the RMSEA, cf. Browne & Cudeck, 1993) to good fit according to the remaining indices. Table 5.4 shows the standardized factor loadings for the three-factor model.

All loadings were significant (α = .01), only the level Rivalry did not fit into the neurotic cluster adequately. An explanation for this lack of fit could be that Rivalry was also substantially correlated with Egocentricity (r = .40, p < .001).

The primitive and neurotic clusters were positively correlated (r = .44, p <

.001). Each of these clusters was negatively correlated with the adaptive cluster (r = −.45, p < .001, and r = −.56, p < .001, respectively).

Table 5.3: Standardized factor loadings, l, of the 9-factor model (N = 736), with for each level the percentage of loadings > .30, indicated by %.

Level/ item l % level/ item l %

Generativity 44 Symbiosis 89

81. Responsibility .56* 31. Dependence .44*

82. Care .61* 32. Parent .60*

83. Authentic self-image: social .18* 33. External self-image .53*

84. Authentic norms: social .44* 34. External norms .55*

85. Integrity .18* 35. Passive need for love .58*

86. Context-related cognitions -.03* 36. Suggestive cognitions .32*

87. Respect for controversial

87.(sub-)cultures .11* 37. Detachment .12*

88. Reorganization .49* 38. Giving up .35*

89. Mourning .12* 39. Lack of basic trust .46*

Solidarity 56 Egocentricity 44

71. Living together .63* 21. Soloist .08*

72. Mate .56* 22. Servant .42*

73. Authentic self-image: relational .18* 23. Overrated self-image .23*

74. Authentic norms: relational .37* 24. Selfish norms .58*

75. Intimacy .65* 25. Mirroring .22*

76. Empathy .30* 26. Self-referring cognitions .39*

77. Respect for the controversial other .21* 27. Disclaiming .23*

78. Alliance .28* 28. Self-overestimation .30*

79. Collectivity .15* 29. Coldness .19*

Individuation 78 Fragmentation 78

61. Productivity .65* 11. Changeability .47*

62. Equal .30* 12. Frame .35*

63. Authentic self image: individual .36* 13. Vague self-image .34*

64. Authentic norms: individual .46* 14. Dichotomous norms .59*

65. Identity .72* 15. Sensation-seeking .44*

66. Self-reflection .14* 16. Non personality-related

16.cognitions .24*

67. Respect for the controversial self .34* 17. Primitive externalization .66*

68. Assertiveness .50* 18. Acting out .36*

69. Primary- process experiences .09* 19. Dissociation .18*

Rivalry 56 Lack of Structure 67

51. Status .59* 1. Bizarre behavior .35*

52. Unattainable love .24* 2. Lack of affection .16*

53. Ideal related self-image .48* 3. Lack of a self-image .15*

54. Excessive ideals .26* 4. Lack of norms .67*

55. Triumph .44* 5. Primary satisfaction of needs .49*

56. Histrionic cognitions .18* 6. Lack of psychological

66.phenomena .21*

57. Reversal .12* 7. Falsification .51*

58. Pretending .34* 8. Impulsive behavior .39*

59. Feelings of sexual insufficiency .42* 9. Disorganization .31*

Resistance 33

41. Defiance .18*

42. Oppressor .42*

43. Norm-related self-image .16*

44. Excessive norms .47*

45. Domination .01*

46. Objectifying cognitions -.05*

47. Elimination .29*

48. Defensiveness .54*

49. Moral masochism .24*

Note. The symbol “*” indicates a significant factor loading, p < .01 (method = Maximum Likelihood, robust).

5.3. Results

Table 5.4: Standardized factor loadings of the 3-factor model (N = 736).

Factor 1 Factor 2 Factor 3

Developmental level Primitive cluster Neurotic cluster Adaptive cluster

Generativity (80) .64*

Solidarity (70) .73*

Individuation (60) .81*

Rivalry (50) .24*

Resistance (40) .48*

Symbiosis (30) .68*

Egocentricity (20) .44*

Fragmentation (10) .78*

Lack of Structure (00) .62*

Note. The symbol “*” indicates a significant factor loading, p < .01 (method = Maximum Likelihood, robust).

5.3.3 Mean DP Levels Scores for Various Patient Groups

The mean DP level scores and standard deviations for the four patient groups are reported in Table 5.5. Differences between patient groups were tested with independent samples t-tests (α = .01), and differences between level scores within each patient group were tested with paired samples t-tests (α = .01).

Table 5.5: Mean scores (standard deviations) of the nine developmental levels for the four groups of patients (possible score range per level: 0-27).

Forensic Psychotherapy Outpatients Normal

inpatients inpatients controls

Developmental level N = 27 N = 468 N = 166 N = 102

Generativity (80) 0.04(0.19)a 0.31(0.61)a 2.80(1.32)b 1.61(1.51)c

Solidarity (70) 0.22(0.51)a 1.63(1.71)q 2.82(2.00)k 3.76(2.71)f

Individuation (60) 1.11(1.09)g 4.17(2.17)h 5.95(2.76)i 7.35(2.73)j

Rivalry (50) 2.59(1.93)kq 3.19(2.52)k 2.88(2.48)k 2.07(1.80)c

Resistance (40) 3.96(2.55)ks 7.40(2.98)m 6.07(2.60)i 4.06(2.52)f k

Symbiosis (30) 4.81(2.75)s 8.60(3.79)o 6.48(3.61)i 2.11(1.77)c

Egocentricity (20) 2.26(2.33)gq 0.84(2.14)rt 0.49(1.27)br 0.33(1.56)r

Fragmentation (10) 2.74(1.72)kq 1.71(2.06)q 0.49(1.12)br 0.11(0.40)r

Lack of Structure (00) 5.30(1.68)s 0.88(1.46)t 0.36(1.05)r 0.07(0.38)r

Note. Means not sharing similar subscripts within each row and column differ at p < .01.

Table 5.5 shows that the patient groups clearly have distinct score patterns, which show a peak on a few adjoining levels. For instance, for the group of psychother- apeutic inpatients the “peak levels” are Symbiosis and Resistance (the means on both the more maladaptive levels and the more adaptive levels for this group are significantly lower). Furthermore, these patients scored significantly higher on Symbiosis and Resistance than the other patient groups, which scored highly on either more maladaptive or more adaptive levels. These score patterns of the dif- ferent groups seem to justify summarizing a patient’s DP into one score indicating the global level of psychosocial functioning. CA is a technique that derives such a scale score from a patient’s DP.

5.3.4 Correspondence Analysis of the DP Level Scores

To further investigate whether there is a bipolar dimension underlying the different levels, on which both the levels and the patients can be ordered in terms of the degree of (mal-) adaptive functioning, CA was performed on the level scores of the 736 patients. Recall that CA results in scores for both levels and patients.

The CA solution for the level scores in two dimensions is displayed in Figure 5.1.

This solution showed that the level scores lie in an arch-shaped pattern, which is marked by the dashed parabola. The CA solution accounted for 55.0 % of the total inertia (37.3 % and 17.7 % for Dimensions 1 and 2, respectively).

The apparent arch-effect in Figure 5.1 was interpreted as support for the theory that an underlying maladaptivity-adaptivity scale exists. The first dimension reflected the theorized order of the levels. The second dimension contrasted the extremes of this scale with the midpoint.

Cross-validation with 10 stratified random splits showed that the scale values presented in Figure 5.1 are sufficiently stable; the correlation between the scale values of both subsamples for each split ranged from .969 to .999 (M = .988) for the first dimension, and from .898 to .990 (M = .958) for the second dimension.

Dimension 1 contrasted the maladaptive levels Lack of Structure, Fragmen- tation, and Egocentricity (with respectively explained inertias .49, .50, and .18) with the adaptive levels Lack of Individuation, Solidarity, and Generativity (with respectively explained inertias .67, .55, and .40). The levels Symbiosis, Resis- tance, and Rivalry were located around the midpoint of the first dimension (with respectively explained inertias .19, .02, and .003).

5.3. Results

3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 -0.50 -1.00 -1.50 -2.00 -2.50 -3.00 -3.50 4.00 3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 -0.50 -1.00

70

30 10

0

80

60

50 40 20

CA dimension 1

CA dimension 2

Figure 5.1: Developmental level scale scores in 2 dimensions resulting from CA of the profile scores. Numbers refer to levels (see Appendices 5.A and 5.B of this chapter). On either side of each scale score, the respectively lowest and highest estimate of the first dimension score resulting from cross-validation are depicted.

The clustering of the developmental levels into three ordered clusters is apparent in Figure 5.1, which was also supported by the 3-factor model discussed in the above. In three out of all 10 cross-validations the positions of Fragmentation and Egocentricity on the first dimension interchanged, which also applied to Resistance and Rivalry.

Note that, in contrast to unipolar scales, bipolar scales also include items on the midpoint of the scale, which necessarily have low correlations with the underlying dimension. Therefore, percentage-explained inertia is not sufficient to judge the quality of these types of scales (cf. Ter Braak & Verdonschot, 1995, p. 274). Alternative criteria, for instance, validation by an external criterion, are often used. For this purpose, we further analyzed the patients’ scale values, which will be discussed in the following.

Like the levels, for each patient the score on the first CA dimension is inter- preted as a score on the maladaptivity-adaptivity scale. ANOVA of the CA scale

scores between the different patient groups revealed significant differences between the groups and a large effect size (scale values: F (3, 732) = 245.10, p < .0001, η²= .50). Paired comparisons among subsequent groups with (Bonferroni corrected) t-tests showed, that each group is significantly different from the groups in closest proximity (forensic inpatients (M = -1.11) against psychotherapeutic inpatients (M = -0.13), p < .0001; psychotherapeutic inpatients against outpatients (M = 0.21), p < .0001; outpatients against normal controls (M = 0.69), p < .0001).

Note that, given the psychiatric complaints in the groups of in- and outpatients, we expected the first to be more maladaptive. These results strongly confirmed the hypothesis that patient groups can be significantly distinguished and ordered by their scores on the bipolar scale underlying the DP. Furthermore, we found a significant correlation of the bipolar scale underlying the Developmental Profile with age (r = .36, p < .001, N = 706), indicating that developmental differences in adults as measured by the DP are partly a natural result of aging.

5.4 Discussion

This study investigated the internal consistency and construct validity of the De- velopmental Profile. The three clusters (adaptive, neurotic and primitive) all had a good reliability, although for separate levels the reliability appeared not to be satisfactory.

Confirmatory factor analysis showed an overall good fit, thereby providing a justification for the organization of item scores into level scores. Furthermore, the CFA results justify the various levels as interrelated subscales, which can be aggregated into three clusters, thus supporting constructs of a primitive maladap- tive cluster (Lack of Structure, Fragmentation, and Egocentricity), a neurotic maladaptive cluster (Symbiosis, Resistance, and Rivalry) and an adaptive cluster (Individuation, Solidarity, and Generativity).

Taken together these results can be considered in line with psychoanalytic developmental theory. This assumes levels of personality organization and distin- guishes healthy personality, neurotic level and borderline level (see for instance, PDM, 2006). Apparently, at the differentiated levels within a cluster it is still more difficult to obtain sufficient reliable scores. This may reflect a measure prob- lem, which could, for instance, be solved by a better operationalization of items.

Further studies on the instrument have to be done with regard to this matter.

Correspondence analysis showed a bipolar scale underlying the Developmen- tal Profile ranging from maladaptive to adaptive psychosocial functioning that

5.4. Discussion

significantly distinguished all different patient groups. There was a significant relationship between the scores on this scale and age. Cross-validations, showed good stability of the CA scale values.

These results confirm the usefulness to distinguish adaptive levels. It also indicates its usefulness to operationalize adaptivity not only by the absence of pathology, but also by the presence of specific adaptive characteristics. In addi- tion, it suggests that personality in general grows to further maturity during life in adulthood. This is concordant with the personality theory of Erikson which was taken as one of the underlying concepts for the DP. It is also in line with in literature reported findings with regard to the development of defense mechanisms (Vaillant, 1993, p. 134ff).

A subgroup of 27 patients (3.5 % of the total sample) was excluded from the CFA and CA (but not from the reliability analysis). These were characterized by a combination of high scores on both level 20 and level 50. This group was identified as an outlying group in the CA analysis, since in the two dimensional solution, the group formed a remote cluster. Inspection of the profile scores of this group showed that all patients had a score of 6 or higher on Egocentricity (level 20). This group had such an outlying score pattern that in the EQS analysis it caused a failure to converge. Thus, although the reported reliabilities of the level and cluster scores do apply to this group, the position of this group in the hierarchy of the DP (in terms of adaptivity and maladaptivity) is ambiguous.

Theoretically this is an interesting finding. Apparently, the order of the levels that follows from the score patterns of all other patients does not apply to this group. That is, this group combines high scores on Egocentricity with high scores on Rivalry (level 50). An explanation for the relatively high scores on Rivalry for this maladaptive group may be found in the concept of the so-called oblivious narcissisism as described by Gabbard (2000). Persons with this type of narcissism do not only believe they are superior but are also inclined to show this in strong competitive behavior. In the DP this is represented in the patterns of rivalry (see Appendices B and C), thus leading to a joint appearance of these two levels.

5.4.1 Limitations and strengths

The study has a number of limitations. First of all, we were not able to mea- sure the interrater reliability of the whole sample, therefore the exact degree of measurement error due to raters in this study is unknown. However, in earlier publications, interrater reliability was reported (Van et al., 2000, 2005) for sub-

samples that are included in this study. The most recent publication on this topic is Polak, Abraham, Van, and Ingenhoven (2010). Also all raters were well trained and their competency was evaluated by one of the registrated DP supervisors before participating in the current study. In addition, it was always possible for raters to discuss questions or doubts with the one of the DP supervisors, includ- ing two of the authors (Robert E. Abraham and Henricus L. Van). We aimed to eliminate rater variability as much as possible by using consensus scores.

Second, we did not compare the DP scores with regular diagnoses of the pa- tient as made during their intake procedure at the participating institutes and departments. However, there was a considerable variability in diagnostic proce- dures across institutes and departments, in particular with regards to personality pathology which was by not measured with other instruments in a comparable way. Therefore we thought it not to be useful to relate DP scores with the clinical Axis II disorders.

Third, the unsatisfactory values of Cronbach’s α coefficient for most of the separate levels of the DP imply that these levels (e.g., Generativity) should be interpreted with caution when using the DP in individual patients in clinical practice.

We used Cronbach’s α coefficient for the reliability analysis, since it is the most common statistical analysis for that purpose. However, there is discussion on which values of alpha are desirable in the field of personality measurement. For instance, Boyle, Stankov, and Cattell (1995, p. 436) argue that low to moderate item homogeneity is preferable, so that each item contributes to the breadth of measurement of a given scale. The authors refer to Kline (1986) and Cattell (1982), who suggested reliabilities in the range 0.3 to 0.7 on the basis that, to obtain maximum validity, items do not need to correlate highly with each other, but rather with an external criterion. Furthermore, the construction of the DP as a matrix that fixes the number and nature of the items of each scale, limits the possibility to redefine problematic items that were identified by the CFA.

Moreover, this prohibits the general approach to improve the reliability of a scale, which is to include more items (cf. Streiner, 2003).

A fourth limitation of this study is that, other psychometric properties, such as test-retest reliability, were not measured. Thus the stability of the DP scores over time is not known. As most of the DP interviews were administrated before the start of therapy in patients with a concurrent axis I disorder as well, one might question to which extent this has influenced the scores. On the other hand, in earlier studies with the DP (Van et al., 2008, 2009) an influence of severity

5.4. Discussion

of concurrent axis I symptoms on the DP scores could be demonstrated neither.

Apparently, the interview protocol that provides specific instruction on how to minimize potential influence of symptoms on the answers has been adequate in this respect.

A strength of the study is the very large number of patients included. It enables us to perform in a more reliable way statistical analysis that in smaller samples would not have been possible, such as a reliable use of factor analysis and the cross-validation of the correspondence analysis.

Second, the DP is a theory-driven instrument that makes use of concepts closely related to the way clinicians think about patients. As such it is an attempt to bridge the gap between personality characteristics that may arise during a psychotherapeutic process and need to be taken into account, and research data.

An example is provided in Appendix 5.B of this chapter that shows how the same behavior of a patient could be placed on various levels of the DP, pending the context.

In conclusion, we believe the DP is a promising instrument. Psychometric studies confirm the developmental perspective at personality, although there is ongoing research to improve the internal consistency reliability of some of the developmental levels that make up the DP. The DP summarizes in one model a broad array of psychodynamic concepts that have been proven useful in daily psychotherapeutic practice. Further research in concordant and prognostic va- lidity is required and may reveal whether the application of the DP in clinical practice can improve selection of psychotherapeutic strategies and interventions in an individual patient and enhance treatment outcome.

5.A The Developmental Profile Matrix

AppendixA TheDevelopmentalProfileMatrix Problemsolving SocialattitudesObjectrelationshipsSelf-imagesNormsNeedsCognitionsThoughts/feelingsActionsMiscellaneous themes 80:Generativity81.Responsibility82.Care83.Authentic self-imagesocial84.Authentic normssocial85.Integrity86.Context-related cognitions87.Respectfor controversial (sub)cultures 88.Reorganization89.Mourning 70:Solidarity71.Living together72.Mate73.Authentic74.Authentic norms relational

75.Intimacy76.Empathy77.Respectforthe controversialother78.Alliance79.Collectivity 60:Individuation61.Productivity62.Equal63.Authentic self-image individual

64.Authentic norms individual

65.Identity66.Self-reflection67.Respectforthe controversialself68.Assertiveness69.Primary-process experiences 50:Rivalry51.Status52.Unattain-able love53.Ideal-related self-image54.Excessiveideals55.Triumph56.Histrionic cognitions57.Reversal58.Pretending59.Feelingsof sexualinsufficiency 40:Resistance41.Defiance42.Oppressor43.Norm-related self-image44.Excessivenorms45.Domination46.Objectifying cognitions47.Elimination48.Defensiveness49.Moral masochism 30:Symbiosis31.Dependence32.Parent33.External self-image34.Externalnorms35.Passive needforlove36.Suggestive cognitions37.Detachment38.Givingup39.Lackofbasictrust 20:Egocentricity21.Soloist22.Servant23.Overrated self-image24.Selfish norms25.Mirroring26.Self-referring cognitions27.Disclaiming28.Self- overestimation29.Coldness 10:Fragmentation11.Changeability12.Frame13.Vague self-image14.Dichotomous norms15.Sensation seeking16.Nonpersonality relatedcognitions17.Primitive externalization18.Actingout19.Dissociation 00:Lackof Structure1.Bizarre behaviour2.Lackof affectivity3.Lackofa self-image4.Lackofnorms5.Primary satisfaction ofneeds 6.Lackof psychological phenomena 7.Falsification8.Impulsive behaviour9.Disorganization Note.Theitemscores(range,03)atonelevelaresummedtoobtainthelevelscore.

The developmental profile

5.B. Description of the Levels of the Developmental Profile

5.B Description of the Levels of the Developmental Profile

AppendixB DescriptionofthelevelsoftheDevelopmentalProfile sngisertneitapehT:ruoivahebfosgninaemelpitlumehtfoelpmaxEnoitpircseDlevellatnempoleveD... evitagenynamootsahfoegrahcniebotsawehtcejorpehtesuaceBlarenegniyteicosrofrosrehtoroferacoT)08(ytivitareneG consequencesfortheenvironment ssimsidylgnorwsawsrekrow-ocsihfoenotahtsleefehesuaceBspihsnoitaleryrotcafsitasyllautumdnagnitsalregnolhsilbatseoT)07(ytiradiloSed Individuation(60)Torealizeone’sownaims,takingintoaccountexistingpossibilities, aswellastheinterestsofothers.Becauseheisnotgiventheresourcesnecessarytocarryouthisworkproperly ,namowronamtludanasaseitilauqnwos’enotuobaytirucesnI)05(yrlaviR withastrongneedtoproveoneselfsociallyBecausehehas‘‘lostface’’afternotbeingselectedforahigherposition eugaellocaybdelttilebsleefehesuaceBymonotuafostcilfnocybdetsefinammodeerfrennifokcaL)04(ecnatsiseR deritersahohw,ssobremrofsihfotroppusehtsessimehesuaceBsrehtomorftnednepedgninoitcnuF)03(sisoibmyS Egocentricity(20)Narcissisticfunctioningwithoverestimatedself-imageandusing relationshipsBecause‘‘thosecluelessidiotsatwork(management)’’refusedtoaccepthis plans Fragmentation(10)Notabletointegrateexperiencesmanifestationsaschangeability, splitting,orprimitiveexternalisation(17)‘‘Fornoreason’’thesamedayhistherapistannounceshisvacation LackofStructure(00)Lackofbasicabilitiessuchexpressingaffectsordisturbancesin realitytestingInanger,andwithoutstoppingtothink,whenhisbossdoesnotagreetohis takingthefollowingdayoff Note.Inthefinalcolumn,anexampleisgiventhatshowshowthesamebehaviorofapatient(here,resigning)couldbeplacedonvariouslevelsoftheDP,pendingthecontext.