Running Head: ASSESSING CHANGES OF PARAMETERS

Assessing changes in

the parameters of common factor models with respect to metric moderator variables

Rajan L. Felisco

Leiden University

Abstract

A multi-groups factor model is often used to study moderator effects. In many cases, the moderator variable is of a metric nature rather than nominal. In this paper, we propose a varying-coefficient factor model. With the application of kernel smoothing, a number of varying-coefficient factor models exist, dependent on the smoothing parameter. The aforementioned factor models were fitted on a set of personality data. Through model selection, a factor model was chosen that best depicts the trends of model parameters of interest, with accompanying confidence bands, across values of a metric moderator variable.

Keywords: moderator, kernel smoothing, multiple group analysis, varying-coefficient model

Acknowledgements

First and foremost, my infinite gratitude to God, the Almighty, for the grit, tenacity and fortitude to finalize this paper. I would like to convey my earnest gratefulness to my advisor Dr.

Henk Kelderman from the Institute of Psychology, section Methodology and Statistics of Leiden

University, for the continuous support and constructive feedback on my Master thesis and for his

zeal and commitment for my research. Furthermore, I would like to thank my second supervisor

Dr. Elise Dusseldorp for her suggestions to further improve my thesis. In addition, I would like

to express my immense appreciation to Ms. Marian de Joode and her colleagues for the provision

of the personality test necessary to interpret the results from the statistical analyses. Lastly, I am

thankful for my family, fellow students and friends for their moral support during the course of

my study.

Table of Contents

Abstract ...2

Acknowledgements ...3

1 Assessing changes in the parameters of common factor models with respect to metric moderator variables ...6

2 Models and Algorithms...10

2.1 A factor model for one value of the moderator variable ...10

2.2 The varying-coefficient factor model ...11

3 Empirical Example...15

3.1 Demographics ...15

3.2 Items ...15

3.3 Statistical Analysis ...17

3.4 Results ...18

3.4.1 Neuroticism: Transformed parameters of the multi-groups factor model ...20

3.4.2 Selection of the best-fitting varying-coefficient factor model ...21

3.4.3 Neuroticism: Transformed parameters of the best-fitting factor model ...22

4 Discussion ...29

5 Conclusion ...30

References ...31

Appendix A Big Five: Facets and items of the remaining factors ...38

Appendix B Confidence bands: Repeated half-sampling bootstrap ...41

Appendix C Transformation of parameters: Deviation contrasts ...43

Appendix D Extraversion: Transformed parameters of the best-fitting factor model ...46

Appendix E Agreeableness: Transformed parameters of the best-fitting factor model ...49

Appendix F Conscientiousness: Transformed parameters of the best-fitting factor model ...52

Appendix G Openness: Transformed parameters of the best-fitting factor model ...55

Appendix H Neuroticism: Table of transformed parameters of the multi-groups factor model ...58

Appendix I Statistical code: Data set and preliminary analysis ...61

Appendix J Statistical code: Neuroticism ...62

Appendix K Statistical code: Extraversion ...80

Appendix L Statistical code: Agreeableness ...97

Appendix M Statistical code: Conscientiousness ...114

Appendix N Statistical code: Openness ...132

1 Assessing changes in the parameters of common factor models with respect to metric moderator variables

Prior to the concept of factor analysis, Galton (Galton, 1886) and Pearson (Pearson, 1896), who introduced linear regression and correlation coefficient respectively, provided the elements necessary to further its notion. Factor analysis is a statistical technique used to represent observed variables through covert constructs called latent variables (Thurstone, 1931).

A distinction is made between exploratory factor analysis and confirmatory factor analysis, in which the former deals with data reduction through factors while the latter allows verifying the model structure for a set of variables through hypothesis testing. Factor analysis has become one of the most extensively employed, quantitative psychological methods during the past century (Brown, 2006), initially developed by psychologists. It was introduced by Spearman in the early twentieth century with his single-factor theory on general intelligence (Spearman, 1904). The common factor model, introduced by Garnett (Garnett, 1919) expanding Spearman’s model and fine-tuned by Thurstone (1935), outlines the relationships between indicators presumed to signify at least a single latent variable (Thurstone, 1947). With the development of factor analysis, Burt (1940), Thurstone (1947) and Thomson (1951) proposed and refined an algorithm to compute the first parameters (factor loadings) of a factor model. Subsequently, factor analysis became a formal statistical procedure with its integration with maximum-likelihood estimation and likelihood-ratio testing (Lawley & Maxwell, 1962).

Aside from data reduction and psychometric evaluation, another extension of factor

analysis is simultaneous parameter estimation in different subpopulations, thereby allowing

comparisons between groups (Sörbom, 1974). Jöreskog pioneered the framework of multi-

groups factor analysis (Jöreskog, 1971). In multi-groups factor analysis, the groups form a

discrete variable which moderates the relationships in a factor analysis (Baron & Kenny, 1986).

The parameters of a factor model may or may not be invariant across values of a moderator variable. The invariance of parameters involving nominal moderator variables is assessed through multi-groups factor analysis by setting certain parameters equal across groups (Bauer &

Hussong, 2009). Meanwhile, factor-analytic techniques addressing metric (i.e. interval and ratio) moderator variables are lacking. For instance, we may want to know how the regression of a test on an intelligence factor changes with age or how the difficulty of a vocabulary item changes through time. Metric variables can be divided into many-category metric and continuous metric variables (Kumar, 2010, p. 73). To date, for factor analysis involving many-category and continuous metric moderator variables, the customary procedure is discretization of the moderator variable, succeeded by multi-groups factor analysis (Baron & Kenny, 1986).

If the parameters of a measurement model are not invariant across values of a moderator

variable, one or more observed indicators of latent variable(s) may be biased, which may make

the comparisons of factor scores invalid. Meanwhile, if parameters of a population model are

not invariant across values of a moderator variable, population properties may change. For

example, factors may merge or diverge while factor means or factor variances may decrease over

age. The former may be of methodological interest and the latter of substantive interest to the

researcher. If moderator variables are categorical with limited number of categories, these

phenomena can be studied with multi-groups factor models. However, when dealing with many-

category or continuous metric moderator variables, multi-groups factor models could prove

problematic due to information loss and loss of power due to discretization of continuous metric

variables (Bagozzi, Baumgartner & Yi, 1992). Furthermore, the variance of parameter estimates

might become unduly large because of information loss. Accordingly, parameter estimates may be better when the metric nature of continuous variables is retained.

The literature presents a number of statistical methods that could tackle factor models with metric moderator variables. A three-way factor analysis, also known as parallel factor analysis or PARAFAC, might be able to portray the trends of the parameters of a factor model. It is a straightforward generalization of the two-dimensional principal component analysis (PCA) (Hopke et al., 1998). With the moderator variable being the third ‘way’ and the two other variables being the observed variable and the latent variable, the PARAFAC model is of an exploratory nature (Harshman & Lundy, 1994). While the PARAFAC model could be accommodated to perform exploratory factor analysis, the next set of models may be easily modified to execute confirmatory factor analysis. Varying-coefficient models (Hastie &

Tibshirani, 1993) could provide the elements to depict the potentially dynamic nature of parameters from a factor model against the values of a moderator variable. They take on diverse forms, with the underlying principle being the modifying effect of a moderator Z on the relationship between an explanatory variable X and a response variable Y (Park, Mammen, Lee &

Lee, 2013). Given the focus on the confirmatory aspect of factor analysis in this paper, varying- coefficient models will be the statistical method of choice.

Varying-coefficient models were developed to replace a number of linear or parametric functions by smooth nonparametric functions (Hastie & Tibshirani, 1993). In the present study, we consider the simplest case of a varying-coefficient model (i.e. the Gaussian model), that has the form

Y = X ₁ β 1 (Z ₁ ) + … + X p β _p (Z _p ) + ϵ, (1)

where E(ϵ) = 0, var(ϵ) = σ². Model (1) specifies that Z 1 , … , Z p modify the coefficients of the X 1 , X ₂ , …, X p through the (unspecified) functions β ₁ ( ), … , β _p ( ) (Hastie & Tibshirani, 1993). The dependence of β( ) on Z denotes a special type of interaction between each Z and X. Therefore, Z could be considered the moderator variable. In Model (1), we kept the original notation by Hastie and Tibshirani (1993). Note, however, that the total number of moderator variables (p) does not need to be equal to the total number of predictors (p). Given its flexibility, Hastie and Tibshirani (1993) showed several examples of a varying-coefficient model. Varying-coefficient models have emerged largely due to their practical use rather than as a purely mathematical extension (Fan & Zhang, 2008). Applications are found in a number of statistical contexts, such as local regression (Fan & Zhang, 1999), generalized linear models (Cai, Fan & Li, 2000) non- linear time series (Cai, Fan & Yao, 2000) and longitudinal (Fan & Zhang, 2000) and functional data analysis (Ramsay & Silverman, 1997). Being depicted as smooth functions, the smoothness is dependent on the metric properties of the moderator variable. Except for a smoothing parameter, the estimation of the trend of the parameters is entirely data-driven. Parameter estimations take place through a number of techniques, namely multiple linear regression, regression splines and general smoothers (Hastie & Tibshirani, 1993).

The present study aims at proposing a new model for metric moderator analysis in the

framework of confirmatory factor analysis by combining the varying-coefficient model with a

factor model, termed the varying-coefficient factor model. The chapter Models and Algorithms

shows the fundamentals of this new model. The model yields smooth functions of selected

model parameters against the values of a metric moderator variable of interest. The model is

illustrated on a set of personality data, using the variable year as the moderator variable of

interest.

2 Models and Algorithms

This chapter discusses formal theories underlying varying-coefficient factor analysis.

Section 2.1 provides the elements of a single-group factor model. Section 2.2 presents an extension of the multi-groups factor model, the so-called varying-coefficient factor model.

2.1 A factor model for one value of a moderator variable

Let 𝒥 be a set of 𝐽 indices 𝑗 with random values 𝑋 _𝑗 for a set ℛ of 𝑅 latent factors 𝑟 with random values 𝐹 _𝑟 and let 𝑍 denote the moderator variable. The measurement model of the common factor model is:

𝑋 _𝑗 (𝑍) = 𝜈 _𝑗 (𝑍) + ∑ _{𝑟 ϵ ℛ} 𝜆 _𝑗𝑟 (𝑍) 𝐹 _𝑟 (𝑍) + 𝐸 _𝑗 (𝑍), 𝑗 𝜖 𝒥 (2) where 𝜈 _𝑗 is the intercept, 𝜆 _𝑗𝑟 is the loading of indicator 𝑗 on factor 𝑟 and 𝐸 _𝑗 the residual of an item. In matrix notation, we have

𝑿(𝑍) = 𝝂(𝑍) + 𝜦(𝑍)𝑭(𝑍) + 𝑬(𝑍) (3)

where 𝑿 = (𝑋 ₁ , … , 𝑋 _𝑗 ) ^𝑇 , 𝑬 = (𝐸 ₁ , … , 𝐸 _𝑗 ) ^𝑇 , and 𝑭 = (𝐹 ₁ , … , 𝐹 _𝑗 ) ^𝑇 . The 𝝂 _𝐽×1 intercept vector and the 𝜦 _𝐽×𝑅 loading matrix can be restricted to specify a particular factor model.

The random variables in the factor model have mean vectors

𝜀(𝑭 | 𝑍) = 𝛼(𝑍), 𝜀(𝑬 | 𝑍) = 0 (4)

and variance matrices

𝑣𝑎𝑟(𝑭 | 𝑍) = 𝛷(𝑍), 𝑣𝑎𝑟(𝑬 | 𝑍) = 𝛹(𝑍), (5) where all the off-diagonal elements of Ψ, the error covariances, are usually all set to zero.

If 𝛩(𝑍) denotes the vector of all model parameters {𝝂(𝑍), 𝜦(𝑍), 𝜶(𝑍), 𝜱(𝑍), 𝜳(𝑍)} and 𝒙 = (𝑥 ₁ , … , 𝑥 _𝑗 ) ^𝑇 is a realization of 𝑿, the parameter estimates are

𝜣 ̂(𝑍) = arg min _𝛩(𝑍) ∑ _{𝑖 ϵ 𝑁} ℓ _𝑖 (𝛩(𝑍); 𝑥 _𝑖 ), (6)

where ℓ _𝑖 is the -2 log-likelihood of the model for an independent sample 𝑖 ϵ 𝑁. To make a factor model identifiable in the analysis (Long, 1983), the metric of a specific factor was set by fixing the factor loading of an item with the highest communality to 1 and fixing its intercept to 0, given a set of items measuring a single factor. This item with the highest communality given a set of items will be further referred as the reference item. For a single factor, the communality can be interpreted as the proportion of variance of an item explained by that single factor (Thompson, 2004, p.148). Therefore, the higher the communality, the better the item as an indicator of a single factor.

2.2 The varying-coefficient factor model

Kernel smoothing was the statistical technique used to depict the functional form of the

trends of the parameters across values of the moderator variable. The regression function, which

illustrates the trend, is estimated by fitting a statistical model separately for each target point 𝑧 ₀

(Hastie, Tibshirani & Friedman, 2009). These target points 𝑧 ₀ are values of the moderator

variable and those that lie between them, with the number of target points chosen as arbitrary. A

factor model is fitted to observations close to the target point 𝑧 ₀ . The localization takes place

through a weighting function 𝐾 _ℎ (𝑧 ₀ , 𝑧 _𝑖 ), assigning a weight to 𝑧 _𝑖 based on its distance from 𝑧 ₀

(Hastie et al., 2009). The subscript i denotes the index of points in which the kernel weight is to

be computed based on its distance from 𝑧 ₀ . The kernels 𝐾 _ℎ are typically indexed by a smoothing

parameter h that dictates the width of the neighborhood of observations to be taken into account

relative to 𝑧 ₀ , so-called bandwidth. Weights can be assigned that gradually diminish with

distance from the target point. Here, a Gaussian kernel function was used. The Gaussian kernel

function (Hastie et al., 2009) is formulated as

𝐾 _ℎ (𝑧 _𝑜 , 𝑧 _𝑖 ) = 𝜙 ( ^{|𝑧 𝑖 – 𝑧} _ℎ ^{0 |} ) = 𝑒 −||𝑧𝑖−𝑧0||²

2ℎ² , (7)

with || ∙ || being the Euclidean norm, h being the bandwidth used and 𝑧 _𝑖 being a point for which a kernel weight is calculated.

In the Gaussian kernel, the bandwidth is defined as the standard deviation of the kernel.

Note that in the discrete-nominal case, the bandwidth corresponding to a nominal multi-groups factor model is close to zero because only the observations belonging to a specific value of a moderator variable are analyzed. For the objective of this paper, we assess various bandwidths to find the best-fitting factor model.

We may want to know to what extent one or more parameters of the factor model depend on the value of a moderator variable 𝑧. For a target point 𝑧 ₀ , this model has maximum- likelihood estimates of the parameters:

𝜣 ̂(𝑧 ₀ ) = arg min _{𝛩(𝑧 0 )} ∑ _{𝑖 ϵ 𝑁} 𝐾 _ℎ (𝑧 ₀ , 𝑧 _𝑖 )ℓ _𝑖 (𝛩(𝑧 ₀ ); 𝑥 _𝑖 ), (8) with the Gaussian kernel weights 𝐾 _ℎ (𝑧 ₀ , 𝑧 _𝑖 ).

Aside from the kernel weights, we also consider sampling weights because, in our example, sample sizes differ across values of the moderator variable. Kernel weights can be combined with sampling weights by multiplying them before calculating the weighted mean (Hastie et al., 2009, p. 194). With the combination of kernel weights and sampling weights, Formula 8 is modified as follows:

𝜣 ̂(𝑧 ₀ ) = arg min _{𝛩(𝑧 0 )} ∑ ^{𝐾 ℎ (𝑧 0 , 𝑧 𝑖 ) 𝑤 𝑖}

𝑥̅(𝑧 ₀ )

𝑖 ϵ 𝑁 ℓ _𝑖 (𝛩(𝑧 ₀ ); 𝑥 _𝑖 ). (9)

where

𝑥̅(𝑧 ₀ ) = ^{∑ 𝑖 ϵ 𝑁 𝐾} _𝑁(𝑧 ^{ℎ (𝑧 0 , 𝑧 𝑖 ) 𝑤 𝑖}

0 ) , (10)

with 𝑤 _𝑖 being the sampling weights and 𝑁(𝑧 ₀ ) the number of observations in a certain target point.

For simplicity, we merely discuss a single moderator variable and a single factor. To find the best-fitting factor model, the data is partitioned into a training set and a test set. Studies (Molinaro, Simon & Pfeiffer, 2005; Steyerberg et al., 2001; Kohavi, 1995; Dobbin & Simon, 2011) indicate that the optimal ratios between the training set and test set range from 60%/40%

to 80%/20%, with the latter also known as the Pareto rule. In our study, the training set and test set contained 70% and 30% of the total observations respectively. Several varying-coefficient factor models with certain bandwidths will have their parameters estimated in the training set.

Varying-coefficient factor models were chosen to have 301 target points 𝑧 ₀ , using the Gaussian kernel with a certain bandwidth between 0.33 and 5. Each target point represents a set of weighted observations being subjected to factor analysis, depending on the bandwidth of the factor model. To make the model identifiable, section 2.1 described the constraints on the factor loading and intercept of an item for a single factor. In this paper, the parameters of the measurement model for a single-factor model consist of thirteen factor loadings (𝜆 _𝑗 ) to be estimated and one constrained factor loading, fourteen residual variances of the items (𝜓 _𝑗 ) and thirteen item intercepts (𝜈 _𝑗 ) to be estimated and one constrained intercept. Furthermore, the factor mean (𝛼) and factor variance (𝜑), which make up the structural portion of the model, are freely estimated. Given the presence of missing values in the data set, model parameters were estimated through Full Information Maximum Likelihood (FIML), with the raw available data being analyzed.

While the training set provides the parameter estimates for different bandwidths, the test

set aims to determine for which bandwidth the varying-coefficient factor model yields the best

fit. Each varying-coefficient factor model yields parameter estimates for 301 target points z 0 on

the moderator variable. In the following empirical example, independent test data are available

for 25 of those target points. We assess the fit of a varying-coefficient factor model by obtaining

the fit of its 25-group factor model with fixed parameters corresponding to the points z ₀ estimated

in the training set.

3 Empirical Example

This chapter describes the characteristics of the Big Five data set from Smits et al. (2011) and the results from data analysis. The methodological question highlights the comparison between the multi-groups factor model and the varying-coefficient factor model, with the latter being a potentially more superior model than the former. Meanwhile, the substantive question accentuates the interpretation of the trends of the parameters of interest. Section 3.1 specifies the demographic aspect of the data and the variables under study, including the moderator variable year. Section 3.2 puts forward the attributes of the personality items, for which the data will be subjected to factor analysis. Section 3.3 elaborates on the statistical analyses. Section 3.4 presents the results for this paper.

3.1 Demographics

The data consists of 74 variables including demographic characteristics and item scores of 8,954 Psychology freshmen from the University of Amsterdam. For each person, there are values on 70 items, gender, age, year of administration (the moderator) and an indicator for the presence of missing data. The data consisted of 69% females (M age = 19.99, age range = 18 – 25) and 31% males (M age = 20.59, age range = 18 – 25). The data were collected from 1982 till 2007, to establish cohort differences in a number of personality factors. Data from 1987 were discarded by Smits et al. (2011) due to missingness of gender and/or age of the respondents. The tests where the data came from were administered during sessions of the so-called ‘Testweek’ at the beginning of an academic year. Figure 1 depicts the sample size per year for the entire data set, with 1982 and 1992 having the lowest and highest number of observations respectively.

3.2 Items

Personality is presumed to be represented by five factors, namely Extraversion, Agreeableness, Conscientiousness, Neuroticism and Openness to experience, termed the Big Five (Tupes & Christal, 1961; McCrae & John, 1992; Cattell, 1996). Table A1 in Appendix A shows the subdivisions of the Big Five, which originated from a renowned personality test known as the Revised Neuroticism-Extraversion-Openness Personality Inventory (NEO-PI-R) (Costa & McCrae, 1992).

Figure 1

A Dutch version of the NEO-PI-R, the 5 Personality Factor Test or 5PFT (Elshout & Akkerman, 1973), was the instrument used to acquire data on the Big Five. It is composed of 14 items for each personality factor, summing up to 70 items. Each item signifies a concise description of behavior. An item is rated from 1 to 7, denoting the degree of applicability of the descriptive to an individual. Nearly 15% of the sample noted at least one missing value on their item scores.

For the illustration of the factor models, we only discuss the factor Neuroticism. It

relates to emotional instability and susceptibility to stress (Caspi, Roberts & Shiner, 2005). It is

subcategorized in six facets (Lord, 2007). Table 1 depicts the fourteen items of Neuroticism and

their characteristics. Due to copyright issues, the exact wording of the items will not be

revealed. Instead, keywords with some definitions of the items are provided. The tables of the items for the remaining factors are tabulated in Appendix A2 (Extraversion), A3 (Agreeableness), A4 (Conscientiousness) and A5 (Openness to Experience).

Table 1

Neuroticism items with some attributes

Item Label Characteristics

1 Temperamental Intensely emotional

2 Tranquil* Showing composure

3 Uptight Tense, often exhausted

4 High-strung Nervous, fidgety

5 Contented* Inner calm

6 Carefree* Undisturbed by circumstances

7 Hypochondriac Health-conscious

8 Distraught Stressed during crisis

9 Upset Confused, hopeless

10 Volatile Unstable emotions leading to impulsiveness

11 Placid* Emotionally stable

12 Distressed Tends to worry excessively

13 Desolate Feels lonely, unfavorable social relationships

14 Sullen Melancholic, depressed

* Reversely worded items; item 9 is the reference item.

Reversely worded items, signified by asterisks, indicate opposing qualities relative to the factor.

Four reversely worded items were noted for the factor Neuroticism. Reversely worded items were recoded such that a high score implies that an item is less applicable to an individual.

3.3 Statistical Analysis

Before we provide the trends and the confidence bands of the parameters from the multi-

groups factor model and the varying-coefficient factor model for the factor Neuroticism, we

discuss the statistical analyses first. The data analysis was done using the statistical

programming environment R (R Core Team, 2014). The packages used were caret for the data

partitioning into training set and test set (Kuhn, 2008), kernlab for the kernel smoothing and

generation of kernel weights (Karatzoglou, Smola, Hornik & Zeileis, 2004), OpenMx for the

confirmatory factor analysis (Boker et al., 2011), sampling for the confidence bands (Tillé &

Matei, 2009) and ggplot2 for the plots (Wickham, 2010). Confidence bands were calculated using the repeated half-sampling bootstrap (Saigo, Shao & Sitter, 2001) described in Appendix B. For a sensible interpretation, the original parameter estimates were transformed using deviation contrasts, with the formulae for the reparameterization presented in Appendix C. The plots of the parameters from the best-fitting factor models for the remaining factors are set in Appendix D (Extraversion with reference item 1), Appendix E (Agreeableness with reference item 6), Appendix F (Conscientiousness with reference item 12) and Appendix G (Openness with reference item 2). Appendix H displays the table of transformed parameter estimates for the multi-groups factor model for Neuroticism. Appendices I - L display the statistical codes.

3.4 Results

This subsection provides the trends and the confidence bands of the parameters from the multi-groups factor model and compares them with those of the best-fitting varying-coefficient factor model for the factor Neuroticism. Subsection 3.4.1 starts with the trends of the parameters and their confidence bands for the customary multi-groups factor model for Neuroticism.

Subsection 3.4.2 illustrates the model selection of the best-fitting factor model for the Big Five.

Finally, section 3.4.3 portrays the model parameters and their confidence bands of the best-fitting factor model for Neuroticism. For Neuroticism, the reference item from the original parameterization is item 9 (Upset).

3.4.1 Neuroticism: Transformed parameters of the multi-groups factor model.

For the nominal multi-groups factor model, factor analysis was employed to data from

each year. Figure 2 (factor loadings), Figure 3 (residual variances) and Figure 4 (intercepts)

make up the measurement model of the multi-groups factor model for the factor Neuroticism.

From Figure 2, the trends of the factor loadings are wobbly with wide confidence bands especially before 1990s. Note the differences in wobbliness of the factor loadings and in the width of the confidence bands between items. The factor loadings of the items 1, 4, 12 and 14 seem to be the wiggliest and display the largest confidence bands.

Figure 2. Reversely worded items: Temperamental, Contented , Carefree, Placid

Figure 3. Reversely worded items: Temperamental, Contented , Carefree, Placid

Figure 4. Reversely worded items: Temperamental, Contented , Carefree, Placid

Like the trends of the factor loadings, we can observe the differences in the wobbliness of the trends and in the width of the confidence bands for the residual variances and intercepts. The trends of the items 1, 4, 12 and 14 showed the most fluctuations and the largest confidence bands among the items.

Figure 5

Figure 6

Figure 5 and Figure 6 depict the factor mean and factor variance respectively for the multi-groups factor model for Neuroticism. The trends vary considerably and the confidence intervals are wide, especially for the factor variance. Larger bands are observed near the boundaries, which were also evident in the confidence bands of the parameters from the measurement model. Given the width of the confidence bands, the parameter estimates of the structural model are imprecise.

3.4.2. Selection of the best-fitting varying-coefficient factor model.

A range of factor models with differing bandwidths were taken into consideration and were discussed in section 2.2. Each varying-coefficient factor model produces a fit. The deviance or the -2 log-likelihood was regarded as the fit statistic. The factor model with the lowest deviance for each personality factor was considered the best-fitting. Table 3.1 shows the bandwidth and the deviance of the best-fitting factor model for each personality factor.

Figure 7 displays the selection of the best-fitting factor model for the factor Neuroticism.

A red dot denotes the deviance for each bandwidth. Though the unrestricted multi-groups factor

model and the multi-groups factor model with measurement invariance do not possess any form of bandwidth, they are shown in Figure 7 with arrows for illustrative purposes. The remaining dots correspond to the deviances of varying-coefficient factor models. The best-fitting factor model for the factor Neuroticism yielded a bandwidth of 1.67, with its deviance displayed in Figure 7.

Table 3.1

Deviance and bandwidth of the best-fitting model for each personality factor

Big Five Factor Bandwidth Deviance

Extraversion 2.00 122530.1

Agreeableness 1. 33 114892.8

Conscientiousness 1.67 124942.2

Neuroticism 1.67 122661.9

Openness to Experience 1.33 118911.8

Figure 7

It is apparent in Figure 7 that both the unrestricted multi-groups factor model and the multi-

groups factor model with measurement invariance show relatively inferior model fits.

3.4.3. Neuroticism: Transformed parameters of the best-fitting factor model.

With the best-fitting factor models chosen for each factor from the previous section, the transformed parameters and confidence bands of the best-fitting factor model for Neuroticism will be shown. The measurement model of the best-fitting factor model for Neuroticism is depicted in Figure 8 (factor loadings), Figure 9 (residual variances) and Figure 10 (intercepts).

Relative to the trends of the factor loadings from the multi-groups factor model, the plots of the factor loadings in Figure 8 from the best-fitting factor model for Neuroticism portray stable and smooth trends with narrower confidence bands. Given the confidence bands, the estimates are relatively accurate, with larger bands near the boundaries. Items 1, 2, 6 and 8 depicted slightly upward trends while items 4, 5, 12 and 13 portrayed somewhat decreasing trends.

Figure 8. Reversely worded items: Temperamental, Contented, Carefree, Placid

Figure 9. Reversely worded items: Temperamental, Contented, Carefree, Placid

Given Figure 9, there is no overall trend in the plots of the residual variances, except for the downward trends of the residual variances for items 2 and 11 and upward trends of the residual variances for items 12, 13 and 14. Relatively larger confidence bands are observed before 1990s.

Like the plots of the factor loadings and residual variances, the trends of the intercepts are

relatively stable compared to the plots from the multi-groups factor model, given Figure 10. A

downward trend of the intercepts is evident for the items 1, 2 and 11 while an upward trend is

observed for the intercept of the item 12. As the intercept increases under a constant factor

score, the item becomes easier. Therefore, the former set of items has become slightly more

difficult through time while the latter item has turned out to be somewhat less difficult.

Figure 10. Reversely worded items: Temperamental, Contented, Carefree, Placid

Figure 11

Figure 11 and Figure 12 exhibit the factor mean and factor variance respectively of the

best-fitting factor model for Neuroticism. Relative to the plots of the factor mean and factor

variance from the multi-groups factor model, the plots of the factor mean and factor variance

from the best-fitting factor model are smoother and portray relatively smaller confidence bands.

Considering Figure 11, we can observe the gradually declining trend of the factor mean.

The factor mean fairly stabilizes during the last two decades of the study period. The literature presents conflicting trends, ranging from increasing (Twenge et al., 2010; Scollon & Diener, 2006), especially during the 1950s to 1990s, to unchanged (André et al., 2010; Sage, 2010) to decreasing (Gentile, Twenge & Campbell, 2012; Reid, 2005) levels of neuroticism. The diverse trends were specifically observed since 1990s, which could be attributed to the use of psychological interventions, medications and the introduction of the digital age. The downward trend of the factor mean might be associated with the Dutch economic growth and the use of social media, especially during the beginning of 2000s, which fostered social connectedness.

Economic conditions and social well-being are namely two aspects linked to measures of neuroticism (Twenge, 2000).

Figure 12

Meanwhile, the trend of the factor variance in Figure 12 is strikingly distinct, with two peaks,

depicting its variability through time. Moreover, the confidence bands are relatively large,

denoting some imprecisions in the factor variance. Dutch students may be less homogenous with

respect to Neuroticism during the end of 1980s and 1990s than the remaining years. That is, the respondents’ relative standing on Neuroticism is fairly different, especially around 1988 and 1999.

As observed in the plots of the parameters from the measurement model as well as the

factor mean and factor variance of the best-fitting factor model, slightly upward as well as

downward trends are apparent. Generally, the items are measurement invariant across values of

the moderator variable. Furthermore, relatively smooth trends and narrower confidence bands

are seen, attributed to the extent of the metric information gained from the moderator variable, in

contrast to the plots of the parameters from the multi-groups factor model. By taking into

account the data from neighboring years relative to a target point, the wobbliness of the trends

observed from the plots of the parameters from the multi-groups factor model is absent and

parameter estimates become more accurate.

4 Discussion

Given the findings, a number of remarks are warranted. Generally, kernel smoothing yielded smooth curves to depict the functional form of the trends. The moderator variable is a discrete metric variable, with the discrete nature of the moderator contributing to the wobbliness of the plots. Relatively large confidence bands were observed at the boundaries. The moderator variable, which consists of multiple cohorts, is not normally distributed, so the non-normality might explain some dents on the plots of the parameters. Studies do not indicate any absolute value for the ratio between the training set and the test set, although several studies suggest that the number of observations in the training set should be larger than those in the test set.

Therefore, the choice of the ratio between the training set and the test set proved to be arbitrary.

Like the choice of the ratio for the data partitioning, the choice of the values for the bandwidths and the number of bandwidths that were taken into account was random. Some studies on kernel smoothing show a plausible range for the bandwidth, which was used to come up with a series of values for the bandwidth. Furthermore, due to the confirmatory nature of the factor analysis employed in this paper, a three-way factor analysis (PARAFAC), being an exploratory technique, is futile to tackle the research objective. However, it may provide valuable initial values for the factor loadings of the factor model.

Prior to the data analysis, a few R packages for the factor analysis were examined,

namely sem, lavaan.survey and OpenMx. The first one, sem, was not suitable because it

does not incorporate sampling weights in the analysis while lavaan.survey does not take

missing values into account. Therefore, OpenMx, with its sophistication and complexity, was

chosen to run the factor analyses. The estimation technique used was Full Information

Maximum Likelihood or FIML, which resulted in slow calculations and heavy computational

load. Moreover, parameter estimation of a factor model through OpenMx is susceptible to instability so a reasonable set of initial values for the parameters and model constraints, such as specification of the lower bound for certain parameters and constraints on the reference item, should be carefully considered. A number of parameter estimations, especially with varying- coefficient factor models, generated a Status 1 code (Mx Status Green), denoting that the final iterate satisfies the optimality conditions of OpenMx but the convergence of the iterates has not yet taken place. It is a minor issue because the optimal solution is found and the estimates are practically close to the correct estimates, with an accuracy higher than 98%. For the generation of confidence bands, a low number of replicates (ten) was chosen due to slow computations, with the small number possibly leading to relatively larger confidence bands. Meanwhile, another form of estimation, without FIML (via the package lavaan.survey in R), was executed. The trends of the parameters from the two estimations were relatively similar. Yet the difference in computational speed between the two techniques was staggering, with the latter being the quickest. The number of missing data relative to the entire data set is insignificant, likely giving rise to the similarity of results.

Future research should contemplate on tackling the relatively large confidence bands at

the boundaries of the trends of factor model parameters. Additionally, establishing a measure for

the extent of measurement invariance for varying-coefficient factor models is a subject for

further research. Meanwhile, for continuous moderator variables, an identical statistical

procedure presented in this paper could be applied in the training set while a new statistical

technique should be employed in the test set. Moreover, future research could consider multiple

moderators and/or multiple factors in the analysis.

5 Conclusion

Given the differences between the multi-groups factor models and various forms of

varying-coefficient factor models and the better fits of the latter relative to the former, varying-

coefficient factor models are comparatively superior relative to multi-groups factor models, for

metric moderator analysis in the framework of confirmatory factor analysis. Varying-coefficient

factor models possess numerous advantages, including flexibility and simplicity of interpretation,

among others. The relatively stable and smooth plots from varying-coefficient factor models

capture trends that are readily visible, compared to the substantially fluctuating tendencies of the

trends from multi-groups factor models. Generally, the items are measurement invariant across

the values of the moderator variable.

References

André, M., Lissner, L., Bengtsson, C., Hallstrom, T., Sundh, V., & Bjorkelund, C. (2010).

Cohort differences in personality in middle-aged women during a 36-year period.

Scandinavian Journal of Public Health, 38, 457-464.

Bagozzi, R. P., Baumgartner, H., & Yi, Y. (1992). State versus action orientation and the theory of reasoned action: An application to coupon usage. Journal of Consumer Research, 18, 505-518.

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic and statistical considerations.

Journal of Personality and Social Psychology, 51(6), 1173-1182.

Bauer, D. J., & Hussong, A. M. (2009). Psychometric approaches for developing

commensurate measures across independent studies: Traditional and new models.

American Psychological Association, 14(2), 101-125.

Boker, S., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T., Spies, T., Estabrook, R., Kenny, S., Bates, T., Mehta, P., & Fox, J. (2011). OpenMx: An open source extended structural equation modeling framework. Psychometrika, 76(2), 306-317.

Brown, T. (2006). Confirmatory Factor Analysis for Applied Research. New York: Guilford Press.

Burt, C. (1940). The factors of the mind. London Univ. Press.

Cai, Z., Fan, J., & Li, R. (2000). Efficient estimation and inferences for varying-coefficient models. Journal of American Statistical Association, 95, 888-902.

Cai, Z., Fan, J., & Yao, Q. (2000). Functional-coefficient regression models for nonlinear time series models. Journal of American Statistical Association, 95, 941-956.

Caspi, A., Roberts, B. W., & Shiner, R. L. (2005). Personality development: Stability and change. Annual Review of Psychology, 56, 453-484.

Cattell, H. E. (1996). The original Big Five: A historical perspective. European Review of Applied Psychology, 46, 5-14.

Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI) professional manual. Odessa, FL:

Psychological Assessment Resources.

DeYoung, C. G., Quilty, L. C. & Peterson, J. B. (2007). Between facets and domains: 10 Aspects of the Big Five, Journal of Personality and Social Psychology, 93(5), 830-896.

Dobbin, K. K. & Simon, R. M. (2011). Optimally splitting cases for training and testing high dimensional classifiers. Biomed Central Medical Genomics, 4(31), 1-8.

Elshout, J. J., & Akkerman, A. (1973). Een Nederlandse test voor vijf

persoonlijkheidsfactoren, de 5PFT. In Drenth, P. J. D., Willems, P. J., & de Wolff, C. (Eds.). Arbeids- en organisatiepsychologie (pp. 49-56). Deventer: Kluver.

Fan, J., & Zhang, W. (1999). Simultaneous confidence bands and hypothesis testing in

varying-coefficient models. Annals of Statistics, 27, 1491-1518.

Fan, J., & Zhang, J-T. (2000). Two-step estimation of functional linear models with

applications to longitudinal data. Journal of the Royal Statistical Society B, 62, 303-322.

Fan, J., & Zhang, W. (2008). Statistical methods with varying coefficient models. Statistics and its interface, 1, 179-195.

Galton F. (1886). Regression towards mediocrity in hereditary stature, The Journal of Anthropological Institute of Great Britain and Ireland, 15, 246-263.

Garnett, J. C. M. (1919). On certain independent factors in mental measurement. Proceedings of the Royal Society of London, Series, 96, 91-111.

Gentile, B., Twenge, J. M., & Campbell, W. K. (2012). Birth cohort differences in the Big Five personality traits: A cross-temporal meta-analysis. Unpublished manuscript,

University of Georgia, Athens.

Harshman, R. A., & Lundy, M. E. (1994). PARAFAC: Parallel factor analysis. Computational Statistics & Data Analysis, 18, 39-72.

Hastie, T., & Tibshirani, R. (1993). Varying-coefficient models. Journal of the Royal Statistical Society. Series B (Methodological), 55(4), 757-796.

Hastie, T., Tibshirani, R., & Friedman, J.(2009). The Elements of Statistical Learning:

Prediction, Inference and Data Mining, Second Edition, Springer Verlag.

Hopke, P. K., Paatero, P., Jia, H., Ross, R. T., & Harshman, R. A. (1998). Three-way

(PARAFAC) factor analysis: examination and comparison of alternative computational methods as applied to ill-conditioned data. Chemometrics and Intelligent Laboratory Systems, 43, 25-42.

Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 43, 409-426.

Karatzoglou, A., Smola, A., Hornik, K., & Zeileis, A. (2004). Kernlab - An S4 package for kernel methods in R. Journal of Statistical Software, 11(9), 1-20.

Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. International Joint Conference on Artificial Intelligence, 14(2), 1137- 1145.

Krueger, R. F. (1999). The structure of common mental disorders. Archives of General Psychiatry, 56, 921-926.

Kuhn, M. (2008). Building predictive models in R using the caret package. Journal of Statistical Software, 28(5), 1-26.

Kumar, R. (2010). Research Methodology: A Step-by-Step Guide for Beginners Third Edition, Sage Publications Ltd, London.

Lawley, D. N. & Maxwell, A. E. (1962). Factor analysis as a statistical method. Journal of Royal Statistical Society. Series D (The Statistician), 12(3), 209-229.

Long, J. S. (1983). Confirmatory Factor Analysis, Sage Publications Inc., Thousand Oaks.

Lord, W. (2007). NEO-PI-R - A Guide to Interpretation and Feedback in a Work

Context. Hogrefe Ltd, Oxford.

Mcrae, R. R., John, O. P. (1992). An introduction to the five-factor model and its applications. Journal of Personality, 60(2), 175-215.

Molinaro, A. M., Simon, R., & Pfeiffer, R. M. (2005). Prediction error estimation: a comparison of resampling methods. Bioinformatics, 21(15), 3301-3307.

Park, B. U., Mammen, E., Lee, Y. K., & Lee, E. R. (2013). Varying-coefficient regression models: A review and new developments. International Statistical Review, doi: 10.1111/

insr.12029.

Pearson, K. (1896). Mathematical contributions to the theory of evolution. III. Regression, heredity and panmixia. Philosophical Transactions of the Royal Society Ser. A, 187, 253- 318.

R Core Team (2014). R: A language and environment for statistical computing. R

Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/.

Ramsay, J. O., & Silverman, B. W. (1997). The Analysis of Functional Data. New York:

Springer.

Reid, M. W. (2005). Searching for agent blue: Birth cohort changes in depressive affect and etiological sociocultural correlates, a cross-temporal meta-analysis. Unpublished master’s thesis, San Diego State University, San Diego, California.

Sage, J. C. (2010). Birth cohort changes in anxiety from 1993-2006: A cross-temporal meta-

analysis. Unpublished master’s thesis, San Diego State University, San Diego, California.

Saigo, H., Shao, J., & Sitter, R. R. (2001). A repeated half-sample bootstrap and balanced repeated replications for randomly imputed data. Survey Methodology, 27(2), 189-196.

Saucier, G., & Goldberg, L. R. (2001). Lexical studies of indigenous personality factors:

Premises, products, and prospects. Journal of Personality, 69, 847–879.

Scollon, C. N. & Diener, E. (2006). Love, work and changes in extraversion and

neuroticism over time. Journal of Personality and Social Psychology, 91, 1152-1165.

Smits, I., & Dolan, C. , Vorst, H., Wicherts, J., & Timmerman, M. (2011). Cohort differences in Big Five personality factors over a period of 25 years. Journal of Personality and Social Psychology, 100(6), 1124-1138. doi: 10.1037/a0022874

Sörbom, D. (1974). A general method for studying differences in factor means and factor structure between groups, British Journal of Mathematical and Statistical Psychology, 27, 229-239.

Spearman, C. (1904). General intelligence, objectively determined and measured. American Journal of Psychology, 15, 201–293.

Steyerberg, E. W., Harrell, F. E. Jr., Borsboom, G. J., Eijkemans, M. J., Vergouwe, Y.,

& Habbema, J. D. (2001). Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. Journal of Clinical Epidemiology, 54(8), 774-781.

Thomson, G. H. (1951). The Geometry of Mental Measurement. London Univ. Press.

Thompson. B. (2004). Communality. In Michael. S. Lewis-Beck, A. Bryman & Tim Futing

Liao (Eds.), The SAGE Encyclopedia of Social Science Research Methods. (pp. 148-149).

Thousand Oaks, CA: Sage Publications, Inc.

Thurstone, L. L. (1931). Multiple factor analysis. Psychological Review, 38, 406-427.

Thurstone, L. L. (1935). The vectors of mind. Chicago: University of Chicago Press.

Thurstone, L. L. (1947). Multiple-factor analysis. Chicago: University of Chicago Press.

Tillé, Y. & Matei, A. (2009). Sampling: Survey Sampling. R package version 2.2. Online:

http://cran.r-project.org/src/contrib/Descriptions/sampling.html

Tupes, E. C., & Christal, R. E. (1961). Recurrent personality factors based on trait

ratings, Technical Report ASD-TR-61-97, Lacklang, Air Force Base, Texas: Personnel Laboratory, Air Force Systems Command.

Twenge, J. M. (2000). The age of anxiety? Birth cohort change in anxiety and neuroticism:

1952-1993. Journal of Personality and Social Psychology, 79(6), 1007-1021.

Twenge, J. M., Gentile, B., DeWall, C. N., Ma, D. S., Lacefield, K., & Schurtz, D. R.

(2010). Birth cohort increases in psychopathology among young American, 1938-2007:

A cross-temporal meta-analysis of the MMPI. Clinical Psychology Review, 30, 145-154.

Wickham, H. (2009). ggplot2: Elegant graphics for data analysis. New York: Springer-Verlag.

Appendix A

Big Five: Facets and items of the remaining factors Table A1

Facets of the Big Five (Costa & McCrae, 1992)

Extraversion Agreeableness Conscientiousness Neuroticism Openness

Warmth Trust Competence Anxiety Fantasy

Gregariousness Straightforwardness Order Angry-Hostility Aesthetics Assertiveness Altruism Dutifulness Depression Feelings

Activity Compliance Achievement Striving

Self- consciousness

Actions Excitement

Seeking

Modesty Self-Discipline Impulsiveness Ideas Positive

Emotions

Tender-Mindedness Deliberation Vulnerability Values

Table A2

Extraversion items with some attributes

Item Label Characteristics

1 Loquacious Talks superfluously

2 Domineering Automatically takes charge

3 Lethargic* Lackluster impression

4 Vigorous Always on the go

5 Reticent* Reserved, enigmatic

6 Exuberant Vibrant, alert

7 Gregarious Tends to like company

8 Demonstrative Frank, open

9 Audacious Risk-taker

10 Taciturn* Silent, introspective

11 Candid Comfortable among strangers

12 Reclusive* Actively seeks solitude

13 Attention-seeking Basks in the limelight

14 Self-effacing* Discreet behavior, modest

* Reversely worded items; item 1 is the reference item

Table A3

Agreeableness items with some attributes

Item Label Characteristics

1 Cynical* Tends toward mistrust

2 Vicious* Malicious, passive-aggressive

3 Egoistic* Self-centered

4 Benevolent Easy-going, gentle

5 Cooperative Compromising

6 Considerate Attentive

7 Sympathetic Free of envy

8 Defiant* Inflexible and rigid in a group

9 Compassionate Shows empathy

10 Condescending* Thinks highly of oneself

11 Team-spirited Avidly works in a team

12 Aloof* Cold, keeps others at arm’s length

13 Trusting Assumes that one is well-intentioned

14 Cordial Warm, pleasant

* Reversely worded items; item 6 is the reference item

Table A4

Conscientiousness items with some attributes

Item Label Characteristics

1 Painstaking Tends toward precision and looks at minute details 2 Apathetic* Indifferent to the daily state of affairs

3 Yielding* Inconsistent, does not plow through

4 Virtuous Possesses integrity

5 Conventional Values adhering to prevailing norms

6 Responsible Prioritizes public interest over personal gain

7 Conformist Obeys dress code according to occasion

8 Orderly Likes neatness

9 Unorthodox* Finds it acceptable to break rules

10 Well-organized Practices time management, efficient

11 Tenacious Persevering, gritty

12 Chaotic* Fussy, not detail-oriented

13 Structured Likes to have clear expectations

14 Diligent Works tirelessly

* Reversely worded items; item 12 is the reference item

Table A5

Openness to Experience items with some attributes

Item Label Characteristics

1 Resourceful Has flashes of progressive ideas, visionary 2 Sophisticated Shows a spectrum of intellectual and cultural interests

3 Pragmatic* Prefers practice over theory

4 Connoisseur Exhibits knowledge of art and has a feel for aesthetics 5 Analytical Investigative, sees the broad picture

6 Nonliterary* Reads little, not studious

7 Unimaginative* Follows trend, expresses non-original perspectives

8 Ingenious Demonstrates intellectual prowess

9 Inquisitive Curious, probing

10 Rational Reasonable, logical

11 Inventive Has an abstract mind

12 Unworldly* Tends toward naiveté, ill-informed

13 Pedantic Gifted in literature

14 Opinionated Forms own opinion with conviction

* Reversely worded items; item 2 is the reference item

Appendix B

Confidence bands: Repeated half-sampling bootstrap

The procedure for the repeated half-sample bootstrap (Saigo, Shao & Sitter, 2001) depends on the number of units in a certain stratum. Denote the sample size in a stratum or number of units in a stratum as N. When the sample size N is even in a particular stratum, the sampling consists of drawing a sample without replacement of size N/2. Each sampled unit is duplicated so the bootstrap sample size is equivalent to the original sample size of that specific stratum. When the number of units is odd, the sampling proceeds in two different ways, with addition (1) and deletion (2) used with probability ¹ ₄ and ³ ₄ respectively when generating replicate samples: (1) For addition, a sample of size (N-1)/2 is drawn without replacement and duplication is performed on each sampled unit to obtain a size N-1. Subsequently, an extra unit is drawn at random from the duplicated sample to reach the sample size N for the odd-sized stratum. (2) For deletion, a sample size of (N-1)/2 + 1 is drawn without replacement and each unit is duplicated to form a sample size of N+1. Given the redundancy of an additional unit, it is omitted at random to achieve the sample size N.

The entire data set consists of 25 strata with 15 and 10 strata having even and odd number of units respectively. Therefore, a bootstrap sample is formed as follows: (1) Determine first whether a stratum is even-sized or odd-sized. (2) If a stratum is even-sized, draw a random sample without replacement of size N/2 and subsequently duplicate the half-sample (i.e. a sample of size N/2). (3) If a stratum is odd-sized, generate a number from a uniform distribution, with a minimum of 0 and a maximum of 1, to establish which of the two methods (having probabilities

1

4 and ³ ₄ ) will be employed. (4) If the generated number is equal or less than ¹ ₄ , apply method 1,

otherwise employ method 2. (5) For method 1, draw a sample of (N-1)/2 without replacement

from an odd-sized stratum, duplicate and finally draw an extra unit at random from the duplicated sample. (6) For method 2, draw a sample of (N-1)/2 + 1 without replacement from an odd-sized stratum, duplicate and finally discard a random unit. Ten bootstrap samples were drawn from the entire data set, with an identical number of observations as the original data set.

These ten bootstrap samples were subjected to varying-coefficient factor analysis, with an identical procedure as the estimation of the parameters in the training set. For each bootstrap sample, the parameters of interest (i.e. factor loadings, residual variances, intercepts, factor mean and factor variance) were estimated. With ten bootstrap samples, there were ten bootstrapped estimates for each parameter of interest. These bootstrapped parameters were then transformed using deviation contrasts (Appendix C).

A pointwise 95% confidence interval was computed for the transformed point estimates, with the transformed point estimates from the original estimation, the z-value (1.96) corresponding to the aforementioned level of confidence and the sampling variability or the standard error of the transformed bootstrapped point estimates being the elements of the interval.

The standard error of a transformed point estimate was calculated as the standard deviation of the

ten transformed bootstrapped estimates for that particular parameter of interest. The lower and

upper bound of a transformed point estimate, which constitute the pointwise confidence bands,

were calculated by subtracting and adding the transformed point estimate from the original

estimation respectively to the product of the z-value for the 95% confidence interval (1.96) and

the standard error of the transformed bootstrapped point estimates.

Appendix C

Transformation of parameters: Deviation contrasts

For simplicity, we formulate the one factor model starting with the true score model:

𝑋 _𝑗 = 𝑇 _𝑗 + 𝐸 _𝑗 , 𝐸 _𝑗 ~ 𝑁(0, 𝜓 _𝑗 ² ), (C1) where 𝑋 _𝑗 is the observed score of indicator 𝑗, 𝑇 _𝑗 is the corresponding true score and 𝐸 _𝑗 an error term with mean zero and variance 𝜓 _𝑗 ² . The one-factor model connects the items’ true scores via a linear scale transformation to a common factor:

𝑇 _𝑗 = 𝜈 _𝑗 + 𝜆 _𝑗 𝐹, 𝐹 ~ 𝑁(𝛼, 𝜑 ² ), (C2) where 𝜈 _𝑗 is an intercept and 𝜆 _𝑗 a loading and 𝐹 is the subject’s common factor with mean 𝛼 and variance 𝜑 ² . There is an indeterminacy in this model because the scale of the latent variable is not known. Since 𝐹 is measured on an unknown interval scale, the model is invariant under linear transformations. If we set

𝐹 ^∗ = 𝑏 + 𝑎𝐹, 𝑎 ^∗ = 𝑏 + 𝑎𝛼, 𝜑 ^∗ = 𝑎𝜑, (C3) which yields

𝑇 _𝑗 = 𝜇 _𝑗 + 𝜆 _𝑗 ^{𝐹 ∗} _𝑎 ^−𝑏 = 𝜇 _𝑗 ^∗ + 𝜆 _𝑗 ^∗ 𝐹 ^∗ , 𝐹 ^∗ ~ 𝑁(𝛼 ^∗ , (𝜎 ^∗ ) ² ),

it is easily seen that this arbitrary linear factor-score transformation yields transformed item parameters

𝜆 _𝑗 ^∗ = ^𝜆 _𝑎 ^𝑗 , 𝜇 ^∗ _𝑗 = 𝜇 _𝑗 − ^𝜆 _𝑎 ^{𝑗 𝑏} , (C4) so that the starred model is observationally equivalent with the non-starred model. This indeterminacy can be removed by putting two constraints on either the factor mean and

variance, e.g.,

𝑎 ^∗ = 0, (𝜎 ^∗ ) ² = 1,

or on the intercepts and the loadings. There are various ways to do the latter. One method is to restrict the parameters of one (reference) item, e.g.

𝜇 _𝑟𝑒𝑓 = 0 𝜆 _𝑟𝑒𝑓 = 1, (C5)

so we have

𝑇 _𝑟𝑒𝑓 = 𝜇 _𝑟𝑒𝑓 + 𝜆 _𝑟𝑒𝑓 𝐹 = 𝐹, (C6)

that is, the factor becomes equal to the true score of one item. This reference item is best chosen to be the one whose score correlates highest with the factor. This is a good restriction to estimate the parameters, but not to present the parameters because the factor becomes equal to the true score of the reference item and becomes too sensitive for specific bias and other idiosyncrasies of the reference item. Therefore, it is better to reparameterize the estimated parameters such that they satisfy restrictions that treat the items equally. In item response theory, we often work with deviation contrasts to address this.

Deviation contrasts of intercepts and loadings can be imposed by setting the arithmetic mean of the intercepts and the geometric mean of the loadings equal to zero and one respectively. This yields the reparameterized model

𝑇 _𝑗 = 𝜇 _𝑗 ^∗ + 𝜆 _𝑗 ^∗ 𝐹 ^∗ , 𝐹 ^∗ ~ 𝑁(𝛼 ^∗ , (𝜎 ^∗ ) ² ) (C7) with constraints

∑ ^𝑛 _𝑗=1 ^𝜇 _𝑛 ^{𝑗 ∗} = 0 , ∏ _𝑗 (𝜆 ^∗ _𝑗 ) ^{1 𝑛} = 1. (C8) Here, the role of the different items is completely symmetric. The question is, how do we get the (starred) parameters (C7) with deviation contrasts (C8) from an arbitrary (non-starred) parameterization

𝑇 _𝑗 = 𝜈 _𝑗 + 𝜆 _𝑗 𝐹, 𝐹 ~ 𝑁(𝛼, 𝜎 ² ).

We first absorb 𝑎 and 𝑏 of (C3) in the item parameters: