A note on the identification of restricted factor loading matrices

(1)

Tilburg University

A note on the identification of restricted factor loading matrices

Bekker, P.A.

Publication date:

1985

Document Version

Publisher's PDF, also known as Version of record

Link to publication in Tilburg University Research Portal

Citation for published version (APA):

Bekker, P. A. (1985). A note on the identification of restricted factor loading matrices. (Research Memorandum

FEW). Faculteit der Economische Wetenschappen.

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners

and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.

• You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal

Take down policy

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately

and investigate your claim.

(2)

CBM

R

7626 1

1985 ~

169 TI; ~SCf íRIFTLNBUREAU

Best~mming, BIBLI TH.~F.K

K .~'I'I : ~ ' I~IiE

~

HOGES:;.~~~0?,

'

TILBUhG

Nr.

faculteit der economische wetenschappen

(3)

(4)

f ~

~`iIi~. l~. -~~

_:,

~~,j

C3~~~-?~ T'r~~E~ti

~

TfL~;~r~G

FEW 169

A NOTE ON THE IDENTIFICATION OF RESTRICTED FACTOR LOADING MATRICES

by Paul A. Bekker

1985

The author would like to thank Arie Kapteyn for his comments and sugges-tions. Financial support by the Netherlands Organization for the Advan-cement of Pure Research (ZWO) is gratefully acknowledged.

(5)

Abstract

It is shown that problems of rotational equivalence of restricted factor loading matrices in orthogonal factor analysis are equivalent to pro-blems of identification in simultaneous equations systems with covarian-ce restrictions. A necessary and sufficient condition for local unique-ness is given and a counterexample is provided to a theorem by J. Algina concerning necessary and sufficient conditions for global uniqueness.

(6)

i

1. Introduction

Consider the orthogonal factor analysis model E- AA' t~,, where E is a positive semi-definite pXp-matrix which can be consistently estí-mated, ~ is a pXp-diagonal matrix of unique variances, A is a pmn-factor loading matrix of full column rank (p ~ m). Identifíability of the ma-trices A and ~ amounts to the question whether these matrices can be solved uniquely from these equations. In what follows, it ís assumed that the matrix ~ is identified. Thus iden[ifiability of A stems from the pruperties of the equations C- E-~- AA'. In general, these equa-tions do not yield a unique solution for A. Identifiability may be aided by invoking prior information represented by linear restrictions on the columns of A-(al,...,am). The identifying equations for A are thus given by

(I)

c - An',

(Z) R~a~ - r~ ~ J - 1.-.-~m~

where R~ i s a k~xp-matrix and r~ a k~-vector, representíng k~ linear restrictions on a,.

J

(7)

2

(8)

3

'L. An AnalogY

The equations in (1) can be seen as a special case of the more general equations C- A~A', where ~ is a positive defínite m~nn-matrix. Let n be partitioned as A' -(Ai,n2), with a curresponding partitioníng of C. It is assumed, without loss of generality, that A1 and C11 are nonsingular mxm-matrices. Furthermore, let A be an arbitrary positive definite (p-m)x(p-m)-matrix. Then

Lc21

c22}A ~

nl o

AZ

I

and H is positive definite. Clearly then

(4) H 1

nl o

nZ I

m o

0 A

~-1

0

0 A 1

r n'

n' ~

1

2

0 I

The classical simultaneous equatíons system of econometrics is giveii by y'B t x'r - E', where y' is a m-vector of endogenous variables, x' is a(p-m)-vector of exogenous variables, B is a nonsingular mXm-ma-trix, r is a(p-m)xm-matrix, and the m disturbances in e' are uncorrela-ted with the exogenous variables in x'. Denoting covariance matrices by E, with subscripts refering to the variables, ít follows that

Í r'

B'~

(5)

0 I

yy yx

rE E

-E E xy xx B o

r

I

EcE

o

0 E

_xx

The analogy with (4) is obvious. The structural coefficients in B and r currespond to A1 and nZ respectively and the covariances of the

distur-bances in EEe correspond to ~-1.

When ~ is unrestricted as in oblique factor analysis, then also

m-1 is unrestricted and necessary and sufficient conditions for global

(9)

simi-4

lar identification conditions for simultaneous equations sys[ems. These conditions were given by many authors, among others by Koopmans and Reiers~l (1950), Anderson and Rubin (1956) and, more recently by Algina (1980) and Geweke and Singleton (1981).

When ~ is restricted to be block diagonal, then also ~ 1 is block diagonal and conditions for identification of A are similar to conditions for identification in simultaneous equations systems where Ece is block diagonal. Conditions for identification in these cases were given by F.M. Fisher (1966). For example, Dunn's condition can be seen as a special case of Fisher's conditions. Conditions for the identiFication uf single rows of (K',t'), or A', were gíven by Hausman and Taylor (1983) and Bekker and Pollock (1984). A necessary and suffí-cient condition for local ídentification, based on the Jacobian matrix of the equations in (5) is given by Bekker and Pollock (1984) and Haus-man, Newey and Taylor (1983). An analogous condition for orthogonal fac-tor analysis is the following. Let K be a permutation matrix, i.e. K is a partitioned matrix of order m2xm2 whose (j,í)-th block is an mxm-matrix which has a unit in the (i,j)-th position and zeros elsewhere: K- E E(e.e:t~e.e'.). Let ttie restrictions (2) be written as R vec(A) - r,

J 1 1 J i j

where R is a E k,xmp-matrix and r is a E k, vector. It is assumed that

j ~ ~ ~

A is a regular point of the matrix:

I 2~-K m

(6)

F(A)

-i.e. the rank of F(A) is constant in an open neighbourhood of A. The condition is the following.

Proposition: A necessary and sufficient conditions for a solution A of the equations (1) and (2) to be locally unique is that rank

{F(A)} - m2.

The proof is omitted.

(10)

(11)

6

3. .A counterexample

Algina's theorem 3 is as follows: Let Ej denote the first (j-1) columns of the mxm identity matrix, then, under the restrictions given by (2), where rj ~ 0, a necessary and sufficient condition for identification of A is that rank(R1A) - m and rank(A'R'.,E,) - m for j- 2,...,m.

J

In order to give a counterexample to this theorem, let A be par-titioned as above, and let A1 be a 3X3 nonsingular matrix which is re-stricted as r ~11 ~12 0 (~) A1 - I 0 ~22 ~23 I; ~31 0 ~33 ~

Civen these restrictions, one of the equations in Ci1 - AlA1 is

(8)

(C11)31 - A31A11'

Considering the equations A1C11A1 - I, it follows that

(9)

(C11)I1~11 } 2(C11)13A11~31 t (C11)33~31 - 1'

Any two solutions Ail) and Ai2j, say, must thus satisfy

(10) a(1)a(1) - a(2)a(2) - (C₃₁ ₁₁ ₃₁ ₁₁ _{11 31}) . and

(11)

(C-1)

~(1)2 } (C 1)

~(1)2 - (C-1)

~(2)2 } (C-1)

~(2)2 .

11 11 11

11 33 31

11 11 11

11 33 31

2

h1u1[iplying (11) by

~11) All) and using (10) yields

(L2) (C-1)

~(1)2~(2)2(a(1)2-a(2)2) - (C 1) (C )2 (a(1)2-a(2)2).

(12)

7

2

Therefore if a(1) ~ a(2) it must hold true that ' L1 11 '

-1

2 (13)

~(1)2~(2)2 - (C11)33(C11)31 .

₁₁

_(C-1)

11 11

This equation shows that if a(1), or -a(1), is a solution for a, then there can be at most two other solutiuns aii) and -aii) which ldiffer only by sign. Analogous expressions hold for the other parameters. Fur-ttiermure, equation (10) shows that, within a single column of A1, the value of one parameter, all say, fixes the value of the other free para-meCer a31. ThereEore, up to column sign changes, there are no more than two distinct solutions for A. As A- C A'-1

1 2 21 1~ each solution for A1

corresponds to a solution for A2. Consequently, if A1 is restricted ac-cording to (7), there are, up to column sign changes, no more than two distinct solutions for A, i.e. A is locally identified.

A nummerical example will show that the two distinct solutions may indeed exist. Let C11 be the following matrix:

81

54 45 ~

(14) C11 -1 100

54

86

5

45

5

50

then two solutions for A1 are given by:

(13)

-8

Of course, within each colwnn signs may be changed.

Dunn's specification of 2eros leads to local identification of A where A is uníque up to column sign changes. The example shows that spe-cification of zeros may also lead to local identification of A where A is not unique up to column sign changes. Further specification of a value, e.g. all - 3.(5)~` leads to uniqueness of A up to sign changes of the second and third columns. If also the values a22 - 9 and a33 -(5)~ are specified, then A is globally unique. This last specification may serve as a counterexample to Algina's theorem 3. The restrictions on the columns, R.a. - r., are such that r. ~ 0, so that, accordíng to the

J ] J ~

theorem, global identification of A implies the existence of at least one culumn, ak say, such that rank(RkA) - 3. That is to say, at least one column must have three specified values. In the example, specifica-tion of only two values in each column leads to global identification. Therefore Algina's conditions are not necessary for identification and

(14)

9

4. Iliscussiou

The sufficient conditions of Dunn (1973), Jennrich (1978), and Algina (1980) for identificatíon of A are all based on the same principle. The columns of A are identified sequentially. In the first step a column is identified by means of linear restrictions Rlal - rl in (2) and

-1

(O,I)H-lal - 0 in (4) and a normalization aiH al - 1 in (4). In the second step another column is ídentified using similar restrictions and an additional bilinear restriction aiH laZ - 0 in (4), which becomes linear if al is previously identified. If al and aZ are previously iden-tified, then the third column has two additional linear restric-tions (al,aZ)'H la3 - 0, etc..

-1

In this procedure the bilinear restrictions A'H A- I in (4) behave as if they were linear restríctions on separate coliunns of A. As all restrictions are linear, apart from the normalizations, the conse-quent conditions for uniqueness of A are relatively simple. However, these conditions, including Algina's condition, are not necessary. The example presented here shows that A may be locally identified while no column of A is previously identified by means of linear restrictions and a normalization. In such cases the restrictions in A'H-lA - I truly be-iiave as bilinear restrictions which cannot be reduced to linear restric-tions, and the columns of A are identified simultaneously instead of sequentially. A necessary and sufficient condition for identification must therefore be a system-wide condition instead of a sequence of rela-tively simple conditions for separate columns.

(15)

10

restrictions on A may affect the identification of ~. Therefore the overall identification problem in confirmatory factor analysis is far

(16)

11

Ketercnces

Algina, J., 1980, A note on identification in the oblique and orthogonal factor analysis models, Psychometrika 45, 393-396.

Anderson, T.W. and H. Rubin, 1956, Statistical inference in factor ana-lysis, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability 5, 111-150, (Jerzy Neyman, ed.), Universi-ty uf California Press.

I3ekker, P.A. and D.S.G. Pollock, 1984, IdentiEication of linear sto-cha5tic models with covaríance restrictions, Research Memorandum L59, Tilburg University.

Dunn, J.l;., 1973, A note on a sufficiency condition for uniqueness of a restricted factor matrix, Psychometrika 38, 141-143.

Fisher, F.M., 1966, The identification problem in econometrics, McGraw-Hill, New Yurk.

Ueweke, .I.I~'. and K..I. Singleton, 1981, Maximum iikelíhood confirmatory factor analysis of economic time series, Internatiunal Economic Re-view 22, 37-54.

Hausman, J.A. and W.E. Taylor, 1983, Identification in linear simulta-neous equations models with covariance restrictions: an instrumental variables interpretation, Econometrica 51, 1527-1549.

Hausman, J.A., W. Newey and W. Taylor, 1983, Efficient estimatíon and identification of simultaneous equation models with covariance re-strictions, MIT Working Paper 331.

Jennrich, R.I., 1978, Rotational equivalence of factor loading matrices with specified values, Psychometrika 43, 421-426.

JSreskog, K.G., 1969, A general approach to confírmatory factor analy-sis, Psychometrika 34, 183-202.

Koopmans, T.C. and 0. Reiersbl, 1950, The identification of structural cliaracteristics, Annals of Math. Stat. 21, 165-181.

Reiers~l, 0., 1950, On the identifiability of parameters in Thurstone's multiple factor analysis, Psychometrika 15, 121-149.

(17)

1

LN 1984 kEEDS VERSCHENEN

138 G.J. Cuypers, J.P.C. Kleijnen en J.W.M. van Rouyen Testing the Mean of an Asymetric Population: Four Yrocedures Evaluated

139 T. Wansbeek en A. Kapteyn

Estimation in a linear model with serially correlated errors when observations are missing

140 A. Kapteyn, S. van de Geer, H. van de Stadt, T. Wansbeek Interdependent preferences: an econometric analysis 141 W.J.H. van Groenendaal

Discrete and continuous univariate modelling 142 J.P.C. Kleijnen, P. Cremers, F. van Belle

The power of weighted and ordinary least squares with estimated unequal variances in experimental design

143 J.P.C. Kleijnen

Supereffícient estimation of power functions in simulatiun experiments

144 P.A. Bekker, D.S.G. Pollock

Identification of linear stochastic models with covariance restrictions.

145 Max D. Merbis, Aart J. de Zeeuw

From structural form to state-space form 146 T.M. Uoup and A.J.J. Talman

A new variable dimension simplicial algorithm to Eind equilibria on the product space of unit simplices.

147 G. van der Laan, A.J.J. Talman and L. Van der Heyden Variable dimension algorithms for unproper labellings. 148 G.J.C.Th. van Schijndel

Dynamic firm behaviour and financial leverage clienteles 149 M. Plattel, J. Peil

The ethico-political and theoretical reconstructíon uf contemporary economic doctrines

150 F.J.A.M. Hoes, C.W. Vroom

Japanese Business Policy: The Cash Flow Triangle an exercise in sociological demystification 151 T.M. Duup, G. van der Laan and A.J.J. Talman

(18)

11

IN 1984 Rh:F.DS VEItSCHENEN (vervnlg)

1'i:? A,I,. Il~~mpi~nius, 1'.G.II. MuLder

1'otal Mortality Analysis of the Rotterdam Sample of the Kaunas-Rotterdam Intervention Study (KRIS)

153 A. kapteyn, Y. Kooreman

A disaggregated analysis of the allocation of time within the household.

154 T. Wansbeek, A. Kapteyn

Statistically and Computationally Efficíent Estimation of the Gravity Model.

155 P.F.P.M. Nederstigt

Uver de kosten per ziekenhuisopname en levensduurmodellen 156 R.R. Meijboom

An input-output like corpurate model including multiple technologies and make-or-buy decisíons

157 P. Kooreman, A. Kapteyn

Estimatiun uf Rationed and Unrationed Household Labor Supply h'unctiuns lJsing Flexible Functional Forms

158 K. Heuts, J. van Lieshout

An implementation of an inventory model with stochastic lead time l59 P.A. Aekker

Comment cn: Identification in the Linear Errors in Variables Model L60 P. Meys

Functies en vormen van de burgerlijke staat

Over parlementarisme, corporatisme en autoritair etatisme 161 J.P.C. Kleijnen, H.M.M.T. Denis, R.M.G. Kerckhoffs

Gfficient estimation of power functions 162 H.L. Theuns

The emergence of research on third world tourism: 1945 to 1970; An introductury essay cum bibliography

163 F. Boekema, L. Verhoef

lle "Grijze" sector zwart op wit

Werklozenprojecten en ondersteunende instanties in Nederland in kaart gebracht

164 G. van der Laan, A.J.J. Talman, L. Van der Heyden Shortest paths for simplicial algorithms

165 J.H.F. Schilderinck

Interregiunal structure of the European Community

(19)

ÍIL

Il~n I'..I.I~.~;. i~l~.~iLendi.jk~;

An exercise in welfare economics (I) 1~~7 L. !-lsner, M.H.C. Yaardekooper

(20)

iv

IN 19t~5 ItF:F.DS VERSCHENEN 16tt T.M. Uoup, A.J.J. Talman

(21)