Journal of Classification 8:93-98 (1991)
Some Notes on the Diagonalization of the Extended
Three-Mode Core Matrix
Piet Brouwer Pietcr M. Kroonenbcrg
Leiden University Leiden University
Abstract: We extend previous results of Krooncnberg and de Leeuw (1980) and Kroonenbcrg (1983, Ch. 5) on transformations of the extended core matrix of the Tucker2 model (Kroonenbcrg and de Leeuw 1980). In particular, it is shown that non-singular transformations to diagonalize the core matrix will lead to PARAFAC solutions (Harshman 1970; Harshman and Lundy 1984), if such solutions exist. Keywords: Three-mode principal component analysis; Parallel factor analysis; PARAFAC; CANDECOMP.
1. Introduction
Many models to analyze three-way data contain a set of parameters to describe relationships between the components (factors, dimensions) of the various ways. These parameters are collected in a (possibly) diagonal three-way 'core matrix' (Tucker 1966) when all three three-ways are reduced, or an 'extended core matrix' (Tucker 1972) when only two ways are reduced. (In this note we will restrict ourselves to extended core matrices, and for simpli-city we will often refer to them as 'core matrices'.) In some three-way models such core matrices arc restricted to be diagonal by definition. Core
This note is a revised version of a paper presented at the 4th European Meeting of the Psychometric Society, Cambridge, U.K., 2-5 July 1985, and is based on the first author's mas-ter thesis.
matrices of non-diagonal models contain far more parameters than diagonal ones. As shown by Carroll and Wish (1974; see also Krooncnbcrg 1983, Ch. 3; Kicrs 1988) models with diagonal core matrices arc nested within models with more general core matrices. One might therefore wonder whether it is possible to transform more general core matrices into (approximately) diago-nal ones, and thus achieve greater parsimony without much loss of fit of the model to the data.
The idea of diagonalizing core matrices of three-way models originates in the multidimensional scaling literature (sec e.g. de Leeuw and Pruzansky 1978, and their references). Based upon an idea of Jan de Leeuw, Krooncn-bcrg (1983, Ch. 5) suggested a diagonalization procedure of the core matrix using non-singular transformations, which is essentially equivalent to apply-ing the PARAFAC/CANDECOMP algorithm (Harshman, 1970; Carroll & Chang, 1970) to the core matrix. In this note we will explore this diagonaliza-tion in greater detail.
2. Theory Diagonalization of Extended Core Matrices
The Tuckcr2 model (a slight generalization of a model for individual differences scaling proposed by Tucker 1972; sec Kroonenbcrg and de Leeuw, 1980) is a way to analyze a three-way data matrix Z = (Z\,Z2,... ,ZK) of the order / xJ x K, where Zk is the £-th frontal / xj
slice with / not necessarily equal to J. The Tuckcr2 model is given as
Zk=AGkB' (1)
in which A and B arc I x P and J x Q component matrices of the first and second way respectively, and Gk is the k-\h P x Q slice of the extended core
matrix, which is of the order P x Q x K. In the TUCKALS2 algorithm used to solve the estimation of the Tucker2 model, the A and B arc initially ortho-normal matrices, while there arc no restrictions on the Gk. However, the
orthonormality restriction functions only to achieve an attractive estimation procedure, and can be relaxed as soon as a solution has been found. For instance to facilitate interpretation, the components may be transformed non-singularly with the inverse transformations applied to the core matrix. An alternative is to simplify the core matrix itself by diagonalizing it. An extended core matrix is called diagonal if each Gk is a square diagonal matrix
of order P x P. (Note that there is no requirement for the Zk to be symmetric
Notes on Diagonalization 95
solution for which the core matrix is diagonal (apart from trivial reflections and permutations). Moreover, the associated components are unique as well. Therefore, there is no transformational freedom with diagonal core matrices. Harshman's parallel factor analysis model (PARAFAC), in fact, starts with the restriction that the extended core matrix is diagonal, i.e. the PARAFAC model can be written as
Zk = ADkB' (k = },...,K), (2)
where A and B are defined as before (without the orthonormality restriction) and the Dk arc diagonal P x P matrices. One way to diagonalize the core
matrix would be to use a procedure which approximates a PARAFAC solu-tion analogously to the procedure of e.g. Cohen (1974) to transform a solusolu-tion of Carroll and Chang's (1970) non-diagonal individual differences scaling model IDIOSCAL to a diagonal INDSCAL solution. The parallel case here would be to seek matrices Dk, S and 7 such that the function
\\Gk-SDkT'\\ (3)
is minimi/.cd for all k ( k = 1 , . . . , K ) simultaneously, which is a decomposi-tion of the core matrix. Simplificadecomposi-tion of the core matrix by nonsingular transformations as in (3) is the same as using the extended core matrix as data for PARAFAC extracting as many factors as there are components for/4 and
B, as we will show below.
Suppose we have data which perfectly fit the PARAFAC model
Zk = ADkB ' (4)
with/1 and B not necessarily orthogonal, and Dk diagonal.
Let ULV' be the Singular Value Decomposition (SVD) of A and P<t>Q the SVD of B. Substitution into (4) gives
Zk = UAV'DkQ4>P' (5)
By defining A* = U,B* = P, and G\ = AV DkQ<b (5) can be written as
exist, and diagonal matrices D\ = D* such that
In other words, the perfect Tucker2 solution can be transformed by a PARAFAC decomposition of the extended core matrix into an overall PARAFAC solution. The uniqueness of the nonsingular transformation of the extended core matrix follows from the uniqueness of the overall PARAFAC solution itself. (Sec Harshman 1972).
Both PARAFAC and Tucker solutions, ZPARAFNC and Z-Tucten may be
seen as subspaces of the space spanned by the data Z. In case of perfect PARAFAC data both subspaces are identical to Z. If the data^ perfectly fit Tucker2 but not PARAFAC with the same dimensionality then Zp^^c is a real subspace of both Z and Zfucker. In general, if ZPAKAfAc £ ^Tucker wc may
write
Z Tucker = ZpARAFAC + £/MRAMC. ZpAK/y.*Ac n E PARAFAC = 0
From this, from the proof given before, and from the uniqueness theorem, it follows that applying PARAFAC to the extended core matrix of TUCKALS2 will produce the same solution as PARAFAC on the data itself for the Z PARAFAC part and no solution for the E PARAFAC part. In other words, given that an overall PARAFAC solution exists, and that the corresponding PARAFAC space is a subspace of the Tucker space, the same overall solution will be found when PARAFAC is applied to the extended core matrix, even with real, noisy data.
If the PARAFAC fit comes close to the TUCKALS2 fit then off-diagonal elements of the core matrix will be (near) zero, and the interpreta-tion can be a fully-Hedged PARAFAC one. If the off-diagonal elements do not vanish, and ipso facto the PARAFAC fit is significantly lower than the TUCKALS2 fit, a PARAFAC interpretation may be less appropriate.
3. Numerical Results
In this section we will illustrate the theoretical results above using the Tongue Shape data published in Harshman, Ladcfoged, and Goldstein (1977).
The data consist of the displacement of 13 points of the tongue (mode B), measured during the enunciation of 10 English vowels (mode A) by 5 speakers (mode C). We used four versions of these data:
Notes on Diagonalization 97
TABLE 1
Tongue Shape Data: Comparison of the Fit of TUCKALS2 plus Core Diagonalization with that of PARAFAC
Overall Data set 1. 1. 3. /. . TS--2f TS--2p TS--2e TS--3p 0 1 d 1 T2+CD .9260 .0000 .8539 .0000 PAR. 0.9262 1.0000 0.8539 1.0000 0 0 0 0 Fit Factor 1 T2+CD .4735 .4628 .4494 .4623 PAR. 0.4753 0.4627 0.4502 0.4623 Factor T2+CD 0 0 n 0 .3691 0 .3696 0 .3984 0 .3376 0 2 PAR. .3684 .3696 .3980 .3377 Factor 3 T2+CD PAR. 0.0956 0.0956 Note: The fit per factor is expressed as its root mean squared contribution to the solution.
2. TS - 2p, perfect data, reconstructed from a two-factor PARAFAC solution of 1;
3. TS - 2e, data obtained by adding uniformly distributed error to 2; 4. TS - 3p, perfect data, reconstructed from a three-factor PARAFAC
solution of 1.
Table 1 shows in paired columns the overall fit and the fit per factor for the TUCKALS analysis + core diagonalixation (T2 + CD; left-hand column) and PARAFAC (PAR.; right-hand column).
4. Discussion
Both theoretical and empirical results show that it can be fruitful to investigate the transformation to diagonality of the extended core matrix of the Tuckcr2 model after a TUCKALS2 solution has been obtained. The increased parsimony of a diagonal core matrix and the unique orientation of components associated with the diagonal core matrix support this contention. Moreover, in some of the data sets examined, it was possible to find a diago-nal core matrix within an essentially nondiagodiago-nal one. This allows the possi-bility of assessing how and where the core matrix is not diagonal. There is no guarantee, as is the case for PARAFAC itself, that a solution can be found.
References
CARROLL, J. D., and CHANG, J. J. (1970). "Analysis of Individual Differences in Multidi-mensional Scaling via an N-way Gcnerali/.ation of "Eckart - Young" Decomposition,"
Psychometrika, 35, 283-319.
CARROLL, J. D., and Wish, M. (1974), "Models and Methods for Three-way Multidimen-sional Scaling," in Contemporary Developments in Mathematical Psychology (Vol. II), Eds. D. H. Krant/., R. C. Atkinson, R. D. Luce, P. Suppes, San Francisco: Freeman, 57-105.
COHEN, L. (1974), "Three-mode Rotation to Approximate INDSCAL Structure (TRIAS)," Paper presented at the Psychometric Society Meeting, Palo Alto, CA.
DE LEEUW, J., and Pruzansky, S. (1978), "A New Computational Method to Fit the Weighted Euclidean Distance Model," Psychometrika, 43, 479-490.
HARSHMAN, R. A. (1970), "Foundations of the PARAFAC Procedure: Models and Condi-tions for an "Explanatory" Multi-mode Factor Analysis," UCLA Working Papers in
Phonelis, 16, 1-84 (Xerox University Microfilms, Ann Arbor, MI; 10,085).
HARSHMAN, R. A. (1972), "Determination and Proof of Minimum Uniqueness Conditions for PARAFAC1," UCLA Working Papers in Phonetics, 22, 111-117 (Xerox University Microfilms, Ann Arbor, MI; 10,085).
HARSHMAN, R. A., LADEFOGED, P., and GOLDSTEIN, L. (1977), "Factor Analysis of Tongue Shapes," Journal of the Acoustical Society of America, 62, 693-707.
HARSHMAN, R. A., and LUNDY, M. E. (1984), "The PARAFAC Model for Three-way Fac-tor Analysis and Multidimensional Scaling," in Research Methods for Mullimode Data
Analysis, Eds. H. G. Law, C. W. Snydcr Jr., J. A. Hattic, and R. P. McDonald, New York:
Pracger, 122-215.
KIERS, H. A. L. (1988), "Comparison of "Anglo-Saxon" and "French" Three-mode Methods, Statistique et Analyse des Données, 13, 14-32.
KROONENBERG, P. M. (1983), Three-mode Principal Component Analysis: Theory and
Applications, Leiden, The Netherlands: DSWO Press.
KROONENBERG, P. M., and DE LEEUW, J. (1980), "Principal Component Analysis of Three-mode Data by Means of Alternating Least Squares Algorithms," Psychomelrika,
45, 69-97.
TUCKER, L. R (1966), "Some Mathematical Notes on Three-mode Factor Analysis,"
Psychometrika, 31, 279-311.
