!
Vol. 36, No. 2, pp. 496–522
COUPLED CANONICAL POLYADIC DECOMPOSITIONS AND (COUPLED) DECOMPOSITIONS IN MULTILINEAR
RANK-(Lr,n, Lr,n, 1) TERMS—PART I: UNIQUENESS∗
MIKAEL SØRENSEN† AND LIEVEN DE LATHAUWER†
Abstract. Coupled tensor decompositions are becoming increasingly important in signal pro-cessing and data analysis. However, the uniqueness properties of coupled tensor decompositions have not yet been studied. In this paper, we first provide new uniqueness conditions for one factor matrix of the coupled canonical polyadic decomposition (CPD) of third-order tensors. Then, we present necessary and sufficient overall uniqueness conditions for the coupled CPD of third-order tensors. The results demonstrate that improved uniqueness conditions can indeed be obtained by taking into account the coupling between several tensor decompositions. We extend the results to higher-order tensors and explain that the higher-order structure can further improve the uniqueness results. We discuss the special case of coupled matrix-tensor factorizations. We also present a new variant of the coupled CPD model called the coupled block term decomposition (BTD). On one hand, the cou-pled BTD can be seen as a variant of coucou-pled CPD for the case where the common factor contains collinear columns. On the other hand, it can also be seen as an extension of the decomposition into multilinear rank-(Lr, Lr, 1) terms to coupled factorizations.
Key words. coupled decompositions, higher-order tensor, parallel factor (PARAFAC), canonical decomposition (CANDECOMP), canonical polyadic decomposition, coupled matrix-tensor factoriza-tion
AMS subject classifications. 15A22, 15A23, 15A69 DOI. 10.1137/140956853
1. Introduction. The coupled canonical polyadic decomposition (CPD) model
(formally defined in subsection 4.1) seems to have been first used in psychometrics
[21,22] as a way of integrating several three-way studies that involve the same stimuli and as a means of coping with missing data in coupled data sets. The technique was also later considered in chemometrics [36]. In recent years coupled canonical polyadic decompositions have had a resurgence in several engineering disciplines. We mention data mining, where they are used as an explorative tool for finding structure in coupled
data sets [3,1], and bioinformatics, where they are used as a tool for fusion of data
obtained by different analytical methods such as nuclear magnetic resonance and
fluorescence spectroscopy [32,48]. In chemometrics it has been suggested that coupled
matrix-tensor factorizations can be used to fuse data obtained by different analytic methods [2]. We also mention that in biomedical engineering several multisubject or data fusion methods that combine different modalities (fMRI, EEG, MEG, etc.) ∗Received by the editors February 13, 2014; accepted for publication (in revised form) by
D. P. O’Leary February 17, 2015; published electronically May 7, 2015. This research was supported by Research Council KU Leuven under GOA/10/09 MaNet and CoE PFV/10/002 (OPTEC); F.W.O. under project G.0427.10, G.0830.14N, G.0881.14N; the Belgian Federal Science Policy Office under IUAP P7 (DYSCO II, Dynamical Systems, Control and Optimization, 2012–2017); and the Euro-pean Research Council under the EuroEuro-pean Union’s Seventh Framework Programme (FP7/2007– 2013)/ERC Advanced Grant BIOTENSORS (339804). This paper reflects only the authors’ views and the Union is not liable for any use that may be made of the contained information.
http://www.siam.org/journals/simax/36-2/95685.html
†Group Science, Engineering and Technology, KU Leuven - Kulak, 8500 Kortrijk,
Bel-gium, and STADIUS Center for Dynamical Systems, Signal Processing and Data Analyt-ics, and iMinds Department Medical Information Technologies, Departement Elektrotechniek (ESAT), KU Leuven, B-3001 Leuven-Heverlee, Belgium (Mikael.Sorensen@kuleuven-kulak.be,
Lieven.DeLathauwer@kuleuven-kulak.be).
496
can be interpreted as coupled CPD problems [19, 28, 9, 20, 29, 4]. Despite their importance, to the best of our knowledge, no algebraic studies of coupled tensor decompositions have been provided so far. In particular, no dedicated uniqueness conditions for coupled CPD problems are available.
Several problems in signal processing involve polyadic decompositions that have factor matrices with collinear columns. A particular case is of block term decompo-sitions, which are decompositions of a tensor in terms of low multilinear rank [13].
We mention applications in array processing [34, 38, 39], wireless communication
[35,10,12,31,37], and blind separation of signals that can be modeled as exponen-tial polynomials [14]. There are also applications in chemometrics [6]. Hence, in the study of the coupled CPD model we should pay special attention to collinearity.
The rest of the introduction presents our notation. Sections2and3briefly review
the CPD and the decomposition into multilinear rank-(Lr, Lr, 1) terms. In section 4
we introduce the coupled CPD and study its uniqueness properties. The results are (i) necessary coupled CPD uniqueness conditions, (ii) sufficient uniqueness conditions for the common factor matrix of the coupled CPD, (iii) sufficient overall uniqueness conditions for the coupled CPD, (iv) extensions to tensors of arbitrary order, and (v) a discussion of the uniqueness properties of the coupled matrix-tensor
factoriza-tion. Section 5 discusses a new coupled CPD model in which the common factor
matrix contains collinear components. The paper is concluded in section6.
1.1. Notation. Vectors, matrices, and tensors are denoted by lowercase bold-face, uppercase boldbold-face, and uppercase calligraphic letters, respectively. The rth
column vector of A is denoted by ar. The symbols⊗ and " denote the Kronecker
and Khatri–Rao product, defined as
A⊗ B := a11B a12B . . . a21B a22B . . . .. . ... . .. , A " B :=' a1⊗ b1 a2⊗ b2 . . . (,
in which (A)mn = amn. The outer product of N vectors a(n) ∈ CIn is denoted by
a(1)◦ a(2)◦ · · · ◦ a(N )∈ CI1×I2×···×IN, such that
)
a(1)◦ a(2)◦ · · · ◦ a(N )*
i1,i2,...,iN
= a(1)i1 a(2)i2 · · · a(N )iN .
The identity matrix, all-zero matrix, and all-zero vector are denoted by IM∈ CM×M,
0M,N ∈ CM×N, and 0M ∈ CM, respectively. The all-ones vector is denoted by
1R= [1, . . . , 1]T∈ CR.
The transpose, Moore–Penrose pseudo-inverse, Frobenius norm, determinant,
range, and kernel of a matrix are denoted by (·)T, (·)†, % · %F, |·|, range (·), and
ker (·), respectively. The cardinality of a set S is denoted by card (S).
MATLAB index notation will be used for submatrices of a given matrix. For example, A(1 : k, :) represents the submatrix of A consisting of the rows from 1 to k
of A. Dk(A)∈ CJ×J denotes the diagonal matrix holding row k of A∈ CI×J on its
diagonal. Given A∈ CI×J, Vec (A)∈ CIJ will denote the column vector defined by
(Vec (A))i+(j−1)I= (A)ij.
The matrix that orthogonally projects onto the orthogonal complement of the
column space of A∈ CI×J is denoted by
PA= II− FFH∈ CI×I,
498 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER
where the column vectors of F constitute an orthonormal basis for range (A).
The Heaviside step function H :Z → {0, 1} is defined as
H[n] = +
0 , n < 0 ,
1 , n≥ 0 .
The rank of a matrix A is denoted by r (A) or rA. The k-rank of a matrix A
is denoted by k (A) or kA. It is equal to the largest integer k (A) such that every
subset of k (A) columns of A is linearly independent. More generally, the k$-rank of
a partitioned matrix A is denoted by k$(A). It is equal to the largest integer k$(A)
such that any set of k$(A) submatrices of A yields a set of linearly independent
columns. The number of nonzero entries of a vector x is denoted by ω (x) in the tensor decomposition literature, dating back to the work of Kruskal [26].
Let Ck
n= k!(n−k)!n! denote the binomial coefficient. The kth compound matrix of
A∈ Cm×nis denoted by C
k(A)∈ CC
k
m×Cnk and its entries correspond to the k-by-k
minors of A, ordered lexicographically. As an example, let A∈ C4×3; then
C2(A) = |A ([1, 2], [1, 2])| |A ([1, 2], [1, 3])| |A ([1, 2], [2, 3])| |A ([1, 3], [1, 2])| |A ([1, 3], [1, 3])| |A ([1, 3], [2, 3])| |A ([1, 4], [1, 2])| |A ([1, 4], [1, 3])| |A ([1, 4], [2, 3])| |A ([2, 3], [1, 2])| |A ([2, 3], [1, 3])| |A ([2, 3], [2, 3])| |A ([2, 4], [1, 2])| |A ([2, 4], [1, 3])| |A ([2, 4], [2, 3])| |A ([3, 4], [1, 2])| |A ([3, 4], [1, 3])| |A ([3, 4], [2, 3])| .
See [23,15] for discussion of compound matrices.
2. Canonical polyadic decomposition. Consider the third-order tensorX ∈
CI×J×K. We say thatX is a rank-1 tensor if it is equal to the outer product of some
nonzero vectors a∈ CI, b∈ CJ, and c∈ CKsuch that x
ijk= aibjck. Decompositions
into a sum of rank-1 terms are called polyadic decompositions (PDs): X = R , r=1 ar◦ br◦ cr. (2.1)
The rank of a tensorX is equal to the minimal number of rank-1 tensors that yield
X in a linear combination. Assume that the rank of X is R; then (2.1) is called
the canonical PD (CPD) of X . The CPD is also known as the PARAllel FACtor
(PARAFAC) [22] and the CANonical DECOMPosition (CANDECOMP) [7]. Let us
stack the vectors{ar}, {br}, and {cr} into the matrices
A = [a1, . . . , aR]∈ CI×R, B = [b1, . . . , bR]∈ CJ×R, C = [c1, . . . , cR]∈ CK×R.
The matrices A, B, and C will be referred to as the factor matrices of the CPD in (2.1). The following subsection presents matrix representations of (2.1) that will be used throughout the paper.
2.1. Matrix representations. Let X(i··)∈ CJ×K denote the matrix such that
(X(i··))jk= xijk; then X(i··)= BDi(A) CT and
CIJ×K ) X
(1):=
-X(1··)T, . . . , X(I··)T.T = (A" B) CT.
(2.2)
More generally, the PD or CPD of the higher-order tensor X ∈ CI1×···×IM has
the matrix representations
(2.3) X(w)= 1 p∈Γw A(p)" 1 q∈Υw A(q) 4 1 r∈Ψw A(r) 5T ,
where A(m)∈ CIm×Rand the sets Γ
w, Υw, and Ψwhave properties Γw6Υw6Ψw=
{1, 2, . . . , M}, Γw7Υw=∅, Γw7Ψw=∅, and Υw7Ψw=∅.
2.2. Uniqueness conditions for one factor matrix of a CPD. A factor
matrix, say C, of the CPD ofX ∈ CI×J×K is said to be unique if it can be
deter-mined up to the inherent column scaling and permutation ambiguities fromX . More
formally, the factor matrix C is unique if all the triplets ( 8A, 8B, 8C) satisfying (2.1)
also satisfy the condition
8
C = CP∆ ,
where P is a permutation matrix and ∆ is a diagonal matrix. One of the first unique-ness conditions for one factor matrix of a CPD was obtained by Kruskal in [26]. In this paper we will make use of the following result.
Theorem 2.1. Consider the PD ofX ∈ CI×J×K in (2.1). If
k (C)≥ 1, min (I, J)≥ R − r (C) + 2,
CR−r(C)+2(A)" CR−r(C)+2(B) has full column rank, (2.4)
then the rank ofX is R and the factor matrix C is unique [15].
Condition (2.4) is more relaxed than Kruskal’s, and the proof of the theorem admits a constructive interpretation [17].
2.3. Overall uniqueness conditions for CPD. The rank-1 tensors in (2.1) can be arbitrarily permuted without changing the decomposition. The vectors within the same 1 tensor can also be arbitrarily scaled provided that the overall rank-1 term remains the same. We say that the CPD is unique when it is only subject to the mentioned indeterminacies. One of the first deep CPD uniqueness results was obtained by Kruskal [26]. For a recent comprehensive study of CPD uniqueness in the
third-order case we refer the reader to [15,16]. Below we state some uniqueness results
for CPD that we will extend to the coupled CPD case. The results are summarized
in Table1.
Table 1
Full column rank (f.c.r.) requirements for different CPD uniqueness conditions. In the case where C has f.c.r., we further distinguish between a sufficient (S) and a necessary and sufficient (N and S) condition.
Thm. 2.2 Thm.2.3 Thm. 2.4 Thm.2.5
Matrices required to have f.c.r. None C C C and A
Condition S N and S S N and S
Together with related results in [16], the following is one of the most relaxed deterministic conditions for CPD uniqueness. It does not require any of the factor matrices to have full column rank.
500 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER
Theorem 2.2. Consider the PD ofX ∈ CI×J×Kin (2.1). Let S denote a subset
of {1, . . . , R} and let Sc ={1, . . . , R} \ S denote the complementary set. Stack the
columns of C with index in S in C(S) ∈ CK×card(S) and stack the columns of C
with index in Sc in C(Sc)
∈ CK×(R−card(S)). Stack the columns of A (resp., B) in
the same order such that A(S) ∈ CI×card(S) (resp., B(S)
∈ CJ×card(S)) and A(Sc)
∈ CI×(R−card(S))(resp., B(S)
∈ CJ×(R−card(S))) are obtained. If =
k (C)≥ 2,
r (CR−rC+2(A)" CR−rC+2(B)) = C
R−rC+2
R ,
and if there exists a subset S of{1, . . . , R} with 0 ≤ card (S) ≤ rC such that1, 2
C(S) has full column rank (rC(S)= card (S)) ,
B(Sc) has full column rank (r
B(Sc )= R− card (S)) , r)-PC(S)C(S c )" A(Sc), P C(S)c(S c) r ⊗ II .* = I + R− card (S) − 1 ∀r ∈ Sc,
then the rank ofX is R and the CPD of X is unique [40].
If one factor matrix has full column rank, say C, then the following condition is not only sufficient but also necessary.
Theorem 2.3. Consider the PD of X ∈ CI×J×K in (2.1). Define E(w) =
>R
r=1wrarb T
r. Assume that C has full column rank. The rank of X is R and the
CPD of X is unique if and only if [42,25,46,14]
r (E(w))≥ 2 ∀w ∈?x∈ CR@@ ω(x) ≥ 2A.
(2.5)
Generically,3 condition (2.5) is satisfied and C has full column rank if R≤ K and
R≤ (I − 1)(J − 1) [42].
In practice, condition (2.5) may not be easy to check. Instead we may resort to the following more convenient result in the case where one factor matrix has full column rank.
Theorem 2.4. Consider the PD ofX ∈ CI×J×K in (2.1). If
=
C has full column rank,
C2(A)" C2(B) has full column rank,
(2.6)
then the rank of X is R and the CPD of X is unique [25, 11,46, 15]. Generically,
condition (2.6) is satisfied if R≤ K and 2R(R − 1) ≤ I(I − 1)J(J − 1) [11,43].
In the case where two factor matrices, say A and C, have full column rank,
Theorems2.3and2.4simplify to the following.
Theorem 2.5. Consider the PD of X in (2.1). Assume that A and C have full
column rank. The rank of X is R and the CPD of X is unique if and only if kB≥ 2
(see, e.g., [27]). Generically, this is satisfied if R≤ min(I, K) and 2 ≤ J.
1Note that the set S in Theorem2.2may be empty, i.e., card (S) = 0 such that S =∅. This
corresponds to the case where PC(S)= IK. 2The last condition states that M
r = [PC(S)C(S c) " A(Sc), P C(S)c(S c) r ⊗ II] has a
one-dimensional kernel for every r ∈ Sc, which is minimal since [nT r, a
(Sc)T
r ]T ∈ ker (Mr) for some
nr∈ Ccard(Sc).
3A tensor decomposition property is called generic if it holds with probability one when the
entries of the factor matrices are drawn from absolutely continuous probability density measures.
3. CPD with collinearity in a factor matrix. We consider PDs of X ∈
CI×J×K that involve collinearities in the factor matrix C of the type
X = R , r=1 Lr , l=1 a(r)l ◦ b(r)l ◦ c(r)= R , r=1 ) A(r)B(r)T*◦ c(r), (3.1) where A(r) = [a(r)1 , . . . , a(r)Lr]∈ CI×Lr, B(r) = [b(r) 1 , . . . , b (r) Lr]∈ C J×Lr. Similarly to
A(r) and B(r), we may define C(r) = 1T
Lr⊗ c
(r) ∈ CK×Lr, i.e., column vector c(r)
is repeated Lr times. Note that, if Lr ≥ 2 for some r ∈ {1, . . . , R}, then the PD of
X cannot be unique (see, e.g., [44]). In cases like this, it is impossible to recover the
individual columns of the factors A(r)and B(r). If the matrices A(r)B(r)T have rank
Lr, then the decomposition (3.1) is also known as the decomposition into multilinear
rank-(Lr, Lr, 1) terms [13].
3.1. Matrix representation. Let us stack the above matrices and vectors into the matrices A =-A(1), . . . , A(R).∈ CI×(!Rr=1Lr), B = -B(1), . . . , B(R).∈ CJ×(!Rr=1Lr), C =-C(1), . . . , C(R).∈ CK×(!R r=1Lr), C(red) =-c(1), . . . , c(R).∈ CK×R,
where “red” stands for reduced. The PD or CPD of the tensorX in (3.1) with collinear
columns in C admits the following matrix representation:
CIJ×K) X (1)= -X(1··)T, . . . , X(I··)T.T = (A" B) CT (3.2)
=-Vec)B(1)A(1)T*, . . . , Vec)B(R)A(R)T*.C(red)T.
(3.3)
3.2. Overall uniqueness conditions for decomposition into multilinear
rank-(Lr, Lr, 1) terms. Let {{ 8A
(n)
}, { 8B(n)}, 8C} yield an alternative
decomposi-tion of X into multilinear rank-(Lr, Lr, 1) terms. The multilinear rank-(Lr, Lr, 1)
tensors in (3.1) can be arbitrarily permuted, and the vectors within the same
cou-pled multilinear rank-(Lr, Lr, 1) tensor can be arbitrarily scaled provided the overall
coupled multilinear rank-(Lr, Lr, 1) term remains the same. We say that the
decom-position into multilinear rank-(Lr, Lr, 1) terms is unique when it is only subject to
the mentioned indeterminacies.
The following uniqueness condition for decomposition ofX into multilinear
rank-(Lr, Lr, 1) terms has been obtained in [13].
Theorem 3.1. Consider the PD ofX ∈ CI×J×K in (3.1). If
k$(A) = R and k$(B) + k (C)≥ R + 2 ,
(3.4)
then the minimal number of multilinear rank-(Lr, Lr, 1) terms is R and the
decompo-sition of X into multilinear rank-(Lr, Lr, 1) terms is unique.
Other related uniqueness results can be found in [13]. For the case where C has full column rank, the following necessary and sufficient uniqueness condition for
decomposition ofX into multilinear rank-(Lr, Lr, 1) terms has been obtained in [14].
Theorem 3.2. Consider the PD of X ∈ CI×J×K in (3.1). Define E(w) =
>R
r=1wrA
(r)B(r)T. Assume that C has full column rank. A necessary and sufficient
502 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER
X
(1)=
c1 a(1)1 b(1)1 +· · · + cR a(1)R b(1)R..
.
X
(N )=
c1 a(N )1 b(N )1 +· · · + cR a(N )R b(N )RFig. 1. Coupled PD of the third-order tensorsX(1), . . . ,X(N).
condition for uniqueness of the decomposition of X into multilinear rank-(Lr, Lr, 1)
terms is that r (E(w)) > max r|wr&=0 Lr ∀w ∈ ? x∈ CR@@ ω(x) ≥ 2A. (3.5)
Generalizing CPD results in [8], generic uniqueness bounds for the BTD have been obtained in [50].
4. New results for coupled CPD. In subsection4.1we introduce some
defini-tions and notation associated with the coupled CPD. Subsection4.2presents necessary
conditions for coupled CPD uniqueness. Subsection4.3presents uniqueness conditions
for the common factor matrix. In subsection4.4we develop sufficient uniqueness
con-ditions for the coupled CPD. Subsection 4.5briefly explains that the results can be
extended to tensors of order greater than three. Subsection 4.6 comments on the
coupled matrix-tensor factorization problem.
4.1. Definitions and notation. We say that a collection of tensors X(n) ∈
CIn×Jn×K, n∈ {1, . . . , N}, admits an R-term coupled polyadic decomposition if each
tensorX(n)can be written as
X(n)= R , r=1 a(n)r ◦ b(n)r ◦ cr, n∈ {1, . . . , N}, (4.1)
with factor matrices
A(n)=- a(n)1 , . . . , a(n)R . ∈ CIn×R, n∈ {1, . . . , N}, B(n)=- b(n)1 , . . . , b(n)R . ∈ CJn×R, n∈ {1, . . . , N}, C =' c1, . . . , cR ( ∈ CK×R.
The coupled PD of the third-order tensors{X(n)} is visualized in Figure1.
We define the coupled rank of{X(n)} as the minimal number of coupled rank-1
tensors that yield {X(n)} in a linear combination. Assume that the coupled rank of
{X(n)} is R; then (4.1) will be called the coupled CPD of{X(n)}.
It is clear that the coupled rank-1 tensors in (4.1) can be arbitrarily permuted and that the vectors within the same coupled rank-1 tensor can be arbitrarily scaled provided the overall coupled rank-1 term remains the same. We say that the coupled CPD is unique when it is only subject to these trivial indeterminacies.
In this paper we will make use of the matrix representation of{X(n)},
X = X(1)(1) .. . X(N )(1) = A(1)" B(1) .. . A(N )" B(N ) CT= FCT∈ C(!N n=1InJn)×K, (4.2) where F = A(1)" B(1) .. . A(N )" B(N ) ∈ C(!Nn=1InJn)×R. (4.3)
4.2. Necessary conditions for coupled CPD uniqueness. Propositions4.1
and4.2following generalize well-known necessary uniqueness conditions for CPD (see,
e.g., [30,44]) to the coupled CPD case.
Proposition 4.1. If the coupled CPD of{X(n)} in (4.1) is unique, then k
C≥ 2.
Proof. Assume that k (C) = 1, say c1and c2are collinear; then linear
combina-tions of c1and c2will yield an alternative coupled CPD of{X(n)} that is not related
via trivial column scaling and permutation ambiguities.
Note that in contrast to ordinary CPD, Proposition 4.1 does not prevent that
kA(n)= 1 and/or kB(n) = 1 for some n∈ {1, . . . , N}. Indeed, the coupled CPD may
be unique in such cases, as will be explained in subsection4.4.
Proposition 4.2. If the coupled CPD of {X(n)} in (4.1) is unique, then F has
full column rank.
Proof. The result follows directly from relation (4.2). Indeed, if F does not have
full column rank, then for any x ∈ ker (F) we obtain X = FCT = F(CT + xyT),
where y∈ CK.
Again, in contrast to ordinary CPD, Proposition 4.2 does not prevent that for
some n ∈ {1, . . . , N} the individual Khatri–Rao product matrices A(n)" B(n) are
rank deficient. This will be further discussed in subsection4.4.
It is well known that the condition kC≥ 2 is generically satisfied if K ≥ 2. Based
on Lemma4.3we explain in Proposition4.4that F generically has full column rank
if >Nn=1InJn ≥ R. Hence, the necessary conditions stated in Propositions 4.1and
4.2are expected to be satisfied under mild conditions.
Lemma 4.3. Given an analytic function f : Cn→ C, if there exists an element
x∈ Cn such that f (x)-= 0, then the set { x | f (x) = 0 } is of Lebesgue measure zero
(see, e.g., [24]).
Proposition 4.4. Consider F ∈ C(!N
n=1InJn)×R given by (4.3). For generic
matrices{A(n)} and {B(n)}, the matrix F has rank min(>Nn=1InJn, R).
Proof. Due to Lemma4.3we just need to find one example where the statement
made in this lemma holds. We give an example in the supplementary material. Another necessary condition for CPD uniqueness is that none of the column
vectors of A" B (similarly for A " C and B " C) in (2.2) can be written as linear
504 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER
combinations of its remaining column vectors [15,14]. Proposition4.5extends the
result to coupled CPD.
Proposition 4.5. Consider the coupled PD ofX(n)∈ CIn×Jn×K, n∈ {1, . . . , N},
in (4.1). Define E(n)(w) = R , r=1 wra(n)r b(n)Tr and Ω = ? x∈ CR@@ ω(x) ≥ 2A. (4.4)
If the coupled CPD of{X(n)} in (4.1) is unique, then
∀w ∈ Ω ∃ n ∈ {1, . . . , N} : r)E(n)(w)*≥ 2 .
(4.5)
Proof. The necessity of r (F) = R has already been mentioned in Proposition4.2.
Assume now that there exists a vector w(r) ∈ CR with ω(w(r)) ≥ 2 such that for
some r∈ {1, . . . , R} we have (4.6) Ba(n)r ⊗ Bb (n) r = R , s=1 w(r) s ) a(n) s ⊗ b(n)s * ∀n ∈ {1, . . . , N} .
Since F has full column rank, its column vectors are linearly independent, that is, > s&=rw (r) s (a(n)s ⊗ b(n)s ) cannot be proportional to a (n) r ⊗ b(n)r for all n∈ {1, . . . , N}, and consequentlyBa(n)r ⊗ Bb (n) r is not proportional to a (n) r ⊗ b(n)r for all n∈ {1, . . . , N}.
This means that factor matrices {{ BA(n)}, { BB(n)}, BC} with property (4.6) yield an
alternative coupled CPD of {X(n)} which is not related to {{A(n)
}, {B(n)}, C} via
the intrinsic column scaling and permutation ambiguities.
In contrast to ordinary CPD, Proposition 4.5 does not prevent that for some
n∈ {1, . . . , N} the individual columns of the matrices A(n)" B(n)may be written as
linear combinations of its remaining column vectors.
4.3. Uniqueness conditions for common factor matrix. This subsection presents conditions that guarantee the uniqueness of the common factor C of the
coupled CPD of {X(n)} in (4.1), even in cases where some of the remaining factor
matrices{A(n)} and {B(n)} contain all-zero column vectors. This is in contrast with
ordinary CPD where kA(n)≥ 2 and kB(n)≥ 2 are necessary conditions.
Proposition4.6is a variant of Theorem2.1for coupled CPD.
Proposition 4.6. Consider the coupled PD ofX(n)∈ CIn×Jn×K, n∈ {1, . . . , N},
in (4.1). W.l.o.g. we assume that min(I1, J1)≥ min(I2, J2)≥ · · · ≥ min(IN, JN).
De-note Q =>Nn=1H [min (In, Jn)− R + rC− 2], where H [·] denotes the Heaviside step
function. Define G(m)= Cm ) A(1)*" C m ) B(1)* .. . Cm ) A(Q)*" C m ) B(Q)* ∈ C( !Q n=1CmInCmJn)×CmR, (4.7) where m = R− rC+ 2. If = k (C)≥ 1, r(G(m)) = Cm R, (4.8)
then the coupled rank of{X(n)} is R and the factor matrix C is unique.
Proof. The result is a technical variant of [15, Proposition 4.3]. It is provided in the supplementary material.
In the case that the common factor matrix C has full column rank, Proposition
4.6directly reduces to the following result. (Compare to Theorem2.4.)
Corollary 4.7. Consider the coupled PD ofX(n)∈ CIn×Jn×K, n∈ {1, . . . , N},
in (4.1). Let G(2) be defined as in (4.7). If
=
C has full column rank,
G(2) has full column rank,
(4.9)
then the coupled rank of{X(n)} is R and the factor matrix C is unique.
If additionally some of the factor matrices in the set{A(n)} also have full column
rank, then Corollary4.7further reduces to the following result. (Compare to Theorem
2.5.)
Corollary 4.8. Consider the coupled PD of {X(n)} in (4.1). Consider also a
subset S of{1, . . . , N} with card (S) = Q. W.l.o.g., we assume that S = {1, . . . , Q}.
If for some Q∈ {1, . . . , N}, we have
(4.10) rC= R , rA(n)= R ∀n ∈ {1, . . . , Q}, ∀r ∈ {1, . . . , R}, ∀s ∈ {1, . . . , R} \ r, ∃ n ∈ {1, . . . , Q} : k)-b(n)r , b(n)s .*= 2 ,
then the coupled rank of{X(n)} is R and the factor matrix C is unique.
Proof. Due to Corollary4.7we know that the coupled rank of{X(n)} is R and
the factor matrix C is unique. We assume that for some Q∈ {1, . . . , N} the matrix
(4.11) G(2)=C)C2 ) A(1)*" C2 ) B(1)**T, . . . ,)C2 ) A(Q)*" C2 ) B(Q)**T DT
has full column rank. As in ordinary CPD [47], we can premultiply each A(n) by a
nonsingular matrix without affecting the rank or the uniqueness of the coupled CPD
of{X(n)}. Hence, w.l.o.g. we can set A(n)='I
R, 0TIn−R,R
(T
. Likewise, as in ordinary
CPD [45], the premultiplication of A(n)by a nonsingular matrix does not affect the
rank of G(2). The problem of determining the rank of G(2) reduces to finding the
rank of H = C2 EC IR 0I1−R,R DF " C2 ) B(2)* .. . C2 EC IR 0IQ−R,R DF " C2 ) B(Q)* .
After removing the all-zero row-vectors of H we need to find the rank of B H =C)IR(R−1) 2 " C2 ) B(1)**T, . . . ,)IR(R−1) 2 " C2 ) B(Q)**T DT .
506 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER Note that C2(B(n)) = [d(n)1,2, . . . , d(n)1,R, d (n) 2,3, . . . , d(n)2,R, . . . , d (n) R−2,R−1, d(n)R−2,R, d(n)R−1,R],
where d(n)p,q = C2([b(n)p , bq(n)]) ∈ CJn(Jn−1)/2. Note also that BH corresponds to a
row-permuted version of a block-diagonal matrix holding the column vectors {Bdp,q}
defined as Bdp,q = [d(1)Tp,q , . . . , d(N )Tp,q ]T ∈ C(
!N
n=1Jn(Jn−1)/2) on its block-diagonal. It
is now clear that BH has full column rank if for every pair (r, s)∈ {1, . . . , R}2 with
r -= s there exists an n ∈ {1, . . . , Q} such that ω(d(n)r,s) ≥ 1. Equivalently, for every
pair (r, s) ∈ {1, . . . , R}2 with r-= s there should exist an n ∈ {1, . . . , Q} such that
k([b(n)r , b(n)s ]) = 2.
In the case where C has full column rank we have the following necessary and
sufficient uniqueness condition for the common factor. (Compare to Theorem2.3.)
Proposition 4.9. Consider the coupled PD ofX(n)∈ CIn×Jn×K, n∈ {1, . . . , N},
in (4.1). Define E(n)(w) and Ω as in (4.4). Assume that C has full column rank. The
coupled rank of{X(n)} is R and the factor matrix C is unique if and only if condition
(4.5) is satisfied.
Proof. The necessity of condition (4.5) has already been demonstrated in
Propo-sition4.5. Let us now prove the sufficiency of condition (4.5) in the case where C has
full column rank. Note that (4.5) implies that F has full column rank. Indeed, if F
is rank deficient, then there exists a vector x∈ CR with property ω(x)≥ 2 such that
>R
r=1xrfr= 0. This will contradict (4.5). Let{{ BA
(n)
}, { BB(n)}, BC} denote the factor
matrices of an alternative coupled CPD ofX(n), n∈ {1, . . . , N}, where BA(n)∈ CIn× "R,
B
B(n)∈ CJn× "R, and BC∈ CK× "R with BR≤ R. Further, let BF∈ C(!Nn=1InJn)× "Rdenote
the alternative F constructed from{ BA(n)} and { BB(n)} such that
(4.12) X = FCT= BF BCT.
Since F has full column rank and C has full column rank by assumption, we know
from (4.12) that R = BR, that BC and BF have full column rank, and that range (C) =
range)CB*.
We obtain from (4.12) the relation
(4.13) FH = BF,
where H = CT( BCT)†∈ CR×Ris nonsingular. This may be expressed in a columnwise
manner as (4.14) Ba(n)r ⊗ Bb (n) r = R , s=1 hsr ) a(n) s ⊗ b(n)s * , r∈ {1, . . . , R}, n ∈ {1, . . . , N} .
Combination of (4.5) and (4.14) now yields that the nonsingular matrix H has exactly
one nonzero entry in every column. This implies that H = PD, where P∈ CR×Ris a
permutation matrix and D∈ CR×Ris a nonsingular diagonal matrix. From (4.12) we
obtain that BC = CPD−1. We conclude that the common factor C is unique.
While Proposition 4.9provides a necessary and sufficient condition for the case
where C has full column rank, Corollaries 4.7 and 4.8 may be easier to check in
practice.
4.4. Sufficient uniqueness conditions for coupled CPD. We first present a
condition in Proposition4.10and Theorem4.11for the case where at least one of the
involved CPDs is unique. Next, in Theorem4.12we extend Theorem2.2to coupled
CPD. It is a more relaxed condition than Proposition4.10and Theorem 4.11since
it requires only that the overall coupled CPD be unique, i.e., none of the individual
CPDs are required to be unique. In Corollary 4.13 and Theorem 4.15 we extend
Theorems2.3and2.4to the coupled CPD case in which the common factor matrix
has full column rank. Finally, in Corollary 4.14we extend Theorem2.5to coupled
CPD. Table2summarizes the organization and structure.
Table 2
Relations between uniqueness conditions for the single CPD and coupled CPD for different rank properties of the common factor matrix C. The coupled CPD, case 1, corresponds to the cases where one of the individual CPDs is unique, while the coupled CPD, case 2, corresponds to the cases where none of the individual CPDs are required to be unique. In the case where C has full column rank, we further distinguish between sufficient (S) conditions and necessary and sufficient (N and S) conditions.
(S) (N and S) (S) (S)
k (C)≥ 2 r (C) = R r (C) = R r (C) = R
Single CPD Thm. 2.2 Thm.2.3 Thm. 2.4 Thm. 2.5
Coupled CPD, case 1 Thm. 4.11 Prop.4.10 Prop.4.10 Cor. 4.14
Coupled CPD, case 2 Thm. 4.12 Thm.4.15 Cor. 4.13 Cor. 4.14
Proposition 4.10. Consider the coupled PD ofX(n)∈ CIn×Jn×K, n∈ {1, . . . , N},
in (4.1). If4
∃ n ∈ {1, . . . , N} : the rank of X(n)is R and the CPD of X(n)is unique,
and if C has full column rank, then the coupled rank of {X(n)} is R and the coupled
CPD of {X(n)} is unique.
Proof. If there exists an integer n∈ {1, . . . , N} such that the rank of X(n)is R and
the CPD of X(n) is unique, then obviously the common factor matrix C is unique.
Compute Y(n) = X(n)(CT)†; then the remaining factor matrices are obtained by
recognizing that the columns of Y(n)are vectorized rank-1 matrices:
min a(n)r ,b(n)r G G Gy(n)r − a(n)r ⊗ b(n)r G G G2 F, r∈ {1, . . . , R}, n ∈ {1, . . . , N}.
Hence, the coupled CPD of {X(n)} is unique and the coupled rank of {X(n)}
is R.
Proposition4.10tells us that a coupled CPD in which the common factor matrix
has full column rank is unique if one of the involved CPDs is unique. This simple observation already demonstrates that a coupled CPD can be unique even if some of
the involved CPDs are individually nonunique. For instance, Proposition 4.10does
not prevent in the coupled CPD that some of the Khatri–Rao products are rank deficient, which is not allowed in the ordinary CPD. As an example, we consider
X = H X(1)(1) X(2)(1) I = C A(1)" B(1) A(2)" B(2) D CT,
4As an example, Theorem2.4states that if r(C
2(A(n))" C2(B(n))) =R(R−1)2 , then the rank of
X(n) is R and the CPD ofX(n)is unique. Alternatively, Theorem2.3states that if r(E(n)(w)) =
!R
r=1wra(n)r b(n)Tr ≥ 2 for all w ∈ Ω = {x ∈ CR
"
" ω(x) ≥ 2}, then the rank of X(n) is R and the
CPD ofX(n)is unique.
508 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER
where A(1)∈ C3×4, A(2)
∈ C3×4, B(1)
∈ C3×4, B(2)
∈ C3×4, and C∈ C4×4. Further,
let a(2)1 ⊗ b(2)1 = a2(2)⊗ b(2)2 ; then generically r(A(2)" B(2)) = 3, and consequently the
CPD ofX(2) is not unique [30]. However, Proposition4.10tells us that the coupled
CPD ofX(1) andX(2)is generically unique.
Theorem4.11considers the more general case where C does not necessarily have
full column rank.
Theorem 4.11. Consider the coupled PD of X(n)∈ CIn×Jn×K, n∈ {1, . . . , N},
in (4.1). Let Sndenote a subset of{1, . . . , R}, and let Snc={1, . . . , R}\Sndenote the
complementary set. Stack the columns of C with index in Snin C(Sn)∈ CK×card(Sn)
and stack the columns of C with index in Sc
n in C(S
c
n)∈ CK×(R−card(Sn)). Stack the
columns of A(n) (resp., B(n)) in the same order such that A(n,Sn) ∈ CIn×card(Sn)
(resp., B(n,Sn) ∈ CJn×card(Sn)) and A(n,Snc) ∈ CIn×(R−card(Sn)) (resp., B(n,Sn) ∈
CJn×(R−card(Sn))) are obtained. If5
∃ n ∈ {1, . . . , N} : the rank of X(n)is R and CPD ofX(n) is unique,
(4.15a)
and for all n∈ {1, . . . , N} there exist an index set Sn with 0≤ card (Sn)≤ rC such
that C(Sn) has full column rank and
=
B(n,Snc) has full column rank,
r)-PC(Sn)C(S c n)" A(n,Snc), P C(Sn)c(S c n) r ⊗ IIn .* = αn ∀r ∈ Snc, (4.15b) where αn= In+ R− card (Sn)− 1, or =
A(n,Snc) has full column rank,
r)-PC(Sn)C(S c n)" B(n,Snc), P C(Sn)c(S c n) r ⊗ IJn .* = βn ∀r ∈ Snc, (4.15c)
where βn= Jn+R−card (Sn)−1, then the coupled rank of {X(n)} is R and the coupled
CPD of{X(n)} is unique. Generically, condition (4.15b) or (4.15c) is satisfied if for
all n∈ {1, . . . , N} we have = R≤ min)Vn+ min (K, R) ,Vn(Wn−1)+WWnn(K−1)+1 * when Vn< R , R≤ (K − 1)Wn+ 1 when Vn≥ R , (4.16)
where Vn= max(In, Jn) and Wn= min(In, Jn).
Proof. We assume that the rank ofX(p) is R and the CPD ofX(p) is unique for
some p ∈ {1, . . . , N}. The overall uniqueness of the CPD of X(p) implies that the
common factor matrix C is unique with property k (C) ≥ 2. We now consider the
individual CPDs of the tensors{X(n)} with matrix representations
X(n)(1) =)A(n)" B(n)*CT, n∈ {1, . . . , N},
as CPDs with a known factor matrix. We know from [40, Theorem 4.8] that the CPD
of the tensor X(n) with known factor C is unique if condition (4.15b) or (4.15c) is
5As an example, if the conditions stated in Theorem2.2are satisfied for some p∈ {1, . . . , N} in
which the roles of A(p), B(p), and C may be interchanged, then the rank ofX(p)is R and the CPD
ofX(p)is unique.
satisfied. We also know from [40, Theorem 4.8] that the CPD of the tensorX(n)with known factor C is generically unique if conditions (4.16) are satisfied. We conclude
that the coupled CPD of {X(n)} linked via the matrix C is unique and the coupled
rank of{X(n)} is R.
Theorem4.11tells us that a coupled CPD is unique under more relaxed conditions
than the individually involved ordinary CPDs even in cases where C does not have full column rank. This also means that some of the involved CPDs are allowed to be individually nonunique. As an example, we consider
X = H X(1)(1) X(2)(1) I = C A(1)" B(1) A(2)" B(2) D CT, where A(1) ∈ C4×5, A(2) ∈ C4×5, B(1) ∈ C4×5, B(2) ∈ C4×5, and C ∈ C4×5.
Furthermore, let b(2)1 = b(2)2 ; then generically k(B(2)) = 1 and consequently the
CPD of X(2) is not unique (see, e.g., [44]). Since C does not have full column rank,
Proposition 4.10does not apply. However, Theorem4.11 tells us that the coupled
CPD ofX(1) andX(2) is generically unique. Note that this result is not obtained by
inverting C as in the proof of Proposition4.10.
Theorem 4.12. Consider the coupled PD of X(n)∈ CIn×Jn×K, n∈ {1, . . . , N}
in (4.1). Let Sndenote a subset of{1, . . . , R}, and let Snc={1, . . . , R}\Sndenote the
complementary set. Stack the columns of C with index in Snin C(Sn)∈ CK×card(Sn)
and stack the columns of C with index in Sc
n in C(S
c
n)∈ CK×(R−card(Sn)). Stack the
columns of A(n) (resp., B(n)) in the same order such that A(n,Sn) ∈ CIn×card(Sn)
(resp., B(n,Sn) ∈ CJn×card(Sn)) and A(n,Snc) ∈ CIn×(R−card(Sn)) (resp., B(n,Sn) ∈
CJn×(R−card(Sn))) are obtained.
If C is unique6with property k (C)≥ 2, and if for all n ∈ {1, . . . , N} there exists
an index set Sn with 0 ≤ card (Sn) ≤ rC such that C(Sn) has full column rank and
condition (4.15b) or (4.15c) is satisfied, then the coupled rank of{X(n)} is R and the
coupled CPD of{X(n)} is unique.
Proof. The necessity of k (C)≥ 2 has already been mentioned in Proposition4.1.
Assuming that the common factor matrix C is unique with k (C)≥ 2, we can consider
the individual CPDs of the tensors{X(n)} as CPDs with a known factor matrix C.
We know from [40, Theorem 4.8] that the CPD of the tensorX(n)with known factor
C is unique if condition (4.15b) or (4.15c) is satisfied. We can now conclude that the
coupled CPD of {X(n)} linked via the matrix C is unique and the coupled rank of
{X(n)} is R.
Note that Theorem 4.12, unlike Proposition 4.10 and Theorem 4.11, does not
assume that the CPD of one of the individual tensorsX(n)is unique. As an example,
we consider X = H X(1)(1) X(2)(1) I = C A(1)" B(1) A(2)" B(2) D CT, where A(1)∈ C4×5, A(2) ∈ C4×5, B(1) ∈ C4×5, B(2) ∈ C4×5, and C∈ C4×5. Further,
let b(1)1 = b(1)2 and b3(2) = b(2)4 ; then generically k(B(1)) = 1 and k(B(2)) = 1.
Consequently the individual CPDs of X(1) and X(2) are not unique, which means
6As an example, if the conditions (4.8) stated in Proposition4.6are satisfied, then the coupled
rank of{X(n)} is R and the common factor matrix C is unique.
510 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER
that neither Proposition 4.10 nor Theorem 4.11 can be used to establish coupled
CPD uniqueness. However, Proposition4.6, together with Theorem4.12, tells us that
the coupled CPD ofX(1) andX(2)is generically unique.
The above example explains that in some cases it is better to first establish
uniqueness of the common factor matrix C via, for instance, Proposition 4.6, and
thereafter establish coupled CPD uniqueness of {X(n)} by treating the individual
CPDs of{X(n)} as CPDs with a known factor C. However, in other cases it is better
to first establish CPD uniqueness of one of the individual tensors, sayX(p), via, for
instance, Theorem 2.2, and thereafter establish coupled CPD uniqueness of {X(n)}
by treating the individual CPDs of{X(n)} as CPDs with a known factor C. As an
example, we consider X = H X(1)(1) X(2)(1) I = C A(1)" B(1) A(2)" B(2) D CT, where A(1)∈ C4×6, A(2) ∈ C4×6, B(1) ∈ C6×6, B(2)
∈ C5×6, and C∈ C3×6. For this
problem Proposition4.6cannot be used since the matrix G(5)is not defined. On the
other hand, Theorem2.2, together with Theorem4.11, tells us that the coupled CPD
ofX(1) andX(2)is generically unique.
Let us now assume that the common factor matrix C has full column rank. In
that case, Theorem4.12reduces to Corollary4.13, which in turn can be understood
as an extension of Theorem 2.4to coupled CPD. Corollary4.13can also be seen as
a generalization of Proposition4.10to the case where none of the involved CPDs are
required to be unique.
Corollary 4.13. Consider the coupled PD ofX(n)∈ CIn×Jn×K, n∈ {1, . . . , N},
in (4.1). Let G(2) be defined as in (4.7). If
=
C has full column rank,
G(2) has full column rank,
(4.17)
then the coupled rank of{X(n)} is R and the coupled CPD of {X(n)} is unique.
Proof. Due to Corollary4.7we know that the coupled rank of{X(n)} is R and
the common factor matrix C is unique when condition (4.17) is satisfied. Assuming that C has full column rank, the remaining factors follow from rank-1 approximations
as explained in the proof of Proposition4.10.
If additionally some of the factor matrices in the set{A(n)} also have full column
rank, then we may use the Corollary4.14following, which can be understood as an
extension of Theorem2.5to coupled CPD.
Corollary 4.14. Consider the coupled PD of{X(n)} in (4.1). Consider also a
subset S of{1, . . . , N} with card (S) = Q. W.l.o.g., we assume that S = {1, . . . , Q}.
If for some Q∈ {1, . . . , N} we have
(4.18) rC= R , rA(n)= R ∀n ∈ {1, . . . , Q}, ∀r ∈ {1, . . . , R}, ∀s ∈ {1, . . . , R} \ r, ∃ n ∈ {1, . . . , Q} : k)-b(n)r , b(n)s .* = 2 ,
then the coupled rank of{X(n)} is R and the coupled CPD of {X(n)} is unique.
Proof. Due to Corollary4.8we know that the coupled rank of{X(n)} is R and the common factor C is unique. Since C is unique and has full column rank, the remaining factors follow from rank-1 approximations as explained in the proof of
Proposition4.10.
Comparison of Theorem 2.5 with condition (4.19) shows that the coupling has
relaxed the uniqueness condition.
Finally, we generalize the necessary and sufficient uniqueness condition (2.5)
stated in Theorem2.3to the coupled CPD problem.
Theorem 4.15. Consider the coupled PD ofX(n)∈ CIn×Jn×K, n∈ {1, . . . , N},
in (4.1). Assume that C has full column rank. The coupled rank of{X(n)} is R and
the coupled CPD of {X(n)} is unique if and only if
∀w ∈ Ω , ∃ n ∈ {1, . . . , N} : r)E(n)(w)*≥ 2 ,
(4.19)
where E(n)and Ω are defined as in (4.4).
Proof. Due to Proposition4.9we know that the coupled rank of{X(n)} is R and
the common factor C is unique if and only if the condition (4.19) is satisfied. Since the common factor C is unique and has full column rank, the remaining factors follow
from rank-1 approximations as explained in the proof of Proposition4.10.
As in the case of ordinary CPD, the conditions in Theorem4.15may be harder
to check than those in Corollary4.13or Corollary4.14.
4.5. Extension to tensors of arbitrary order. The uniqueness properties of the CPD of higher-order tensors are not just a straightforward generalization of those for third-order tensors. As a matter of fact, they are conceptually quite different. We note that the idea of simultaneously considering different matrix representations of the CPD of a single higher-order tensor for the case where one factor matrix has full column rank was first considered in [46]. As our contribution, first we generalize the idea to cases where none of the involved factor matrices are required to have full column rank. In fact, based on the connection between coupled CPD and higher-order tensors, we even demonstrate that the (coupled) CPD of a higher-order tensor(s) can be unique despite collinearities in all factor matrices. Second, we extend the coupled CPD framework in subsections 4.3 and 4.4 to tensors of arbitrary order. More specifically, we demonstrate that by taking into account both the coupled and higher-order structures, improved uniqueness conditions are obtained.
We consider coupled PDs of X(n) ∈ CI1,n×···×IMn,n×K, n ∈ {1, . . . , N}, of the
form X(n)= R , r=1 a(1,n)r ◦ · · · ◦ a(Mr n,n)◦ cr, n∈ {1, . . . , N}. (4.20)
The factor matrices are
A(m,n)=- a(m,n)1 , . . . , a(m,n)R . ∈ CIm,n×R, m∈ {1, . . . , M n}, n ∈ {1, . . . , N}, C =' c1, . . . , cR ( ∈ CK×R.
The coupled PD of tensors{X(n)} of arbitrary order is visualized in Figure2.
Note that the tensorsX(n)may have different orders M
nand different sizes Im,n.
As a special case, we have the case of a single tensor (N = 1) of order M ≥ 4. Our
key idea is that, if one or more tensors have order Mn ≥ 4, then we may combine
512 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER X(1) = ⊗ · · · ⊗ c1 b(M1,1) 1 a(1,1)1 +· · · + ⊗ · · · ⊗ cR b(M1,1) R a(1,1)R .. . X(N ) = ⊗ · · · ⊗ c1 b(MN,N ) 1 a(1,N )1 +· · · + ⊗ · · · ⊗ cR b(MN,N ) R a(1,N )R
Fig. 2. Coupled PD of tensorsX(1), . . . ,X(N)in which b(Mn,n)
r = a(2,n)r ⊗ · · · ⊗ a(Mr n,n).
the coupled third-order CPD results discussed in subsections4.2–4.4with results for
higher-order tensors [46]. More precisely, uniqueness results may be obtained by reducing the associated higher-order PDs to coupled third-order PDs. Namely, we simultaneously consider several matrix representations of the form
(4.21) X(w,n)= 1 p∈Γw,n A(p,n)" 1 q∈Υw,n A(q,n) CT =)A[w,n] " B[w,n]*CT, where A[w,n] = J p∈Γw,nA (p,n) ∈ CIˆw,n×R with ˆIw,n = Kp∈Γw,nIp,n, B [w,n] = J q∈Υw,nA (q,n) ∈ CJˆw,n×R with ˆJ
w,n =Kq∈Υw,nIq,n, and the sets Γw,n and Υw,n
have properties Γw,n6Υw,n={1, 2, . . . , Mn} and Γw,n7Υw,n =∅. Let us assume
that there are Wn sets{Γw,n} and {Υw,n} for each n ∈ {1, . . . , N}. We collect the
matrices{X(w,n)} into the matrix
X =-X(1)T, X(2)T, . . . , X(N )T.T, X(n)=-X(1,n)T, X(2,n)T, . . . , X(Wn,n)T.T, such that (4.22) X = FCT, F = F(1) F(2) .. . F(N ) , F (n)= A[1,n]" B[1,n] A[2,n]" B[2,n] .. . A[Wn,n] " B[Wn,n] .
We now ignore the Khatri–Rao structure of A[w,n] and B[w,n] and treat (4.22) as a
matrix representation of a set of coupled third-order CPDs.
For establishing uniqueness, we may resort to the different results in subsection 4.4. For the results that make use of G(R−rC+2), we may work with the
follow-ing generalization. We limit ourselves to the LWn sets {Γw,n} and {Υw,n} for each
n ∈ {1, . . . , N} in which min(Kp∈Γw,nIp,n,
K
q∈Υw,nIq,n) ≥ R − rC+ 2. Define
G(R−rC+2,#Wn)∈ C( !Wn# w=1C R−rC+2 $ p∈Γw,n Ip,nC R−rC+2 $ q∈Υw,n Iq,n)×C R−rC+2 R as follows: G(R−rC+2,#Wn)= CR−rC+2 ) A[1,n]*" CR−rC+2 ) B[1,n]* .. . CR−rC+2 ) A[#Wn,n] * " CR−rC+2 ) B[#Wn,n] * , n∈ {1, . . . , N}.
The following matrix generalizes G(R−rC+2) in (4.7):
G(m)= G(m,#W1) .. . G(m,#WN) ∈ C !N n=1( !Wn# w=1C$mp∈Γw,n Ip,nC m $ q∈Υw,n Iq,n)×C m R , (4.23)
where m = R− rC+ 2. In the extensions of Theorems4.11and4.12, it suffices to
check condition (4.15b) or (4.15c) for one of the LWn matrix representations.
As an example, consider the fourth-order tensorsX(n)∈ CI×J×K×L, n∈ {1, 2},
with PDs, (4.24) X(n)= R , r=1 a(n) r ◦ b(n)r ◦ c(n)r ◦ dr, n∈ {1, 2}, in which I = 4, J = 5, K = 4, L = 3, R = 4, a(1)2 = a(1)3 , b(1)1 = b(1)3 , c(1)1 = c(1)2 ,
c(1)3 = c(1)4 , a(2)1 = a(2)4 , b1(2) = b(2)2 = b(2)3 , and c(2)3 = c(2)4 . Note that generically
kA(1) = kB(1) = kC(1) = kA(2) = kB(2) = kC(2) = 1 and kD ≥ 2. The existing
CPD uniqueness conditions for higher-order tensors stated in [33,46,5] do not apply.
Similarly, the uniqueness conditions for coupled CPD based on third-order tensors (i.e., if we ignore the fourth-order structure by combining two modes) discussed in
subsection4.4do not apply either. We now explain that by simultaneously exploiting
both the higher-order and coupled structures of the PDs in (4.24), coupled CPD uniqueness can be established. Generically D has rank 3. Denote
G(n)= C3 ) A(n)*" C3 ) B(n)" C(n)* C3 ) B(n)*" C3 ) C(n)" A(n)* C3 ) C(n)*" C3 ) A(n)" B(n)* .
Using Lemma 4.3 it can be verified that, although the matrices G(1) and G(2) are
rank deficient, the matrix G = [G(1)T, G(2)T]T generically has full column rank.
Thus, Proposition4.6tells us that by taking into account the higher-order structure
and the coupling betweenX(1)andX(2), uniqueness of D can be established. Using
Lemma 4.3it also can be verified that E(1) = B(1)" C(1) and E(2) = B(2)" C(2)
generically have full column rank and that the matrix [D" A(n), dr⊗ II] generically
has a one-dimensional kernel for every r ∈ {1, 2, 3, 4} and n ∈ {1, 2}. Invoking
Theorem4.12we can conclude that the coupled CPD ofX(1) andX(2)(in which the
Khatri–Rao structure of E(1) and E(2) has been ignored) is unique. Consequently,
the factors {A(n)}, {E(n)}, and D are unique despite the collinearities in the factor
matrices. Finally, the rank-1 structure of the columns of E(n)= B(n)" C(n)implies
that{B(n)} and {C(n)} are also unique.
514 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER
We now demonstrate that (coupled) CPD of higher-order tensors can even be unique despite collinearities in all factor matrices; i.e., the factor matrices of the PDs
of{X(n)} in (4.20) may satisfy k
C = 1 and kA(m,n) = 1 for all m∈ {1, . . . , Mn}, for
all n ∈ {1, . . . , N}. For this reason Proposition4.1does not extend to higher-order
tensors in an obvious manner. Note that the existing CPD uniqueness conditions for
higher-order tensors stated in [33,46,5] do not apply in this case. As an example,
consider N = 1 and the PD ofX ∈ CI×J×K×Lgiven by
(4.25) X =
R , r=1
ar◦ br◦ cr◦ dr,
in which a1= a2, b1= b3, c3= c4, d2= d3, and I = J = K = L = R = 4. Since
kA = kB= kC= kD= 1 the results discussed in subsection4.4cannot be applied in
a direct manner. We will establish uniqueness by reducing the fourth-order PD to a
coupled third-order PD and by following a deflation argument. Generically rA = 3.
Using Lemma4.3it can be verified that
CCRR−r−rAA+2+2(B)(C)" C" CRR−r−rAA+2+2(C(B" D)" D) CR−rA+2(D)" CR−rA+2(B" C) = CC33(B)(C)" C" C33(C(B" D)" D) C3(D)" C3(B" C)
generically has full column rank. Proposition 4.6implies that the factor matrix A
is unique. The next step is to demonstrate that the rank-1 term a4◦ b4◦ c4◦ d4is
unique. The PD ofX in (4.25) has matrix representation
X = (A" B) (C " D)T = (A" B) ET, E = C" D.
Lemma4.3can also tell us that E generically has full column rank and that generically
r ([A" B, ar⊗ IJ]) = R + J− Γr(A) ,
where Γ1(A) = Γ2(A) = 2 and Γ3(A) = Γ4(A) = 1. Since Γ4(A) = 1, [41,
Proposition 5.2] tells us that the vectors b4 and e4 are unique. As a consequence of
the rank-1 structure of e4we also know that c4and d4 are unique. We subtract the
unique rank-1 term,
Y = X − a4◦ b4◦ c4◦ d4=
3 , r=1
ar◦ br◦ cr◦ dr.
The PD ofY has the factor matrices A(2) = [a
1, a2, a3], B(2) = [b1, b2, b3], C(2) =
[c1, c2, c3], and D(2)= [d1, d2, d3]. The matrix C(2)generically has full column rank.
Using Lemma4.3it can be verified that
C2 ) A(2)*" C2 ) B(2)" D(2)* C2 ) B(2)*" C2 ) D(2)" A(2)* C2 ) D(2)*" C2 ) A(2)" B(2)*
generically has full column rank. Due to Corollary4.13 we now know that the
re-maining factors A(2), B(2), C(2), and D(2) are unique.
More generally, for the case of coupled CPD of higher-order tensors it is possible in some cases to establish coupled CPD uniqueness via a sequence of deflation steps. See the supplementary material for a brief discussion.
4.6. Coupled matrix-tensor factorization. A simple case of coupled decom-positions is the coupled matrix-tensor factorization, admitting a matrix representation of the form X = H X(1)(1) X(2)(1) I = C A(1)" B(1) A(2) D CT. (4.26)
Because of its simplicity, (4.26) is common in the analysis of multiview data [49,1,18,
2]. Note that coupled matrix-tensor factorization is a special case of coupled CPD.
Indeed, define B(2)= [1, 1, . . . , 1]∈ C1×R; then (4.26) can also be written as
X = H X(1)(1) X(2)(1) I = C A(1)" B(1) A(2)" B(2) D CT,
which is of form (4.2), so that several of the results presented in this paper can be applied.
A notable limitation of the coupled matrix-tensor factorization (4.26) is that
in order to guarantee the uniqueness of A(2), the common factor C must have full
column rank. More precisely, if C has full column rank, then A(2)follows from A(2)=
X(2)(1)(CT)†. On the other hand, if C does not have full column rank, then there will
be an intrinsic indeterminacy between A(2)and C. Indeed, when C does not have full
column rank, the null space of C is not empty. Any vector y∈ ker (C) will generate an
alternative coupled matrix-tensor factorization X in which X(2)(1)= (A(2)+ xyT)CT,
where x∈ CI2.
5. Coupled CPD with collinearity in common factor. We consider coupled
PDs ofX(n)∈ CIn×Jn×K, n∈ {1, . . . , N}, of the following form:
X(n)= R , r=1 L,r,n l=1 a(r,n)l ◦ b(r,n)l ◦ c(r)= R , r=1 ) A(r,n)B(r,n)T*◦ c(r). (5.1)
On one hand, this is an extension of (3.1) to the coupled case. On the other hand, it is a variant of the coupled PD in (4.1) for collinearity constrained C. If the matrices A(r,n)B(r,n)T have rank Lr,n, then (5.1) is a coupled decomposition into multilinear
rank-(Lr,n, Lr,n, 1) terms. We will briefly call this a coupled block term decomposition
(BTD). The coupled BTD of the third-order tensors{X(n)} is visualized in Figure3.
The coupled multilinear rank-(Lr,n, Lr,n, 1) tensors in (5.1) can be arbitrarily
per-muted, and the vectors/matrices within the same coupled multilinear
rank-(Lr,n, Lr,n, 1) tensor can be arbitrarily scaled provided the overall coupled multilinear
rank-(Lr,n, Lr,n, 1) term remains the same. We say that the coupled BTD is unique
when it is only subject to the mentioned indeterminacies.
In this section we limit the exposition to third-order tensors. Analogous to the
coupled CPD in subsection4.5, the coupled BTD and its associated properties can
be extended to tensors of arbitrary order. In the supplementary material we briefly explain that it can be reduced to a set of coupled BTDs of third-order tensors.
516 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER X(1) = c(1) A(1,1) L1,1 B(1,1)T +· · · + c(R) A(R,1) LR,1 B(R,1)T .. . X(N ) = c(1) A(1,N ) L1,N B(1,N )T +· · · + c(R) A(R,N ) LR,N B(R,N )T
Fig. 3. Coupled BTD of the third-order tensorsX(1), . . . ,X(N).
5.1. Matrix representation. Denote Rtot,n=>Rr=1Lr,n. The coupled PD of
the tensors{X(n)} of the form (5.1) has the factor matrices
A(r,n)=- a(r,n)1 , . . . , aL(r,n)r,n .∈ CIn×Lr,n, A(n)=' A(1,n), . . . , A(R,n) (∈ CIn×Rtot,n, n∈ {1, . . . , N}, B(r,n)=- b(r,n)1 , . . . , b(r,n)Lr,n .∈ CJn×Lr,n, B(n)=' B(1,n), . . . , B(R,n) (∈ CJn×Rtot,n, n∈ {1, . . . , N}, C(red)=' c(1), . . . , c(R) (∈ CK×R, (5.2) C(n)=-1T Lr,n⊗ c (1), . . . , 1T LR,n⊗ c (R).∈ CK×Rtot,n, (5.3)
and matrix representation
X =-X(1)T(1) , . . . , X(N )T(1) .T= F(red)C(red)T∈ C(!N n=1InJn)×K, (5.4) where F(red)∈ C(!N n=1InJn)×Ris given by F(red)= Vec)B(1,1)A(1,1)T* · · · Vec)B(R,1)A(R,1)T* .. . . .. ... Vec)B(1,N )A(1,N )T* · · · Vec)B(R,N )A(R,N )T* . (5.5)
Denote Lr,max= maxn∈{1,...,N}Lr,nand Rext=>Rr=1Lr,max, where “ext” stands
for extended. By appending all-zero column vectors to A(r,n) and B(r,n), (5.4) may
also be expressed as
X = F(ext)C(ext)T ∈ C(!N
n=1InJn)×K,
(5.6)
where F(ext)= HE B A(1)" BB(1) FT , . . . , E B A(N )" BB(N ) FTIT ∈ C(!N n=1InJn)×Rext, (5.7) C(ext)=-1T L1,max⊗ c (1), . . . , 1T LR,max⊗ c (R).∈ CK×Rext, (5.8) in which B A(r,n)=-A(r,n), 0In,(Lr,max−Lr,n) . ∈ CIn×Lr,max, B A(n)= C B A(1,n), . . . , BA(R,n) D ∈ CIn×Rext, n∈ {1, . . . , N}, B B(r,n)=-B(r,n), 0Jn,(Lr,max−Lr,n) . ∈ CJn×Lr,max, B B(n)= C B B(1,n), . . . , BB(R,n) D ∈ CJn×Rext, n∈ {1, . . . , N}.
5.2. Uniqueness conditions for coupled CPD with collinearity in
com-mon factor. Let {{ 8A(r,n)}, { 8B(r,n)}, {8c(r)}} yield an alternative coupled BTD of
the tensors {X(n)} in (5.1). We say that the coupled BTD of{X(n)} is unique if it
is unique up to a permutation of the coupled multilinear rank-(Lr,n, Lr,n, 1) terms
{( 8A(r,n)B8(r,n)T)◦ 8c(r)} and up to the following indeterminacies within each term: 8
A(r,n)= α(r,n)A(r,n)Hr,n, 8B (r,n)
= β(r,n)B(r,n)H−1r,n, 8c(r)= γ(r)c(r),
where Hr,n ∈ CLr,n×Lr,n are nonsingular matrices and α(r,n), β(r,n), γ(r) ∈ C are
scalars satisfying α(r,n)β(r,n)γ(r)= 1, r∈ {1, . . . , R}, n ∈ {1, . . . , N}. From (5.4) it is
clear that uniqueness requires kC(red)≥ 2. From (5.4) it is also clear that F(red)must
have full column rank in order to guarantee the uniqueness of the coupled BTD of
{X(n)}. Proposition5.1below extends the necessary conditions stated in Propositions
4.1,4.2, and4.5to coupled BTD.
Proposition 5.1. Consider the coupled PD of X(n), n ∈ {1, . . . , N}, in (5.1).
Define E(n)(w) =>Rr=1wrA(r,n)B(r,n)T and Ω =
?
x∈ CR@@ ω(x) ≥ 2A. If the coupled
BTD of{X(n)} in (5.1) is unique, then
(i) kC(red)≥ 2,
(ii) F(red) has full column rank,
(iii) for all w∈ Ω , ∃ n ∈ {1, . . . , N} : r(E(n)(w)) > maxr|wr&=0Lr,n.
Proof. The proof is analogous to that of Propositions4.1,4.2, and4.5.
Proposition5.2tells us that this is generically true if F(red)has at least as many
rows as columns.
Proposition 5.2. Consider F(red)∈ C(!N
n=1InJn)×Rgiven by (5.5). For generic
matrices{A(r,n)} and {B(r,n)}, the matrix F(red) has rank min(>Nn=1InJn, R).
Proof. Due to Lemma4.3we just need to find one example for which the
propo-sition holds. Since the coupled CPD (4.1) is a particular case of (5.4), a
particu-lar example is the matrix F(red) in (4.3). (Formally, we take a(r,n)l = 0In for all
l∈ {2, . . . , Lr,n}, for all r ∈ {1, . . . , R}, for all n ∈ {1, . . . , N}.) The proposition now
follows directly from Proposition4.4.
We will now discuss extensions of Theorems 4.11, 4.12, and 4.15 to the case
where the common factor matrix contains collinear components. The generalizations
518 MIKAEL SØRENSEN AND LIEVEN DE LATHAUWER
of Proposition4.10and Corollary4.13follow immediately and are therefore not
con-sidered in this section.
Theorem 5.3 can be seen as a version of Theorem4.11 for the case where the
common factor matrix contains collinear columns.
Theorem 5.3. Consider the coupled PD ofX(n), n∈ {1, . . . , N}, in (5.1). Let S
n
denote a subset of{1, . . . , R}, and let Sc
n={1, . . . , R} \ Sndenote the complementary
set. Stack the columns of C(red)with index in Snin C(red,Sn)∈ CK×card(Sn)and stack
the columns of C(red) with index in Sc
n in C(red,S
c
n)∈ CK×(R−card(Sn)). Stack A(r,n)
(resp., B(r,n) and C(r,n)) in the same order such that A(n,Sn)
∈ CIn×(
!
p∈SnLp,n)
(resp., B(n,Sn) ∈ CJn×(!p∈SnLp,n) and C(n,Sn) ∈ CK×(!p∈SnLp,n)) and A(n,Snc) ∈
CIn×(!p∈Sc nLp,nt)(resp., B(n,S c n)∈ CJn×( ! p∈ScnLp,n) and C(n,S c n)∈ CK×( ! p∈ScnLp,n))
are obtained. Denote D(n,Scn)= P
C(red,Sn)C(n,S c n). If 7
∃ p ∈ {1, . . . , N} : the minimal number of rank-(Lr,p, Lr,p, 1)
terms inX(p) is R and the decomposition ofX(p) into
rank-(Lr,p, Lr,p, 1) terms is unique,
(5.9a)
and for every n∈ {1, . . . , N} there exists an index set Sn⊆ {1, . . . , R} with property
0≤ card (Sn)≤ rC(red), such that
B(n,Sc
n) has full column rank
) rB(n,Scn) =>p ∈Sc nLp,n * , r)-D(n,Snc)" A(n,Scn), d(n,Snc) r ⊗ IIn .* = αr,n ∀r ∈ Snc, (5.9b) where αr,n= In+>p∈Sc nLp,n− Lr,n, or
A(n,Snc) has full column rank
) rA(n,Scn )=>p ∈Sc nLp,n * , r)-D(n,Snc)" B(n,Snc), d(n,Scn) r ⊗ IJn .* = βr,n ∀r ∈ Snc, (5.9c) where βr,n= Jn+>p∈Sc
nLp,n− Lr,n, then the minimal number of coupled multilinear
rank-(Lr,n, Lr,n, 1) terms is R and the coupled BTD of{X(n)} is unique.
Proof. Assume that there exists an integer p∈ {1, . . . , N} such that the minimal
number of rank-(Lr,p, Lr,p, 1) terms inX(p) is R and the decomposition ofX(p) into
rank-(Lr,p, Lr,p, 1) terms is unique. Since C(red)is unique and the condition (5.9b) or
(5.9c) is satisfied for every n∈ {1, . . . , N}, we know from [41, Proposition 5.2] that
the coupled BTD of{X(n)} is unique and the minimal number of coupled multilinear
rank-(Lr,n, Lr,n, 1) terms is R.
Theorem 5.4below can be interpreted as a version of Theorem4.12for the case
where the common factor matrix contains collinear columns.
Theorem 5.4. Consider the coupled PD ofX(n), n∈ {1, . . . , N}, in (5.1). Let S
n
denote a subset of{1, . . . , R} and let Sc
n={1, . . . , R} \ Sndenote the complementary
7As an example, if r(C
Rtot,p−rC(red)+2(A(p))" CRtot,p−rC(red)+2(B(p))) = C
Rtot,p−rC(red)+2
Rtot,p ,
then Theorem 2.1 tells us that the rank ofX(p) is!R
r=1Lr,p and the factor C(p) is unique up
to a column permutation and scaling. The uniqueness of the decomposition of X(p) into
rank-(Lr,p, Lr,p, 1) terms now follows from condition (5.9b) or (5.9c), as explained by Proposition 5.2 in
[41].
