Delin Chu

Delin Chu

^

and Bart De Moor

^y

Department of Electrical Engineering

Katholieke Universiteit Leuven Kardinaal Mercierlaan 94

B-3001 Leuven, Belgium August 13, 1998

Abstract

Recently, Chu, Funderlic and Golub [ SIAM J. Matrix Anal. Appl., 18:1082{1092, 1997] presented a variational formulation for the quotient singular value decomposition (QSVD) of two matrices

^{A 2 Rnm;C 2 Rpm}

which is a generalization of that one for the ordinary singular value decomposition (OSVD) and characterizes the role of two orthogonal matrices in QSVD. In this paper, we give an alternative derivation of this variational formulation and extend it to establish an analogous variational formulation for the Restricted Singular Value Decomposition (RSVD) of Matrix Triplets

^A2Rnm;B2

Rn^l;C2Rp^m

which provides new understanding of the orthogonal matrices appearing in this decomposition.

Keywords:

OSVD, QSVD, RSVD, Generalized Singular Value, Variational Formu- lation, Stationary Value, Stationary Point.

AMSsubject classication:

65F15, 65H15.

1 Introduction

The ordinary singular value decomposition (OSVD) of a given matrix A

²

R

^nm

is U

^T

AV =

"

r

a

m

^?

r

a

r

a

 0

n

^?

r

a

0 0

#

; (1)

with

U =

^h

r

a

n

^?

r

a

n U

¹

U

^{2 i}

; V =

^h

r

a

m

^?

r

a

m V

¹

V

^{2 i}

;

 = diag

^f



¹

;

;

r^ag

; 

^{1 }



<sup>2 


</sup>



r^a

> 0 ; r

a

= rank( A ) ;

where U;V are orthogonal matrices. The 

¹

;

;

r^a

are the non-trivial singular values of A , and the columns of U

¹

and V

¹

are, respectively, the non-trivial left and right singular vectors of A . In this paper,

^k
k

denotes the 2

^?

norm of a vector. The following theorem is well-known [4]:

Delin.Chu@esat.kuleuven.ac.be

yBart.Demoor@esat.kuleuven.ac.be

1

Theorem 1 Given A

²

R

^nm

with OSVD (1).

(a) Consider the optimization problem

y⁼Ax; y

max

⁶⁼⁰

k

y

^k

k

x

^k

: (2)

Then the non-trivial singular values 

¹

;

;

r^a

of A are precisely the stationary values, i.e., the functional evaluations at the stationary points, of (2). And, let the stationary points in (2) corresponding to the stationary values 

¹

;

;

r^a

be

"

x

¹

y

¹

#

;

;

"

x

r^a

y

r^a

#

, then V

¹

=

^{h k}_x<sup>x11k


kx</sup>_x^{rarak i}

:

Moreover, if n = m = r

a

, then

U

¹

=

^{h ky}_y<sup>11k


ky</sup>_y^rara k

i

: (b) Consider the dual optimization problem

y^T=x

max

^TA; y⁶⁼⁰

k

y

^k

k

x

^k

: (3)

Then the non-trivial singular values 

¹

;

;

r^a

of A are precisely the stationary values of (3). And, let the stationary points in (3) corresponding to the stationary values 

¹

;

;

r^a

be

"

x

¹

y

¹

#

;

;

"

x

r^a

y

r^a

#

, then

U

¹

=

^{h kx}_x<sup>11k


kx</sup>_x^rara k

i

: Moreover, if n = m = r

a

, then

V

¹

=

^{h ky}_y<sup>11k


ky</sup>_y^rraa k

i

:

Recently, in Chu, Funderlic and Golub [1] Theorem 1 has been generalized to the Quotient Singular Value Decomposition (QSVD) [3, 5, 6, 7, 8, 9, 10, 11, 13, 14] of two matrices A

²

R

^nm

;C

²

R

^pm

based on the relationship between QSVD of two matrix A;C and the eigendecomposition of the matrix pencil ( A

^T

A;C

^T

C ).

The purposes of this paper are twofold. Firstly, we present an alternative derivation of the variational formulation in [1] directly based on the QSVD of two matrices A;C . Then we extend this result to the Restricted Singular Value Decomposition (RSVD)[8, 9, 10, 11, 15, 16] of matrix triplets and obtain a analogous variational formulation which provides new understanding of the orthogonal matrices appearing in this decomposition.

In order to prove our main results, we will establish two condensed forms based on orthog- onal matrix transformations. The QSVD of two matrices and the RSVD of matrix triplets can be obtained and the variational formulation for QSVD and RSVD can be proved directly based on these two condensed forms.

In this paper, we use the following notation:

 S

1

( M ) denotes a matrix with orthogonal columns spanning the right nullspace of a

matrix M ;

 T

1

( M ) denotes a matrix with orthogonal columns spanning the right nullspace of a matrix M

^T

;



M

^?

denotes the orthogonal complement of the space spanned by the columns of M ;



Unless noted, we do not distinguish between a matrix with orthogonal columns and the space spanned by its columns.

We also use the following notation for any given matrices A;B;C with compatible sizes:

denote

r

a

= rank( A ) ; r

b

= rank( B ) ; r

c

= rank( C ) ; r

ab

= rank

^h

A B

ⁱ

;  r

ac

= rank

"

A C

#

; r

abc

= rank

"

C A B 0

#

; k

¹

= r

abc^?

r

b^?

r

c

; k

²

= r

ab

+ r

c^?

r

abc

;

k

³

=  r

ac

+ r

b^?

r

abc

; k

⁴

= r

a

+ r

abc^?

r

ab^?

 r

ac

:

2 A Variational Formulation for QSVD

Nowadays, several generalizations of the OSVD have been proposed and analysed. One that is well-known is the generalized SVD as introduced by Paige and Saunders in [5], which was proposed by De Moor and Golub [11] to rename as the QSVD. Another one is the RSVD, introduced in its explicit form by Zha [16] and further developed and discussed by De Moor and Golub [8].

In this section we will give an alternative proof for the variational formulation for the QSVD of [1] based directly on QSVD itself. Firstly we present a condensed form to derive QSVD of two matrices.

Lemma 2 Given matrices A

²

R

^nm

;C

²

R

^pm

. Then there exist three orthogonal matrices U

a²

R

^nn

;W

²

R

^mm

;V

c²

R

^pp

such that

U

_Ta

AW =

2

6

4

r 

ac^?

r

c

r

a

+ r

c^?

r 

ac

r 

ac^?

r

a

m

^?

r 

ac

 r

ac^?

r

c

A

¹¹

A

¹²

0 0

r

a

+ r

c^?

r 

ac

0 A

²²

0 0

n

^?

r

a

0 0 0 0

3

7

5

;

(4) V

_Tc

CW =

2

6

4

r 

ac^?

r

c

r

a

+ r

c^?

r 

ac

r 

ac^?

r

a

m

^?

r 

ac

p

^?

r

c

0 0 0 0

r

a

+ r

c^?

r 

ac

0 C

²²

0 0

 r

ac^?

r

a

C

³¹

C

³²

C

³³

0

3

7

5

; where A

¹¹

;A

²²

;C

²²

and C

³³

are nonsingular.

Proof. See Appendix A.

Let the OSVD of A

²²

C

^22?1

be

U

^22T

A

²²

C

^22?1

V

²²

= diag

^f



¹

;

;

s^g

=: S

A

; s = r

a

+ r

c^?

r 

ac

; (5)

where U

²²

;V

²²

are orthogonal matrices, 

^1



<sup>2 


</sup>



s

> 0. Dene

U := U

a

diag

^f

I

^r^ac?r^c

;U

²²

;I

n^?r^ag

; (6) V := V

c

diag

^f

I

p^?r^c

;V

²²

;I

r^ac?r^ag

: (7) X =

2

6

6

6

4

I 0 0 0

0 I 0 0

?

C

^33?1

C

^{31 ?}

C

^33?1

C

³²

C

^33?1

0

0 0 0 I

3

7

7

7

5 2

6

6

6

4

A

^{?111 ?}

A

^?111

A

¹²

C

^22?1

V

²²

0 0 0 C

^22?1

V

²²

0 0

0 0 I 0

0 0 0 I

3

7

7

7

5

; (8) Then, as a direct consequence of the condensed form (4), we have the following well-known QSVD theorem.

Theorem 3 ( QSVD Theorem ) Let A

²

R

^nm

;C

²

R

^pm

, there exist orthogonal matrices U

²

R

^nn

;V

²

R

^pp

and nonsingular matrix X such that

U

^T

AX =

2

6

4

r 

ac^?

r

c

r

a

+ r

c^?

r 

ac

r 

ac^?

r

a

m

^?

r 

ac

r 

ac^?

r

c

I 0 0 0

r

a

+ r

c^?

r 

ac

0 S

A

0 0

n

^?

r

a

0 0 0 0

3

7

5

;

V

^T

CX =

2

6

4

r 

ac^?

r

c

r

a

+ r

c^?

r 

ac

r 

ac^?

r

a

m

^?

r 

ac

p

^?

r

c

0 0 0 0

r

a

+ r

c^?

r 

ac

0 I 0 0

r 

ac^?

r

a

0 0 I 0

3

7

5

; (9)

where S

A

is of the form (5), and U;V and X can be chosen to be given by (6), (7) and (8), respectively. 

i

;i = 1 ;

;s are dened to be the non-trivial generalized singular values of two matrices A;C .

According to the uniqueness theorem in [16], we only need to characterize matrices U;V given by (6) and (7) in order to characterize the role of orthogonal matrices in QSVD. Let U;V be given by (6) and (7) and partition these two orthogonal matrices by

U =

^h

r 

ac^?

r

c

r

a

+ r

c^?

r 

ac

n

^?

r

a

U

¹

U

²

U

^{3 i}

; (10)

V =

^h

p

^?

r

c

r

a

+ r

c^?

r 

ac

 r

ac^?

r

a

V

¹

V

²

V

^{3 i}

: (11)

(12) Then, from Lemma 2 we have

U

³

=

^T1

( A ) ; U

¹

=

^T1?

( A

^S1

( C )) ; (13) V

¹

=

^T1

( C ) ; V

³

=

^T1?

( C

^S1

( A )) : (14) Hence, in order to characterize the role of orthogonal matrices U;V in QSVD, it should only characterize the role of U

²

;V

²

in QSVD.

The following variational formulation has been established in [1] to characterize U

²

and

V

²

.

Theorem 4 Given A

²

R

^nm

;C

²

R

^pm

. Consider the optimization problem max

2

6

4

A

^T

C

^T

TT

1

( A ) 0 0

^T1T

( C )

3

7

5

"

x

?

y

#

=0; x⁶⁼⁰

k

y

^k

k

x

^k

: (15)

Then the non-trivial generalized singular values 

¹

;

;

s

of two matrices A;C are precisely the stationary values for the problem (15). Furthermore, let

"

x

¹

?

y

¹

#

;

;

"

x

s

?

y

s

#

be sta- tionary points of the problem (15) with corresponding stationary values 

¹

;

;

s

, then

U

²

=

^{h k}_x<sup>x11k


kx</sup>_x^{ssk i}

; V

²

=

^{h ky}_y<sup>11k


k</sup>_y^{yssk i}

: Proof. We prove Theorem 4 by the following three arguments.



Argument 1 Firstly, we characterize orthogonal matrices U

²²

;V

²²

in (5). Consider the optimization problem

x^T2A²²⁼

max

y^2TC²²; x²⁶⁼⁰

k

y

^2k

k

x

^2k

: (16)

Since A

²²

;C

²²

are both nonsingular, by Theorem 1 the 

¹

;

;

s

, i.e., the singular values of the matrix A

²²

C

^22?1

are precisely the stationary values of the problem (16), and, if

"

x

¹²

?

y

²¹

#

;

;

"

x

^s2

?

y

^2s

#

are the stationary points of the problem (16) with corresponding stationary values 

¹

;

;

s

, then

U

²²

=

^{h kx}_x¹²¹

2 k

x^s2

kx^s2k

i

; V

²²

=

^{h ky}_y²¹¹

2 k

y^s2

ky^s2k

i

:



Argument 2 Secondly, let

F

=

^f

"

x

?

y

#

j

x

²

R

ⁿ

;y

²

R

^p

;U

_Ta

x =

2

6

6

6

4

r 

ac^?

r

c

0 r

a

+ r

c^?

 r

ac

x

²

n

^?

r

a

0

3

7

7

7

5

;

V

_Tc

y =

2

6

6

6

4

p

^?

r

c

0 r

a

+ r

c^?

 r

ac

y

²

r 

ac^?

r

a

0

3

7

7

7

5

; x

^T

A = y

^T

C;x

⁶

= 0

^g

: Consider the optimization problem

max

"

x

?

y

#

2F k

y

^k

k

x

^k

: (17)

Obviously, we have that

"

x

?

y

#

is a stationary point of the problem (17) with stationary value  if and only if

"

x

²

?

y

²

#

is a stationary point of the problem (16) with the same stationary point  and furthermore

U

_Ta

x =

2

6

6

6

4

 r

ac^?

r

c

0 r

a

+ r

c^?

r 

ac

x

²

n

^?

r

a

0

3

7

7

7

5

; V

_Tc

y =

2

6

6

6

4

p

^?

r

c

0 r

a

+ r

c^?

 r

ac

y

²

r 

ac^?

r

a

0

3

7

7

7

5

:



Argument 3 Finally, for any x

²

R

ⁿ

;y

²

R

^p

, partition U

_Ta

x =

2

6

6

6

4

 r

ac^?

r

c

x

¹

r

a

+ r

c^?

r 

ac

x

²

n

^?

r

a

x

³

3

7

7

7

5

; V

_Tc

y =

2

6

6

6

4

p

^?

r

c

y

¹

r

a

+ r

c^?

 r

ac

y

²

r 

ac^?

r

a

y

³

3

7

7

7

5

:

Since

U

_Ta^T1

( A ) =

2

6

4

r 

ac^?

r

c

0 r

a

+ r

c^?

r 

ac

0 n

^?

r

a

I

3

7

5

; V

_Tc^T1

( C ) =

2

6

4

p

^?

r

c

I r

a

+ r

c^?

 r

ac

0 r 

ac^?

r

a

0

3

7

5

; it is easy to know that

"

x

?

y

#

2F

if and only if

2

6

4

A

^T

C

^T

TT

1

( A ) 0 0

^T1T

( C )

3

7

5

"

x

?

y

#

= 0 ; x

⁶

= 0 :

Note that

U

_Ta

U

²

=

2

6

4

 r

ac^?

r

c

0 r

a

+ r

c^?

r 

ac

U

²²

n

^?

r

a

0

3

7

5

; V

_Tc

V

²

=

2

6

4

p

^?

r

c

0 r

a

+ r

c^?

r 

ac

V

²²

r 

ac^?

r

a

0

3

7

5

; thus, Theorem 4 follows directly from the above Arguments 1, 2 and 3.

3 A Variational Formulation for RSVD

In Section 2 we have derived the QSVD of two matrices A;C based on the condensed form

(4). Now we will establish the RSVD of a matrix triplet ( A;B;C ) via an analogous condensed

form.

Lemma 5 Given A

²

R

^nm

;B

²

R

^nl

;C

²

R

^pm

. Then there exist orthogonal matrices P

²

R

^nn

;Q

²

R

^mm

;U

b ²

R

^ll

;V

c²

R

^pp

such that

PAQ =

2

6

6

6

6

6

6

6

4

k

¹

k

²

k

³

k

⁴

r 

ac^?

r

a

m

^?

 r

ac

k

¹

A

¹¹

A

¹²

0 0 0 0

k

²

0 A

²²

0 0 0 0

k

³

A

³¹

A

³²

A

³³

A

³⁴

0 0

k

⁴

A

⁴¹

A

⁴²

0 A

⁴⁴

0 0

r

ab^?

r

a

0 0 0 0 0 0

n

^?

r

ab

0 0 0 0 0 0

3

7

7

7

7

7

7

7

5

;

PBU

b

=

2

6

6

6

6

6

6

6

4

l

^?

r

b

k

³

k

⁴

r

ab^?

r

a

k

¹

0 0 0 B

¹⁴

k

²

0 0 0 B

²⁴

k

³

0 B

³²

B

³³

B

³⁴

k

⁴

0 0 B

⁴³

B

⁴⁴

r

ab^?

r

a

0 0 0 B

⁵⁴

n

^?

r

ab

0 0 0 0

3

7

7

7

7

7

7

7

5

; (18)

V

_Tc

CQ =

2

6

6

6

4

k

¹

k

²

k

³

k

⁴

r 

ac^?

r

a

m

^?

r 

ac

p

^?

r

c

0 0 0 0 0 0

k

²

0 C

²²

0 0 0 0

k

⁴

C

³¹

C

³²

0 C

³⁴

0 0

r 

ac^?

r

a

C

⁴¹

C

⁴²

C

⁴³

C

⁴⁴

C

⁴⁵

0

3

7

7

7

5

; where A

¹¹

;A

²²

;A

³³

;A

⁴⁴

, B

³²

, B

⁴³

, B

⁵⁴

, C

²²

, C

³⁴

and C

⁴⁵

are nonsingular.

Proof. See Appendix B.

Let the OSVD of B

^43?1

A

⁴⁴

C

^34?1

be

U

^44T

B

^43?1

A

⁴⁴

C

^34?1

V

⁴⁴

= diag

^f



¹

;

;

k^4g

=: S

A

; (19) where U

⁴⁴

;V

⁴⁴

are orthogonal matrices, 

^1



<sup>2 


</sup>



k⁴

> 0. Dene

U := U

b

2

6

6

6

4

I

l^?r^b

I

k³

U

⁴⁴

I

r^ab?r^a

3

7

7

7

5

; (20)

V := V

c

2

6

6

6

4

I

p^?r^c

I

k²

V

⁴⁴

I

^r^ac?r^a

3

7

7

7

5

: (21)

Similarly to Theorem 3, from Lemma 5 directly, we have

Theorem 6 ( RSVD Theorem ) Given A

²

R

^nm

;B

²

R

^nl

;C

²

R

^pm

. Then there exist

nonsingular matrices X

²

R

^nn

;Y

²

R

^mm

and orthogonal matrices U

²

R

^ll

;V

²

R

^pp

such that

X

^T

AY =

2

6

6

6

6

6

6

6

4

k

¹

k

²

k

³

k

⁴

r 

ac^?

r

a

m

^?

r 

ac

k

¹

I 0 0 0 0 0

k

²

0 I 0 0 0 0

k

³

0 0 I 0 0 0

k

⁴

0 0 0 S

A

0 0

r

ab^?

r

a

0 0 0 0 0 0

n

^?

r

ab

0 0 0 0 0 0

3

7

7

7

7

7

7

7

5

;

X

^T

BU =

2

6

6

6

6

6

6

6

4

l

^?

r

b

k

³

k

⁴

r

ab^?

r

a

k

¹

0 0 0 0

k

²

0 0 0 0

k

³

0 I 0 0

k

⁴

0 0 I 0

r

ab^?

r

a

0 0 0 I

n

^?

r

ab

0 0 0 0

3

7

7

7

7

7

7

7

5

; (22)

V

^T

CY =

2

6

6

6

4

k

¹

k

²

k

³

k

⁴

 r

ac^?

r

a

m

^?

r 

ac

p

^?

r

c

0 0 0 0 0 0

k

²

0 I 0 0 0 0

k

⁴

0 0 0 I 0 0

r 

ac^?

r

a

0 0 0 0 I 0

3

7

7

7

5

;

where S

A

is of the form (19), and U;V can be chosen to be given by (20) and (21), respectively,



¹

;

;

k⁴

are dened to be the non-trivial restricted singular values of matrix triplets A;B;C . From the uniqueness theorem in [16], we only need to consider matrices U;V given by (20) and (21) in order to characterize the role of orthogonal matrices in RSVD. Let U;V be dened by (20) and (21), respectively and partition

U =

^h

l

^?

r

b

k

³

k

⁴

r

ab^?

r

a

U

¹

U

²

U

³

U

^{4 i}

; V =

^h

p

^?

r

c

k

²

k

⁴

r 

ac^?

r

a

V

¹

V

²

V

³

V

^{4 i}

: We have

U

¹

=

^S1

( B ) ; U

⁴

=

^S1?

(

^T1T

( A ) B ) ; V

¹

=

^T1

( C ) ; V

⁴

=

^T1?

( C

^S1

( A )) : Furthermore, if we dene

¹

:= C

^S1

(

^T1T

( B ) A ) ;

²

:= A

^S1

(

^T1T

( B ) A ) ;

³

:= (

^T1

(

^2S1

(

¹

)))

^T

B;

then,

¹

= V

c^

2

6

6

6

4

k

³

k

⁴

r 

ac^?

r

a

m

^?

r 

ac

p

^?

r

c

0 0 0 0

k

²

0 0 0 0

k

⁴

0 C

³⁴

0 0

r 

ac^?

r

a

C

⁴³

C

⁴⁴

C

⁴⁵

0

3

7

7

7

5

;

²

= P

^{T }

2

6

6

6

6

6

6

6

4

k

³

k

⁴

r 

ac^?

r

a

m

^?

r 

ac

k

¹

0 0 0 0

k

²

0 0 0 0

k

³

A

³³

A

³⁴

0 0

k

⁴

0 A

⁴⁴

0 0

r

ab^?

r

a

0 0 0 0

n

^?

r

ab

0 0 0 0

3

7

7

7

7

7

7

7

5

;

³

=

2

6

6

6

6

6

4

l

^?

r

b

k

³

k

⁴

r

ab^?

r

a

k

¹

0 0 0 B

¹⁴

k

²

0 0 0 B

²⁴

k

⁴

0 0 B

⁴³

B

⁴⁴

r

ab^?

r

a

0 0 0 B

⁵⁴

n

^?

r

ab

0 0 0 0

3

7

7

7

7

7

5

U

_Tb

: (23)

Hence, from (23) we have

^h

U

¹

U

^{2 i}

=

^S1

(

³

) :

Thus, in order to characterize the role of orthogonal matrices U;V in RSVD we only need to characterize the role of U

³

;V

³

of U;V in RSVD. This can be done by the following variational formulation.

Theorem 7 Given matrices A

²

R

^nm

;B

²

R

^nl

;C

²

R

^pm

. Consider the optimization problem

max

2

6

6

6

6

6

6

6

6

4

A B

ST

1

(

"

A C

#

) 0

ST

1

( A ) C

^T

C 0 0

^S1T

( B ) 0

^T1T

( B

^S1?

(

³

)) B

3

7

7

7

7

7

7

7

7

5

"

x

?

y

#

=0; x⁶⁼⁰

k

y

^k

k

Cx

^k

: (24)

Then the stationary values for the problem (24) are precisely the non-trivial generalized singu- lar values 

¹

;

;

k⁴

of the matrix triplet A;B;C . Moreover, if

"

x

¹

?

y

¹

#

;

;

"

x

k⁴

?

y

k⁴

#

are the stationary point of the problem (24) corresponding to the stationary values 

¹

;

;

k⁴

, respectively, then

U

³

=

^{h ky}_y<sup>11k


ky</sup>_y^kk44 k

i

:

Proof. Same as the proof of Theorem 4, we prove part (a) by the following three arguments.



Argument 1 Firstly, we characterize U

⁴⁴

in (19). Consider the optimization problem

A⁴⁴x⁴⁼

max

B⁴³y³; y³⁶⁼⁰

k

y

^3k

k

C

³⁴

x

^4k

: (25)

Since A

⁴⁴

;B

⁴³

;C

³⁴

are nonsingular, so by Theorem 1, the stationary values of the prob- lem (25) are precisely 

¹

;

;

k⁴

, i.e., the singular values of the matrix B

^43?1

A

⁴⁴

C

^34?1

, and, if let the corresponding stationary points be

"

x

¹⁴

?

y

³¹

#

;

;

"

x

^k44

?

y

^3k4

#

, then U

⁴⁴

=

^{ ky}_y¹³¹

3 k

y^k34

ky^k34k



:



Argument 2 Secondly, dene

F

:=

^f

"

x

?

y

#

k

x

²

R

^m

;y

²

R

^l

;U

_Tb

y =

2

6

6

6

4

l

^?

r

b

0

k

³

0

k

⁴

y

³

r

ab^?

r

a

0

3

7

7

7

5

; Q

^T

x =

2

6

6

6

6

6

6

6

4

k

¹

0

k

²

0

k

³

x

³

k

⁴

x

⁴

 r

ac^?

r

a

x

⁵

m

^?

r 

ac

0

3

7

7

7

7

7

7

7

5

;

C

⁴³

x

³

+ C

⁴⁴

x

⁴

+ C

⁴⁵

x

⁵

= 0 ;Ax = By;y

⁶

= 0

^g

: Consider the optimization problem

max

"

x

?

y

#

2F k

y

^k

k

Cx

^k

: (26)

Since A

³³

;A

⁴⁴

;B

⁴³

and C

⁴⁵

are nonsingular, so a simple calculation yields that the problem (26) are equivalent to the problem (25) in the sense that the stationary values of the problem (26) are precisely the stationary values of the problem (25), i.e., 

¹

;

;

k⁴

, and,

"

x

?

y

#

is the stationary point of the problem (26) if and only if

"

x

⁴

?

y

³

#

is the stationary point of the problem (25) with same stationary value.



Argument 3 Thirdly, for any x

²

R

^m

;y

²

R

^l

, denote

U

_Tb

y =

2

6

6

6

4

l

^?

r

b

y

¹

k

³

y

²

k

⁴

y

³

r

ab^?

r

a

y

⁴

3

7

7

7

5

; Q

^T

x =

2

6

6

6

6

6

6

6

4

k

¹

x

¹

k

²

x

²

k

³

x

³

k

⁴

x

⁴

 r

ac^?

r

a

x

⁵

m

^?

r 

ac

x

⁶

3

7

7

7

7

7

7

7

5

:

Since

ST

1

(

"

C A

#

) x = 0

⁽⁾

x

⁶

= 0;

Ax = By =

⁾

x

¹

= 0 ; x

²

= 0 ; y

⁴

= 0;

ST

1

( A ) C

^T

Cx =

⁽⁾

C

⁴¹

x

¹

+ C

⁴²

x

²

+ C

⁴³

x

³

+ C

⁴⁴

x

⁴

+ C

⁴⁵

x

⁵

= 0;

ST

1

( B ) y = 0

⁽⁾

y

¹

= 0 : From (23), we also know

T

T

1

( B

^S1?

( )) By = 0

⁽⁾

y

²

= 0 : Therefore, we have that

"

x

?

y

#

2F

if and only if

2

6

6

6

6

6

6

6

6

4

A B

ST

1

(

"

A C

#

) 0

ST

1

( A ) C

^T

C 0 0

^S1T

( B ) 0

^T1T

( B

^S1?

(

³

)) B

3

7

7

7

7

7

7

7

7

5

"

x

?

y

#

= 0 ; y

⁶

= 0 :

Note that

U

_Tb

U

³

=

2

6

6

6

4

l

^?

r

b

0

k

³

0

k

⁴

U

⁴⁴

r

ab^?

r

a

0

3

7

7

7

5

;

so, Theorem 7 follows directly from the above Arguments 1, 2 and 3.

Similarily, we also have the dual result of Theorem 7 which characterizes the non-trivial generalized singular values 

¹

;

;

k⁴

and the matrix V

³

in (21). For the sake of simplicity, we omit it here.

4 Conclusion

In this paper, we have studied generalized singular value decompositions. We have given an alternative proof of the variational formulation for the QSVD in [1] and established an analogous variational formulation for the RSVD which provides new understanding of the orthogonal matrices appearing in this decomposition.

5 Acknowledgement

Some of Delin Chu's work was done during his visit in the Department of Mathematics at

The University of Bielefeld in Germany in April 1998. He is grateful to Professor L.Elsner for

his kind hospitality and nancial support. He also thanks Professor L.Elsner for his reading

and correcting the rst version of the present paper.

Appendix A

In this appendix we prove Lemma 2 constructively.

Proof. We prove Lemma 2 by 4 steps as follows:

Step 1: Perform simulaneous row and column compression:

U

^1T

AW

¹

=:

"

r

a

r 

ac^?

r

a

m

^?

r 

ac

r

a

A

¹¹

0 0

n

^?

r

a

0 0 0

#

;

V

^1T

CW

¹

=:

2

6

4

r

a

r 

ac^?

r

a

m

^?

r 

ac

p

^?

r

c

0 0 0

r

a

+ r

c^?

r 

ac

C

²¹

0 0

 r

ac^?

r

a

C

³¹

C

³³

0

3

7

5

with A

¹¹

;C

³³

nonsingular and C

²¹

full row rank.

Step 2: Perform a column compression:

C

²¹

W

²

=:

^h

r 

ac^?

r

c

r

a

+ r

c^?

r 

ac

0 C

^{22 i}

with C

²²

nonsingular. Set

A

¹¹

W

²

=:

^h

r 

ac^?

r

c

r

a

+ r

c^?

r 

ac

A

¹¹

A

^{12 i}

; C

³¹

W

²

=:

^h

r 

ac^?

r

c

r

a

+ r

c^?

r 

ac

C

³¹

C

^{32 i}

: Step 3: Perform a row compression:

U

^3T

A

¹¹

=:

"

r 

ac^?

r

c

A

¹¹

r

a

+ r

c^?

r 

ac

0

#

with A

¹¹

nonsingular. Set

U

^3T

A

¹²

=:

"

r 

ac^?

r

c

A

¹²

r

a

+ r

c^?

 r

ac

A

²²

#

: Step 4: Set

U

a

:= U

¹

"

U

³

I

#

; W := W

¹

"

W

²

I

#

; V

c

:= V

¹

:

Then orthogonal matrices U

a

;V

c

and W satisfy (4).