Generalized eigenvectors and sets of nonnegative matrices

Generalized eigenvectors and sets of nonnegative matrices

Citation for published version (APA):

Zijm, W. H. M. (1980). Generalized eigenvectors and sets of nonnegative matrices. (Memorandum COSOR; Vol. 8003). Technische Hogeschool Eindhoven.

Document status and date: Published: 01/01/1980

Document Version:

Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.

• The final author version and the galley proof are versions of the publication after peer review.

• The final published version features the final layout of the paper including the volume, issue and page numbers.

Link to publication

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:

www.tue.nl/taverne Take down policy

If you believe that this document breaches copyright please contact us at: openaccess@tue.nl

providing details and we will investigate your claim.

PROBABILITY THEORY, STATISTICS AND OPERATIONS RESEARCH GROUP

,

Memorandum COSOR 80-03

Generalized eigenvectors and sets of nonnegative matrices

by W.H.M. Zijm

Eindhoven, April 1980 .

by

W.H.M. Zijm

O. Abstract

In this paper we present extensions of the Perron-Frobenius theory for square irreducible nonnegative matrices. After showinq a generalization to reducible matrices, we extend the theory to sets of nonneqative matrices, which play an important role in several dynamic proqramminq recursions (e.q. Markov

decision processes) and in mathematical economics (e.q. Leontief substitution systems). We consider a finite set M of (in general reducible) matrices, which is generated by all possible interchanqes of correspondinq rows,

selected from a fixed finite set of square nonneqative matrices. A simultaneous block-trianqular decomposition of the set of matrices will be presented and characterized in terms of the maximal spectral radius and the maximal index, associated with this maximal spectral radius, usinq the concept of qeneralized eiqenvectors. As a by-product of our analysis we obtain a qeneralization of

Howard's policy iteration method. This paper extends earlier results of Sladky [7J and Zijm [8J.

1. Introduction

Nonnegative matrices play an important role in the analysis of certain dynamic programming recursions in e.g. Markov decision processes and in Leontief models with alternative techniques (compare Howard [4],

Burmeister and Oobell [1]). This paper extends results of Sladky [7] and Zijm [8J concerning the structure of nonnegative matrices and sets of nonnegative matrices.

We consider a finite set M of (in general reducible) NxN-matrices. This set is generated by all possible interchanges of corresponding rows, selected from a fixed finite set of nonnegative NxN-matrices. In this paper we study a simultaneous block-triangular representation of all matrices in M and give characterizations in terms of their spectral radii and the index of their spectral radii, using so-called generalized eigenvectors (compare Rothblum

[6]).

Concordant with the dynamic programming background of the problem, we

will refer to the indices l, •••

,N

as states and we will call S

=

{It •••

,N} the state space.

In particular we prove the existence of a matrix P e M and of a partition {Ok; k

=

l"",m} of the state space S, such that

1. [P]ij 0 for i € Ok' j € 0t' with t < k, for all P €M (here [P]ij

denotes the ij-th element of Pl.

Furthermore

and

max 0 (P (k»

P€M

= 0 (p. (k) ),. k = 1 _{, ••• ,m}

(where p(k) denotes the restriction of p to 0kxOk and o(p(k» is defined as the spectral radius of p(k), k

=

l"",m).

2. For k

=

l, ••• ,m there exist sequences of semi-positive (column)-vectors { (k) _{uR. ; ~} n

=

1 _{" " , v} } i h

k w t

l, ••• , 'V

(k) (k)

Here ~1 » 0 (strictly positive) and ~Vk+l

=

0, whereas v

k denotes the index of p(k) with respect to (J(p(k)}. The vectors

\.l~k),

I,

=

l,.t"v

k are

A (k)

called generalized eigenvectors of P ; k

=

l, ••• ,m.

For the special case that M contains exactly one matrix, the above result already gives a strong generalization of the Perron-Frobenius theory for

·(k)

irreducible matrices (note that P is reducible, with index V

k' and that (k)

the generalized eigenvector of highest order, \.1

1 ' is strictly positive).

For this case a number of important results may also be found in Rothblum [6J. A first classification for sets of square reducible nonnegative matrices, with the interchangeability property as described in the introduction, was given by Sladky [7J, for a very special case (V

k

=

1, k

=

l, ••• ,m). A b1ock-triangular decomposition for the set of matrices M in the general case was described in Zijm

[eJ.

However, the characterization in terms of nonnegative generalized eigenvectors is believed to be new, and appears to be very

useful in the study of a large number of dynamic programming recursions (Zijm [9]).

We conclude this section with a short overview of the organization of the rest of the paper. After summarizing some definitions and notational conventions we mention a few spectral properties of nonnegative matrices, together with a result, proved in Zijm

[eJ,

which may be viewed as a

starting point for the analysis in this paper (section 2). In section 3 we give a characterization of one nonnegative matrix in terms of its spectral radius and its index, using generalized eigenvectors. In section 4 these results are extended to the whole set of matrices, as expounded above. Finally, as a by-product of our analysis we obtain a generalization of Howard's policy iteration method (in the appendix, see also Howard [4]).

2. Preliminaries

N

We will work in the Euclidean space ~ • Matrices, resp. (column)-vectors are denoted by upper, resp. lower, case letters. We say that a matrix A is nonnegative (positive) - written A ~ 0 (A » 0) - if all its entries are nonnegative (positive). We say that A is semipositive - written A > 0 - if A ~ 0 and A ~ O. We write A ~

a

(A »

a,

A >

a)

if A -

a

~ 0 (A -

a

» 0, A -

a

> 0). Similar definitions apply to vectors.

For matrices, [AJ. denotes the i-th row of A, [AJ

ij the ij-th element of A.

~ D D

For a vector c, [cJ. again denotes its i-th element. For Des, A , resp. C ,

~

denotes the restriction of the matrix A, resp. the vector c, to DxD, resp. D. D

If D is a proper subset, then we call A a principal minor of A.

Let cr(A) be the spectral radius of a square nonnegative matrix A

#

0, then cr(A) equals the largest positive eigenvalue of A and we can choose the

corresponding eigenvector ~(A), such that we have ~(A) > 0 (Perron-Frobenius theorem). If A is irreducible, then ~(A) » 0 and cr(A) is simple.

Furthermore, we have

Lemma 1. The spectral radius cr(B) of any principal minor B (of order < N) of the nonnegative matrix A (of order N) does not exceed the spectral radius cr(A) of A. If A is irreducible then cr(B) < cr(A). If A is reducible, then cr(B)

=

cr(A) for al least one irreducible principal minor B.

A proof of lemma 1 may be found in Gantmacher [3J.

k . k

Define ~-CA) to be the null space of (A - 0 (A) I) • Then the index v CA) of A, with respect to cr(A), is the smallest nonnegative integer k such that

Nk(A) = Nk+1(A). It follows that

veAl

S N and that

N1(A) c N2{A) c ••• C NV(A) (A)

=

~(A)

for k

~

V(A) (compare e.g. Dunford

#

#

#

and Schwartz [2]) •. The elements of NV(A) (A) are called generalized eigen-vectors.

Furthermore we have

Lemma 2. Let P be a square nonnegative matrix with spectral radius o(P) and suppose there exists a strictly positive right-eigenvector, ~(P) say,

*

associated with o(P). Then there exists a matrix P , defined by

*

and PP P P

*

= a (P) P. Furthermore the matrix (0 (P) I - P + P ) is non-singular

*

(I is the identity matrix).

Proof. The matrix P, defined by

(p]

is stochastic (all row-sums equal to one). For stochastic matrices the result is well-known (Kemeny and Snell [5]). Using the inverse

transformation, the result for P is translated straightforwardly into a result for P.

Lemma 1 and lemma 2 will prove to be very useful in characterizing non-negative matrices which will be the aim of the next section.

3. Characterization of a nonnegative matrix

o

In this section we present a block-triangular decomposition for a specific matrix, and we characterize it in terms of its spectral radius and the generalized eigenvectors associated with it. We follow the terminology in Rothblum [6]. Let P be an NxN nonnegative matrix. Referring to the indices

l, .•• ,N as states, we say that state i has access to state j (or j is accessible from i), if for some integer n ~ 0, [pn]ij > O. An irreducible class is a set of states, such that each state has access to each other state. This implies that we may speak of having access to (from) an

irreducible class if there is access to (from) some state in the irreducible class. Irreducible classes are partially ordered by the accessibility

relation. We say that pc is irreducible, if C is an irreducible class with respect to P.

In the sequel of this paper a class will always mean an irreducible class. A class C of a square nonnegative matrix P is called basic if O(pc}

=

Q(~), otherwise it is called nonbasic (i.e. o(pc) < O(P); compare lemma 1). We have the following result (Zijm [8]):

Lemma 3. Let P be a square nonnegative matrix with spectral radius 0 and

index v. Then, after eventually permuting the states, we may write

Q

1

P (1) '"' P (0) ;

P 1 _{v- ,v-}1··· P 1 1

_.

_{v- ,} P _v,v P _{v , v - v ,} 1··· P 1

(0)

with cr(Pi,i)

= crJ i

= 1, •••

,v,

and cr(P ) < cr. If

v

~ 2 then every basic class of Pi,ihas access to some basic class of P_i-_l,i-l' for i = 2, •••

,v.

Finally each Pi,i possesses a strictly positive riqht eiqenvector,

associated with cr (i = l, •.. ,v) and Pi,j = 0 for i,j = l, ••• ,v and i < j.

Next we present a characterization of the block-triangular structure of P in terms of generalized eigenvectors.

Lemma 4. There exist vectors ~(I),~(2), ••. ,~(v), such that

(1)

P(1)~(k)

=

cr~(k)

+

~(k+1)J

k = l, •••

,v,

(~(v+l)

=

0)

where pel) is defined as in lemma 3. If

~i(k)

denotes the restriction of

~(k) to the states of Pi,i (i,k

=

1, ••• ,v), then

~k(k) » 0, ~i (k)

=

0 for i < k, i,k = 1, ••.

,v •

Proof. Define, for n

= l, •••

,v

p

n,n P n,n-1 •••.••• P n, 1

P _{n- ,n-}1 l··· P _{n- ,} 1 1

...

.

(v) (1) (1) (2) (v) .

hence R

=

P • We will prove (1) for R ,R , ••• ,R subsequently by induction. For n

= 1 the result is obvious since R(l)

= P possesses a

1,1

strictly positive right eiqenvector, Yl (1) say, with respect to

a

(lemma 3). (n-l)

Hence, suppose n ~ 2 and let the result hold for R , with the vectors y(l), •.• ,y(n-l),y(n)

= O. We will now prove (1) for R

Cn). Consider the following equations

(2) p xen) n,n P _n,n x(n-l) +P _n,n-lY ten-l) n-P _n,n x (n-2) + P _n,n-t Y _n-1 (n-2) + P _n,n-2Y _n-2 (n-2)

=

O'x(n)

=

ax(n-1}+x(n) = ax(n-2)+x(n-l) P x(1) n,n + P n,n-lY n-1 (1) +P n,n-2Y n-2(1) + ••• +P 1Y1(1) - axel) +x(2) • n, Here x(t), ••• ,x(n) are the unknown vectors. By lemma _, A in the appendix there

*

exists a unique solution of (2), together with P x(1)

=

O. Hence, certainly n,n

there will be a (not unique) solution of (2), which we denote by:

(3) xCi) - x (1); i

=

l, ••• ,n

n

*

(recall that P exists, by lemma 3 and lemma 2). From the first equation n,n

of (2) we derive

x (n)

n

=

P

*

n,n n

x (n)

*

and multiplying the second equation of {2} with P yields {lemma 2}:

n,n

(4) x (n) _n

=

P

*

_{n,n n,n-}P t Y _n-· 1 (n-l) •

Since P possesses a strictly positive right eigenvector, associated with n,n

a, every non-basic class of P has access to some basic class of P •

n,n

n,n

Furthermore, every basic class of P

n,n has access to some basic class of P _n-l,n-l' Together with y _n-l(n-1) » 0 this implies

x (n) » 0 • n

Combining the equations (2) with the induction hypothesis, and defining

y'(k) =: [Xn(k)] ; k

=

1,.~.,n,

y(k)

y' (n+l) = 0

(n)

the result follows immediately for R , with y'(l) , ••• ,y'(n),y'(n+1}.

Having proved the induction step, the proof of the theorem is complete, i.e.

Corollary. The vectors ~(1),~(2), ••. ,~(v) of lemma 4 may be chosen such that ~i (k) » 0 for i ~ k, ~i (k)

=

0 for i < k.

Proof. If {~(l), ••• ,~(v)} satisfies (1), then the same holds for {w(l), ••• ,w(v)}, defined by

w(v) == ~(v)

w(k) = ~(k) + aw(k+l) ; k == l, ••• ,v-1, a E' JR.

Since ~k(k) » 0 and ~i (k)

=

0 for i < k; i,k == 1, ••• ,v, it is possible to choose a so large that wi (k) » 0 for i,k

=

l, ••• ,v; i ~ k.

0

(1)

Lemma 4 and its corollary characterize P , a very particular principal minor of P. However, it will be obvious that we may continue in the same way with p(O), i.e. defining basic classes and index of p(O) with respect to a(p(O» (strictly smaller that

a),

we may decompose p(O) according to

lemma 4 again. Continuing in this way, we finally obtain a decomposition of P, such that - taking into account the corollary above - we have

Theorem 5. Let P be an NxN nonnegative matrix. Then there exists a partition {Ol""'Om} of S == {l, ••• ,N} such that

1. [PJ

ij

= 0 for i

E' Ok' j E' 0t' with t < k,

t,

k == 1, ••• ,m. > a (P (m) )

(here pCk) denotes the restriction of P to Ok x Ok)'

3. For k == l, ••• ,m there exist sequences of semi-positive (column)-vectors { (k) _{~t ; t}

₌

}

l " " , vk with

; t == l, ... , v_k •

(k)

0, whereas IV

k denotes the index of P with

1, ... ,m.

o

We end this section with the remark that lemma 4 may also be found in

Rothblum [6J, although the proof given here is slightly different. The block-triangular representation (which was not given in [6J) and the extensions of theorem 5 are given in order to emphasize the fact that parts of the matrix

are completely characterized already by smaller spectral radii. Moreover, our approach enables us to extend our results to the whole set of matrices, which will be the aim of the next section.

4. Characterization of the set M of nonnegative matrices

We now return to the situation of the set of matrices M with the inter-changeability property as described in the introduction. Our main goal will be to establish a result more or less similar to theorem S. To be specific we want to prove:

~

Theorem 6. There exist a matrix P ~ M and a partition {DI, ••• ,D

m} of S

=

{l, .•• ,N} such that

1. For all P ~ M: [PJ

ij

=

0 for i € Dk, j € Dt, with t < k; k,t

=

l, ••• ,m.

2. max oCp(k»

=

o(pCk»; k

=

l, ••• ,m, whereas P€M

(pCk) denotes the restriction of P to Dk x D

k, for P € M).

3. For k

=

l, ••• ,m there exist sequences of semi-positive (column)-vectors { (k) _{~t ; ~} n

=

I

_"",v

} ith

k

w

t

=

_l"",vk

Here

~:k)

» 0,

~~k~l

=

0, whereas "k denotes the index of pCk) with

~(k) k

respect to a (P ); k

=

1, ••• ,m.

IJ

Before proving this theorem, we need some more results concerning sets of nonnegative matrices. The following lemma is the analogue of lemma 3 in section 3:

Lemma

7.

There exists a partition {CO,CI' ••• ,C

v} of S

=

{l, ••• ,N}

such that by possibly permuting the states we may write for all P:

P v,v P

v,v-

I ... P

v,

I

P I

_{v- ,v-}

1···P I 1

_{v- ,}

....

_...

,

(0) .

(here P , resp. Pk,k' denotes the restriction of

P

to Co X

CO'

resp. C

k x Ck' for k

=

l, •••

,v,

Vp€M' Furthermore, Pk,t

=

0 for

t

> k~ k,t

=

l, •••

,v

and

v [p

J .,

= 0 for i e: CO; j € U C

k' for all P € M).

1J k=l

Moreover, there exist a matrix P € M, and strictly positive vectors ~k'

k

=

l, ••• ,v, such that

with

0,

k

=

l, ••• ,m, max a(PO,O} < a •

P€M

-Finally, if v ~ 2, then every basic class of Pk,k has access to some basic class of Pk- 1,k-l' for k

=

2, ••• ,v.

o

Proof. See Zijm [8J, theorem 6. In fact, that theorem also gives the further decomposition of the matrices P € M, restricted to CO' but for our goal the

formulation given here is sufficient and more convenient.

Remark 1. In Zijm [8J it is proved that v is precisely the index of P with respect to a

=

o(P) (lemma 1). Compare also Rothblum [6J.

Remark 2. It is important to notice that a matrix P with

max Pk k~k

=

Pk,k~k

=

a(Pk,k)~k and Pk ,l

=

0 for 1 > k, Vp ' is not unique in

Pe:M '

-general. However, there is at least one matrix P with these properties and such that every basic class of Pk,k has access to some basic class of P k-l, k-l' for k

=

2, ••• , v.

As in the preceding section we will restrict out attention to the principal (1)

minors P , VPEM* We have

Lemma 8. Let S be partitioned (and eventually permuted) as in lemma 7.

.

-

-

-Then there exist vectors U(1),U(2), ••• ,U(v) and a matrix P E M such that

(5 ) max P (1) U(k) = P . - ( 1 ) · U(k) = au(k) + U(k+l), k . = l, •••

,v

(U(v+l)

=

0) • PEM

Furthermore, if we denote by ui(k) the restriction of U(k) to C i, i,k

=

l, ••• ,v, we have

Uk(k) » 0, U

i (k)

=

0 for i < k, i,k

=

1, ••• ,v •

Proof. The proof will follow almost the same lines as that of lemma 4.

Define for n

=

1, ••• ,v and for each P:

R(n) =: P P n,n P n,n-1 •••••• P n, 1 P 1 _{n- ,n-}l··· P 1 1 _{n- ,}

,

.

, ,

(according to the decomposition of the state space S, indicated in lemma 7). The proof will be by induction again. For n

=

1 we have, according to lemma 7:

max R(O (1) "'" R(l)y (1)

=

P y 1 1

P

for some strictly positive Yl (1) and some matrix R(l) on C

1 XCI'

Now suppose n

~

2 and let (5) hold for all

~n-l}

and some R(n-l) , with the vectors y{l), ••• ,y(n-l),y(n)

=

O. Consider the following functional

(6)

r

max{P x(n)} P€M n,n max{P x(n-1)+P lY . 1 (n-l)} P€M n,n n,n- n-max{P x(n-2)+P lY l(n-2)+p 2Y 2(n-2)} P M € . n,n n,n- • n - . n,n- • n-= axen) = ax (n-l) +x (n)

= ax(n-2)+x(n-l)

max{P x(1) + P lY 1 (1) + P 2Y 2 (1)+ ••• +P lY_l (1)}= axel) + x(2) P€M n,n n,n- n- n,n- n- n,

where x(1),x(2), ••• ,x(n) are the unknown vectors. Since P, the matrix defined in lemma 7, satisfies the following inequality

(compare the proof of lemma 4), it follows from lemma B in the appendix that there exists a solution of (6) which specifies the maximizing matrix rows, denoted by [P]i' i € C

n and the vectors x(k); k

= l, ••• ,n. Suppose

x (k)

=

x (k), k

=

l, .•• ,n •

n

Then x (n) » 0 (lemma B).

n

Combining the functional equations (6) with the induction hypothesis and defining [ X (k)

1

y I (k)

=

n , k = l, ••• , n , y(k) y' (n+l)

=

0 and

the result (5) follows for all R(n) and a(n) , with the vectors P

Corollary. The vectors ~(l), ••• ,~(v) of lemma 8 may be chosen in such a way that ~i (k) » 0 for i ~ k, Pi (k)

=

0 for i < k.

Proof. Analogous to the proof of the corollary after lemma 4.

It will be clear that we may decompose Co in exactly the same way as we did with S in lemma 7 (recall that max a(P

O 0) <a), and prove lemma 7 and 8

PeM '

o

again. Continuing in this way we will get the block-triangular decomposition, described in theorem 6.

In fact, we already established this block-triangular decomposition in an earlier paper (Zijm [8]). However, the characterization in terms of

generalized eigenvectors is new and appears to be very useful in the study of the· asymptotic behaviour of several dynamic programming recursions

Appendix

In this appendix we will establish the existence of the solutions of tha sets of equations in the proofs of lemma 4 and lemma 8. The proof of lemma B

appears to be a generalization of Howard's policy-iteration method (Howard

[4J). First we have

Lemma A. Let P be a square nonnegative matrix, with spectral radius

a,

and let P possess a strictly positive right eigenvector, associated with o. Then the set of equations

(a) Px(n)

=

ax(n) Px(n-l) + r(n-l) = ox(n-l) + x(n) Px(n-2) + r(n-2)

= ox(n-2) + x(n-l)

Px(1) + r(l)

*

P x(1)

= ox(l)

=

0 + x(2)

possesses a unique solution. (here rei), i

=

1, ••• ,n-l are column vectors.)

*

Proof. Multiplying both sides of the equations with P yields (lemma 2):

*

*

(b) P x(k)

=

P r(k-l), k

= 2, ••• ,n •

Repeated application of the first equation gives

*

(c) x(n) = P x(n) •

Combining (a), (b) and (c) it is easy to determine the unique solution of the set of equations (a)

*

rX(nl

=

P dn-l) x (n-1)

₌

(Ol-P+P)

*

-1 (r(n-l) +P r(n-2)

*

- x (n) )

*

-1

_*

x(2)

₌

(ol - P +P) (r(2) +Pr(1)· - x(3» x (1) = (ol - P +P)

*

-1 (r(1) - x(2»

*

Lemma B. Suppose we have a finite set Z of nonnegative NxN-matrices, with the interchangeability property as described in the introduction.

Let cr

=

max cr(P) and suppose there exists a matrix Po €Zand a strictly P€Z

positive vector y such that Poy

= maxPy

= cry. With every matrix P

€ Z there

P€z.

is associated a set of vectors {r(P;k), k

=

O, ••• ,n-l} and we suppose

Then the set of functional equations

(d) max {Px(n)} P€Z = oxen) max {Px{n-l) + r{P;n-l)}

=

ox(n-l) + x(n) P€Z max {px(n-2) + r(P;n-2)}

=

crx(n-2) + x(n-l) P€z

l

max {Px(1) P€Z + r (P; 1)}

=

ox(l) + x(2)

possesses a solution {p,x(1),x(2), ••• ,x(n)} = (p,x(1),x(2), ••• ,x(n)} with x(n) » 0 •

Proof. By lemma A the set of equations

pox(n)

= crx(n)

P Ox(n-l) + rCPO,n-l)

= ox(n-i) + x(n)

p ox(n-2) + r(po,n-2)

=

ox(n-2) + x(n-l) = Ox (1) + x(2)

=

0

possesses a unique solution (x(l), ••• ,x(n»

=

(x(P

O;l), ••• ,x(PO;n», with

*

X{POin)

=

P

E(n) P

1x(PO,n) ... max _pez {px(PO,n)}

1!: (n-1) P 1 x (PO; n-l) + r(P₁ ;n-l)

=

max {px(P O,n-l) + r{P;n-l)} Pe:A n E(n-2) P 1 x (P 0 ;n':' 2) + r{P 1,n-2) ... max {Px{PO,n-2)

+

r{P;n-2)} peA 1 n-E(1) PI x(P 0; 1) + r (P

1 ;1)

=

max {px(PO,I)

+

r (P, I)} • Pe:A

2

Here ~cz denotes the set of matrices, which maximize the right-hand side of equation E (k), k = 1, ••• ,n. Hence An ~ A

n_1 ~... :;) A1• We choose PI

n

equal to Po if Po E: Al

=

n

~.

k=1

Finally we define the sequence of vectors {~O(k); k = 1, ••• ,n} by

F(n) P

1x(Pa;n) ... O'x(Pa;n)

+

~O(n)

F (n-1) P

1x(PO;n-l) + r (P I; n-l) == O'x(PO,n-1)

+

x(Pam)

+

~O (n-1)

F(n-2) P

lx(PO;n-2) + r(Pl,n-2) ... O'x(PO,n-2)

+

x(PO,n-l) + ~O(n-2)

F (1)

It follows immediately that ~O(n) ~ 0 and that [~O(j)Ji

= 0,

j = k+l, ••• ,n. implies [~O(k)]i ~ a, for i € {l, ••• ,N}, k e {l, ••• ,n-l}.

Furthermore

has access to some basic class of P

l" Since PlY S O'y and y » 0 it is c~ear that P1 possesses no connected basic classes, which implies that with P

l and

0' there is associated a strictly positive eigenvector again. By lemma A

there exists a sequence of vectors {x{P

G(n) = C1X(P 1,n) G (n-1) P 1x(P1,n-1) + r(P1,n-1) = C1X(P 1 ,n-1) + x(P1,n) G(n-2) P 1x(P1,n-2) + r(P1,n-2)

=

C1x{P1,n-2) + xeP1 In-I) G (1) P 1x(PI,l) + r(P1,l)

= C1X(P

1,1) + x(PI,2) G(O) P

*

1x(P1,1)

=

o •

We will prove the following assertions: 1. x(PI;n) ~ x(PO;n) •

2. x(Pt,j)

=

x(Po,j) for j

=

k+l, ••• ,n implies x(P

1,k) ~ x(PO,k) 3. x(P₁,j)

=

x(Po,j) for j

=

l, ••• ,n if and only if PI

= Po •

Ad 1.

Denote by D the set of states which belong to a basic class of Pl' Since PI possesses no connected basic classes and x(n;P

O) » 0 we have [~O(~)]i = 0 for i € D, hence [~O(n-l)]i ~ 0 for i € D. Multiplying F(n-l) by PI implies:

(h)

In the same way we derive from G(n-l) and G(n) :

(i)

Repeated application of Fen) gives (recall that ~o(n) ~ 0):

(j)

Combination of (h), (i) and (j) yields: x(P

1,n) ~ x(PO;n) » O. Ad 2.

Define 6x(j)

= x(Pt;j) - x(Po;j), for j

=

1, ••• ,n. Then, by F(j) and G(j) we have

P_I6X(j)

= C16x(j) + bX(j+l) -

~O(j); j

=

l, ••• ,n

hence bX(j)

=

0 for j

=

k+l, ••• ,n implies: ~O{j)

=

0 for j

=

k+l, ••• ,n, and we may conclude Po € ~+l and ~O(k) ~ O.

From F(k) and G(k) we have, since 6x(k+l)

= 0:

(1)

*

*

Multiplying both sides with P

1 gives Pl~O(k)

= O. Since

~O(k) ~ 0 we have

For k ~ 2 we proceed as follows: [~O(k)Ji

= 0

for i ~ D implies

*

[~O(k-l)Ji ~ 0, for i ~ D, hence Pl~OCk-l) ~O. Multiplying equations F(k-l)

*

and G(k-l) with P

l and subtracting gives:

By (1) and ~O(k) ~ 0 we find

*

Combining these results yields 6x(k) ~ Pl~O(k-l) ~O.

*

*

For k

=

1 we choose [P

1Ji

= [POJ! for i

€ D, hence [PlJi

= [POJ! for i

€ D, which implies [X(P1i1)Ji

=

[X(POil)J

i for i ~ D (since the equations for i € D completely determine the solution ([x(1)]i i i € D}). This implies:

By (1), ~O(l) ~ 0 and the definition of 6x(1) we conclude

Ad 3.

As above we have, by F(j) and G(j): P

16x(j)

=

a6x(j) + 6X(j+l) - ~O(j); j

=

l, ••• ,n ,

hence 6x(j)

=

0, j

=

1, ••• ,n implies ~O(j)

=

0, j

=

l, ••• ,n, in which case we choose P

1 equal to PO' The inverse implication is trivial.

Once having 1, 2 and 3 it is immediately clear that the policy-iteration method, suggested by .the equations E and G is not cycling. If P

l ~ Po we may apply the improvement procedure again. Since Z is finite the lemma is

References

[IJ Burmeister, E. and A. Dobell, Mathematical theories of Economic Growth, MacMillan, New York, 1970.

[2J Dunford, N. and J.T. Schwartz, Linear Operators, Part I, Interscience, New York, 1958.

[3J Gantmacher, F.R., Applications of the theory of matrices, translated from Russian by J.L. Brenner, Interscience, New York, 1959.

[4J Howard, R.A., Dynamic programming and Markov processes, Wiley, New York, 1960.

[5J Kemeny, J.G. and J.L. Snell, Finite Markov chains, Van Nostrand, New York, 1960.

[6J Rothblum, U.R., Algebraic, Eigenspaces of nonnegative matrices, Linear Algebra and its applications ~, 281-292, 1975.

[7J Sladky, K., Successive approxi.mation methods for dynamic programming models, Proceedings of the Third Formator Symposium on

Mathematical Methods for the analysis of Large-Scale Systems, 171-189, Prague, 1979.

[8J Zijm, W.H.M., On nonnegative matrices in dynamic programming I, Memorandum COSOR 79-10, Eindhoven University of Technology, 1979.

[9J Zijm, W.H.M., Asymptotic behavior of dynamic programming recursions with general nonnegative matrices, Memorandum COSOR 80-04,