R-theory for countable reducible nonnegative matrices

R-theory for countable reducible nonnegative matrices

Citation for published version (APA):

Zijm, W. H. M. (1981). R-theory for countable reducible nonnegative matrices. (Memorandum COSOR; Vol. 8101). Technische Hogeschool Eindhoven.

Document status and date: Published: 01/01/1981

Document Version:

Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.

• The final author version and the galley proof are versions of the publication after peer review.

• The final published version features the final layout of the paper including the volume, issue and page numbers.

Link to publication

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:

www.tue.nl/taverne

Take down policy

If you believe that this document breaches copyright please contact us at: openaccess@tue.nl

providing details and we will investigate your claim.

Department of Mathematics

PROBABILITY THEORY, STATISTICS AND OPERATIONS RESEARCH GROUP

Memorandum COSOR 81-01

R-theory for countable reducible nonnegative matrices

by W.H.M. Zijm

Eindhoven, February 1981

by

W.H.M. Zijm

O. Abstract

In this paper we generalize Vere-Jones R-theory to reducible nonnegative matrices of countably infinite dimension. We discuss 8-excessive functions and eigenvectors, associated with the convergence parameter R. Furthermore we show how generalized eigenvectors, associated with R, can be constructed. Under special conditions, related to strong ergodicity in Markov chains,

these generalized eigenvectors can be chosen nonnegative. The results are extended to sets of nonnegative matrices which often appear in optimization problems (e.g. Markov decision chains, Leontief substitution systems).

I. lntroduction

NOllnegativl:! matrices play all important roJe in several interesting and frequently studied prublems, e.g. in probability theory (Markov chains, bran<.:hing processes) and in mathelllati<.:al economics (input-output analysis). To quote just a ft'w references, see Chung 13], Harris [5] and Gale [4J.

Tile most fundamental toui for the investigation of these problems proves to be the well-known Perrou-Frobeniu5 theorem. This result states that the spectral radius of a square, finite-dimensional nonnegative matrix (i.e. the absolute value of the largest eigenvalue of that matrix) is an eigenvalue, and that nonnegative left- and right-eigenvectors can be associated widl it. Moreover, if the matrix is irreducible*, the eigen-vectors can be chosen to be strictly positive.

These results have been generalized in several ways by e.g. Birkhoff [2], Karlin [9J, MandlandSeneta [IIJ,Rothblum [12J, Sladky [14J,[15J, Vere-Jones [17], [18J and Zijm [20J, [2IJ. For this paper the results ofVere-Jones are of special importance. In fI71,IIS] Vere-Jones studies irreducible nonnegative matrices of countably infinite dimension. A basic role in his analysis is played by the paramelt'r I{, tit" cunUllon convergence radius of the series

(n) n . (n)

L: p .. z (where p.. IS til" ii-th element of the n-th iterate of a

n=O 1J IJ

-I

matrix P). 14itl1 R we ,:an dssu('i,lte a strictly positive vector u, such

I -) . , , -I F h

t 1at Pu ;;, R u and und,,~r Spt!C 1'.I! CUndl.tlons Pu '" R u. urt ermore, x IS a strictly positive veclor sucll that Px ;;, ax for an irreducible non-negative matrixP and a constant

B

(such a vector is called S-excessive*),

-I -I

then a ~ R (although R may be strictly smaller than the spectral radius of P; for an example see Seneta I I J], p. 169). This is of particular

importance for the exploitation ot certain contraction properties of non-negative matrices and for the construction of a-excessive functions with

an (almost) absolutely soul lest pxcpssivity factor 13. More generally,

the main advantage of Vere-JoIlL's' l{-theory is that the results are inde-pendent of any norm ChUSl'1l j II advance; on the contrary, once having

calculated a B-excessive fllnction with (almost) smallest excessivity factor

IS, we are able to define il w('igill"d supremum norm (Wessels [19J), such that certain contraction pruperl i.,"; :In' <oplimally exploited.

In the next section we first introduce some notations and definitions, after which we briefly summarize the main results of Vere-Jones [18J, which are of particular importance for this paper. In section 3 we discuss nonneg.;l.tive matrices with (almost) optimal S-excessive functions. We also show how, under some restrictions, generalized eigenvectors (associated with R-1) can be con-structed, and (under one additional constraint) that they can be chosen nonnegative. The conditions are separately discussed in section

4;

one of them is related to strong ergodicity in Markov chains. The results in section 3 generalize those of Rothblum [12J for finite-dimensional non-negative matrices, a case in which all necessary conditions are trivially fulfilled. In section 5 we turn to sets of nonnegative matrices and show how the results of section 3 can be generalized. This is of particular importance for applications to various problems in optimization theory. We conclude the paper with some remarks concerning relaxation of the conditions and with indication of relations with other literature.

2. Preliminaries

Let P denote a nonnegative matrix of countably infinite dimension. By

(n) h .. h 1 n h (n)

p .. we mean t e ~J-t e ement of P ; we assume throughout t at p .. < 00

1J (0) 1J

(n = 1,2, ••• ). Furthermore Pij == 0ij (the Kronecker delta; 0ij

=

1 if i

=

j, 0 .• == 0 otherwise). An extensive use will be made of so-called

1.J (

taboo-transitions

HP'~)'

where H 1S a subset of the indices, a so-called

b d f ·

~J

(0) l' f

~.

t

H; (0) 0 . f . d f

t a 0 0 se. t W e e 1.ne HP • • !J. • ~ p. .

=

,

... 1. cHan '-- or

1.J 1.J H 1.J n > == (2. 1 ) (n) HP _1.J .•

I

i I ' .•. ,in-I " H p. . 1. n_IJ

H may be empty; in this case it ~s omitted from the notation. If H consists of a single state k we write

called first entrance transitions

f~~)

1.J

(2.2) • p •. (n)

J ~J n=0,1,2, •••

k

P~~)'

Furthermore we define

so-~J

by

(n) (n)

Furthermore we write Hf .. for. HP " , etc. etc. (all these concepts are _{1.J J,1.J} defined in Chung [3J for homogeneous Markov chains).

Motivated by the theory of Markov chains we call S

=

{1,2, ••• } the state space. We say that state i has access to state j (j is accessible from i)

i f p

~r:)

>

°

for some n. I f any two states have access to each other we

~J

call P irreducible, otherwise reducible. If P is reducible then it is pos-sible to partition the state space into subsets, C

I, C2, ••• say, such

that the restriction of P to C

k is irreducible, for all k. Such an irre-ducible subset will be called a class. For the classes a partial ordering exists; we say that C

k has access to C,t i f that is so for some (equiva-lently any) state i E C

k and some (equivalently any) state j E Ct'

Let x be a (column)vector, its components are then denoted by x .. \ole write ~

x > 0 if x is nonnegative , x » 0 if x is strictly positive (all components

=

positive). We say that x is semi-positive, written x > 0, if x ~

°

and x ~ 0. Similar we write x ~ y if x - y ~ 0, etc.

A 8-excessive function for a nonnegative matrix P ~s a semi-positive vector x such that Px ;; f3x. The constant f3 is called the excessivity factor.

We next summarize some results of the R-theory for nonnegative matrices (Vere-Jones [17J,[18]). Define for a nonnegative matrix P

(2.3)

HP •• _1J (z) i,j E S ,

where H c S is some taboo-set. Consider especially P .. (z): let R .. be its

1.J 1.J

convergence radius and define

(2.4) R = inf R •.

i, j 1.J

In order to avoid trivialities we assume R > O. It can be shown that R < 00

and that R ••

=

R for all i,j E S if P is irreducible.

1.J

In the same way we may define the series HF .. (z) by

1.J 00

(2.5) HF .. (z)

=

\' L Hf .. (n) n z

1J n=1 1J i,j

E S •

It is easy to prove, using a first entrance decomposition (Chung [3J),that

P .. (z) = 0 .. + F .. (z)P .. (z)

from which we immediately deduce

(2.6) F .. (R-)

=

lim F .. (z) <

11 z-+R 11

i E S •

Furthermore we have F .. (R-) < 00, for all i,j

J1 S, if P is irreducible.

We say that state i is R-recurrent if F .. (R-)

=

I, R-transient if

11

F .. (R-) < 1. A further classification can be reached by taking into 11

account the series _In •• (z), defined by

J:i 1J (2.7) co ( ) \ n fen) n-l Hm •• z

=

L H iJ' Z 1J n= 1

\\Te say that an R-recurrent state i is R-positive if

(2.8) m .. (R-) = lim m .. (z) < 00 ,

11 z-+R 11

and R-null if m .. (R-)

=

co. It is well-known that for irreducible matrices 11

the above mentioned characteristics are "solidarity properties". Hore precisely we have (Vere-Jones [18J).

Lemma I: Let P be irreducible. Then all states are either R-recurrent or R-transient. Moreover, in case of R-recurrency, all states are either R-positive or R-null.

Another example of a solidari property 1S the period of the states in

the case of an irreducible nonnegative matrix. We define for state i the integer d. as the greatest common divisor (g.c.d.) of those indices n >

a

1

for which

p~~)

> O. It is well-known that all numbers d. are equal if P

11 1

is irreducible, hence we may speak of a common period d of all states. We say that P is aperiodic if d

=

I, otherwise P is called periodic. Obviously a further classification of

by considering higher moments. Define

R-positive states can be obtained

(k) Hm .. (z) by 1J (2.9) (k) Hm .. (z) 1J

L

(n) f(n+I)-k n-k k H ij z k :: 0,1,2, •••• n=k

o

Note that

.JU~?)(z)

- HF .. (z);

.JU~~)(z)

= Hm .. (z). We have the following

ti 1J 1J H 1J 1J

result:

Lemma 2: Let P a nonnegative matrix with convergence radius R and let

H

m~~)(R-)

< 00 for all k and for i,j E S, where H c S is some taboo set. 1J

Define jH = H

u

{j} • Then for k > 1 =

(2.10)

.JU~~)

(i-) =

H 1J

Proof: For

Izl

< R we have

\' (k) z L p·o .JU_{o '} (z)

=

2/jH 1", 11 "'J = 00 \' (n) f(n+2)-k L k H ij n-k 00 (n+l )-k ~ z

=

L n=k \' n f(n+2)-k (n+l)-k + L (k-l) H ij z = n=k 00

I

(n+l) f(n+2)-k z (n+I)-k -n=k-l k H ij 00 00 i,j E S • (n+J)-k z

=

(n+l) f(n+2)-k k H ij (n+1)-k z -00 (n+l)

I

(k~

I) _{H ij} f(n+2)-k z n=k-I f(n+I)-k

I

(n) n-k

_L

n f(n+l)-(k-I} n-(k-I) = z _(k-I) z n=k k H ij n=k-I H ij (k) (k-I) Hm .. 1J (z) - Hm. . 1J (z) - k =

(the first equality follows from Fubini's theorem for absolutely convergent series). Hence (k) Hm." (z) 1J = (k-l) \' (k) Hm.. _1J (z) + z _!l.,/jH L p,o HmoJ' _{1", '"} (z) ~ for

1

zl

< R ='

since the series are monotone increasing in z. We then find

(k) Hm.. 1J (R-)

The inverse inequality follows analogously (or by Fatou's lemma). This proves the lemma.

The following result can be found in Seneta [13J.

Lemma 3: Let P be an irreducible nonnegative matrix with convergence radius R and let x be a B-excessive function for P. Then x is strictly positive

-1

whereas furthermore

e

> R • If x is normalized in such a way that x = I

=

s

for some s E S, then x > u(S), where u(S) is defined by

=

(2. 11 ) u.(I3) :::: (I -

o.

)F. (13 -I ) +

o.

1 1S 1S 1S

Clearly u(S) is also strictly positive and

i E S •

-I

S-excessive, for S ~ R • In

-I

particular we have that u(R ) is an eigenvector, associated with R , i f - -1

o

and only if P is R-recurrent.

o

A final word about the convergence radius R. Clearly a finite irreducible nonnegative matrix has spectral radius R-1; such a matrix is always

R-positive. But even irreducible countably infinite nonnegative matrices may have spectral radius strictly larger than R-1 (for an example, see Seneta [13J, p. 169).

In the following section we will discuss extensions of the R-theory to reducible nonnegative matrices.

3. Reducible nonnegative matrices

In this section we will discuss strictly positive S-excessive functions

-I

and give conditions under which we may choose S to be equal to R ,or, stronger, under which there exist strictly positive eigenvectors associated with R-1, for a reducible nonnegative matrix. After that we will consider

-)

generalized eigenvectors, associated with R ,and show that they can also be chosen nonnegative, under special conditions. All the conditions we

need appear to hold trivially in the finite case. The results in this section extend those of Rothblum [12J. Moreover, we obtain a semi-probabilistic interpretation for eigenvectors, generalized eigenvectors and p-excessive functions.

As noticed already in the introduction of partial ordering for the irre-ducible classes C_I' C_Z •••. of P exists in terms of accessibility relations. Let C

k and

C~

be two of these classes. We may define R(k) as the common convergence radius of all P,.(z), i,j E C

k• Similar we can define

R(~).

1J

Furthermore if iJ,jl E C

k, i2,j2 E Cn then R. ,

=

R, . , hence we may also N 1₁J₁ 1_ZJ2

speak of a common convergence radius R(k,£). This was noticed by Tweedie [16J, who also proved

(3. 1 )

But even if there do not exist more than two irreducible classes, C(k) en

C(O _{say, we may} h ' . _{ave str1ct 1nequa 1ty 1n} I' . (3 _• I) _{, contrary to w at may} h

occur in finite-dimensional situations.

The main goal of this paper is to show the structure of (almost) optimal p-excessive functions and (generalized) eigenvectors, not to treat the most general cases extensively. Therefore, and in order to simplify the proofs in this section, we will work under some restrictions (which may be considerably relaxed; compare the last section). We assume for any nonnegative matrix P, discussed in this section:

Condition I: P possesses only a finite number of irreducible classes.

We then have the following theorem (partly analogous to Lemma 3):

Theorem 4: Let P be a nonnegative matrix with convergence radius R. Let the final classes (i.e. classes which do not have access to any other irreducible class) be

rily chosen state in for a > R-1 the vector

given by C1, ..• ,C_k• C.(j

=

I, ...• k) and

J

u(A,a), defined by

Let t. be a fixed, but

arbitra-J define A

=

{tl •••• ' t_k}. Then (3.2) u,,(A,a.) = 1. k _I

I

{c.

+ (l - O't )F't (a )} • 1 1t, 1 . , 1 . J= J J J i E S

is strictly positive and a-excessive. Moreover, if x is any other strictly positive S-excessive vector, then S > R-1 and

=

(3.3) i E S •

Proof: That u(A,a) is strictly positive and a-excessive follows from

k 1 -I 1

I

F'

t (a- ) >

a

for all i E S, F ( a ) < I, F (a-) =

a

for j

1

t and

. 1 1. . t.t. tJ.to

J= J J J '"

(3.4) _{Fit. (a )} -I _a -I ₀₄ \' L P'nFo _{1.", ",t.} (a - 1 - 1 ) + a 'P't _{1. .} j = I, ... , k; i E S •

J "'Tt. J J

J

Let now x be a strictly positive, S-excessive function. We will prove by induction that for m = 1,2, •..

(3.5) x. >

1. i E S

from which the result follows immediately.

For i E A (3.5) holds trivially for all m. For i t A we have

-I k -I k _{f~ I)}

x. _1.

₌

> 13 _{. 1 1. t .}

I

p. X_{t .} = S

I

_{1.t. Xt .}

J= J _J j=1 _J J

hence (3.5) holds for m= J • Suppose it holds for m

=

_{mO' Then for}

-I k -I X. > f3

I

_{Pit. x} + S

I

_PiR,X t ~ 1.

=

t. j=1 J J 9vtA -I k _{f ~1) + S-l} mo k _f(h)S-h > _S

I

Xt .

I

Pit

I

I

x = l.t. tt. t. j=1 J J 9vI.A h=J j=1 J J -I k _£~I) k mO _{f~h+l) S-(h+l)} > _f3

I

_x +

_{I I}

_Xt. = ₁ 1.t. t. 1.t. J J j=l h=J J J k mO+l f~h) -h

I

L

_1.t.

s

x t. j=1 h=l _{J J}

Hence (3,5) holds for all m E :IN,

i

t

A

Theorem 5: Let P, R and A be defined as in Theorem 4. Then a strictly positive, R-1-excessive function exists if and only if F'

t (R-) <

~

for

_} 1 •

j

=

l, .•• ,k; i E S. The vector u(A,R ), defined by J

-} u. (A,R )

1 i E S

is an eigenvector of P if and only if all final irreducible classes are R-recurrent and F. (R-) < 00 for j = l, ..• ,k; i E S.

1t.

J

-1

Proof: Obviously the inequality (3.3) holds also for 8

=

R ,from which the first statement follows immediately. Since we have F _t·t. (R-)

=

I if C. is R-recurrent, F t (R-) < 1 if C. is R-transient,

t~ ~econd

state-J t. . J

ment is also obvious, J J D

Remark: Clearly any non-final class of P is R-transient, since for these classes there exist strictly positive, R-1-excessive vectors which are not eigenvectors (compare Seneta [13J, tho 6.2), However, the condition "the final classes are R-recurrent, the non-final classes are R-transient" is in general not sufficient for the existence of strictly positive eigen-vectors, associated with R-1 (contrary to the finite case).

Remark: If P possesses a strictly positive eigenvector, associated with R-l , then we call P semi-stochastic, in view of the stochasticity of

Q

= (q .. ) . . S ' defined by

1J 1 oJ E

(3.6) q ..

=

Ru. -} (A,R -J )p .. u. (A,R ) -1

1J 1 1J J i,j E S ,

Remark: The conditions of the choice of t. in

J

F'

t (R-) < 00 for all i ,( S are obviously independent

1 •

theJfinal class C. This follows from Ftt.(R-) <

00,

J Ft.£(R-) < ~ for £ E C. and ] J (3.7) F. (R-) = F. (R-) _{+ t. F i£ (R-) FR,t •} (R-) 1t. £ 1t. ] _{J J J} (3.8) _{F i£ (R-) t . F i£ (R-) +} 9, F . _1t. (R- ) F p, (R- ) t. J J J

Furthermore, if C is any non-final class, then ~n fact we only have to assume F'

t (R-) < 00 for one particular state i E C. This follows from ~

.

(3.7) if wJ take ~ E C (since then t.Fi~(R-)

=

Fi~(R-) < 00).

J

The question arises what we can say if the conditions of Theorem 5 are not fulfilled, i.e. if not F't (R-) < 00, j ::: I, ..• ,k; i E S. Before

~

.

answering this question we

fir~t

discuss another classification of the states (in the case of general reducible matrices),

Let {CI" •. ,C_n} be a sequence of irreducible classes of P such that for each k E {I, ••• ,n - J} ther exist states i

k E Ck, jk E Ck+1 such that p. . > O. Such a sequence is called a chain. C) is called the initial

~kJk

class, C

n the final class, of the chain {CI, .. "Cn}. The length of a chain is defined as the number of R-recurrent classes it contains. The index v of the matrix P is defined as the length of its longest chain of irreducible classes. We say that a class C has depth k if k is the length of the longest chain in which class C is initial; the depth v

of state i is the depth of the class which contains i. (Notice that a semi-stochastic matrix has index equal to one).

In order to simplify the notations and the proof of the following theorem we make one additional assumption (but all results can be proved without

this assumption; compare section 6), We assume for the rest of this section.

Condition 2: The nonnegative matrix P, with convergence radius R and index v, contains precisely one R-recurrent class with depth k (k ::: I, ••• ,v).

Finally we need one technical definition:

Definition: Let B c S be some taboo-set, {tl, ••• ,t

k} a finite set of states, {nl""'~} a finite set of nonnegative integers and {a) ••••• a_{k } a set of}

constants. Let the vector x be defined by

x. 1 k

I

j=1 (n, ) a, • m. J (R-) J B 1t. J

Then the first derivative of x is the vector x', defined by

x! :::

1

k (n.+l)

I

a, • m. J (R-)

We now have the following result:

Theorem 6: Let P be a nonnegative matrix with convergence radius Rand index v. Choose in each R-recurrent class C. (j ... I, ••• ,v) a state t.,

J J let B (3.9) (3. 10) F. (R-) < IX) B ~t. J j

=

I •... , v i € S sup{

(Bm~rt)

(R-» (BF. (R-» -I} < <X> • 1. • ~ t. 1. J J j

=

I, ... ,v; r

=

1,2, ... (r) -]

(here we define (Bmit. (R-»(BFit. (R-» equal to zero if BFit.(R-) m 0).

J J J

Then there exist semi-positive vectors w(I), ••• ,w(v) such that

-] Pw( I) == R w(l) (3.11) Pw(k)

=

R-I(w(k) + w(k - I» k ... 2, •••

,v •

We have w.(k) > 0 i f v. > k, w.(k)

=

0 i f v. <k, for k ... 1, ... , i € S • 1 1 . = ~ 1.

Proof: Let {D1 •••• D ,D I} be a partition of the state space S such that

--- v v+

Dk contains all states with depth (v - k + I), for k

=

I, ••• ,v + I. Let B be ordered in such a way that B n Dk

=

{t

k} , k ... I, ••• ,v (that there exist precisely v R-recurrent classes follows from condition 2). Obviously there is precisely one final class in Dk (i.e. with respect to the restric-tion of P to D

k), namely the R-recurrent class, hence by (3.9) and Theorem 5 it follows that there exists a strictly positive eigenvector, associated

-I

with R ,for the restriction of P to Dk (for k

=

I, •.• ,v; not for k

=

v + I). Furthermore Dk is not accessible form D_k+

1 and each state in Dk has access to some state in D_k+_l (k = I, ••• ,v - I).

Now define the vector x(l) by

F. (R-)

HI i € S •

Obviously xi(I)'" 0 for i € S\D

I and RP xCI) == x(1). Define x(2) by

x. (2)

where ct

21 LS determined such that xt, (2) = O. By Lemma 2 we have

RPx(2) = x(2) + ct

21 x(l)

(note that x. (2) = F. (R-) for i E D 2;

L Lt2 (2)

Continuing in this way we obtain a sequence of vectors x(k), k = I, ... ,v such that (3.12) x. (k)

=

BF. (R-) 1. Ltk k-J

I

Uk x! (r) r=1 r L v+l k = I, ... ,v; i E S •

We have x.(k) = 0 for i E U D . x.(k) = F. (R-) for i E D

k• The

1. r=k+1 r ' L l.tk

constants ct_kr are determined in such a way that Xtr{k)

=

0 for r = I, ••. ,k-J. First we determine ctk k-l from x (k) = 0, _{then ctk k-2 from xt} (k) = 0,

, t_{k _1} ' k-2

etc. Using Lemma 2 we find the relations:

(3.13) _R

I

jiB

p .. x~ (r)

1.J J = x!(r) - x.(r) 1. 1. r = I, ... ,v; i E S •

By (3.12), (3.13) and the choice of the constants ct

kr we obtain: (3.14) k-I RPx(k) = x(k) +

I

ctkrx(r) r=1 Now define (3.15) y(v) xCv) and for k v - I, v - 2, .•• ,1:

(3.16) y(k) RPy(k + I) - y(k + 1) •

k 1, ••• ,v.

Then the sequence {y(I), ••• ,y(v)} clearly satisfies (3.11) and y.(k) = 0

v+ I 1.

for i E U D (k = 1, ..• , v). Notice that xi (k) > 0 for i E Dk (k = I, ••• ,v).

r=k+1 r

-1

=

(F t (R-»(m_t t (R-» t k-1, k k-l' k-I

Combining (3.14), (3.15), (3.16) and the fact that uk,k-l > 0 for

k

=

2, •••

,v

it follows that

(3.17)

y.

(k) >

a

1 for i € Dk, k

=

I, ...• v •

(in particular Yi(l) >

a

for i E D

I; Yi(1) = 0 for i E S\D1). Notice

that with {y(I), ••• ,y(v)} also the sequence {w(I), ••• ,w(v)} satisfies (3.11), where w(k) is defined by

w(l) = y( I) (3. 18)

w(k) y(k) + Bw(k - 1) k = 2 .... ,v

Note that (3.10) implies immediately that

s~p{(Bmi~:(R-»(Bm~~:(R-»-I}

< 00

1 J J

for t ~ r, j

=

I, ••• ,v • Together with (3.17) this implies that it is possible to choose B so large that all the vectors w(k),k

=

l, ... ,v

k

become semi-positive. More precisely w.(k) > 0 for i E U D

v+ I ' 1 r= 1 r

w.(k) = 0 for i E U D (k

=

l, ••• ,v). This completes the proof.

1 r

r=k+1

Again it can be proved that the conditions of Theorem 6 do not depend explicitly on a particular choice of t.

J

last remark after Theorem 5). Condition

k-j k-]

€ C., j = I, ••• ,v (compare the

J

(3.9) means in fact the following: let ~ = (u D.) \ B = u (D.\{t.}),then

I J j=1 J J

the restriction of P to ~ u Dk is semi-stochastic (k = I, •.• ,v).

A version of Theorem 6 without condition 2 can be proved with slightly more sophisticated methods (Le. there may be more than one R-recurrent class of depth k, k

=

I •.•• ,v). In that case we obtain the full generalization of results of Rothblum [12] for finite nonnegative matrices. It can be shown (as will become clear in the next section) that in this finite case

all assumptions of Theorem 6 are trivially fulfilled. The advantage of the proof of Theorem 6 is that it shows much better the nature of (gene-ralized) eigenvectors, associated with R-1 (each element a linear combi-nation of certain power series, based on taboo-transitions and calculated at the convergence radius). In the next section we give a separate dis-cussion of the assumptions of Theorem 6, in particular (3.8).

4. The conditions: strong ergodicity

In this section we will briefly discuss the conditions of Theorem 5 and 6. In particular we show that assumption (3.10) is related to strong ergodi-city of certain stochastic matrices, associated with a homogeneous Markov chain. We start with a definition.

Definition: A stochastic matrix P is called strongly ergodic if

kim

IIpn - QII = 0 where Q is a constant stochastic matrix (that is, a sto-chastic matrix with identical rows) and II •• II denotes the usual sup-norm. Obviously any finite Markov chain with one final, aperiodic class is strongly ergodic. For countably infinite stochastic matrices we have the following result:

Lemma 7: Let P be a stochastic matrix, associated with a Markov chain with one final recurrent class with period d. Choose one state s in that final class. Then

m~l)(1)

00 nf~n)

(4. 1 ) sup sup

I

< 00

i 18 i n=1 1S

if and only i f pd is strongly ergodic.

Lemma 7 was proved by Huang and Isaacson [8J in a more general context (but only for the case d = I). A direct proof of Lemma 7 can be found in Zijm [23J.

We can also proof:

Lemma 8: Let P be a stochastic matrix, associated with a Markov chain with one final recurrent class. Choose one state s in that final class. Then sup

m~l)(I)

< 00 implies sup

m~k)(l)

< 00 for k

=

1,2, ..••

i 1S i 18

Proof: By induction to k. For k

=

I the result holds trivially. Suppose

we have sup

m~k)(1)

< 00 for k = 1,2, ..• ,q, q € :N. By

1.S

i

m~I)(1)

<:t> (n) 00 <Y> _f~r) <Y> (n)

L

n

₌

_L

_L

₌

_{L L}

_sPij

1.S

n=O n=O r=n+J 1.S n=O j:fs

00

we find sup

I

_L

_sPij (n) < "". By the induction hypothesis:

i n=O j;'s

co <Y> 00

00 > sup

L

I

p~X:)m~q) (1) sup

I I

en)

I

(k)f~k+l)-q

=

sPij

i n=O }Is s 1.J J s i n=O j:fs k=q q JS

""

co

f~n+k)+l-q <:t> n+k

= sup

_{I I}

(k) _{= sup}

_L

_I

(Q.) f~n+k)+I-q ₌₌

i n=O k=q q 1.S i n+k=q Q.=q q 1.S

00

f~n+k+l)-q <:t> f~r+l )-(q+l) _m~q+l)(l)

I

(n+k+l)

I

r

= sup sup _(q+l) ::: sup

q+1 1.S 1.S 1.S

1. n+k=q i r=q+l 1.

(n+l) n Q, _{k - 1)}

(here we used =

I

_(k-l) for k > I, n >

.

k _i/,=k-) =

The proof can now be completed by induction.

Let us now return to the nonnegative matrices which were discussed in the preceding section, in particular in Theorem 6. Recall that we obtained a partition {D1, ••• ,D

v' Dv+l } of the state space S such that Dk contained

all indices of depth (v - k + 1). We fix one k E {l, ••• ,v} and eliminate one state t. from the R-recurrent class C. in D., for j == I, ••• ,k - I,

J J J

that is: each state in C. \ {t.} becomes R-transient, since

J J

F (R-) < F (R-)

tj H k-I H

== I for Q, E

for i E u (D. \ {t.}), otherwise eliminate the states which do not have

j= 1 J J

access to Dk after skipping {tl, ••• ,t

k_I}. Condition (3.9) now simply means

k

that P, restricted to

(.u

D_k) \ {t1, ••• ,t

k_I}, is semi-stochastic, whereas

J=! )

condition (3.10) states that this matrix can be transformed into a stochas-tic matrix (use (3.6» which is strongly ergodic (or satisfies the con-dition of Lemma 7).

Furthermore, Lemma 9 implies that in Theorem 6 we only have to assume

(4.2) sup

{(Bm~tl)(R-»)(BF.

(R-»-I} <<<>

1 . 1 t. j = I,.,.,v

1 J J

instead of (3.10).

In the case of finite nonnegative matrices it is clear that all assumptions are trivially fulfilled (Le. the related finite stochastic matrices are always strongly ergodic or satisfy the condition of Lemma

71.

We take once again a closer look at condition (3.9). Note that in particular F (R-) < 00, Since t t t v-I v-I' v hence F t t (z) v-I' v

=

t F_t t (z) + F_t t (z).F t t (z)

v-I v-I' v v-I' v-I v-l v

t F t ,t (z)

v-I v-I v

F t t (z)

=

-;----=:---~-v-I' v - Ft t (z)

v-I' v-I

we find, by multiplying with (R - z) and then taking the limit

F (R-) t \)_ It\)_ I ' tv lim (R - z)F (z)

=

-"7':"'":,....-;---z+R _{tv_I,tv m}(1) (R-) t t v-I' v-I

In the same way

or, since lim (R - z)2F (z) < 00 z+R _{tv_t,t v} t

=

I, ... ,\) - 1 P .. (z)

=

F .. (z)P .. (z) 1J 1J JJ lim (R - z)t+lp (z) z+R t_{v_2}, t_v F .. (z) 1J - F .. (z) JJ i =I j,

Izi

< R < 00 t

=

I, ••• ,v - 1 •

I zl

< R

In other words: R is a pole (maximally of order v) of the power series P .. (z), i,j E S.

~J

An interpretation of (3.9) can also be discussed by using both "last exittl

and "first entrance" transitions. Last exit transitions

~~I?)

are defined by

1J

(4.3) ~~I?) = . p .. (n) n

=

0,1,2, ...

~J 1 1.J

and as before we define L .. (z) by

1J 00

~~~)zn

(4.4) L. . (z)

₌

I

.

1J _n=1 1J

A complete analysis of irreducible, countably infinite, nonnegative matrices can be given by means of L .. (z) instead of F .. (z) (compare e.g. Seneta [13]).

1J 1J

In particular it can be shown that for an irreducible nonnegative matrix P with convergence radius R the vector y, defined by

j E S

(where t is some fixed state), is an elementwise finite, strictly positive, left-eigenvector of P, associated with R-1 •

Now consider the case of a reducible matrix P with convergence radius R and index 2, which contains only two R-recurrent classes, V and W say, and no transient classes. Fix state t E V and SEW (V has access to W).

Condition (3.9) becomes

F. (R-) < 00

t ~s i e S

or equivalently F (R-) < 00 (assumption for only one state in V; compare

t ts

the last remark after Theorem 5). But we have

F (R-) t ts R _ieV\{t}

I

L .(R-)p .. F. (R-) _{t1. 1J JS} + R _iEV\{t}

I

L .(R-)p. _{t1. 1S} jEW\{S} + R I p .F. (R-) + Rp jeW\{s} tJ JS ts

=

R

L

icv jeW L . (R- ) p .. F. (R-) tl 1J JS

since L (R-) = F (R-)

=

I. In other words: the part of the matrix which

tt ss

specifies the transitions from one recurrent class to another one must be more or less "normar', i.e. the innerproduct with the left-eigenvector, associated with the first class V, and the right-eigenvector, associated with W, must be finite. Similar remarks can be made in more general cases.

5. Sets of nonnegative matrices

In this section we deal with a set M of countably infinite nonnegative matrices and show how some of the results of section 3 can be extended. Following Seneta [13J. p. 59 we make two assumptions with respect to M

(which we assume to hold for the rest of this section).

00

Condition 3: M is compact (regarded as a subset of R with the usual topo-logy of component-wise convergence).

Condition 4: To each vector y ~ 0 (element-wise finite) and any two matrices P I ,P2 E M there exists a matrix P3 E M such that P3y ~ PlY' P3y ~ PZY'

The following operator is frequently used in a dynamic programming context and in Markov decision processes

(5. 1 ) Ux

where x is a nonnegative vector. Note that conditions 3 and 4 imply for each nonnegative vector x the existence of a matrix P E M such that Ux

=

Px. This property is usually called the "optimal choice property" (Mandl and Seneta [IIJ).

We say that a subset A of S is communicating with respect to M if for each pair i,j E A there exist an integer n ; I and matrices PI""'P_n E M such that (p)PZ, ••. ,P ) .. > 0 (compare Bather [IJ). The case in which S itself

n ~J

~s communicating with respect to M has been discussed by Kennedy [IOJ. In order to develop an R-theory for sets of nonnegative matrices we first have to specify what we mean by R. Define a strategy ~ as a sequence of matrices: n-th step ~

=

(P"P_{Z" " )} with transition

t~~)(n)

by ~J P. E M. 1.

(5.2) t (.n.)(1T) = (P P P )

I 2 ••• . .

1J n 1J i,j €

S;

n ~

and

t~?)(1T)

= 0 ..• Let furthermore

1J 1J (5.3)

t~r;)

= 1J (n) sup t.. (1T) 1J i,j E S; n ~ 1

In order to avoid trivialities we assume

t~r;)

< 00 for each i,j and n.

Let R •. (M) be the radius of convergence

of1~he

series

E

t~r;)zn,

1,J E S

1J n=O 1J

and define

(5.4) R(M) inf R .. (M) • 1J

i,j

Kennedy [IOJ showed that R .. (M)

=

R(M) for all i,j E S if S is communicating.

1J

In that case R is also the common convergence radius of the series T,.(z), 1J i,j E S, defined by (5.5) T .. (z) = sup 1J 'IT 00

L

n=O (n) n t.. (n)z • 1J

Taboo-transitions can be defined again. Let RES be some taboo-set and

~ = (P

I,P2" " ) a strategy. For i,j E Sand n ~ 2 set

(5.6) and let more let (5.7) (5.8) (n) Rt .. 1J (1T) == (J ) Rt" (1T) = 1J (n) Hf.. (1T) 1J RT, . (z) 1J HF .. (z) = 1J

L

(PI)" (P₂)· . , . lR 111 1\12 1 1, .. · , ln_1,"

(P 1) . , • We omit the subscript R i f H is empty.

Further-1J (n)

with jR = R u {j} and define 'Ht.. (1T) J 1J 00 (n) n sup

I

Ht .. (1T)Z 1T n=1 1J 00 f (n) ( ) n • 'RT, . (z) = sup

_L

_R" 'IT Z J 1J _{'IT n=1} 1J

The following results were proved by Kennedy [IOJ in the communicating case but is is easy to show that they hold in general:

Lemma 9: For each z, 0 < z < R(M) and each fixed j

=

(5.9) (5.10) (5.11) F •• (z) 1J F .. (z)

=

1J sup {z

L

P'kFk'(z) + zp .. (1 - F .. (z»} PEM k 1 J 1J JJ sup PEM 00

{ I

n=O (n) nl,

~

(n) n} p .. Z L p .. z q n=O JJ i :f j F .. (z) "" {(T .. (z) - I)/(T .. (Z»} JJ JJ / JJ i E S

Let R := R(H). We see that F .. (z) < I for z < R, hence F .. (R-) < I. As

JJ JJ

=

before we classify state j as R-recurrent if F .. (R-) "" I and as R-transient JJ

i f F .. (R-) < 1.

JJ

We have seen that the role of an irreducible class in the "one-matrix" case

1S now played by a communicating set. Again we will refer to such a

commu-nicating set as to a class. In particular: if S is communicating, then

R .. (M) = R(M) =: R, 0 < F .. (R-) < 1, 0 < F .. (R-) < 00 for i,j E S (Kennedy

1J 11

=

1J

[IOJ). We assume now, analogous to condition I:

Condition 5: S contains only a finite number of communicating classes, with respect to M.

Obviously, condition 5 holds if there is at least one matrix P € M with only a finite number of irreducible classes.

A $-excessive function for the set M is defined as a semi-positive vector

x such that sup Px 2 Sx. Again S is called the excessivity factor. PEM

-Analogous to Theorems 4 and 5 we now have:

Theorem 10: LetMbe a set of nonnegative matrices, with R(M) "" R. Let the final classes be given by C

1' •.•• C • Choose t. _] n J E C. J (j = I, ... ,k) and let

A = {t1' .•.• t_k}. Then for a > R the vector u(A,a), defined by

(5.12) u. (A.a) 1 k _I

I

{c.

+ (1 -

o.

)F' t (a )} . I 1t. 1t. 1 . J= J J J i E S

o

is strictly positive and a-excessive. If x is any other strictly positive,

-I

8-excessive vector, then

8

> Rand

(5.13) i E S •

Theorem 11: Let M, R and A be defined as in Theorem 10. Then a strictly positive, R-1-excessive function exists if and only if F'

t (R-) <

~

for 1 .

j = I, ... , k; i E S. We have

(5.14 ) R sup Pu(A,R-1)

=

u(A,R-1) PEM

J

o

if and only if all :~nal classes are R-recurrent a~~ Fit. (R-) < ~ (j

=

i S). (here u(A,R ) is defined by (5.12) for ex

=

R ). J

I, •.• ,k;

The proofs of Theorems Ii and 12 are completely analogous to those of Theorems 4 and 5 and will be omitted.

As before we may define series Hm.. (z); (k) k == 1 .2, •.• Set

1J 00 (k) _{f(n+1 )-k( ) n-k} (5.15) _{Hm.. (z)} sup

I

(n) H .. 7T Z i,j E S

.

1J k 1J 7f n=k

Chain-structures can also be defined. A chain is now a sequence of

communi-o

eating classes. {C 1 ' ••• , C_n} say. such k = I, ... ,n - I. The index v(M) of M

that t. . > 0 for some i

k E 1

kJk

is the length of the longest

Ck • j k E Ck+ 1 ' chain of communicating classes (where length is again defined as the number of R-recurrent classes in the chain). Class C has depth k if k is the length of the longest chain in which C is initial; the dept

v.

of state i is the

1 depth of the class which contains i.

Suppose now, analogous to condition 2:

Condition 6: Let M be a set of nonnegative matrices with R(M) = R, v(M) == v.

The following analogon of Theorem 6 can be proved:

Theorem 12: Let M be a set of nonnegative matrices, R(M)

=

Rand v(M

=

v. Choose in each R-recurrent class C. a state t. (j

=

I, ... ,v), let

J J (5. IS) (5.16) F. (R-) < 00 B 1.t. J J

=

I, ... i E S (r) -I sup {(Bm. (R-»(BF't (R-» } i 1t_j 1. j < 00 j l, ... ,v; r

=

1,2, ••••

Then there exist semi-positive vectors w(I) •.•. ,w(v) such that

(5.17.1) R sup Pw(l)

=

w(l) PEM

(5.17.k) R sup Pw(k) PE~_I

w(k) + w(k - I) k 2, .•. , v

with Al = {pIPEM, RPw(I) = weI)} and ~

=

{pIPE~_l' RPw(k)

=

w(k) + w(k- J)},

k 2, ••• ,v. Furthermore w.(k) > 0 if

v.

> k, w.(k)

=

0 if

v.

< k, for

1. 1.

=

1 1.

k= I, ••. ,\); i IE S.

A proof of Theorem 12 (which is essentially harder than that of Theorem 6) can be found in Zijm [24J. Again a version of this theorem can be proved without condition 6. Also the other assumptions can be considerably relaxed again. Theorem 6 is extremely important for applications in optimization problems (Markov decision processes. Leontief substitution systems); e.g. for the construction of accurate bounds for the value of a system. or to obtain a polynomial expansion for it, etc. (compare also Zijm [22]).

6. Extensions and remarks

In this final section we indicate how several assumptions can be relaxed. First of all it follows immediately from the proofs of Theorems 4 and 5

o

that only a finite number of final irreducible classes has to be assumed (instead of condition 1). In the same way we need in Theorem 6 only a finite number of R-recurrent classes (an infinite number of R-transient classes can be allowed). Furthermore we noticed already that Theorem 6 can be proved without condition 2. The conditions are then related to the so-called Doeblin-condition for countable

stochastic matrices (compare e.g. Hordijk [7J), which can be seen as an analogue of strong ergodicity in the multichained, periodic case

(Zijm [23J),

Theorem 4 can even be proved without assuming the existence of final classes. We then work as follows: choose a set of classes, {C

t,C2, •• ,} say, such that C. is not accessible from C. for i ~ j. Suppose that the

1 J

set is maximal (i.e. it cannot be extended without violating the non-accessibility condition), Let DO be the union of C

I, C2, ••• and all

classes which have access to u C.and define recursively Ok as the

maxi-i 1

mal set of classes which are accessible in at most one step from D_k

-t

and such that the classes in Ok \ D

k-1 do not have access to each other

(k = 1,2, •.• ). Note that D

k-1 cOk (k = 1,2, ••• ). If we assume thgt Dk contains only a finite number of final classes with respect to P k, the restriction of P

Dk

such P Yk(S) ~

to Ok' then there exist strictly positive vectors Yk(S),

-I

Yk<S) for

S >

R , k

=

0,1,2, •••• For each E > 0 the

r

ekYk(S) is now strictly positive and S-excessive. k=O

Finally we state without proof that Theorem 6 can be proved if we allow

v

=

00 (chains of countably infinite length). Furthermore it is clear

that analogues of the above remarks hold with respect to the results in section 6.

The results of this paper are related to work of Hordijk [7], concerning sets of countably infinite stochastic matrices (in fact the vectors y(k), defined by y.(k)

=

m~~)(I)

for i E S, k

=

0,1,2, •.• and fixed j form a

1 1J

set of Lyapunov-functions) and to results in potential theory (Hordijk [7J, Seneta [13]). Strictly positive S-excessive functions are treated extensively in a paper by van Hee and Wessels [6J, in which they give a lower bound for the excessivity factor which is, unfortunately, norm-dependent. The question wether P, defined by

1

p = sup sup lim sup

{p~~)}n

PEM i,j n + 00 1J

-)

ha.li to be R ,defin.e~ by

(compare (5.2) until 5.4».

Finally we note that applications of strictly positive a-e~cessive functions can be found in Wessels [19J.

REFERENCES

[IJ Bather, J., Optimal decision procedures for finite Markov chains II, Adv. Appl. Prob.

1,

521-540 (1973).

[2J Birkhoff, G., Extensions of Jentzsch's theorem, Trans. Amer. Math. Soc., 85, 219-228 (1957).

[3J Chung, K.L., Markov chains with stationary transition prohabilities, Springer-Verlag, Berlin-Heidelberg (1960).

[4J Gale, D., The theory of linear economic models, McGraw-Hill, New York (1960).

[5J Harris, Th.E., The theory of branching processes, Springer-Verlag, Berlin-Heidelberg (1963.

[6J Hee, K.M. van and J. Wessels, Markov decision processes and strongly excessive functions, Stoch. Proc. Appl., ~ (I), 59-76 (1978).

[7J Hordijk, A., Dynamic programming and Markov potential theory, Math. Centrum,.Amsterdam (1974).

[8J Huang, C.C. and D. Isaacson, Ergodicity using mean visit times, J. Lond. Math. Soc. ~, 570-576 (1977).

[9J Karlin, S., Positive operators, J. Math. and Mech. ~, 907-937 (1959).

[IOJ Kennedy, D.P., On sets of countable nonnegative matrices and Markov

decision processes, Adv. Appl. Prob.

!£,

633-646 (1978).

[ I I ] Mandl, P. and E. Seneta, The theory of nonnegative matrices in a dynamic programming problem, Austral.J. Statist.

!!,

85-96 (1969).

[12J Rothblum, U.G., Algebraic Eigenspaces of nonnegative matrices, Lin. Alg. and Appl. ~. 281-292 (1975).

[13J Seneta, E., Nonnegative matrices, Allen and Unwin, London (1973).

[14J Sladky, K., On dynamic programming recursions for multiplicative Markov decision chains, Math. Program. Study ~, 216-226 (1976).

[15J Sladky, K., Bounds on discrete dynamic programming recursions I, Models with nonnegative matrices, Kybernetica ~ (6), 27-40 (1980).

[16] Tweedie, R.L., Truncation procedures for nonnegative matrices, J. Appl. Probe ~,,311-320 (1971).

[17] Vere-Jones, D., Geometric ergodicity in denumerable }furkov chains, Quart. J. Math. Oxford (2)

11,

7-28 (1962).

[18] Vere-Jones, D., Ergodic properties of nonnegative matrices I, Pacific J. Math., ~, 361-386 (1967).

[19J Wessels, J., Markov programming by successive approximations with respect to weighted supremum norms, J. Math. Anal. Appl. 58, 326-335 (1977).

[20J Zijm, W.H,.M., On nonnegative matrices in dynamic programming, Mem. COSOR 79-10, Eindhoven Unive~sity of Technology (1979), submitted for publication.

[21J Zijm, W.H.M., Generalized eigenvectors and sets of nonnegative

matrices, Mem. COSOR 80-03, Eindhoven University of Technology (1980), submitted for publication.

[22J Zijm, W.H.M., Asymptotic behaviour of the utility vector in a dynamic programming model, Mem. COSOR 80-04, Eindhoven University of Techno-logy (1980), submitted for publication.

[23] Zijm, W.H.M., Strong ergodicity and the Doeblin-condition (in prepa-ration) •

[24J Zijm, W.H.M., Dynamic programming and nonnegative matrices (forth-coming).