Recurrence and conservativity for Markov processes

Recurrence and conservativity for Markov processes

Citation for published version (APA):

Simons, F. H., & Overdijk, D. A. (1977). Recurrence and conservativity for Markov processes. (Memorandum COSOR; Vol. 7702). Technische Hogeschool Eindhoven.

Document status and date: Published: 01/01/1977

Document Version:

Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.

• The final author version and the galley proof are versions of the publication after peer review.

• The final published version features the final layout of the paper including the volume, issue and page numbers.

Link to publication

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:

www.tue.nl/taverne Take down policy

If you believe that this document breaches copyright please contact us at: openaccess@tue.nl

providing details and we will investigate your claim.

Department of Hathematics

PROBABILITY THEORY, STATISTICS AND OPERATIONS RESEARCH GROUP

Memorandum COSOR 77-02

Recurrence and conservativity for Markov processes

by

F.H. Simons and D.A. Overdijk

Eindhoven, January 1977 The Netherlands

by

F.R. Simons and D.A. Overdijk

I. Introduction

+

Let (X,E) be a measurable space, and let M be the class of non negative extended real valued measurable functions on (X,E). A Markov process on (X,E) is a mapping P of M+ into itself such that

00 I ) P(

I

n==) a. f ) n n 2) PI s 1. 00

I

n=J a. Pf for all a. ~ 0, f E n n n n + H ,

In the sequel, all sets and functions will be measurable. If we put P(x,A) == PIA (x), then P(.,.) is the (sub) transition probability

describing the process on the state space (X,I). Conversely, every (sub) transition probability on (X,E) induces a mapping of M+ into itself satisfying the conditions I and 2.

If PI =1= 1, then we can adjoin to the state space an "infinite" state 00

with P(x,{oo})

=

.I-PI(x) for all x E X, and P (oc,{oc})

=

1. In this way we obtain a Markov process P on X U {oo} which satisfies PI == 1.

Hence we may and shall assume that the Harkov process P which we shall consider in the sequel satisfies PI

=

I.

For every set A we define the

embedded process

Q A by

00

QAf

=

I

(PIA,)k PIA

f

k=O

+

for all f E H .

Here AI stands for X\A, and IA for multiplication by the indicator function of the set A.

we conclude

QA1(x) + lim (PI ,)n lex)

n-+<x> A I for all x E X.

Now it is easily verified that the operator Q

A indeed satisfies the

conditions 1 and 2. Moreover, we note

(I. J) iff lim (PIA,)n I(x)

=

0 .

n-+<x> (1. 2) If A c B

Suppose that we also have a probability measure m on the state space

(X,E). The Markov process P is said to be

non singuZar'

with respect

to m if for every A E E with meA) = 0 we have m (P(· ,A) > 0)

=

O.

In this case we shall say that P is a Markov process on (X,E,m).

If f=g m-a.e .. , then Pf

=

Pg m-a.e., hence we can consider the operator P

also as a mapping of M+(m), the space of equivalence classes of

m-almost equal non negative extended real valued measurable functions,

into itself, satisfying the conditions I and 2.

The following theorem is due to E.Hopf (see e.g.[3], chapter II).

Theorem 1.1. Let P be a Markov process ana probability space (X,E,m).

Then there exists a mod m unique partion X

=

e u D of the state space X

such that on the

oonservative part

e we have

co

I

pnlA

=

0 or 00 m-a.e. on X for all Ace,

n=O

and on the

dissipative part

D there exists a sequence Bk

+

D [m]

such that

n

P IB < 00 m-a.e. on X for every k.

k

A Markov process on

(X,E,m)

is said to be

oonservative

if m(D)

=

0

and

diss'ipative

i f m(e)

=

O.

Theorem 1.2. Let P be a conservative Markov process on (X,I,m). Then for all A E I we have QA1=1 m-a.e. on A.

Proof. Combine in [3J chapter II, theorem B with (3.3).

In general, if P is a Markov process on (X,I,m), we shall call a set A with the property QA1=1 m-a.e. on A a

recurrent set.

Hence, form-a.a.

initial states x € A the process will return to A with probability 1. Now for a moment we fix our attention to Markov processes on

(R,B),

where

B

is the a-algebra of the Borel sets of R.

Sometimes, a state x € R ~s said to be recurrent (see e.g. [IJ , definition 3.31, going back to [2J)if for every open interval A

with x E A the probability that starting 1n x the process will return

to the interval A infinitely often is 1. We can give this definition also in terms of the embedded process as follows.

Proposition 1.1. Let P be a Markov process on

(R,B).

A state x is recurrent if and only if QA1(x) = 1 for every open interval A with x € A.

Proof. Let

f

be the Markov measure on the realization space starting

x

~n x. If x is recurrent, then we have for every open interval A with x E A

~ (X E A' for all n 2 I) = lim (PI ,)n I(x) = 0,

x n n-+a> A

and therefore by (1. I) we have Q

A 1 (x) = I.

Conversely, suppose QA1(x) =1 for all open intervals A with x € A. If x were not recurrent then there exists an open interval A with x E A and an integer N such that

f (X E A' for all n > N) >

° .

x n

If we choose the integer N as small as possible, then we have

w

(X. ~ x for 1 ~ i ~ N, X E A' for all n > N)

x ~ n

= W (X E A' for all n > N) .

x n

It follows that

lim ~

(IX.-xl

> 0 for I $ 1 $ N, X E A' for all n > N)

= W (X E At for all n > N) > 0 •

x n

Hence there exists an interval B

=

(x-6,x+S) c A such that

W (X ~ B for all n ~ I) > O.

x n

By (1.1) we then have QBI(x) < I, contradiction.

A process on (RsB) for which all states are recurrent 18 said to be

a recurrent process. Then all open intervals are recurrent (even

pointwise). If the process moreover non singular with respect to

e.g. the Lebesgue measure

A,

the question arises whether or not the

process is conservative.

Although for a conservative process all sets are recurrent, a condition

like the recurrent sets are dense in L is not sufficient to imply

conservativity. In § 3 we shall give an example of a Markov process

on (~,B) which is recurrent and non singular with respect to the Lebesgue measure, but which nevertheless is dissipative.

On the other hand, if there exists an equivalent invariant measure and a partition of X into recurrent sets each having finite invariant measure, then the process is conservative. This will be shown in the

next section.

2. Recurrent sets and conservativity

Throughout this section P will be a Markov process on a probab ity

space (X,L,m), satisfying PI=I m-a.e. Therefore all statements on sets and functions in this section will have to be interpreted modulo a m-null set.

First we show that also for a dissipative process there are "many" recurrent sets.

Theorem 2.1. Let P be a dissipative Markov process on a probability

space (X,E,m). Then the recurrent sets are dense in L.

Proof. Let Bl t X such that

K

IB < 00 for all k. k

n lim (PI D ' ) n-+oo _k hence by (1.1)

Q

B k ::;; lim n-+<» == 1 on X

It follows by (1.2) that for every A C L the set A U Dk is recurrent.

Since we can choose the set Dk as small as we want, it follows that

the recurrent sets are dense in L.

o

In order to obtain a condition on the class of recurrent sets which

implies conservativity we need the following results.

ProEosition 2.1. Let P be a Markov process on (X, z) •

Then for every A E l: and every f E M + we have

00 00

L

pn I f

_I

_QAn _f. n=l A n=l 00 Proof.

I

pn I f A c=l 00 =: \' (P(I AI + I

»

n-i PIAf L _A n=l 00 k J k

=

_I

_L

(PIA I) PIA,··ePI_A,) p PIAf

n=\ k + ••• k +p=n 1 p 00

=

I

p=l 00

o

I t is easily verified that if P is non singular with respect to m. then

every embedded process

Q

A is non singular with respect to m. The

Proposition 2.2. Let P be a Harkov process on a probability space (X,L,m) with conservative part C.

Then the conservative part of X with respect to the embedded process

Q

A is A n C.

Proof. For every subset B of A n C we have by proposition 2.1 and

theorem 1.1 00

I

n=O I = B

I

on

x,

n=O

hence A n C must belong to the conservative part for QA'

Let D be the dissipative part of X for P, and let the sequence Bk be as in theorem 1.1. Then

00 00 00

I

=

I

I

n=1 n=1 n=J

hence D must belong to the dissipative part for QA'

00

Finally.

I.

Q~

1 A' nC = 0,

n=l

hence AI n C must belong to the dissipative part for Q

A,

0

Recall that by the Radon-Nikodym theorem there is a one-to-one

correspondence between M+(m) and the non negative measures absolutely +

continuous with respect to m. For every f E M (m) we can also define

a measure ~f « m by

~f(A)

=

J

f (PIA) dm for all A E L:.

Put fP

=

d~ d~ , then the mapping f + fP ~s a mapping of M (m) into +

itself such that

For details the reader is referred to the book of Foguel [3J.

A measure IJ « m is said to be

invariant

under P if dlJ P = dlJ.

dm dm

The following proposition shows that the restriction of an invariant measure to a recurrent set of finite invariant measure is an invariant measure for the embedded process on that recurrent set.

This is a slight extension of a result of Foguel and Lin([4], lemma 2.4 ).

Proposition 2.3. Suppose fP=f for some

recurrent set with

fA

f dm < 00 •

Then fIA

Q

_A

=

fIA •

+

f E M (m), and let A be a

Proof. From fP f we conclude for every n

n

k

I

f IA (PI A' ) + f lA' (PIA') n f.

k=O

n _k

I

f IA (PI A' ) PIA $ fPI

A = fIA'

k=O

f IA Q_A $ fIA •

o

It easily follows from theorem 1.1 that a process admitting a finite

equivalent invariant measure, i.e. for which there exists a function

f E £I(m) withfP = f and f > 0, must be conservative, no matter what

we know beforehand of the class of recurrent sets. A similar theorem. which hardly uses recurrent sets, holds

a-finite equivalent invariant measure.

there exists an infinite

Theorem 2.2. Let P be a Markov process on (X,L,m). Suppose that there

exist a function f E M+(m) with f > 0 and fP = f, and a sequence of

00

recurrent sets A with X = u

1 A and

n n= n

r

J

A f dm < 00 for all n •

Then the process P is conservative. Proof. For every n the function fIA

n

18 invariant under

Q

A by

n

proposition 2.3. Moreover, this function 1S positive on A and integrable,

n

hence Q_A is conservative on An' By proposition 2.2, also P must be

n

conservative on every A , and therefore on X •

n

o

The condition given in this theorem is sufficient, but not necessary. Ornstein [6J has given an example of a conservative transformation T,

and therefore of a Markov process P by defining Pf = foT, for which

there does not exist a finite or a-finite equivalent invariant measure. As an application of theorem 2.2 we have that the random walk on

(R,B,A)

with ~

=

0 is conservative. In fact, Chung and Fuchs ([2J, see

e.g. [IJ, theorem 3.38) have shown that every interval is recurrent, and

the Lebesgue measure A is invariant.

3. An example

In this section we shall give an example of a dissipative Markov process on

«O,I),B,A)

for which every state is recurrent.

Then every interval is a recurrent set, and it follows that there exists

an algebra of recurrent sets generating

B.

Another example of a dissipative

Markov process for which there exists a generating algebra of recurrent sets was given in [7J. However this example did not supply a counterexample

to the conjecture that for Markov processes on

(R,B,A)

recurrency for

every interval would imply conservativity.

In the sequel, (X,~) will stand for

«O,I),B).

The process P which

we shall study is defined by

Pf(x) = _{p f Ox} +

₀

+ ( 1 - p) fOx)

for all f E: M + and for all x E X, where p is a number with

o

< p <

It is easily verified that P indeed is a Markov process on (X,~),

moreover is non singular with respect to the Lebesgue measure A.

I .

which

Actually, this process is a backward process for the transformation

Tx

=

2x(mod I) on (0,1). It was shown in [7J that all backward processes

for this transformation are dissipative, except for p

=

~, where the

process is conservative. Hence for p ~

!

there exists for every E > 0

CS)

I

pnJA(x) < CS) for A-almost all x X.

n=O

We shall prove the following stronger pointwise statement~ which impl S

the dissipativity of the process

P.

Property J. I f P ~ ~, then for every I:: > 0 there exis ts a set A tll

A(A) > 1 - I:: such that

I

converges uniformly on (0,1).

n=O

Moreover we shall show that, despite the dissipativity of the process, all states are recurrent. This follows from

Property 2. For every interval A we have

QA1(x)

=

1 for all x E X.

In order to be able to perform our calculations with the operator P, we introduce the measurable space

CS)

n

n=l

Every point x has a unique dyadic expansion x=0.x

1x2 •••• , if we agree

that a dyadic expansion is not allowed to end in a sequence consisting

of the number 1 only. Then the mapping ~: X ~

r.

defined by ~(O.xlx2"')

=

(x

1,x2, .•. ) is an injective bimeasurable mapping, and

r. \

~(X) is a

countable set. Hence with every measurable function f on ~(X) there

corresponds a unique measurable function fQ~ on X, and conversely,

with every measurable function g on X there corresponds a unique

-\

measurable function go~ on ~(X).

We shall denote both f and fo~ with the same symbol f; f(') is the

function on X, and f(·,·, ••• )the corresponding function on ~(X).

Let ~ be the probability on {O,l} determined by ~ {I}

=

p, and let Ap

p P

be the corresponding product measure on (r.,~).

Then A

(r.\,(X})=O

for everyp and by means of the mapping ~, the

p

probability A corresponds with a probability, also denoted by A,

p p

on

(X,L).

Note that A! is the Lebesgue measure A on

(X,L).

2 '

Throughout, (w

1,w2, ••. ) will be an element w of

r.,

and (x1,x2, ••. )

+

Proposition 3.1. For all f E tl and all n E ~ we have

Proof. We proceed by induction on n. For n=l we obtain

J

f(W I,xl,x2,···)Ap(dw)

=

J

f ( 1 ,x 1 ,x_Z ' ... ) \ (dw) +

f

f(0,x 1,x2, .. ·) {w 1

= \}

{w1=0}

=

p f(!x + !) + (1 - p) f(!x)

=

Pf (x)

Now suppose that we have shown for some N ~ I

Then, using the result for n:: I, we obtain

A (dw)

P

pN+1f(x)

=

J

[J

f(wl,· .. ,wN,nl,xl,x2, ..

.)\~dw)

J\(d

n)

=

J

f _{(wI'" ,wN'wN}+ _{1 ,xl ,x2 ,·· • )Ap (dw).}

o

Proposition 3.2. For every A EL and every n E N we have

Proof. Induction on n, us~ng proposition 3.1.

o

We now choose a, for the moment still arbitrary, increasing sequence N

1, N2, ••• in ~ and a number a with 0 < a <I. Then we define for every

kEN Nk Ak

=

{ w E Q

_I

I

_w < a _(N k - Nk- 1) } (NO

=

0), n=Nk_I+I n 00 Dk

=

n A n=k n

These sets are defined as sets in

n,

but as we have seen before, we nay consider them as well as sets in X.

Proposition 3.3. For every x E X, m E ~ and £ ~ m we have

00 00

I

n=N +) £, Proof. Suppose Nk < n N k+1 , with k ~ £ ~ m.

Then by proposition 3.1. we obtain

hence

pn 1 (x)

D m

from which the statement follows.

o

Our last preliminary is the following estimation, which was suggested to the authors by J.G.F. Thiemann.

Proposition 3.4. If 0 < a < p < I, then there exists a number s

such that 0 < s < and

n Proof.

₌

_Z

(~) i=O 1 ~

L

O::;i::;an 1 a n (i) ( )_a)n-1 n s ai (l_a)n-i for all n E. IN .

p

(r:) 1 (l-p) n-1 (~)i I-a n-i

=

_L

p ( - )

O::;;i::;an 1 p I-p

~

r

(~) p i (I_p)n-i (~)an ( - ) I-a (l-a)n

.

O::;i::;an 1 p I-p

Put _s = (1:) a _{( I-a)} I-p I-a

.

_{For p ~ a we have} a

log s

=

a log 1: + (I-a) log I

a I-a

< log (a' £ + (I-a) I-p)

=

0,

a I-a

hence 0 <s <1. It follows that

L

(~) p1 (l_p)n-l ~ sn for all n E ~ •

O~i:::;an

o

Proof of property 2. Because of (1.2) we may assume that the interval A consists of those points x E X for which the dyadic expansion

starts with al,."a m,

n

By (1.1) we have to show that ~~ (PIA') lex)

=

0 for all x E X.

Since this limit is the limit of a monotone sequence of functions nm

it suffices to show lim (PIA') I (x)

=

0 for all x E X.

n-+co

Because of proposition 3.2 we have

n-l ::;

J

II I A' (wkm+ 1 ' ••• ,wnm,xj ,x2'" • )A (dw) k=O p

J

n-I

=

II _{1 AI (wkm+ I' ""w(k+l)m)} A (dw) k=O p n-l

J

lA' TI _{(wkm+ I ' ••• ,W (k+ 1 )w)}

A

(dw) 1<:=0 p

=

(A (A,»n p

It follows that lim (PIA,)nm l(x)

=

0 for all x E X.

n""'"

D

Proof of property 1. We shall g1ve the proof only for ~< p < I;

if 0 < P <

i

the proof is similar.

l~ow we take N = ~ k(k+ I) and CI. such that

!

< CI. < p. Then by

k

proposition 3.4 we find for every k

'"

for some s with 0 < s < 1. Since the series

I

(k+l)sk converges,

k=O

co

it follows from proposition 3.3 that

I

pn In

n=O m

converges uniformly on (0,1) for every m.

We still have to snow that

A(D )

t I if m ~ "'. This again follows

m

from proposition 3.4:

AI (A_k) =

I

(~)

(D

k = 1

-

I

(~)(Dk

2 _O:S;i<ak 1 _{O:S;i:s;(I-a)k} 1

1 k for some t with 0 < t < 1 •

~

-

_t

"" 00

Hence A I (n ) = II

_A!

_(~) :::: II (I-tk) t I if m ~ "".

D

References

[)J Breiman, L.: Probability. Reading, Massachusets: Addison-Wesley 1968.

[2J Chung, K.L. and W.H.J. Fuchs: On the distribution of sums of

random variables. Hem. A.M.S. nr. 6, 1-)2 (1951).

[3J Foguel, S.R.: The ergodic theory of Markov processes. New York:

Van Nostrand 1969.

[4J Foguel, S.R. and M. Lin: Some ratio limit theorems for Markov

operators. Z. Waht'sch. verw. Geb. 23, 55-66 (1972).

[5]

Lin,

M.: On quasi-compact Markov operators. The Annals of

Probability ~, 464-475 (1974).

[6J Ornstein, D.S.: On invariant measures. Bull. A.M.S. 66,

297-300 (1960).

[7J Simons, F.R.: Recurrence properties and periodicity for Markov

processes. Ph.D. thesis, Technological University