Sweep-out sets for Markov processes

Sweep-out sets for Markov processes

Citation for published version (APA):

Simons, F. H., & Overdijk, D. A. (1977). Sweep-out sets for Markov processes. (Memorandum COSOR; Vol. 7704). Technische Hogeschool Eindhoven.

Document status and date: Published: 01/01/1977 Document Version:

Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers) Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.

• The final author version and the galley proof are versions of the publication after peer review.

• The final published version features the final layout of the paper including the volume, issue and page numbers.

Link to publication

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:

www.tue.nl/taverne Take down policy

If you believe that this document breaches copyright please contact us at: openaccess@tue.nl

providing details and we will investigate your claim.

Department of Mathematics

PROBABILITY THEORY, STATISTICS AND OPERATIONS RESEARCH GROUP

Memorandum COSOR 77-04

Sweep-out sets for Markov processes

by

F.R. Simons and D.A. Overdijk

Eindhoven, February 1977 The Netherlands

Sweep-out sets for Markov processes by

F.H. Simons and D.A. Overdijk

For measurable transformations on a probability space it is known that the existence of arbitrary small sweep-out sets is equivalent to aperiodicity for the transformation [2J. In this note we shall investigate the existence of arbitrary small sweep-out sets for Markov processes on a probability space. We shall show that arbitrary small sweep-out sets always exist, unless the state space contains an invariant set on which the process behaves pointwise as in a finite state space.

Let P be a Markov process on a probability space (X,E,m), i.e. P is a

mapping of M+, the space of equivalence classes of almost everywhere equal

non negative extended real valued measurable functions, into itself such that

P(

l

ex f )

=

n=l n n _n=l

/,

PI s 1. ex Pf n n + (ex ~ 0, f EM), n n

Here, as in the sequel, all statements on sets and functions have to be inter-preted modulo m-null sets. Moreover, we shall assume that all sets we shall

. . l.n M+.

meet wl.ll be measurable and unless otherwise stated all functl.ons are

For reasons of convenience we shall also assume that the process P which we

shall consider satisfies PI

=

I.

For every A E E we define the embedded prooess Q

A by

Here IAmeans multiplication by the indicator function,6f the set A, and

AI

=

X \ A. The value QA1(x) may be interpreted as the probability that

starting in x, the process will visit the set A. From

(2) 1 on a set B .. lim (PIA') 1 = 0 on B . n

n-+<><>

As a preliminary we first note that if B ~ A, and if we embed the

process Q_A on the set B, we then obtain the process QB'

+

Lemma 1. I f B c A, then for all f E M we have

Proof. _QBf

₌

= ""

l.

(QAIB,)nQAIBf. n=O 00

l.

(PI_B, )~IBf n=O 00

L

_{(PIA' + PIAnB ,) PIAnBf} n

n=O

00 k

kl

=

I

l.

(PIA') _{PIAnB , PIAnB,(PIA,) PPIAnBf}

n=O kl+···+kp+p=n+1

00 00 00

kl k

L I

I

_{(PIA') PIAnB ,} (PIA') PPIAIBf

p=l k =0 k =0

1 p

00

1.

p-l +

= _{(QAIB') QAIBf for every f}

EM.

p=1

Definition 1. A set A is said to be a B~eep-out set for a set B if QA] =

on B.

A sweep-out set for X is usually called a sweep-out set.

Definition 2. A set R is said to be inva:r'iant i f PI

=

I on R. R

0

Lemma 2. If A E L is a sweep-out set and if R 1S an invariant set, then A n R

is a sweep-out set for R.

+

Proof. Since R 1S invariant we have PI

R, ~ lR" Therefore for all f E M

3

-00

IRQAnR 1 = IR

L

(PI(AnR)') P1 AnR n n=O 00

I

n = I _{(PIRI(AnR)') PI AnR} R n=O 00

I

n = IR (PIRI_A,) PIRI_A n=O 00

I

n

IR (PIA') PIA

=

IRQAl

=

IR •

n=O

Recall that by Hopf's decomposition theorem there exists a partition

of X into a oonservative part C and a dissipative papt D, characterized by

~

i)

I

pnlA ~ 0 or 00 on X for all A c C,

n=O

ii) there exists a sequence Dk t D such that

00

2

pn, < 00 on X for every k •

n=O Dk

For details the reader ~s referred to the book of Foguel [1], chapter II.

0

The conservative part C is invariant under P, hence for every sweep-out set A the set A nc is sweep-out set for C by .lemma 2. This shows one half of the next lemma.

Lemma 3. There exists an arbitrary small sweep-out set if and only if there exists an arbitrary small sweep-out set for C.

Proof. Choose any E > 0, and let A be a sweep-out set for C with meA) <

t.

Let the sequence Dk be as ~n condition ii) in the decomposition theorem and

determine k such that m(D \ D~) < ~. Put B = D \ D~, Then

Hence by (2) we have QCuBI = on X.

00

L

n=O

(Q 1 )"1 J I

CuB (AuB) , «CuB AuB

By (I) we have

It follows that QAuB I

=

QCuBI

set with meA u B) < E.

I on X, and therefore A u B is a sweep-out

By this lemma, we may suppose ~n the sequel that the process P is

con-servative, i.e. X

=

C. Since for any invariant set R we have PI

R ~ IR, we have in this case by [IJ, (2,9), PI

R

=

IR and it is easily seen that the

invariant sets now form a a-algebra, which we shall denote by L .• Moreover,

~

o

a function f E M+ is invariant under P, i.e. Pf

=

f, if and only if the function

f is L. -measurable- ([IJ, chapter III theorem A). We shall denote the class

~

+

of non negative invariant functions by M ••

~

4 . M+ M+.

Lemma . If P 1S conservative, then for all f E and g E we have

1

P(fg)

=

gPf •

Proof. It suffices to show the lemma for g

=

IR with R ELi.

Since PI_R

=

I_R, we have that P(fI

R)

=

0 on R' and P(fIR,)

=

0 on R.

Then from Pf

=

P(fI

R) + P(fIR,) we conclude that

Definition 3. For every R E L. with meR) > 0 we define

1

s(R)

=

sup{n

I

there exists a partition of R into n sweep-out sets for R} •

As a corollary of lemma 2 we note that if S is an invariant subset of R of

positive measure, then s(S) ~ s(R).

o

Theorem I. Let P be conservative. There exists a partition (X

I'X2' •.. ,Xoo) of

X into invariant sets such that for I ~ n ~ 00 for every invariant set

ReX of positive measure we have s(R)

=

n.

5

-Proof. For every n E N we define

R == {A E E.I V B c A ... (B == f) v s (B) = n)}.

n 1 BEE.

1

Using the fact that E.

1 is a a-algebra, it 1S easily verified that R n is

a a-ring. Let X be the (mod m) largest _n

variant subset R of X we have s(R)

=

n

n

element of R , then for every in-n

or meR) == 0.

Put Xoo

=

X \ U X_n' then Xoo is invariant. Suppose m(X_oo) > 0. nEE

If we can show sex ) ₀₀

=

00 _, then for all invariant. subsets R of X we have ₀₀

s(R)

=

00 or meR)

=

0, and the theorem will have been proved.

Suppose seX ) == N < 00. Define R =. {A c X

I

A E E., meA)

=

°

v s(A) > N}.

00 00 1

Again it is easily verified that R is a a-ring. Moreover, since Xoo n ~

=

f),

we have that R contains an element of positive measure, and therefore contains

a largest element R of positive measure. Then for all invariant subsets S of

X~\R we have s(S) ==

N

or m(S)

=

0, therefore X()O\R should belong to ~. It

fol-lows that m(X \R) ₀₀ == 0, R == X and therefore s(X ) _{00 00}

=

s(R) ;:: N+ 1. Contradiction,

hence s (X ) _()O

=

00.

o

Obviously, there exist arbitrary small sweep-out sets for Xoo' Therefore we still have to investigate whether or not there exist arbitrary small sweep-out sets for each of the sets X , 1 ~ n < 00.

. n

In the following we shall use the notation EIA for the class of

inter-sections of the elements of the a-algebra E with the set A~

We start with two technical results.

Lemma 5. Suppose X

=

Xn (n < (0) and let (A1, ••• ,A

n) be a sweep-out set

parti-tion of X. Then we have tiA.

=

E. IA. for I ~ j ~ n.

J 1 J

Proof. Obviously, we have EIA. ~ t.IA .•

J 1 J

Take A E riA. and let A* be the smallest invariant set containing A. Then,

J

since P is conservative, we have Q I

=

lA* by Foguel (3.3), and A is a

sweep-*

A

out set for A •

Put B

=

(A* n A.) \ A, and let B* be the smallest invariant set containing B.

* * J * *

Then B c A , and by lemma 2 the sets Al n B , ••• ,A. 1 n B ,

*

*

*

J-

*

A n B ,B,A. I n B , •.. ,A n Bare n+I disjoint sweep-out sets for B • It

J+

*

n

follows that m(B )

=

m(B)

=

0, and A

=

A. n A* E r.IA..

0

I

I

I

Lemma 6. Suppose X = Xn (n < 00), and let (i1, •.• ,A_n) be a sweep-out set

partition of X. Then for every j, I $ j $ n,and for every f E M+ there

.

*

M+.

ex~sts a unique function f. E such that

*

IA f

=

IA f . • • . J J J Moreover we have f~ = J J ~

Proof. For every B €E there exists by lemma 5 a set B* E E. such that

~

Therefore for such that lA.f

J

+

*

every f E

M

there exists at least one invariant function f.

*

J = IA f .• Then j J = 00 \ n

*

~ (PIA ,) PIA f J. n=O j j 00

= f; Y. (PIA.,)npIA . (by lemma 4)

n=O J J

* f*.

= f.QA 1 =

J j J

*

This implies the uniqueness of the function f., and completes the proof of

J

the lemma.

o

Now we show that for each of the sets X , n < 00 there do not exist arbitrary

n

small sweep-out sets.

Theorem 2. Suppose X = X (n < (0). There exists a sweep-out set S for X such n

that for every sweep-out set B we have m(B) ~ m(S).

Proof. Let (AI, .•• ,A

n) be a sweep-out set partition of X. For every R E Li

7

-v.(R) = meA. n R) for I ~ j ~ n .

J J

Since v. is a measure absolutely continuous with respect to m on (X,E.), the

J ~

dv.

Radon-Nikodym derivatives g. = - d J-exist on (X, E.).

J m ~

Put g = min(gl, ... ,gn),and define the function k by

k(x) = min{j

I

g.(x) = g(x)} for all x E X. Then k is E.-measurable. It

J ~

follows by lemma 2 that the set A. n {k=j} is a sweep-out set for {k=j}.

J n Since u j=l n {k=j} = X, ,the set S = u j=l A. n {k=j} is a sweep-out set. J

*

Let B be another sweep-out set. Put B. = B n A. for I ~ j ~ n, and let B.

J J J

be the smallest invariant set containing B .•

J

Then we have

m(B)

n n

L

m(B.) =

l.

m(B~ n A.) (by lemma 5)

j=l J j=l J J ~ ~

=

n

L

j=l n

L

j=l fg . (x)m(dx)

*

J B. J

J

g(x)m(dx) B. J n

*

fg(x)m(dX) (since u B. j=l n /. gJ' (x) l{k=J·}(x)m(dx) j=l n J =

L

meA. n {k=j}) = m(S) . J j=l X) Theorem 3. Suppose X equivalent to m.

X (n < 00). There exists an invariant probability

n

Proof. Let (AI, ••• ,A

n) be a sweep-out set partition of X. By lemma 6, we

have for every j, 1 s j s n, and for every f E M+ IA.QA.f=IA.f, fromwhich

J J J

we conclude lA.QA.

=

IA.' (Here we use the notation as given in Foguel, chapter 1.)

J J J

It follows by [IJ, chapter VI theorem C that there exists a function f. with

J

o

< f. < ~,with f.P

=

f. and IA f. = lA • (Note that in theorem C the condition

J J J j J j

E. is trivial is superfluous.) Take f

=

min(f1, ••• ,f ) then 0 < f ~ I, and

~ n

fP ~ f.P

=

f. for all j, hence fP

s

f.

J J

Since PI

=

I and f is integrable, it follows that fP

=

f. The function f is

the Radon-Nikodym derivative of an invariant equivalent finite measure and

normalisation yields an equivalent invariant probability.

o

If P is an irreducible Markov process on a finite state space X with n

elements, then X

=

X , since every state is a sweep-out set. The process is

n

described by a n~n-matrix of transition probabilities. We shall show now that

if X

=

X , _n the process actually can be described pointwise by a nxn-matrix which for every point gives the transition probabilities to the n states which can be reached under the action of the process.

More precisely, we shall show that the Markovprocess _. (X _n ,E,m,P) is isomorphic

with a Markov process (Q,f,jJ,M), where the Markov process M is defined

point-wise on Q by a transition probability such that for every state WE Q the

smallest (pointwise) invariant set containing W consists of n different states,

and the process is described on that invariant set by a nxn-matrix. Moreover

each entry of that matrix is a measurable function on (Q,F).

From now on we suppose X

=

X . We first define the space Q and the a-algebra

F.

n Q={{x,j) f = {A E Q X E X, I ~ j s n} {x

I

(x,j) E A} E E. for 1 ~ J s n} • ~

Now fix a sweep~out set partition (Al, .•• ,A

n) of X, and define the mapping

<1>: X -+ Q by

<j>(x)

=

(x,j) i f x E A . •

J

-I It ~s easily verified that this mapping is measurable, and therefore jJ

=

m<l> is a probability on (Q,f). Furthermore, by lemma 5, the mapping <1>-1: f[]J] -+ E[m] is an isomorphism between these two measure algebra's, hence if we associate

9

-with a function f on fl the function f 0 q, on X, we obtain an isomorphism

between the class of ~-almost everywhere equal non negative measurable

functions on nand

M+.

We now choose representatives for the functions QA.PIA. such that for

1. J

every x E X the matrix

Q_A PIA (x) 1 1 Q_A PIA (x) 1 n M -x Q_A PIA (x) _{QA PIA (x)} n I n n

is an irreducible transition matrix. Since the functions QA.PIA. E

M:

1. J

(lemma 6) we have that the entries of the matrix M are measurable functions

on (n,F).

For every non negative measurable function f on (n,F) we define the function Mf by Mf (x, 1) Mf(x,n) Q A PIA (x) f(x,l) J n Q A PIA (x) f(x,n) n n

Again it is a straightforward verification that in this way a Markov process

M on (n,F,~) is defined, which is pointwise given by the transition matrix M • _x

Theorem 4. Suppose X

=

X (n<oo). Then the Markov process (X,I:,m,P) is

iso-n

morphic with the Markov process (n,F,~,M).

Proof. Let f be a non negative measurable function on fl. The only thing we

still have to show is

P(f 0 q,)(x)

=

Mf(q,(x» for m-almost all x EX.

*

*

Put f.(x)

=

f(x,j), then because of the construction of F the function f. is an

J J

invariant function on X for I ~ j ~ n. Since

f 0 q,

we have

n

I

IA

f~,

P(f 0 4»

=

n

I

PIA

f~

=

j=1 j J n

l.

fJ~PIA

j=1 j by lemma 4.

On the other hand, we have

n Mf(x,i)

=

I

fJ~(x)QA.PIA.

(x) j=l 1. J hence n n Mf 0 q, =

_I

_L

*

lA, LQ_A P1 A 1 1. j=1 J 1. ' J ' n n

I

*

_I

= f. _{IA,QA,PI A,} j=l J i=1 1. 1. _J n

*

n

=

I

L

t

_{IA,PI A, (by lemma 6)}

J=1 J i=l 1. _J

n

I

*

=

_{f.PI A}

j=l J J '

Finally, we remark that if the process P is deterministic, i.e. for

o

. every A E Ii we have PIA

=

IB for some BEE, and if X

=

Xn (n < co), then we

may assume that for every x E X the irreducible matrix M has entries 0 and 1

x

only, and therefore represents a transformation T on {(x,I), ••. ,(x,n)} with

x

Tn _x

=

I. It follows that there exists an invertible transformation T on (O,F)

such that Mf

=

f 0 T for all functions in 0, and Mnf

=

f 0 Tn

=

f. Hence

for a deterministic process P on (X,E,m) we obtain that if there do not exist arbitrary small sweep-out sets, there must be an invariant set X with m(X ) _{. n n} > 0

such that pn

=

I on X • However, in general, we do not have a similar periodic

n

behaviour of the process if there do not exist arbitrary small sweep-out sets. This is easily shown, e.g. for the process P on { 1,2} determined by the matrix

( 1

tJ,

we have pn = P ;% I for all n.

References.

I. Foguel, S.R.: The ergodic theory of Markov processes. Van Nostrand Mathema-tical Studies #21, New York, Van Nostrand Reinhold 1969. 2. Helmberg, G., Simons, F.H.t Aperiodic transformations. Z. Wahrsch. verw.